2023-11-28T11:36:41.119 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5 2023-11-28T11:36:41.119 DEBUG:teuthology.run:Teuthology command: teuthology --name neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi --verbose --owner scheduled_neesingh@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} --archive /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962 -- /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962/orig.config.yaml 2023-11-28T11:36:41.161 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T11:36:41.242 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962 branch: wip-neesingh-testing-231123 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7469962' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-neesingh-testing-231123 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: issue_54557 sha1: 17c45c858c4cb3105f973d12c6c74f8d5037aa89 owner: scheduled_neesingh@teuthology priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4140 sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 sleep_before_teardown: 0 suite: fs suite_branch: issue_54557 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: 17c45c858c4cb3105f973d12c6c74f8d5037aa89 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi057.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA6+4W7Ok7NoF6bVdQc/c/LOeeQAcwpH+6HftP+t3kY+m8ErJj7jYo3zzEqgHMOHWK+VUvA6Fl7wmHXamG87gGs= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBrSn8IPPHQCmw++kQtET3aogXGzcc8RD5mq/rRSwIrRlek5htPlT3joY4zEeml9nV4qK/jmu/p0jz1EVyYBsYI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 5ae09a5cc7064ca2bf994df19f3006b79794789a timestamp: 2023-11-28_11:27:19 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-28T11:36:41.243 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa; will attempt to use it 2023-11-28T11:36:41.243 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks 2023-11-28T11:36:41.244 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-28T11:36:41.245 INFO:teuthology.task.internal:Checking packages... 2023-11-28T11:36:41.269 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f5f6028b893ad41357cd06e827373f0fe2a1232' 2023-11-28T11:36:41.269 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-28T11:36:41.269 INFO:teuthology.packaging:ref: None 2023-11-28T11:36:41.269 INFO:teuthology.packaging:tag: None 2023-11-28T11:36:41.269 INFO:teuthology.packaging:branch: wip-neesingh-testing-231123 2023-11-28T11:36:41.269 INFO:teuthology.packaging:sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:36:41.269 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-neesingh-testing-231123 2023-11-28T11:36:41.421 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2697.g2f5f6028 2023-11-28T11:36:41.421 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-28T11:36:41.427 INFO:teuthology.task.internal:no buildpackages task found 2023-11-28T11:36:41.428 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-28T11:36:41.462 INFO:teuthology.task.internal:Saving configuration 2023-11-28T11:36:41.478 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-28T11:36:41.505 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:36:41.528 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 11:31:37.858995', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA6+4W7Ok7NoF6bVdQc/c/LOeeQAcwpH+6HftP+t3kY+m8ErJj7jYo3zzEqgHMOHWK+VUvA6Fl7wmHXamG87gGs='} 2023-11-28T11:36:41.550 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 11:31:37.857232', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBrSn8IPPHQCmw++kQtET3aogXGzcc8RD5mq/rRSwIrRlek5htPlT3joY4zEeml9nV4qK/jmu/p0jz1EVyYBsYI='} 2023-11-28T11:36:41.550 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-28T11:36:41.556 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-28T11:36:41.556 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-28T11:36:41.556 INFO:teuthology.run_tasks:Running task console_log... 2023-11-28T11:36:41.741 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f746e62ea60>, signals=[15]) 2023-11-28T11:36:41.742 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-28T11:36:41.747 INFO:teuthology.task.internal:Opening connections... 2023-11-28T11:36:41.748 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T11:36:41.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:36:41.818 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T11:36:41.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:36:41.886 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-28T11:36:41.892 DEBUG:teuthology.orchestra.run.smithi057:> uname -m 2023-11-28T11:36:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64 2023-11-28T11:36:41.916 DEBUG:teuthology.orchestra.run.smithi057:> cat /etc/os-release 2023-11-28T11:36:41.978 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="CentOS Stream" 2023-11-28T11:36:41.979 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="8" 2023-11-28T11:36:41.979 INFO:teuthology.orchestra.run.smithi057.stdout:ID="centos" 2023-11-28T11:36:41.979 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE="rhel fedora" 2023-11-28T11:36:41.979 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="8" 2023-11-28T11:36:41.979 INFO:teuthology.orchestra.run.smithi057.stdout:PLATFORM_ID="platform:el8" 2023-11-28T11:36:41.979 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-28T11:36:41.979 INFO:teuthology.orchestra.run.smithi057.stdout:ANSI_COLOR="0;31" 2023-11-28T11:36:41.979 INFO:teuthology.orchestra.run.smithi057.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-28T11:36:41.979 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://centos.org/" 2023-11-28T11:36:41.979 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T11:36:41.979 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T11:36:41.979 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-28T11:36:41.980 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2023-11-28T11:36:42.000 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-11-28T11:36:42.022 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-11-28T11:36:42.022 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-11-28T11:36:42.084 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2023-11-28T11:36:42.084 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8" 2023-11-28T11:36:42.084 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2023-11-28T11:36:42.084 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2023-11-28T11:36:42.084 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8" 2023-11-28T11:36:42.084 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-11-28T11:36:42.084 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-28T11:36:42.085 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-11-28T11:36:42.085 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-28T11:36:42.085 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2023-11-28T11:36:42.085 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T11:36:42.085 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T11:36:42.085 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-28T11:36:42.085 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-11-28T11:36:42.107 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-28T11:36:42.117 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-28T11:36:42.154 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-28T11:36:42.154 DEBUG:teuthology.orchestra.run.smithi057:> test '!' -e /home/ubuntu/cephtest 2023-11-28T11:36:42.157 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2023-11-28T11:36:42.178 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-28T11:36:42.184 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-28T11:36:42.184 DEBUG:teuthology.orchestra.run.smithi057:> test -z $(ls -A /var/lib/ceph) 2023-11-28T11:36:42.219 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2023-11-28T11:36:42.250 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-28T11:36:42.299 INFO:teuthology.run_tasks:Running task kernel... 2023-11-28T11:36:42.314 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-11-28T11:36:42.314 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-11-28T11:36:42.314 DEBUG:teuthology.orchestra.run.smithi057:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T11:36:42.315 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T11:36:42.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:36:42.335 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2023-11-28T11:36:42.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:36:42.337 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2023-11-28T11:36:42.397 INFO:teuthology.orchestra.run.smithi181.stdout:4.18.0-489.el8.x86_64 2023-11-28T11:36:42.397 INFO:teuthology.task.kernel:Running kernel on smithi181: 4.18.0-489.el8.x86_64 2023-11-28T11:36:42.397 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum install -y kernel 2023-11-28T11:36:42.398 INFO:teuthology.orchestra.run.smithi057.stdout:4.18.0-489.el8.x86_64 2023-11-28T11:36:42.399 INFO:teuthology.task.kernel:Running kernel on smithi057: 4.18.0-489.el8.x86_64 2023-11-28T11:36:42.399 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum install -y kernel 2023-11-28T11:37:56.802 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 8 - AppStream 487 kB/s | 34 MB 01:11 2023-11-28T11:38:06.960 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 8 - BaseOS 16 MB/s | 55 MB 00:03 2023-11-28T11:38:15.660 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 8 - Extras 48 kB/s | 18 kB 00:00 2023-11-28T11:38:16.143 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 8 - Extras common packages 24 kB/s | 6.9 kB 00:00 2023-11-28T11:38:16.437 INFO:teuthology.orchestra.run.smithi057.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-11-28T11:38:17.745 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-11-28T11:38:21.771 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 420 kB/s | 24 kB 00:00 2023-11-28T11:38:25.612 INFO:teuthology.orchestra.run.smithi057.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-11-28T11:38:25.612 INFO:teuthology.orchestra.run.smithi057.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-11-28T11:38:25.906 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2023-11-28T11:38:25.907 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-11-28T11:38:25.907 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2023-11-28T11:38:25.907 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-11-28T11:38:25.907 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2023-11-28T11:38:25.907 INFO:teuthology.orchestra.run.smithi057.stdout: kernel x86_64 4.18.0-526.el8 baseos 10 M 2023-11-28T11:38:25.907 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2023-11-28T11:38:25.907 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-core x86_64 4.18.0-526.el8 baseos 43 M 2023-11-28T11:38:25.907 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-modules x86_64 4.18.0-526.el8 baseos 36 M 2023-11-28T11:38:25.907 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T11:38:25.907 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2023-11-28T11:38:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-11-28T11:38:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:Install 3 Packages 2023-11-28T11:38:25.908 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T11:38:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 90 M 2023-11-28T11:38:25.908 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 96 M 2023-11-28T11:38:25.909 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2023-11-28T11:38:27.355 INFO:teuthology.orchestra.run.smithi057.stdout:(1/3): kernel-4.18.0-526.el8.x86_64.rpm 8.0 MB/s | 10 MB 00:01 2023-11-28T11:38:28.430 INFO:teuthology.orchestra.run.smithi057.stdout:(2/3): kernel-modules-4.18.0-526.el8.x86_64.rpm 15 MB/s | 36 MB 00:02 2023-11-28T11:38:29.097 INFO:teuthology.orchestra.run.smithi057.stdout:(3/3): kernel-core-4.18.0-526.el8.x86_64.rpm 14 MB/s | 43 MB 00:02 2023-11-28T11:38:29.097 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:38:29.097 INFO:teuthology.orchestra.run.smithi057.stdout:Total 28 MB/s | 90 MB 00:03 2023-11-28T11:38:29.813 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2023-11-28T11:38:30.244 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2023-11-28T11:38:30.244 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2023-11-28T11:38:31.558 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2023-11-28T11:38:31.581 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2023-11-28T11:38:34.607 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2023-11-28T11:38:35.085 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : kernel-core-4.18.0-526.el8.x86_64 1/3 2023-11-28T11:38:38.853 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-core-4.18.0-526.el8.x86_64 1/3 2023-11-28T11:38:39.343 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : kernel-modules-4.18.0-526.el8.x86_64 2/3 2023-11-28T11:38:43.024 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-modules-4.18.0-526.el8.x86_64 2/3 2023-11-28T11:38:43.294 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : kernel-4.18.0-526.el8.x86_64 3/3 2023-11-28T11:39:06.679 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-core-4.18.0-526.el8.x86_64 3/3 2023-11-28T11:39:06.705 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-modules-4.18.0-526.el8.x86_64 3/3 2023-11-28T11:39:07.053 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-4.18.0-526.el8.x86_64 3/3 2023-11-28T11:39:07.053 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : kernel-4.18.0-526.el8.x86_64 1/3 2023-11-28T11:39:07.053 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : kernel-core-4.18.0-526.el8.x86_64 2/3 2023-11-28T11:39:07.845 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : kernel-modules-4.18.0-526.el8.x86_64 3/3 2023-11-28T11:39:07.845 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T11:39:07.845 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2023-11-28T11:39:07.845 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-4.18.0-526.el8.x86_64 kernel-core-4.18.0-526.el8.x86_64 2023-11-28T11:39:07.845 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-modules-4.18.0-526.el8.x86_64 2023-11-28T11:39:07.845 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T11:39:07.846 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2023-11-28T11:39:08.006 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T11:39:08.236 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-4.18.0-526.el8.x86_64 2023-11-28T11:39:08.236 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-526.el8.x86_64 2023-11-28T11:39:08.237 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-526.el8.x86_64 2023-11-28T11:39:08.237 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-28T11:39:08.237 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-28T11:39:08.237 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-11-28T11:39:08.237 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi057.front.sepia.ceph.com, path=None, version=distro) 2023-11-28T11:39:08.237 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T11:39:08.456 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-4.18.0-526.el8.x86_64 2023-11-28T11:39:08.456 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-526.el8.x86_64 2023-11-28T11:39:08.457 DEBUG:teuthology.orchestra.run.smithi057:> sudo rpm -qi grub2-tools 2023-11-28T11:39:08.491 INFO:teuthology.orchestra.run.smithi057.stdout:Name : grub2-tools 2023-11-28T11:39:08.491 INFO:teuthology.orchestra.run.smithi057.stdout:Epoch : 1 2023-11-28T11:39:08.492 INFO:teuthology.orchestra.run.smithi057.stdout:Version : 2.02 2023-11-28T11:39:08.492 INFO:teuthology.orchestra.run.smithi057.stdout:Release : 129.el8 2023-11-28T11:39:08.492 INFO:teuthology.orchestra.run.smithi057.stdout:Architecture: x86_64 2023-11-28T11:39:08.492 INFO:teuthology.orchestra.run.smithi057.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-11-28T11:39:08.492 INFO:teuthology.orchestra.run.smithi057.stdout:Group : System Environment/Base 2023-11-28T11:39:08.492 INFO:teuthology.orchestra.run.smithi057.stdout:Size : 9265586 2023-11-28T11:39:08.492 INFO:teuthology.orchestra.run.smithi057.stdout:License : GPLv3+ 2023-11-28T11:39:08.492 INFO:teuthology.orchestra.run.smithi057.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-11-28T11:39:08.492 INFO:teuthology.orchestra.run.smithi057.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-11-28T11:39:08.492 INFO:teuthology.orchestra.run.smithi057.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-11-28T11:39:08.492 INFO:teuthology.orchestra.run.smithi057.stdout:Build Host : kbuilder.bsys.centos.org 2023-11-28T11:39:08.492 INFO:teuthology.orchestra.run.smithi057.stdout:Relocations : (not relocatable) 2023-11-28T11:39:08.493 INFO:teuthology.orchestra.run.smithi057.stdout:Packager : CentOS BuildSystem 2023-11-28T11:39:08.493 INFO:teuthology.orchestra.run.smithi057.stdout:Vendor : CentOS 2023-11-28T11:39:08.493 INFO:teuthology.orchestra.run.smithi057.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-28T11:39:08.493 INFO:teuthology.orchestra.run.smithi057.stdout:Summary : Support tools for GRUB. 2023-11-28T11:39:08.493 INFO:teuthology.orchestra.run.smithi057.stdout:Description : 2023-11-28T11:39:08.493 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T11:39:08.493 INFO:teuthology.orchestra.run.smithi057.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-28T11:39:08.493 INFO:teuthology.orchestra.run.smithi057.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-28T11:39:08.493 INFO:teuthology.orchestra.run.smithi057.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-28T11:39:08.493 INFO:teuthology.orchestra.run.smithi057.stdout:hardware devices. 2023-11-28T11:39:08.493 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T11:39:08.493 INFO:teuthology.orchestra.run.smithi057.stdout:This subpackage provides tools for support of all platforms. 2023-11-28T11:39:08.494 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-28T11:39:08.494 INFO:teuthology.task.kernel:Updating grub on smithi057 to boot 4.18.0-526.el8.x86_64 2023-11-28T11:39:08.495 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-28T11:39:10.091 INFO:teuthology.orchestra.run.smithi057.stderr:Generating grub configuration file ... 2023-11-28T11:39:12.353 INFO:teuthology.orchestra.run.smithi057.stderr:done 2023-11-28T11:39:12.355 DEBUG:teuthology.orchestra.run.smithi057:> mktemp 2023-11-28T11:39:12.385 INFO:teuthology.orchestra.run.smithi057.stdout:/tmp/tmp.zWbIkY6gq1 2023-11-28T11:39:12.386 DEBUG:teuthology.orchestra.run.smithi057:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zWbIkY6gq1 2023-11-28T11:39:12.477 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0666 /tmp/tmp.zWbIkY6gq1 2023-11-28T11:39:12.581 DEBUG:teuthology.orchestra.remote:smithi057:/tmp/tmp.zWbIkY6gq1 is 6KB 2023-11-28T11:39:12.630 DEBUG:teuthology.orchestra.run.smithi057:> rm -fr /tmp/tmp.zWbIkY6gq1 2023-11-28T11:39:12.648 DEBUG:teuthology.orchestra.run.smithi057:> sudo /bin/ls /boot/loader/entries || true 2023-11-28T11:39:12.722 INFO:teuthology.orchestra.run.smithi057.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-11-28T11:39:12.722 INFO:teuthology.orchestra.run.smithi057.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-11-28T11:39:12.722 INFO:teuthology.orchestra.run.smithi057.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-11-28T11:39:12.722 INFO:teuthology.orchestra.run.smithi057.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-526.el8.x86_64.conf 2023-11-28T11:39:12.723 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-526.el8.x86_64 2023-11-28T11:39:12.855 DEBUG:teuthology.orchestra.run.smithi057:> sudo shutdown -r now 2023-11-28T11:39:21.561 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - AppStream 224 kB/s | 34 MB 02:36 2023-11-28T11:39:30.169 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - BaseOS 23 MB/s | 55 MB 00:02 2023-11-28T11:39:38.244 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - Extras 64 kB/s | 18 kB 00:00 2023-11-28T11:39:38.578 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - Extras common packages 45 kB/s | 6.9 kB 00:00 2023-11-28T11:39:38.879 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-11-28T11:39:40.422 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2023-11-28T11:39:42.857 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-28T11:39:42.858 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-11-28T11:39:42.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:39:44.213 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 403 kB/s | 24 kB 00:00 2023-11-28T11:39:47.821 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-11-28T11:39:47.821 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-11-28T11:39:48.110 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-11-28T11:39:48.111 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-11-28T11:39:48.111 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2023-11-28T11:39:48.111 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-11-28T11:39:48.111 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-11-28T11:39:48.111 INFO:teuthology.orchestra.run.smithi181.stdout: kernel x86_64 4.18.0-526.el8 baseos 10 M 2023-11-28T11:39:48.111 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-11-28T11:39:48.112 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-core x86_64 4.18.0-526.el8 baseos 43 M 2023-11-28T11:39:48.112 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-modules x86_64 4.18.0-526.el8 baseos 36 M 2023-11-28T11:39:48.112 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T11:39:48.112 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-11-28T11:39:48.112 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-11-28T11:39:48.112 INFO:teuthology.orchestra.run.smithi181.stdout:Install 3 Packages 2023-11-28T11:39:48.112 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T11:39:48.113 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 90 M 2023-11-28T11:39:48.113 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 96 M 2023-11-28T11:39:48.113 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-11-28T11:39:49.414 INFO:teuthology.orchestra.run.smithi181.stdout:(1/3): kernel-4.18.0-526.el8.x86_64.rpm 9.0 MB/s | 10 MB 00:01 2023-11-28T11:39:50.464 INFO:teuthology.orchestra.run.smithi181.stdout:(2/3): kernel-modules-4.18.0-526.el8.x86_64.rpm 17 MB/s | 36 MB 00:02 2023-11-28T11:39:51.015 INFO:teuthology.orchestra.run.smithi181.stdout:(3/3): kernel-core-4.18.0-526.el8.x86_64.rpm 16 MB/s | 43 MB 00:02 2023-11-28T11:39:51.015 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:39:51.016 INFO:teuthology.orchestra.run.smithi181.stdout:Total 31 MB/s | 90 MB 00:02 2023-11-28T11:39:51.692 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-11-28T11:39:52.114 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-11-28T11:39:52.115 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-11-28T11:39:53.343 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-11-28T11:39:53.372 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-11-28T11:39:56.153 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-11-28T11:39:56.678 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : kernel-core-4.18.0-526.el8.x86_64 1/3 2023-11-28T11:40:00.219 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-core-4.18.0-526.el8.x86_64 1/3 2023-11-28T11:40:00.635 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : kernel-modules-4.18.0-526.el8.x86_64 2/3 2023-11-28T11:40:04.062 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-modules-4.18.0-526.el8.x86_64 2/3 2023-11-28T11:40:04.325 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : kernel-4.18.0-526.el8.x86_64 3/3 2023-11-28T11:40:17.378 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2023-11-28T11:40:26.379 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-11-28T11:40:26.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:40:26.590 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-core-4.18.0-526.el8.x86_64 3/3 2023-11-28T11:40:26.615 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-modules-4.18.0-526.el8.x86_64 3/3 2023-11-28T11:40:27.017 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-4.18.0-526.el8.x86_64 3/3 2023-11-28T11:40:27.017 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-4.18.0-526.el8.x86_64 1/3 2023-11-28T11:40:27.018 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-core-4.18.0-526.el8.x86_64 2/3 2023-11-28T11:40:27.779 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-modules-4.18.0-526.el8.x86_64 3/3 2023-11-28T11:40:27.779 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T11:40:27.779 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-11-28T11:40:27.779 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-4.18.0-526.el8.x86_64 kernel-core-4.18.0-526.el8.x86_64 2023-11-28T11:40:27.779 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-modules-4.18.0-526.el8.x86_64 2023-11-28T11:40:27.779 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T11:40:27.779 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-11-28T11:40:27.921 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T11:40:28.141 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-4.18.0-526.el8.x86_64 2023-11-28T11:40:28.141 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-526.el8.x86_64 2023-11-28T11:40:28.141 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-526.el8.x86_64 2023-11-28T11:40:28.141 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-28T11:40:28.142 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-28T11:40:28.142 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-11-28T11:40:28.142 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi181.front.sepia.ceph.com, path=None, version=distro) 2023-11-28T11:40:28.142 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T11:40:28.351 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-4.18.0-526.el8.x86_64 2023-11-28T11:40:28.351 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-526.el8.x86_64 2023-11-28T11:40:28.351 DEBUG:teuthology.orchestra.run.smithi181:> sudo rpm -qi grub2-tools 2023-11-28T11:40:28.384 INFO:teuthology.orchestra.run.smithi181.stdout:Name : grub2-tools 2023-11-28T11:40:28.385 INFO:teuthology.orchestra.run.smithi181.stdout:Epoch : 1 2023-11-28T11:40:28.385 INFO:teuthology.orchestra.run.smithi181.stdout:Version : 2.02 2023-11-28T11:40:28.385 INFO:teuthology.orchestra.run.smithi181.stdout:Release : 129.el8 2023-11-28T11:40:28.385 INFO:teuthology.orchestra.run.smithi181.stdout:Architecture: x86_64 2023-11-28T11:40:28.385 INFO:teuthology.orchestra.run.smithi181.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-11-28T11:40:28.385 INFO:teuthology.orchestra.run.smithi181.stdout:Group : System Environment/Base 2023-11-28T11:40:28.385 INFO:teuthology.orchestra.run.smithi181.stdout:Size : 9265586 2023-11-28T11:40:28.385 INFO:teuthology.orchestra.run.smithi181.stdout:License : GPLv3+ 2023-11-28T11:40:28.385 INFO:teuthology.orchestra.run.smithi181.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-11-28T11:40:28.385 INFO:teuthology.orchestra.run.smithi181.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-11-28T11:40:28.385 INFO:teuthology.orchestra.run.smithi181.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-11-28T11:40:28.385 INFO:teuthology.orchestra.run.smithi181.stdout:Build Host : kbuilder.bsys.centos.org 2023-11-28T11:40:28.386 INFO:teuthology.orchestra.run.smithi181.stdout:Relocations : (not relocatable) 2023-11-28T11:40:28.386 INFO:teuthology.orchestra.run.smithi181.stdout:Packager : CentOS BuildSystem 2023-11-28T11:40:28.386 INFO:teuthology.orchestra.run.smithi181.stdout:Vendor : CentOS 2023-11-28T11:40:28.386 INFO:teuthology.orchestra.run.smithi181.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-28T11:40:28.386 INFO:teuthology.orchestra.run.smithi181.stdout:Summary : Support tools for GRUB. 2023-11-28T11:40:28.386 INFO:teuthology.orchestra.run.smithi181.stdout:Description : 2023-11-28T11:40:28.386 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T11:40:28.386 INFO:teuthology.orchestra.run.smithi181.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-28T11:40:28.386 INFO:teuthology.orchestra.run.smithi181.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-28T11:40:28.386 INFO:teuthology.orchestra.run.smithi181.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-28T11:40:28.386 INFO:teuthology.orchestra.run.smithi181.stdout:hardware devices. 2023-11-28T11:40:28.386 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T11:40:28.387 INFO:teuthology.orchestra.run.smithi181.stdout:This subpackage provides tools for support of all platforms. 2023-11-28T11:40:28.387 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-28T11:40:28.387 INFO:teuthology.task.kernel:Updating grub on smithi181 to boot 4.18.0-526.el8.x86_64 2023-11-28T11:40:28.388 DEBUG:teuthology.orchestra.run.smithi181:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-28T11:40:29.682 INFO:teuthology.orchestra.run.smithi181.stderr:Generating grub configuration file ... 2023-11-28T11:40:31.888 INFO:teuthology.orchestra.run.smithi181.stderr:done 2023-11-28T11:40:31.890 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2023-11-28T11:40:31.919 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.HYnHYwdhDI 2023-11-28T11:40:31.919 DEBUG:teuthology.orchestra.run.smithi181:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HYnHYwdhDI 2023-11-28T11:40:31.993 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.HYnHYwdhDI 2023-11-28T11:40:32.142 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.HYnHYwdhDI is 6KB 2023-11-28T11:40:32.192 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.HYnHYwdhDI 2023-11-28T11:40:32.211 DEBUG:teuthology.orchestra.run.smithi181:> sudo /bin/ls /boot/loader/entries || true 2023-11-28T11:40:32.276 INFO:teuthology.orchestra.run.smithi181.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-11-28T11:40:32.277 INFO:teuthology.orchestra.run.smithi181.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-11-28T11:40:32.277 INFO:teuthology.orchestra.run.smithi181.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-11-28T11:40:32.277 INFO:teuthology.orchestra.run.smithi181.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-526.el8.x86_64.conf 2023-11-28T11:40:32.278 DEBUG:teuthology.orchestra.run.smithi181:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-526.el8.x86_64 2023-11-28T11:40:32.415 DEBUG:teuthology.orchestra.run.smithi181:> sudo shutdown -r now 2023-11-28T11:41:02.448 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-28T11:41:02.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-11-28T11:41:02.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:41:26.406 DEBUG:teuthology.orchestra.remote:timed out 2023-11-28T11:41:38.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-11-28T11:41:38.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:41:38.818 DEBUG:teuthology.orchestra.run.smithi057:> true 2023-11-28T11:41:39.104 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-11-28T11:41:39.104 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-526.el8.x86_64"... 2023-11-28T11:41:39.104 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2023-11-28T11:41:39.126 INFO:teuthology.orchestra.run.smithi057.stdout:4.18.0-526.el8.x86_64 2023-11-28T11:41:39.126 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-526.el8.x86_64 vs 4.18.0-526.el8.x86_64 2023-11-28T11:41:39.126 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-28T11:41:39.127 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-28T11:41:40.127 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-28T11:41:40.127 DEBUG:teuthology.orchestra.run.smithi057:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-28T11:41:40.214 INFO:teuthology.orchestra.run.smithi057.stdout:ttyS1 2023-11-28T11:41:40.229 DEBUG:teuthology.parallel:result is None 2023-11-28T11:42:02.452 DEBUG:teuthology.orchestra.remote:timed out 2023-11-28T11:42:11.453 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-11-28T11:42:11.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:42:11.798 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-11-28T11:42:12.076 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-11-28T11:42:12.076 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-526.el8.x86_64"... 2023-11-28T11:42:12.076 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2023-11-28T11:42:12.092 INFO:teuthology.orchestra.run.smithi181.stdout:4.18.0-526.el8.x86_64 2023-11-28T11:42:12.092 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-526.el8.x86_64 vs 4.18.0-526.el8.x86_64 2023-11-28T11:42:12.093 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-28T11:42:12.093 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-28T11:42:13.093 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-28T11:42:13.094 DEBUG:teuthology.orchestra.run.smithi181:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-28T11:42:13.201 INFO:teuthology.orchestra.run.smithi181.stdout:ttyS1 2023-11-28T11:42:13.216 DEBUG:teuthology.parallel:result is None 2023-11-28T11:42:13.216 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-28T11:42:13.223 INFO:teuthology.task.internal:Creating test directory... 2023-11-28T11:42:13.223 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T11:42:13.226 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T11:42:13.248 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-28T11:42:13.267 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-28T11:42:13.301 INFO:teuthology.task.internal:Creating archive directory... 2023-11-28T11:42:13.301 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T11:42:13.303 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T11:42:13.342 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-28T11:42:13.351 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-28T11:42:13.352 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T11:42:13.378 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T11:42:13.413 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:42:13.416 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:42:13.425 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:42:13.426 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:42:13.428 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-28T11:42:13.435 INFO:teuthology.task.internal:Configuring sudo... 2023-11-28T11:42:13.435 DEBUG:teuthology.orchestra.run.smithi057:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T11:42:13.470 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T11:42:13.503 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-28T11:42:13.511 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-28T11:42:13.512 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T11:42:13.546 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T11:42:13.567 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:42:13.658 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:42:13.724 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:42:13.774 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:42:13.826 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T11:42:13.826 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T11:42:13.900 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:42:13.946 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:42:13.991 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:42:14.071 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:42:14.110 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T11:42:14.110 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T11:42:14.174 DEBUG:teuthology.orchestra.run.smithi057:> sudo service rsyslog restart 2023-11-28T11:42:14.176 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2023-11-28T11:42:14.263 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:42:14.290 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:42:14.621 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-28T11:42:14.629 INFO:teuthology.task.internal:Starting timer... 2023-11-28T11:42:14.629 INFO:teuthology.run_tasks:Running task pcp... 2023-11-28T11:42:14.654 INFO:teuthology.run_tasks:Running task selinux... 2023-11-28T11:42:14.687 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-28T11:42:14.687 DEBUG:teuthology.orchestra.run.smithi057:> sudo service auditd rotate 2023-11-28T11:42:14.774 INFO:teuthology.orchestra.run.smithi057.stdout:Rotating logs: 2023-11-28T11:42:14.776 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2023-11-28T11:42:14.898 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2023-11-28T11:42:14.899 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-28T11:42:14.900 DEBUG:teuthology.orchestra.run.smithi057:> /usr/sbin/getenforce 2023-11-28T11:42:14.949 INFO:teuthology.orchestra.run.smithi057.stdout:Permissive 2023-11-28T11:42:14.949 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2023-11-28T11:42:15.031 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2023-11-28T11:42:15.031 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi057.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive'} 2023-11-28T11:42:15.032 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:42:15.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:42:15.067 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:42:15.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:42:15.095 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-28T11:42:15.095 DEBUG:teuthology.orchestra.run.smithi057:> sudo /usr/sbin/setenforce permissive 2023-11-28T11:42:15.144 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2023-11-28T11:42:15.362 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-28T11:42:15.372 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-28T11:42:15.372 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-28T11:42:15.389 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-28T11:42:15.390 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi057.front.sepia.ceph.com,smithi181.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-28T11:53:31.716 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi057.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com')] 2023-11-28T11:53:31.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-11-28T11:53:31.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:53:31.791 DEBUG:teuthology.orchestra.run.smithi057:> true 2023-11-28T11:53:31.869 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-11-28T11:53:31.869 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-11-28T11:53:31.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:53:31.937 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-11-28T11:53:32.006 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-11-28T11:53:32.006 INFO:teuthology.run_tasks:Running task clock... 2023-11-28T11:53:32.017 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-28T11:53:32.017 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:53:32.017 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:53:32.020 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:53:32.020 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:53:32.056 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:53:32.072 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:53:32.080 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:53:32.095 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:53:32.138 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: ntpd: command not found 2023-11-28T11:53:32.151 INFO:teuthology.orchestra.run.smithi057.stdout:506 Cannot talk to daemon 2023-11-28T11:53:32.165 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2023-11-28T11:53:32.168 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:53:32.176 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2023-11-28T11:53:32.184 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:53:32.190 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:53:32.203 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:53:32.250 INFO:teuthology.orchestra.run.smithi057.stderr:bash: ntpq: command not found 2023-11-28T11:53:32.255 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:53:32.255 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2023-11-28T11:53:32.255 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:53:32.256 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:53:32.256 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:53:32.256 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:53:32.274 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-11-28T11:53:32.278 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:53:32.278 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-11-28T11:53:32.278 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:53:32.278 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:53:32.278 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:53:32.278 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:53:32.279 INFO:teuthology.run_tasks:Running task install... 2023-11-28T11:53:32.290 DEBUG:teuthology.task.install:project ceph 2023-11-28T11:53:32.290 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}} 2023-11-28T11:53:32.290 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'} 2023-11-28T11:53:32.290 INFO:teuthology.task.install:Using flavor: default 2023-11-28T11:53:32.296 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-28T11:53:32.296 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-28T11:53:32.297 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:53:32.297 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:53:32.300 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:53:32.300 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:53:32.441 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:53:32.441 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:53:32.457 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:53:32.457 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:53:32.559 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:53:32.560 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T11:53:32.560 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:53:32.596 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:53:32.597 DEBUG:teuthology.orchestra.run.smithi057:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:53:32.623 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:53:32.623 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T11:53:32.624 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:53:32.652 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:53:32.653 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:53:32.673 DEBUG:teuthology.orchestra.run.smithi057:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:53:32.722 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:53:32.760 DEBUG:teuthology.orchestra.run.smithi057:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:53:32.793 INFO:teuthology.orchestra.run.smithi057.stdout:check_obsoletes = 1 2023-11-28T11:53:32.794 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum clean all 2023-11-28T11:53:32.799 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:53:32.865 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2023-11-28T11:53:32.866 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2023-11-28T11:53:33.110 INFO:teuthology.orchestra.run.smithi057.stdout:76 files removed 2023-11-28T11:53:33.136 INFO:teuthology.orchestra.run.smithi181.stdout:76 files removed 2023-11-28T11:53:33.142 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:53:33.165 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:53:33.888 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 252 kB/s | 80 kB 00:00 2023-11-28T11:53:34.012 INFO:teuthology.orchestra.run.smithi057.stdout:ceph packages for x86_64 199 kB/s | 80 kB 00:00 2023-11-28T11:53:34.289 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 26 kB/s | 9.4 kB 00:00 2023-11-28T11:53:34.354 INFO:teuthology.orchestra.run.smithi057.stdout:ceph noarch packages 32 kB/s | 9.4 kB 00:00 2023-11-28T11:53:34.556 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 6.2 kB/s | 1.5 kB 00:00 2023-11-28T11:53:34.721 INFO:teuthology.orchestra.run.smithi057.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2023-11-28T11:56:04.156 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - AppStream 234 kB/s | 34 MB 02:29 2023-11-28T11:56:13.155 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - BaseOS 20 MB/s | 55 MB 00:02 2023-11-28T11:56:20.933 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 8 - AppStream 211 kB/s | 34 MB 02:46 2023-11-28T11:56:21.063 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - Extras 124 kB/s | 18 kB 00:00 2023-11-28T11:56:21.388 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - Extras common packages 48 kB/s | 6.9 kB 00:00 2023-11-28T11:56:22.690 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 8 - PowerTools 5.8 MB/s | 6.6 MB 00:01 2023-11-28T11:56:24.133 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-11-28T11:56:25.367 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2023-11-28T11:56:29.333 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 375 kB/s | 24 kB 00:00 2023-11-28T11:56:31.117 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 8 - BaseOS 17 MB/s | 55 MB 00:03 2023-11-28T11:56:33.666 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:56:33.667 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:56:33.681 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-11-28T11:56:33.681 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-11-28T11:56:33.682 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-11-28T11:56:33.683 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-11-28T11:56:33.684 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:56:33.686 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:56:33.689 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:56:33.693 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:56:33.941 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-11-28T11:56:33.954 INFO:teuthology.orchestra.run.smithi181.stdout:==================================================================================================== 2023-11-28T11:56:33.955 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-11-28T11:56:33.955 INFO:teuthology.orchestra.run.smithi181.stdout:==================================================================================================== 2023-11-28T11:56:33.955 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-11-28T11:56:33.955 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-11-28T11:56:33.955 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:56:33.955 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:56:33.955 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:56:33.955 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:56:33.955 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:56:33.955 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:56:33.955 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:56:33.955 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:56:33.956 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:56:33.956 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:56:33.956 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:56:33.956 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:56:33.956 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:56:33.956 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:56:33.956 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:56:33.956 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.190-1.el8 baseos 62 k 2023-11-28T11:56:33.956 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-11-28T11:56:33.956 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-11-28T11:56:33.956 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:56:33.956 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:56:33.957 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:56:33.957 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-11-28T11:56:33.957 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-11-28T11:56:33.957 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:1.1.1k-11.el8 baseos 3.2 M 2023-11-28T11:56:33.957 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 10.23-2.module_el8+572+929c87ac appstream 1.6 M 2023-11-28T11:56:33.957 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 10.23-2.module_el8+572+929c87ac appstream 901 k 2023-11-28T11:56:33.957 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 10.23-2.module_el8+572+929c87ac appstream 5.1 M 2023-11-28T11:56:33.957 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:56:33.957 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:56:33.957 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:56:33.957 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:56:33.957 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:56:33.957 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:56:33.958 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:56:33.958 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-11-28T11:56:33.958 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2023-11-28T11:56:33.958 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-11.el8 baseos 144 k 2023-11-28T11:56:33.958 INFO:teuthology.orchestra.run.smithi181.stdout: acl x86_64 2.2.53-3.el8 baseos 83 k 2023-11-28T11:56:33.958 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-debuginfod-client x86_64 0.190-1.el8 baseos 77 k 2023-11-28T11:56:33.958 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf x86_64 0.190-1.el8 baseos 232 k 2023-11-28T11:56:33.958 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libs x86_64 0.190-1.el8 baseos 305 k 2023-11-28T11:56:33.958 INFO:teuthology.orchestra.run.smithi181.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-11-28T11:56:33.958 INFO:teuthology.orchestra.run.smithi181.stdout: libacl x86_64 2.2.53-3.el8 baseos 35 k 2023-11-28T11:56:33.958 INFO:teuthology.orchestra.run.smithi181.stdout: libacl-devel x86_64 2.2.53-3.el8 baseos 85 k 2023-11-28T11:56:33.958 INFO:teuthology.orchestra.run.smithi181.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-11-28T11:56:33.959 INFO:teuthology.orchestra.run.smithi181.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-11-28T11:56:33.959 INFO:teuthology.orchestra.run.smithi181.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-11-28T11:56:33.959 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:56:33.959 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:56:33.959 INFO:teuthology.orchestra.run.smithi181.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-11-28T11:56:33.959 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-11-28T11:56:33.959 INFO:teuthology.orchestra.run.smithi181.stdout: openssl x86_64 1:1.1.1k-11.el8 baseos 737 k 2023-11-28T11:56:33.959 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs x86_64 1:1.1.1k-11.el8 baseos 1.5 M 2023-11-28T11:56:33.959 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-11-28T11:56:33.959 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-11-28T11:56:33.959 INFO:teuthology.orchestra.run.smithi181.stdout: xfsdump x86_64 3.1.8-7.el8 baseos 334 k 2023-11-28T11:56:33.959 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-11-28T11:56:33.959 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-11-28T11:56:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-11-28T11:56:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:56:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:56:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:56:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:56:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:56:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:56:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:56:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:56:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-11-28T11:56:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:56:33.960 INFO:teuthology.orchestra.run.smithi181.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2023-11-28T11:56:33.961 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-11-28T11:56:33.961 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-11-28T11:56:33.961 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:56:33.961 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-11-28T11:56:33.961 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2023-11-28T11:56:33.961 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:56:33.961 INFO:teuthology.orchestra.run.smithi181.stdout: libpq x86_64 13.5-1.el8 appstream 198 k 2023-11-28T11:56:33.961 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2023-11-28T11:56:33.961 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-11-28T11:56:33.961 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:56:33.961 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-11-28T11:56:33.961 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:56:33.961 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-11-28T11:56:33.962 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-11-28T11:56:33.962 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:56:33.962 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-11-28T11:56:33.962 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-11-28T11:56:33.962 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-11-28T11:56:33.962 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-11-28T11:56:33.962 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-11-28T11:56:33.962 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-11-28T11:56:33.962 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-11-28T11:56:33.962 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-11-28T11:56:33.962 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-11-28T11:56:33.962 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:56:33.963 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-11-28T11:56:33.963 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:56:33.963 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:56:33.963 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:56:33.963 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:56:33.963 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:56:33.963 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:56:33.963 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-11-28T11:56:33.963 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:56:33.963 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:56:33.963 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-11-28T11:56:33.963 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:56:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:56:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:56:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-11-28T11:56:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-11-28T11:56:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-11-28T11:56:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-11-28T11:56:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:56:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:56:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-11-28T11:56:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-11-28T11:56:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:56:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:56:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-11-28T11:56:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-11-28T11:56:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-11-28T11:56:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:56:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:56:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-11-28T11:56:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-11-28T11:56:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-11-28T11:56:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-11-28T11:56:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-11-28T11:56:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-11-28T11:56:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:56:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-11-28T11:56:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:56:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:56:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-11-28T11:56:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:56:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:56:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:56:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:56:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:56:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:56:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:56:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:56:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-11-28T11:56:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:56:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-11-28T11:56:33.967 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:56:33.967 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-43.el8 appstream 64 k 2023-11-28T11:56:33.967 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-11-28T11:56:33.967 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-11-28T11:56:33.967 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling module streams: 2023-11-28T11:56:33.967 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql 10 2023-11-28T11:56:33.967 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T11:56:33.967 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-11-28T11:56:33.967 INFO:teuthology.orchestra.run.smithi181.stdout:==================================================================================================== 2023-11-28T11:56:33.967 INFO:teuthology.orchestra.run.smithi181.stdout:Install 128 Packages 2023-11-28T11:56:33.967 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 21 Packages 2023-11-28T11:56:33.967 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T11:56:33.968 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 212 M 2023-11-28T11:56:33.968 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-11-28T11:56:34.601 INFO:teuthology.orchestra.run.smithi181.stdout:(1/149): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 18 kB/s | 6.4 kB 00:00 2023-11-28T11:56:35.252 INFO:teuthology.orchestra.run.smithi181.stdout:(2/149): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.3 MB/s | 845 kB 00:00 2023-11-28T11:56:35.360 INFO:teuthology.orchestra.run.smithi181.stdout:(3/149): ceph-immutable-object-cache-18.0.0-269 1.5 MB/s | 165 kB 00:00 2023-11-28T11:56:36.060 INFO:teuthology.orchestra.run.smithi181.stdout:(4/149): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 3.0 MB/s | 5.4 MB 00:01 2023-11-28T11:56:36.261 INFO:teuthology.orchestra.run.smithi181.stdout:(5/149): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 2.5 MB/s | 2.3 MB 00:00 2023-11-28T11:56:36.378 INFO:teuthology.orchestra.run.smithi181.stdout:(6/149): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 4.9 MB/s | 1.5 MB 00:00 2023-11-28T11:56:37.370 INFO:teuthology.orchestra.run.smithi181.stdout:(7/149): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 4.1 MB/s | 4.6 MB 00:01 2023-11-28T11:56:38.220 INFO:teuthology.orchestra.run.smithi181.stdout:(8/149): ceph-osd-18.0.0-2697.g2f5f6028.el8.x86 9.6 MB/s | 18 MB 00:01 2023-11-28T11:56:38.337 INFO:teuthology.orchestra.run.smithi181.stdout:(9/149): ceph-selinux-18.0.0-2697.g2f5f6028.el8 208 kB/s | 24 kB 00:00 2023-11-28T11:56:38.629 INFO:teuthology.orchestra.run.smithi181.stdout:(10/149): ceph-common-18.0.0-2697.g2f5f6028.el8 4.9 MB/s | 22 MB 00:04 2023-11-28T11:56:38.779 INFO:teuthology.orchestra.run.smithi181.stdout:(11/149): cephfs-mirror-18.0.0-2697.g2f5f6028.e 1.7 MB/s | 257 kB 00:00 2023-11-28T11:56:38.904 INFO:teuthology.orchestra.run.smithi181.stdout:(12/149): libcephfs-devel-18.0.0-2697.g2f5f6028 244 kB/s | 30 kB 00:00 2023-11-28T11:56:39.059 INFO:teuthology.orchestra.run.smithi181.stdout:(13/149): libcephfs2-18.0.0-2697.g2f5f6028.el8. 4.9 MB/s | 767 kB 00:00 2023-11-28T11:56:39.205 INFO:teuthology.orchestra.run.smithi181.stdout:(14/149): libcephsqlite-18.0.0-2697.g2f5f6028.e 941 kB/s | 137 kB 00:00 2023-11-28T11:56:39.313 INFO:teuthology.orchestra.run.smithi181.stdout:(15/149): librados-devel-18.0.0-2697.g2f5f6028. 1.3 MB/s | 139 kB 00:00 2023-11-28T11:56:39.447 INFO:teuthology.orchestra.run.smithi181.stdout:(16/149): libradosstriper1-18.0.0-2697.g2f5f602 3.5 MB/s | 474 kB 00:00 2023-11-28T11:56:40.106 INFO:teuthology.orchestra.run.smithi181.stdout:(17/149): ceph-radosgw-18.0.0-2697.g2f5f6028.el 4.8 MB/s | 13 MB 00:02 2023-11-28T11:56:40.125 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 8 - Extras 57 kB/s | 18 kB 00:00 2023-11-28T11:56:40.198 INFO:teuthology.orchestra.run.smithi181.stdout:(18/149): librgw2-18.0.0-2697.g2f5f6028.el8.x86 8.7 MB/s | 6.5 MB 00:00 2023-11-28T11:56:40.240 INFO:teuthology.orchestra.run.smithi181.stdout:(19/149): python3-ceph-argparse-18.0.0-2697.g2f 334 kB/s | 45 kB 00:00 2023-11-28T11:56:40.340 INFO:teuthology.orchestra.run.smithi181.stdout:(20/149): python3-ceph-common-18.0.0-2697.g2f5f 826 kB/s | 117 kB 00:00 2023-11-28T11:56:40.365 INFO:teuthology.orchestra.run.smithi181.stdout:(21/149): python3-cephfs-18.0.0-2697.g2f5f6028. 1.5 MB/s | 190 kB 00:00 2023-11-28T11:56:40.466 INFO:teuthology.orchestra.run.smithi181.stdout:(22/149): python3-rados-18.0.0-2697.g2f5f6028.e 2.9 MB/s | 371 kB 00:00 2023-11-28T11:56:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 8 - Extras common packages 38 kB/s | 6.9 kB 00:00 2023-11-28T11:56:40.516 INFO:teuthology.orchestra.run.smithi181.stdout:(23/149): python3-rbd-18.0.0-2697.g2f5f6028.el8 2.4 MB/s | 364 kB 00:00 2023-11-28T11:56:40.617 INFO:teuthology.orchestra.run.smithi181.stdout:(24/149): python3-rgw-18.0.0-2697.g2f5f6028.el8 709 kB/s | 107 kB 00:00 2023-11-28T11:56:40.650 INFO:teuthology.orchestra.run.smithi181.stdout:(25/149): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 707 kB/s | 94 kB 00:00 2023-11-28T11:56:40.784 INFO:teuthology.orchestra.run.smithi181.stdout:(26/149): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 1.3 MB/s | 180 kB 00:00 2023-11-28T11:56:40.942 INFO:teuthology.orchestra.run.smithi181.stdout:(27/149): ceph-grafana-dashboards-18.0.0-2697.g 146 kB/s | 23 kB 00:00 2023-11-28T11:56:41.284 INFO:teuthology.orchestra.run.smithi181.stdout:(28/149): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 359 kB/s | 123 kB 00:00 2023-11-28T11:56:41.559 INFO:teuthology.orchestra.run.smithi181.stdout:(29/149): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 6.3 MB/s | 1.7 MB 00:00 2023-11-28T11:56:41.618 INFO:teuthology.orchestra.run.smithi181.stdout:(30/149): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 3.4 MB/s | 3.4 MB 00:01 2023-11-28T11:56:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:(31/149): ceph-mgr-modules-core-18.0.0-2697.g2f 2.1 MB/s | 236 kB 00:00 2023-11-28T11:56:41.817 INFO:teuthology.orchestra.run.smithi181.stdout:(32/149): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 526 kB/s | 47 kB 00:00 2023-11-28T11:56:41.886 INFO:teuthology.orchestra.run.smithi181.stdout:(33/149): ceph-prometheus-alerts-18.0.0-2697.g2 213 kB/s | 15 kB 00:00 2023-11-28T11:56:41.978 INFO:teuthology.orchestra.run.smithi181.stdout:(34/149): ceph-volume-18.0.0-2697.g2f5f6028.el8 2.8 MB/s | 263 kB 00:00 2023-11-28T11:56:42.087 INFO:teuthology.orchestra.run.smithi181.stdout:(35/149): cephadm-18.0.0-2697.g2f5f6028.el8.noa 763 kB/s | 83 kB 00:00 2023-11-28T11:56:42.170 INFO:teuthology.orchestra.run.smithi181.stdout:(36/149): cephfs-top-18.0.0-2697.g2f5f6028.el8. 217 kB/s | 18 kB 00:00 2023-11-28T11:56:42.646 INFO:teuthology.orchestra.run.smithi181.stdout:(37/149): ceph-mgr-diskprediction-local-18.0.0- 6.8 MB/s | 7.4 MB 00:01 2023-11-28T11:56:43.437 INFO:teuthology.orchestra.run.smithi181.stdout:(38/149): ceph-test-18.0.0-2697.g2f5f6028.el8.x 9.1 MB/s | 47 MB 00:05 2023-11-28T11:56:43.518 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 8 - PowerTools 2.3 MB/s | 6.6 MB 00:02 2023-11-28T11:56:44.338 INFO:teuthology.orchestra.run.smithi181.stdout:(39/149): flex-2.6.1-9.el8.x86_64.rpm 355 kB/s | 320 kB 00:00 2023-11-28T11:56:44.838 INFO:teuthology.orchestra.run.smithi181.stdout:(40/149): jq-1.6-8.el8.x86_64.rpm 407 kB/s | 203 kB 00:00 2023-11-28T11:56:45.038 INFO:teuthology.orchestra.run.smithi181.stdout:(41/149): bison-3.0.4-10.el8.x86_64.rpm 288 kB/s | 688 kB 00:02 2023-11-28T11:56:45.094 INFO:teuthology.orchestra.run.smithi057.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-11-28T11:56:45.330 INFO:teuthology.orchestra.run.smithi181.stdout:(42/149): libpq-13.5-1.el8.x86_64.rpm 402 kB/s | 198 kB 00:00 2023-11-28T11:56:46.288 INFO:teuthology.orchestra.run.smithi181.stdout:(43/149): librdkafka-0.11.4-3.el8.x86_64.rpm 283 kB/s | 354 kB 00:01 2023-11-28T11:56:46.870 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-11-28T11:56:46.889 INFO:teuthology.orchestra.run.smithi181.stdout:(44/149): libtool-2.4.6-25.el8.x86_64.rpm 455 kB/s | 709 kB 00:01 2023-11-28T11:56:47.297 INFO:teuthology.orchestra.run.smithi181.stdout:(45/149): lttng-ust-2.8.1-11.el8.x86_64.rpm 257 kB/s | 259 kB 00:01 2023-11-28T11:56:47.339 INFO:teuthology.orchestra.run.smithi181.stdout:(46/149): oniguruma-6.8.2-2.el8.x86_64.rpm 416 kB/s | 187 kB 00:00 2023-11-28T11:56:49.238 INFO:teuthology.orchestra.run.smithi181.stdout:(47/149): automake-1.16.1-8.el8.noarch.rpm 105 kB/s | 740 kB 00:07 2023-11-28T11:56:51.206 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 317 kB/s | 24 kB 00:00 2023-11-28T11:56:55.489 INFO:teuthology.orchestra.run.smithi181.stdout:(48/149): openblas-threads-0.3.15-4.el8.x86_64. 610 kB/s | 4.9 MB 00:08 2023-11-28T11:56:55.996 INFO:teuthology.orchestra.run.smithi057.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:56:55.997 INFO:teuthology.orchestra.run.smithi057.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:56:56.013 INFO:teuthology.orchestra.run.smithi057.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-11-28T11:56:56.013 INFO:teuthology.orchestra.run.smithi057.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-11-28T11:56:56.014 INFO:teuthology.orchestra.run.smithi057.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-11-28T11:56:56.015 INFO:teuthology.orchestra.run.smithi057.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-11-28T11:56:56.016 INFO:teuthology.orchestra.run.smithi057.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:56:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:56:56.022 INFO:teuthology.orchestra.run.smithi057.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:56:56.023 INFO:teuthology.orchestra.run.smithi057.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:56:56.313 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2023-11-28T11:56:56.330 INFO:teuthology.orchestra.run.smithi057.stdout:==================================================================================================== 2023-11-28T11:56:56.330 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2023-11-28T11:56:56.330 INFO:teuthology.orchestra.run.smithi057.stdout:==================================================================================================== 2023-11-28T11:56:56.330 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2023-11-28T11:56:56.330 INFO:teuthology.orchestra.run.smithi057.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-11-28T11:56:56.330 INFO:teuthology.orchestra.run.smithi057.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:56:56.330 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:56:56.330 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:56:56.330 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:56:56.330 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:56:56.331 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:56:56.331 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:56:56.331 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:56:56.331 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:56:56.331 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:56:56.331 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:56:56.331 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:56:56.331 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:56:56.331 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:56:56.331 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:56:56.331 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf-devel x86_64 0.190-1.el8 baseos 62 k 2023-11-28T11:56:56.332 INFO:teuthology.orchestra.run.smithi057.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-11-28T11:56:56.332 INFO:teuthology.orchestra.run.smithi057.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-11-28T11:56:56.332 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:56:56.332 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:56:56.332 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:56:56.332 INFO:teuthology.orchestra.run.smithi057.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-11-28T11:56:56.332 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-11-28T11:56:56.332 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-devel x86_64 1:1.1.1k-11.el8 baseos 3.2 M 2023-11-28T11:56:56.332 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql x86_64 10.23-2.module_el8+572+929c87ac appstream 1.6 M 2023-11-28T11:56:56.332 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-contrib x86_64 10.23-2.module_el8+572+929c87ac appstream 901 k 2023-11-28T11:56:56.332 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-server x86_64 10.23-2.module_el8+572+929c87ac appstream 5.1 M 2023-11-28T11:56:56.332 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:56:56.333 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:56:56.333 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:56:56.333 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:56:56.333 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:56:56.333 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:56:56.333 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:56:56.333 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-11-28T11:56:56.333 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrading: 2023-11-28T11:56:56.333 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-11.el8 baseos 144 k 2023-11-28T11:56:56.333 INFO:teuthology.orchestra.run.smithi057.stdout: acl x86_64 2.2.53-3.el8 baseos 83 k 2023-11-28T11:56:56.333 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-debuginfod-client x86_64 0.190-1.el8 baseos 77 k 2023-11-28T11:56:56.333 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf x86_64 0.190-1.el8 baseos 232 k 2023-11-28T11:56:56.334 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libs x86_64 0.190-1.el8 baseos 305 k 2023-11-28T11:56:56.334 INFO:teuthology.orchestra.run.smithi057.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-11-28T11:56:56.334 INFO:teuthology.orchestra.run.smithi057.stdout: libacl x86_64 2.2.53-3.el8 baseos 35 k 2023-11-28T11:56:56.334 INFO:teuthology.orchestra.run.smithi057.stdout: libacl-devel x86_64 2.2.53-3.el8 baseos 85 k 2023-11-28T11:56:56.334 INFO:teuthology.orchestra.run.smithi057.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-11-28T11:56:56.334 INFO:teuthology.orchestra.run.smithi057.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-11-28T11:56:56.334 INFO:teuthology.orchestra.run.smithi057.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-11-28T11:56:56.334 INFO:teuthology.orchestra.run.smithi057.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:56:56.334 INFO:teuthology.orchestra.run.smithi057.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:56:56.334 INFO:teuthology.orchestra.run.smithi057.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-11-28T11:56:56.334 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-11-28T11:56:56.334 INFO:teuthology.orchestra.run.smithi057.stdout: openssl x86_64 1:1.1.1k-11.el8 baseos 737 k 2023-11-28T11:56:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-libs x86_64 1:1.1.1k-11.el8 baseos 1.5 M 2023-11-28T11:56:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-11-28T11:56:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-11-28T11:56:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: xfsdump x86_64 3.1.8-7.el8 baseos 334 k 2023-11-28T11:56:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-11-28T11:56:56.335 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2023-11-28T11:56:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-11-28T11:56:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:56:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:56:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:56:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:56:56.335 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:56:56.336 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:56:56.336 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:56:56.336 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:56:56.336 INFO:teuthology.orchestra.run.smithi057.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-11-28T11:56:56.336 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:56:56.336 INFO:teuthology.orchestra.run.smithi057.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2023-11-28T11:56:56.336 INFO:teuthology.orchestra.run.smithi057.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-11-28T11:56:56.336 INFO:teuthology.orchestra.run.smithi057.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-11-28T11:56:56.336 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:56:56.336 INFO:teuthology.orchestra.run.smithi057.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-11-28T11:56:56.336 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2023-11-28T11:56:56.336 INFO:teuthology.orchestra.run.smithi057.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:56:56.337 INFO:teuthology.orchestra.run.smithi057.stdout: libpq x86_64 13.5-1.el8 appstream 198 k 2023-11-28T11:56:56.337 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2023-11-28T11:56:56.337 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-11-28T11:56:56.337 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:56:56.337 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-11-28T11:56:56.337 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:56:56.337 INFO:teuthology.orchestra.run.smithi057.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-11-28T11:56:56.337 INFO:teuthology.orchestra.run.smithi057.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-11-28T11:56:56.337 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:56:56.337 INFO:teuthology.orchestra.run.smithi057.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-11-28T11:56:56.337 INFO:teuthology.orchestra.run.smithi057.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-11-28T11:56:56.337 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-11-28T11:56:56.338 INFO:teuthology.orchestra.run.smithi057.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-11-28T11:56:56.338 INFO:teuthology.orchestra.run.smithi057.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-11-28T11:56:56.338 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-11-28T11:56:56.338 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-11-28T11:56:56.338 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-11-28T11:56:56.338 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-11-28T11:56:56.338 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:56:56.338 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-11-28T11:56:56.338 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:56:56.338 INFO:teuthology.orchestra.run.smithi057.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:56:56.338 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:56:56.339 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:56:56.339 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:56:56.339 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:56:56.339 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-11-28T11:56:56.339 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:56:56.339 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:56:56.339 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-11-28T11:56:56.339 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:56:56.339 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:56:56.339 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:56:56.339 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-11-28T11:56:56.340 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-11-28T11:56:56.340 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-11-28T11:56:56.340 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-11-28T11:56:56.340 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:56:56.340 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:56:56.340 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-11-28T11:56:56.340 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-11-28T11:56:56.340 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:56:56.340 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:56:56.340 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-11-28T11:56:56.340 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-11-28T11:56:56.340 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-11-28T11:56:56.341 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:56:56.341 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:56:56.341 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-11-28T11:56:56.341 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-11-28T11:56:56.341 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-11-28T11:56:56.341 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-11-28T11:56:56.341 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-11-28T11:56:56.341 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-11-28T11:56:56.341 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:56:56.341 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-11-28T11:56:56.341 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:56:56.341 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:56:56.342 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-11-28T11:56:56.342 INFO:teuthology.orchestra.run.smithi057.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:56:56.342 INFO:teuthology.orchestra.run.smithi057.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:56:56.342 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:56:56.342 INFO:teuthology.orchestra.run.smithi057.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:56:56.342 INFO:teuthology.orchestra.run.smithi057.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:56:56.342 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:56:56.342 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:56:56.342 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:56:56.342 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-11-28T11:56:56.342 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:56:56.342 INFO:teuthology.orchestra.run.smithi057.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-11-28T11:56:56.343 INFO:teuthology.orchestra.run.smithi057.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:56:56.343 INFO:teuthology.orchestra.run.smithi057.stdout: uuid x86_64 1.6.2-43.el8 appstream 64 k 2023-11-28T11:56:56.343 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-11-28T11:56:56.343 INFO:teuthology.orchestra.run.smithi057.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-11-28T11:56:56.343 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling module streams: 2023-11-28T11:56:56.343 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql 10 2023-11-28T11:56:56.343 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T11:56:56.343 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2023-11-28T11:56:56.343 INFO:teuthology.orchestra.run.smithi057.stdout:==================================================================================================== 2023-11-28T11:56:56.343 INFO:teuthology.orchestra.run.smithi057.stdout:Install 128 Packages 2023-11-28T11:56:56.343 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrade 21 Packages 2023-11-28T11:56:56.343 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T11:56:56.344 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 212 M 2023-11-28T11:56:56.344 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2023-11-28T11:56:56.687 INFO:teuthology.orchestra.run.smithi057.stdout:(1/149): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 33 kB/s | 6.4 kB 00:00 2023-11-28T11:56:56.822 INFO:teuthology.orchestra.run.smithi181.stdout:(49/149): postgresql-contrib-10.23-2.module_el8 676 kB/s | 901 kB 00:01 2023-11-28T11:56:57.033 INFO:teuthology.orchestra.run.smithi057.stdout:(2/149): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 2.4 MB/s | 845 kB 00:00 2023-11-28T11:56:57.096 INFO:teuthology.orchestra.run.smithi057.stdout:(3/149): ceph-immutable-object-cache-18.0.0-269 2.6 MB/s | 165 kB 00:00 2023-11-28T11:56:57.255 INFO:teuthology.orchestra.run.smithi057.stdout:(4/149): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 7.0 MB/s | 5.4 MB 00:00 2023-11-28T11:56:57.463 INFO:teuthology.orchestra.run.smithi057.stdout:(5/149): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 7.4 MB/s | 1.5 MB 00:00 2023-11-28T11:56:57.814 INFO:teuthology.orchestra.run.smithi057.stdout:(6/149): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 3.2 MB/s | 2.3 MB 00:00 2023-11-28T11:56:57.956 INFO:teuthology.orchestra.run.smithi057.stdout:(7/149): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 9.3 MB/s | 4.6 MB 00:00 2023-11-28T11:56:58.481 INFO:teuthology.orchestra.run.smithi057.stdout:(8/149): ceph-common-18.0.0-2697.g2f5f6028.el8. 11 MB/s | 22 MB 00:01 2023-11-28T11:56:58.531 INFO:teuthology.orchestra.run.smithi057.stdout:(9/149): ceph-selinux-18.0.0-2697.g2f5f6028.el8 490 kB/s | 24 kB 00:00 2023-11-28T11:56:59.331 INFO:teuthology.orchestra.run.smithi057.stdout:(10/149): ceph-radosgw-18.0.0-2697.g2f5f6028.el 9.5 MB/s | 13 MB 00:01 2023-11-28T11:56:59.440 INFO:teuthology.orchestra.run.smithi057.stdout:(11/149): cephfs-mirror-18.0.0-2697.g2f5f6028.e 2.3 MB/s | 257 kB 00:00 2023-11-28T11:56:59.532 INFO:teuthology.orchestra.run.smithi057.stdout:(12/149): libcephfs-devel-18.0.0-2697.g2f5f6028 333 kB/s | 30 kB 00:00 2023-11-28T11:56:59.999 INFO:teuthology.orchestra.run.smithi057.stdout:(13/149): libcephfs2-18.0.0-2697.g2f5f6028.el8. 1.6 MB/s | 767 kB 00:00 2023-11-28T11:57:00.132 INFO:teuthology.orchestra.run.smithi057.stdout:(14/149): libcephsqlite-18.0.0-2697.g2f5f6028.e 1.0 MB/s | 137 kB 00:00 2023-11-28T11:57:00.266 INFO:teuthology.orchestra.run.smithi057.stdout:(15/149): librados-devel-18.0.0-2697.g2f5f6028. 1.0 MB/s | 139 kB 00:00 2023-11-28T11:57:00.483 INFO:teuthology.orchestra.run.smithi057.stdout:(16/149): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 6.6 MB/s | 18 MB 00:02 2023-11-28T11:57:00.558 INFO:teuthology.orchestra.run.smithi057.stdout:(17/149): libradosstriper1-18.0.0-2697.g2f5f602 1.6 MB/s | 474 kB 00:00 2023-11-28T11:57:00.642 INFO:teuthology.orchestra.run.smithi057.stdout:(18/149): python3-ceph-argparse-18.0.0-2697.g2f 537 kB/s | 45 kB 00:00 2023-11-28T11:57:00.697 INFO:teuthology.orchestra.run.smithi181.stdout:(50/149): postgresql-10.23-2.module_el8+572+929 140 kB/s | 1.6 MB 00:11 2023-11-28T11:57:00.775 INFO:teuthology.orchestra.run.smithi057.stdout:(19/149): python3-ceph-common-18.0.0-2697.g2f5f 879 kB/s | 117 kB 00:00 2023-11-28T11:57:00.917 INFO:teuthology.orchestra.run.smithi057.stdout:(20/149): python3-cephfs-18.0.0-2697.g2f5f6028. 1.3 MB/s | 190 kB 00:00 2023-11-28T11:57:01.143 INFO:teuthology.orchestra.run.smithi057.stdout:(21/149): python3-rados-18.0.0-2697.g2f5f6028.e 1.6 MB/s | 371 kB 00:00 2023-11-28T11:57:01.676 INFO:teuthology.orchestra.run.smithi057.stdout:(22/149): ceph-test-18.0.0-2697.g2f5f6028.el8.x 15 MB/s | 47 MB 00:03 2023-11-28T11:57:01.701 INFO:teuthology.orchestra.run.smithi057.stdout:(23/149): python3-rbd-18.0.0-2697.g2f5f6028.el8 652 kB/s | 364 kB 00:00 2023-11-28T11:57:01.727 INFO:teuthology.orchestra.run.smithi057.stdout:(24/149): python3-rgw-18.0.0-2697.g2f5f6028.el8 2.1 MB/s | 107 kB 00:00 2023-11-28T11:57:01.827 INFO:teuthology.orchestra.run.smithi057.stdout:(25/149): librgw2-18.0.0-2697.g2f5f6028.el8.x86 4.8 MB/s | 6.5 MB 00:01 2023-11-28T11:57:01.852 INFO:teuthology.orchestra.run.smithi057.stdout:(26/149): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 628 kB/s | 94 kB 00:00 2023-11-28T11:57:01.911 INFO:teuthology.orchestra.run.smithi057.stdout:(27/149): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 2.1 MB/s | 180 kB 00:00 2023-11-28T11:57:01.936 INFO:teuthology.orchestra.run.smithi057.stdout:(28/149): ceph-grafana-dashboards-18.0.0-2697.g 277 kB/s | 23 kB 00:00 2023-11-28T11:57:01.986 INFO:teuthology.orchestra.run.smithi057.stdout:(29/149): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 13 MB/s | 3.4 MB 00:00 2023-11-28T11:57:02.012 INFO:teuthology.orchestra.run.smithi057.stdout:(30/149): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.2 MB/s | 123 kB 00:00 2023-11-28T11:57:02.154 INFO:teuthology.orchestra.run.smithi057.stdout:(31/149): ceph-mgr-modules-core-18.0.0-2697.g2f 1.6 MB/s | 236 kB 00:00 2023-11-28T11:57:02.246 INFO:teuthology.orchestra.run.smithi057.stdout:(32/149): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 519 kB/s | 47 kB 00:00 2023-11-28T11:57:02.413 INFO:teuthology.orchestra.run.smithi057.stdout:(33/149): ceph-mgr-diskprediction-local-18.0.0- 17 MB/s | 7.4 MB 00:00 2023-11-28T11:57:02.446 INFO:teuthology.orchestra.run.smithi057.stdout:(34/149): ceph-prometheus-alerts-18.0.0-2697.g2 73 kB/s | 15 kB 00:00 2023-11-28T11:57:02.480 INFO:teuthology.orchestra.run.smithi057.stdout:(35/149): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 3.2 MB/s | 1.7 MB 00:00 2023-11-28T11:57:02.505 INFO:teuthology.orchestra.run.smithi057.stdout:(36/149): cephadm-18.0.0-2697.g2f5f6028.el8.noa 1.4 MB/s | 83 kB 00:00 2023-11-28T11:57:02.530 INFO:teuthology.orchestra.run.smithi057.stdout:(37/149): cephfs-top-18.0.0-2697.g2f5f6028.el8. 360 kB/s | 18 kB 00:00 2023-11-28T11:57:02.556 INFO:teuthology.orchestra.run.smithi057.stdout:(38/149): ceph-volume-18.0.0-2697.g2f5f6028.el8 1.8 MB/s | 263 kB 00:00 2023-11-28T11:57:02.756 INFO:teuthology.orchestra.run.smithi181.stdout:(51/149): postgresql-server-10.23-2.module_el8+ 883 kB/s | 5.1 MB 00:05 2023-11-28T11:57:03.481 INFO:teuthology.orchestra.run.smithi181.stdout:(52/149): python3-jinja2-2.10.1-3.el8.noarch.rp 742 kB/s | 538 kB 00:00 2023-11-28T11:57:03.615 INFO:teuthology.orchestra.run.smithi181.stdout:(53/149): python3-jsonpatch-1.21-2.el8.noarch.r 199 kB/s | 27 kB 00:00 2023-11-28T11:57:03.722 INFO:teuthology.orchestra.run.smithi057.stdout:(39/149): flex-2.6.1-9.el8.x86_64.rpm 274 kB/s | 320 kB 00:01 2023-11-28T11:57:03.740 INFO:teuthology.orchestra.run.smithi181.stdout:(54/149): python3-jsonpointer-1.10-11.el8.noarc 160 kB/s | 20 kB 00:00 2023-11-28T11:57:04.009 INFO:teuthology.orchestra.run.smithi181.stdout:(55/149): openblas-0.3.15-4.el8.x86_64.rpm 290 kB/s | 4.7 MB 00:16 2023-11-28T11:57:04.049 INFO:teuthology.orchestra.run.smithi181.stdout:(56/149): python3-mako-1.0.6-14.el8.noarch.rpm 509 kB/s | 157 kB 00:00 2023-11-28T11:57:04.299 INFO:teuthology.orchestra.run.smithi181.stdout:(57/149): python3-markupsafe-0.23-19.el8.x86_64 133 kB/s | 39 kB 00:00 2023-11-28T11:57:04.564 INFO:teuthology.orchestra.run.smithi057.stdout:(40/149): jq-1.6-8.el8.x86_64.rpm 241 kB/s | 203 kB 00:00 2023-11-28T11:57:05.406 INFO:teuthology.orchestra.run.smithi057.stdout:(41/149): bison-3.0.4-10.el8.x86_64.rpm 239 kB/s | 688 kB 00:02 2023-11-28T11:57:05.448 INFO:teuthology.orchestra.run.smithi057.stdout:(42/149): libpq-13.5-1.el8.x86_64.rpm 224 kB/s | 198 kB 00:00 2023-11-28T11:57:05.490 INFO:teuthology.orchestra.run.smithi057.stdout:(43/149): automake-1.16.1-8.el8.noarch.rpm 248 kB/s | 740 kB 00:02 2023-11-28T11:57:05.724 INFO:teuthology.orchestra.run.smithi181.stdout:(58/149): python3-numpy-f2py-1.14.3-10.el8.x86_ 158 kB/s | 225 kB 00:01 2023-11-28T11:57:06.091 INFO:teuthology.orchestra.run.smithi181.stdout:(59/149): python3-prettytable-0.7.2-14.el8.noar 120 kB/s | 44 kB 00:00 2023-11-28T11:57:06.549 INFO:teuthology.orchestra.run.smithi057.stdout:(44/149): lttng-ust-2.8.1-11.el8.x86_64.rpm 245 kB/s | 259 kB 00:01 2023-11-28T11:57:06.808 INFO:teuthology.orchestra.run.smithi181.stdout:(60/149): python3-pyOpenSSL-19.0.0-1.el8.noarch 144 kB/s | 103 kB 00:00 2023-11-28T11:57:07.094 INFO:teuthology.orchestra.run.smithi057.stdout:(45/149): librdkafka-0.11.4-3.el8.x86_64.rpm 210 kB/s | 354 kB 00:01 2023-11-28T11:57:07.274 INFO:teuthology.orchestra.run.smithi057.stdout:(46/149): oniguruma-6.8.2-2.el8.x86_64.rpm 258 kB/s | 187 kB 00:00 2023-11-28T11:57:08.191 INFO:teuthology.orchestra.run.smithi181.stdout:(61/149): python3-pyasn1-0.3.7-6.el8.noarch.rpm 91 kB/s | 126 kB 00:01 2023-11-28T11:57:08.266 INFO:teuthology.orchestra.run.smithi057.stdout:(47/149): libtool-2.4.6-25.el8.x86_64.rpm 251 kB/s | 709 kB 00:02 2023-11-28T11:57:09.842 INFO:teuthology.orchestra.run.smithi181.stdout:(62/149): python3-pyasn1-modules-0.3.7-6.el8.no 67 kB/s | 110 kB 00:01 2023-11-28T11:57:10.475 INFO:teuthology.orchestra.run.smithi181.stdout:(63/149): python3-pytz-2017.2-11.el8.noarch.rpm 90 kB/s | 57 kB 00:00 2023-11-28T11:57:13.067 INFO:teuthology.orchestra.run.smithi181.stdout:(64/149): python3-numpy-1.14.3-10.el8.x86_64.rp 424 kB/s | 3.7 MB 00:09 2023-11-28T11:57:14.375 INFO:teuthology.orchestra.run.smithi181.stdout:(65/149): python3-werkzeug-0.12.2-4.el8.noarch. 350 kB/s | 457 kB 00:01 2023-11-28T11:57:15.242 INFO:teuthology.orchestra.run.smithi181.stdout:(66/149): socat-1.7.4.1-1.el8.x86_64.rpm 373 kB/s | 323 kB 00:00 2023-11-28T11:57:15.576 INFO:teuthology.orchestra.run.smithi181.stdout:(67/149): uuid-1.6.2-43.el8.x86_64.rpm 192 kB/s | 64 kB 00:00 2023-11-28T11:57:15.851 INFO:teuthology.orchestra.run.smithi181.stdout:(68/149): xmlstarlet-1.6.1-20.el8.x86_64.rpm 257 kB/s | 71 kB 00:00 2023-11-28T11:57:16.226 INFO:teuthology.orchestra.run.smithi181.stdout:(69/149): elfutils-libelf-devel-0.190-1.el8.x86 166 kB/s | 62 kB 00:00 2023-11-28T11:57:16.418 INFO:teuthology.orchestra.run.smithi181.stdout:(70/149): fuse-2.9.7-17.el8.x86_64.rpm 433 kB/s | 83 kB 00:00 2023-11-28T11:57:16.552 INFO:teuthology.orchestra.run.smithi181.stdout:(71/149): keyutils-libs-devel-1.5.10-9.el8.x86_ 361 kB/s | 48 kB 00:00 2023-11-28T11:57:16.844 INFO:teuthology.orchestra.run.smithi181.stdout:(72/149): krb5-devel-1.18.2-22.el8.x86_64.rpm 1.9 MB/s | 561 kB 00:00 2023-11-28T11:57:16.945 INFO:teuthology.orchestra.run.smithi181.stdout:(73/149): libaio-devel-0.3.112-1.el8.x86_64.rpm 187 kB/s | 19 kB 00:00 2023-11-28T11:57:17.070 INFO:teuthology.orchestra.run.smithi181.stdout:(74/149): libcom_err-devel-1.45.6-5.el8.x86_64. 310 kB/s | 39 kB 00:00 2023-11-28T11:57:17.237 INFO:teuthology.orchestra.run.smithi181.stdout:(75/149): libgfortran-8.5.0-21.el8.x86_64.rpm 3.8 MB/s | 645 kB 00:00 2023-11-28T11:57:17.362 INFO:teuthology.orchestra.run.smithi181.stdout:(76/149): libquadmath-8.5.0-21.el8.x86_64.rpm 1.4 MB/s | 172 kB 00:00 2023-11-28T11:57:17.479 INFO:teuthology.orchestra.run.smithi181.stdout:(77/149): librabbitmq-0.9.0-4.el8.x86_64.rpm 402 kB/s | 47 kB 00:00 2023-11-28T11:57:17.604 INFO:teuthology.orchestra.run.smithi181.stdout:(78/149): libselinux-devel-2.9-8.el8.x86_64.rpm 1.6 MB/s | 200 kB 00:00 2023-11-28T11:57:17.713 INFO:teuthology.orchestra.run.smithi181.stdout:(79/149): libsepol-devel-2.9-3.el8.x86_64.rpm 803 kB/s | 87 kB 00:00 2023-11-28T11:57:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:(80/149): libuuid-devel-2.32.1-43.el8.x86_64.rp 799 kB/s | 100 kB 00:00 2023-11-28T11:57:17.947 INFO:teuthology.orchestra.run.smithi181.stdout:(81/149): libverto-devel-0.3.2-2.el8.x86_64.rpm 166 kB/s | 18 kB 00:00 2023-11-28T11:57:18.097 INFO:teuthology.orchestra.run.smithi181.stdout:(82/149): libzstd-devel-1.4.4-1.el8.x86_64.rpm 292 kB/s | 44 kB 00:00 2023-11-28T11:57:18.464 INFO:teuthology.orchestra.run.smithi181.stdout:(83/149): openssl-devel-1.1.1k-11.el8.x86_64.rp 8.7 MB/s | 3.2 MB 00:00 2023-11-28T11:57:18.589 INFO:teuthology.orchestra.run.smithi181.stdout:(84/149): pcre2-devel-10.32-3.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-11-28T11:57:18.706 INFO:teuthology.orchestra.run.smithi181.stdout:(85/149): pcre2-utf16-10.32-3.el8.x86_64.rpm 1.9 MB/s | 229 kB 00:00 2023-11-28T11:57:18.832 INFO:teuthology.orchestra.run.smithi181.stdout:(86/149): pcre2-utf32-10.32-3.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-11-28T11:57:18.966 INFO:teuthology.orchestra.run.smithi181.stdout:(87/149): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.8 MB/s | 249 kB 00:00 2023-11-28T11:57:19.117 INFO:teuthology.orchestra.run.smithi181.stdout:(88/149): python3-cryptography-3.2.1-6.el8.x86_ 4.1 MB/s | 641 kB 00:00 2023-11-28T11:57:19.216 INFO:teuthology.orchestra.run.smithi181.stdout:(89/149): python3-jwt-1.6.1-2.el8.noarch.rpm 434 kB/s | 43 kB 00:00 2023-11-28T11:57:19.358 INFO:teuthology.orchestra.run.smithi181.stdout:(90/149): python3-oauthlib-2.1.0-1.el8.noarch.r 1.1 MB/s | 155 kB 00:00 2023-11-28T11:57:19.492 INFO:teuthology.orchestra.run.smithi181.stdout:(91/149): python3-pycparser-2.14-14.el8.noarch. 818 kB/s | 109 kB 00:00 2023-11-28T11:57:19.592 INFO:teuthology.orchestra.run.smithi181.stdout:(92/149): python3-requests-oauthlib-1.0.0-1.el8 427 kB/s | 43 kB 00:00 2023-11-28T11:57:19.734 INFO:teuthology.orchestra.run.smithi181.stdout:(93/149): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 527 kB/s | 74 kB 00:00 2023-11-28T11:57:19.851 INFO:teuthology.orchestra.run.smithi181.stdout:(94/149): zlib-devel-1.2.11-21.el8.x86_64.rpm 497 kB/s | 58 kB 00:00 2023-11-28T11:57:20.109 INFO:teuthology.orchestra.run.smithi181.stdout:(95/149): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-11-28T11:57:20.185 INFO:teuthology.orchestra.run.smithi181.stdout:(96/149): liboath-2.6.2-3.el8.x86_64.rpm 784 kB/s | 59 kB 00:00 2023-11-28T11:57:20.252 INFO:teuthology.orchestra.run.smithi181.stdout:(97/149): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-11-28T11:57:20.385 INFO:teuthology.orchestra.run.smithi181.stdout:(98/149): python3-asyncssh-2.7.0-2.el8.noarch.r 3.5 MB/s | 476 kB 00:00 2023-11-28T11:57:20.444 INFO:teuthology.orchestra.run.smithi181.stdout:(99/149): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 748 kB/s | 44 kB 00:00 2023-11-28T11:57:20.503 INFO:teuthology.orchestra.run.smithi181.stdout:(100/149): python3-beautifulsoup4-4.6.3-2.el8.1 3.1 MB/s | 185 kB 00:00 2023-11-28T11:57:20.570 INFO:teuthology.orchestra.run.smithi181.stdout:(101/149): python3-cachetools-3.1.1-4.el8.noarc 495 kB/s | 33 kB 00:00 2023-11-28T11:57:20.663 INFO:teuthology.orchestra.run.smithi181.stdout:(102/149): python3-certifi-2018.10.15-7.el8.noa 176 kB/s | 16 kB 00:00 2023-11-28T11:57:20.738 INFO:teuthology.orchestra.run.smithi181.stdout:(103/149): python3-cheroot-8.5.2-1.el8.noarch.r 2.3 MB/s | 173 kB 00:00 2023-11-28T11:57:20.804 INFO:teuthology.orchestra.run.smithi181.stdout:(104/149): python3-cherrypy-18.4.0-1.el8.noarch 5.6 MB/s | 384 kB 00:00 2023-11-28T11:57:20.848 INFO:teuthology.orchestra.run.smithi057.stdout:(48/149): postgresql-10.23-2.module_el8+572+929 127 kB/s | 1.6 MB 00:12 2023-11-28T11:57:20.871 INFO:teuthology.orchestra.run.smithi181.stdout:(105/149): python3-google-auth-1.1.1-10.el8.noa 1.2 MB/s | 81 kB 00:00 2023-11-28T11:57:20.930 INFO:teuthology.orchestra.run.smithi181.stdout:(106/149): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-11-28T11:57:20.997 INFO:teuthology.orchestra.run.smithi181.stdout:(107/149): python3-jaraco-functools-2.0-4.el8.n 264 kB/s | 18 kB 00:00 2023-11-28T11:57:21.097 INFO:teuthology.orchestra.run.smithi181.stdout:(108/149): python3-kubernetes-11.0.0-6.el8.noar 12 MB/s | 1.2 MB 00:00 2023-11-28T11:57:21.147 INFO:teuthology.orchestra.run.smithi181.stdout:(109/149): python3-logutils-0.3.5-11.el8.noarch 974 kB/s | 49 kB 00:00 2023-11-28T11:57:21.214 INFO:teuthology.orchestra.run.smithi181.stdout:(110/149): python3-more-itertools-7.2.0-3.el8.n 879 kB/s | 59 kB 00:00 2023-11-28T11:57:21.290 INFO:teuthology.orchestra.run.smithi181.stdout:(111/149): python3-natsort-7.1.1-2.el8.noarch.r 803 kB/s | 60 kB 00:00 2023-11-28T11:57:21.357 INFO:teuthology.orchestra.run.smithi181.stdout:(112/149): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2023-11-28T11:57:21.407 INFO:teuthology.orchestra.run.smithi181.stdout:(113/149): python3-portend-2.6-1.el8.noarch.rpm 312 kB/s | 16 kB 00:00 2023-11-28T11:57:21.482 INFO:teuthology.orchestra.run.smithi181.stdout:(114/149): python3-repoze-lru-0.7-6.el8.noarch. 448 kB/s | 34 kB 00:00 2023-11-28T11:57:21.577 INFO:teuthology.orchestra.run.smithi181.stdout:(115/149): python3-routes-2.4.1-12.el8.noarch.r 2.0 MB/s | 196 kB 00:00 2023-11-28T11:57:21.650 INFO:teuthology.orchestra.run.smithi181.stdout:(116/149): python3-rsa-4.9-2.el8.noarch.rpm 885 kB/s | 64 kB 00:00 2023-11-28T11:57:21.708 INFO:teuthology.orchestra.run.smithi181.stdout:(117/149): python3-simplegeneric-0.8.1-17.el8.n 321 kB/s | 19 kB 00:00 2023-11-28T11:57:21.759 INFO:teuthology.orchestra.run.smithi181.stdout:(118/149): python3-singledispatch-3.4.0.3-18.el 477 kB/s | 24 kB 00:00 2023-11-28T11:57:21.817 INFO:teuthology.orchestra.run.smithi181.stdout:(119/149): python3-tempora-1.14.1-5.el8.noarch. 503 kB/s | 29 kB 00:00 2023-11-28T11:57:21.901 INFO:teuthology.orchestra.run.smithi181.stdout:(120/149): python3-trustme-0.6.0-4.el8.noarch.r 330 kB/s | 27 kB 00:00 2023-11-28T11:57:21.968 INFO:teuthology.orchestra.run.smithi181.stdout:(121/149): python3-waitress-1.4.3-1.el8.noarch. 3.7 MB/s | 255 kB 00:00 2023-11-28T11:57:22.035 INFO:teuthology.orchestra.run.smithi181.stdout:(122/149): python3-webob-1.8.5-1.el8.1.noarch.r 3.7 MB/s | 251 kB 00:00 2023-11-28T11:57:22.085 INFO:teuthology.orchestra.run.smithi181.stdout:(123/149): python3-websocket-client-0.56.0-5.el 1.2 MB/s | 61 kB 00:00 2023-11-28T11:57:22.144 INFO:teuthology.orchestra.run.smithi181.stdout:(124/149): python3-webtest-2.0.33-1.el8.noarch. 1.4 MB/s | 85 kB 00:00 2023-11-28T11:57:22.228 INFO:teuthology.orchestra.run.smithi181.stdout:(125/149): python3-zc-lockfile-2.0-2.el8.noarch 275 kB/s | 23 kB 00:00 2023-11-28T11:57:22.361 INFO:teuthology.orchestra.run.smithi181.stdout:(126/149): thrift-0.13.0-2.el8.x86_64.rpm 13 MB/s | 1.7 MB 00:00 2023-11-28T11:57:22.886 INFO:teuthology.orchestra.run.smithi181.stdout:(127/149): librados2-18.0.0-2697.g2f5f6028.el8. 7.0 MB/s | 3.7 MB 00:00 2023-11-28T11:57:23.245 INFO:teuthology.orchestra.run.smithi181.stdout:(128/149): librbd1-18.0.0-2697.g2f5f6028.el8.x8 9.9 MB/s | 3.6 MB 00:00 2023-11-28T11:57:23.354 INFO:teuthology.orchestra.run.smithi181.stdout:(129/149): NetworkManager-initscripts-updown-1. 1.3 MB/s | 144 kB 00:00 2023-11-28T11:57:23.454 INFO:teuthology.orchestra.run.smithi181.stdout:(130/149): acl-2.2.53-3.el8.x86_64.rpm 827 kB/s | 83 kB 00:00 2023-11-28T11:57:23.563 INFO:teuthology.orchestra.run.smithi181.stdout:(131/149): elfutils-debuginfod-client-0.190-1.e 709 kB/s | 77 kB 00:00 2023-11-28T11:57:23.673 INFO:teuthology.orchestra.run.smithi181.stdout:(132/149): elfutils-libelf-0.190-1.el8.x86_64.r 2.1 MB/s | 232 kB 00:00 2023-11-28T11:57:23.797 INFO:teuthology.orchestra.run.smithi181.stdout:(133/149): elfutils-libs-0.190-1.el8.x86_64.rpm 2.4 MB/s | 305 kB 00:00 2023-11-28T11:57:23.922 INFO:teuthology.orchestra.run.smithi181.stdout:(134/149): iproute-6.2.0-5.el8.x86_64.rpm 6.7 MB/s | 854 kB 00:00 2023-11-28T11:57:24.047 INFO:teuthology.orchestra.run.smithi181.stdout:(135/149): libacl-2.2.53-3.el8.x86_64.rpm 278 kB/s | 35 kB 00:00 2023-11-28T11:57:24.156 INFO:teuthology.orchestra.run.smithi181.stdout:(136/149): libacl-devel-2.2.53-3.el8.x86_64.rpm 783 kB/s | 85 kB 00:00 2023-11-28T11:57:24.265 INFO:teuthology.orchestra.run.smithi181.stdout:(137/149): libblkid-2.32.1-43.el8.x86_64.rpm 2.0 MB/s | 220 kB 00:00 2023-11-28T11:57:24.382 INFO:teuthology.orchestra.run.smithi181.stdout:(138/149): libfdisk-2.32.1-43.el8.x86_64.rpm 2.1 MB/s | 254 kB 00:00 2023-11-28T11:57:24.490 INFO:teuthology.orchestra.run.smithi181.stdout:(139/149): libmount-2.32.1-43.el8.x86_64.rpm 2.1 MB/s | 236 kB 00:00 2023-11-28T11:57:24.599 INFO:teuthology.orchestra.run.smithi181.stdout:(140/149): libsmartcols-2.32.1-43.el8.x86_64.rp 1.6 MB/s | 179 kB 00:00 2023-11-28T11:57:24.699 INFO:teuthology.orchestra.run.smithi181.stdout:(141/149): libuuid-2.32.1-43.el8.x86_64.rpm 989 kB/s | 99 kB 00:00 2023-11-28T11:57:24.825 INFO:teuthology.orchestra.run.smithi181.stdout:(142/149): openssl-1.1.1k-11.el8.x86_64.rpm 5.8 MB/s | 737 kB 00:00 2023-11-28T11:57:24.967 INFO:teuthology.orchestra.run.smithi181.stdout:(143/149): openssl-libs-1.1.1k-11.el8.x86_64.rp 10 MB/s | 1.5 MB 00:00 2023-11-28T11:57:25.142 INFO:teuthology.orchestra.run.smithi181.stdout:(144/149): util-linux-2.32.1-43.el8.x86_64.rpm 14 MB/s | 2.5 MB 00:00 2023-11-28T11:57:25.215 INFO:teuthology.orchestra.run.smithi057.stdout:(49/149): openblas-threads-0.3.15-4.el8.x86_64. 277 kB/s | 4.9 MB 00:17 2023-11-28T11:57:25.267 INFO:teuthology.orchestra.run.smithi181.stdout:(145/149): util-linux-user-2.32.1-43.el8.x86_64 822 kB/s | 103 kB 00:00 2023-11-28T11:57:25.376 INFO:teuthology.orchestra.run.smithi181.stdout:(146/149): xfsdump-3.1.8-7.el8.x86_64.rpm 3.0 MB/s | 334 kB 00:00 2023-11-28T11:57:25.509 INFO:teuthology.orchestra.run.smithi181.stdout:(147/149): xfsprogs-5.0.0-12.el8.x86_64.rpm 8.1 MB/s | 1.1 MB 00:00 2023-11-28T11:57:29.265 INFO:teuthology.orchestra.run.smithi057.stdout:(50/149): openblas-0.3.15-4.el8.x86_64.rpm 218 kB/s | 4.7 MB 00:22 2023-11-28T11:57:31.823 INFO:teuthology.orchestra.run.smithi057.stdout:(51/149): postgresql-contrib-10.23-2.module_el8 82 kB/s | 901 kB 00:10 2023-11-28T11:57:34.139 INFO:teuthology.orchestra.run.smithi057.stdout:(52/149): python3-jinja2-2.10.1-3.el8.noarch.rp 232 kB/s | 538 kB 00:02 2023-11-28T11:57:34.340 INFO:teuthology.orchestra.run.smithi057.stdout:(53/149): python3-jsonpatch-1.21-2.el8.noarch.r 133 kB/s | 27 kB 00:00 2023-11-28T11:57:34.515 INFO:teuthology.orchestra.run.smithi057.stdout:(54/149): python3-jsonpointer-1.10-11.el8.noarc 114 kB/s | 20 kB 00:00 2023-11-28T11:57:35.148 INFO:teuthology.orchestra.run.smithi057.stdout:(55/149): python3-mako-1.0.6-14.el8.noarch.rpm 248 kB/s | 157 kB 00:00 2023-11-28T11:57:35.474 INFO:teuthology.orchestra.run.smithi057.stdout:(56/149): python3-markupsafe-0.23-19.el8.x86_64 119 kB/s | 39 kB 00:00 2023-11-28T11:57:41.982 INFO:teuthology.orchestra.run.smithi057.stdout:(57/149): postgresql-server-10.23-2.module_el8+ 312 kB/s | 5.1 MB 00:16 2023-11-28T11:57:43.023 INFO:teuthology.orchestra.run.smithi057.stdout:(58/149): python3-numpy-f2py-1.14.3-10.el8.x86_ 216 kB/s | 225 kB 00:01 2023-11-28T11:57:43.265 INFO:teuthology.orchestra.run.smithi057.stdout:(59/149): python3-prettytable-0.7.2-14.el8.noar 182 kB/s | 44 kB 00:00 2023-11-28T11:57:43.782 INFO:teuthology.orchestra.run.smithi057.stdout:(60/149): python3-pyOpenSSL-19.0.0-1.el8.noarch 200 kB/s | 103 kB 00:00 2023-11-28T11:57:44.349 INFO:teuthology.orchestra.run.smithi181.stdout:(148/149): python3-babel-2.5.1-7.el8.noarch.rpm 112 kB/s | 4.8 MB 00:43 2023-11-28T11:57:44.557 INFO:teuthology.orchestra.run.smithi057.stdout:(61/149): python3-pyasn1-0.3.7-6.el8.noarch.rpm 163 kB/s | 126 kB 00:00 2023-11-28T11:57:45.099 INFO:teuthology.orchestra.run.smithi057.stdout:(62/149): python3-pyasn1-modules-0.3.7-6.el8.no 203 kB/s | 110 kB 00:00 2023-11-28T11:57:45.416 INFO:teuthology.orchestra.run.smithi057.stdout:(63/149): python3-pytz-2017.2-11.el8.noarch.rpm 180 kB/s | 57 kB 00:00 2023-11-28T11:57:49.259 INFO:teuthology.orchestra.run.smithi057.stdout:(64/149): python3-numpy-1.14.3-10.el8.x86_64.rp 277 kB/s | 3.7 MB 00:13 2023-11-28T11:57:50.508 INFO:teuthology.orchestra.run.smithi057.stdout:(65/149): python3-babel-2.5.1-7.el8.noarch.rpm 231 kB/s | 4.8 MB 00:21 2023-11-28T11:57:51.099 INFO:teuthology.orchestra.run.smithi057.stdout:(66/149): python3-werkzeug-0.12.2-4.el8.noarch. 249 kB/s | 457 kB 00:01 2023-11-28T11:57:51.416 INFO:teuthology.orchestra.run.smithi057.stdout:(67/149): uuid-1.6.2-43.el8.x86_64.rpm 202 kB/s | 64 kB 00:00 2023-11-28T11:57:51.775 INFO:teuthology.orchestra.run.smithi057.stdout:(68/149): xmlstarlet-1.6.1-20.el8.x86_64.rpm 197 kB/s | 71 kB 00:00 2023-11-28T11:57:52.000 INFO:teuthology.orchestra.run.smithi057.stdout:(69/149): socat-1.7.4.1-1.el8.x86_64.rpm 216 kB/s | 323 kB 00:01 2023-11-28T11:57:52.226 INFO:teuthology.orchestra.run.smithi057.stdout:(70/149): elfutils-libelf-devel-0.190-1.el8.x86 138 kB/s | 62 kB 00:00 2023-11-28T11:57:52.376 INFO:teuthology.orchestra.run.smithi057.stdout:(71/149): keyutils-libs-devel-1.5.10-9.el8.x86_ 320 kB/s | 48 kB 00:00 2023-11-28T11:57:52.443 INFO:teuthology.orchestra.run.smithi057.stdout:(72/149): fuse-2.9.7-17.el8.x86_64.rpm 188 kB/s | 83 kB 00:00 2023-11-28T11:57:52.586 INFO:teuthology.orchestra.run.smithi057.stdout:(73/149): libaio-devel-0.3.112-1.el8.x86_64.rpm 130 kB/s | 19 kB 00:00 2023-11-28T11:57:52.718 INFO:teuthology.orchestra.run.smithi057.stdout:(74/149): libcom_err-devel-1.45.6-5.el8.x86_64. 293 kB/s | 39 kB 00:00 2023-11-28T11:57:52.760 INFO:teuthology.orchestra.run.smithi057.stdout:(75/149): krb5-devel-1.18.2-22.el8.x86_64.rpm 1.4 MB/s | 561 kB 00:00 2023-11-28T11:57:52.886 INFO:teuthology.orchestra.run.smithi057.stdout:(76/149): libquadmath-8.5.0-21.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2023-11-28T11:57:52.994 INFO:teuthology.orchestra.run.smithi057.stdout:(77/149): librabbitmq-0.9.0-4.el8.x86_64.rpm 434 kB/s | 47 kB 00:00 2023-11-28T11:57:53.103 INFO:teuthology.orchestra.run.smithi057.stdout:(78/149): libgfortran-8.5.0-21.el8.x86_64.rpm 1.6 MB/s | 645 kB 00:00 2023-11-28T11:57:53.128 INFO:teuthology.orchestra.run.smithi057.stdout:(79/149): libselinux-devel-2.9-8.el8.x86_64.rpm 1.5 MB/s | 200 kB 00:00 2023-11-28T11:57:53.220 INFO:teuthology.orchestra.run.smithi057.stdout:(80/149): libsepol-devel-2.9-3.el8.x86_64.rpm 744 kB/s | 87 kB 00:00 2023-11-28T11:57:53.245 INFO:teuthology.orchestra.run.smithi057.stdout:(81/149): libuuid-devel-2.32.1-43.el8.x86_64.rp 853 kB/s | 100 kB 00:00 2023-11-28T11:57:53.329 INFO:teuthology.orchestra.run.smithi057.stdout:(82/149): libverto-devel-0.3.2-2.el8.x86_64.rpm 166 kB/s | 18 kB 00:00 2023-11-28T11:57:53.354 INFO:teuthology.orchestra.run.smithi057.stdout:(83/149): libzstd-devel-1.4.4-1.el8.x86_64.rpm 403 kB/s | 44 kB 00:00 2023-11-28T11:57:53.512 INFO:teuthology.orchestra.run.smithi057.stdout:(84/149): pcre2-devel-10.32-3.el8.x86_64.rpm 3.7 MB/s | 605 kB 00:00 2023-11-28T11:57:53.630 INFO:teuthology.orchestra.run.smithi057.stdout:(85/149): pcre2-utf16-10.32-3.el8.x86_64.rpm 1.9 MB/s | 229 kB 00:00 2023-11-28T11:57:53.688 INFO:teuthology.orchestra.run.smithi057.stdout:(86/149): openssl-devel-1.1.1k-11.el8.x86_64.rp 8.9 MB/s | 3.2 MB 00:00 2023-11-28T11:57:53.747 INFO:teuthology.orchestra.run.smithi057.stdout:(87/149): pcre2-utf32-10.32-3.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-11-28T11:57:53.789 INFO:teuthology.orchestra.run.smithi057.stdout:(88/149): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.4 MB/s | 249 kB 00:00 2023-11-28T11:57:53.880 INFO:teuthology.orchestra.run.smithi057.stdout:(89/149): python3-jwt-1.6.1-2.el8.noarch.rpm 465 kB/s | 43 kB 00:00 2023-11-28T11:57:53.935 INFO:teuthology.orchestra.run.smithi057.stdout:(90/149): python3-cryptography-3.2.1-6.el8.x86_ 3.3 MB/s | 641 kB 00:00 2023-11-28T11:57:53.989 INFO:teuthology.orchestra.run.smithi057.stdout:(91/149): python3-oauthlib-2.1.0-1.el8.noarch.r 1.4 MB/s | 155 kB 00:00 2023-11-28T11:57:54.040 INFO:teuthology.orchestra.run.smithi057.stdout:(92/149): python3-pycparser-2.14-14.el8.noarch. 1.0 MB/s | 109 kB 00:00 2023-11-28T11:57:54.082 INFO:teuthology.orchestra.run.smithi057.stdout:(93/149): python3-requests-oauthlib-1.0.0-1.el8 464 kB/s | 43 kB 00:00 2023-11-28T11:57:54.148 INFO:teuthology.orchestra.run.smithi057.stdout:(94/149): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 687 kB/s | 74 kB 00:00 2023-11-28T11:57:54.291 INFO:teuthology.orchestra.run.smithi057.stdout:(95/149): zlib-devel-1.2.11-21.el8.x86_64.rpm 278 kB/s | 58 kB 00:00 2023-11-28T11:57:54.691 INFO:teuthology.orchestra.run.smithi057.stdout:(96/149): liboath-2.6.2-3.el8.x86_64.rpm 147 kB/s | 59 kB 00:00 2023-11-28T11:57:54.908 INFO:teuthology.orchestra.run.smithi057.stdout:(97/149): libunwind-1.3.1-3.el8.x86_64.rpm 347 kB/s | 75 kB 00:00 2023-11-28T11:57:54.933 INFO:teuthology.orchestra.run.smithi057.stdout:(98/149): gperftools-libs-2.7-9.el8.x86_64.rpm 390 kB/s | 306 kB 00:00 2023-11-28T11:57:55.033 INFO:teuthology.orchestra.run.smithi057.stdout:(99/149): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 436 kB/s | 44 kB 00:00 2023-11-28T11:57:55.177 INFO:teuthology.orchestra.run.smithi057.stdout:(100/149): python3-beautifulsoup4-4.6.3-2.el8.1 1.3 MB/s | 185 kB 00:00 2023-11-28T11:57:55.209 INFO:teuthology.orchestra.run.smithi057.stdout:(101/149): python3-asyncssh-2.7.0-2.el8.noarch. 1.5 MB/s | 476 kB 00:00 2023-11-28T11:57:55.293 INFO:teuthology.orchestra.run.smithi057.stdout:(102/149): python3-cachetools-3.1.1-4.el8.noarc 282 kB/s | 33 kB 00:00 2023-11-28T11:57:55.310 INFO:teuthology.orchestra.run.smithi057.stdout:(103/149): python3-certifi-2018.10.15-7.el8.noa 163 kB/s | 16 kB 00:00 2023-11-28T11:57:55.410 INFO:teuthology.orchestra.run.smithi057.stdout:(104/149): python3-cheroot-8.5.2-1.el8.noarch.r 1.4 MB/s | 173 kB 00:00 2023-11-28T11:57:55.460 INFO:teuthology.orchestra.run.smithi057.stdout:(105/149): python3-cherrypy-18.4.0-1.el8.noarch 2.5 MB/s | 384 kB 00:00 2023-11-28T11:57:55.552 INFO:teuthology.orchestra.run.smithi057.stdout:(106/149): python3-google-auth-1.1.1-10.el8.noa 572 kB/s | 81 kB 00:00 2023-11-28T11:57:55.577 INFO:teuthology.orchestra.run.smithi057.stdout:(107/149): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-11-28T11:57:55.669 INFO:teuthology.orchestra.run.smithi057.stdout:(108/149): python3-jaraco-functools-2.0-4.el8.n 151 kB/s | 18 kB 00:00 2023-11-28T11:57:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:(109/149): python3-logutils-0.3.5-11.el8.noarch 390 kB/s | 49 kB 00:00 2023-11-28T11:57:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:(110/149): python3-kubernetes-11.0.0-6.el8.noar 4.0 MB/s | 1.2 MB 00:00 2023-11-28T11:57:55.928 INFO:teuthology.orchestra.run.smithi057.stdout:(111/149): python3-more-itertools-7.2.0-3.el8.n 440 kB/s | 59 kB 00:00 2023-11-28T11:57:55.962 INFO:teuthology.orchestra.run.smithi057.stdout:(112/149): python3-natsort-7.1.1-2.el8.noarch.r 656 kB/s | 60 kB 00:00 2023-11-28T11:57:56.055 INFO:teuthology.orchestra.run.smithi057.stdout:(113/149): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2023-11-28T11:57:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:(114/149): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2023-11-28T11:57:56.138 INFO:teuthology.orchestra.run.smithi057.stdout:(115/149): python3-repoze-lru-0.7-6.el8.noarch. 403 kB/s | 34 kB 00:00 2023-11-28T11:57:56.191 INFO:teuthology.orchestra.run.smithi057.stdout:(116/149): python3-routes-2.4.1-12.el8.noarch.r 1.8 MB/s | 196 kB 00:00 2023-11-28T11:57:56.230 INFO:teuthology.orchestra.run.smithi057.stdout:(117/149): python3-rsa-4.9-2.el8.noarch.rpm 699 kB/s | 64 kB 00:00 2023-11-28T11:57:56.272 INFO:teuthology.orchestra.run.smithi057.stdout:(118/149): python3-simplegeneric-0.8.1-17.el8.n 224 kB/s | 19 kB 00:00 2023-11-28T11:57:56.314 INFO:teuthology.orchestra.run.smithi057.stdout:(119/149): python3-singledispatch-3.4.0.3-18.el 286 kB/s | 24 kB 00:00 2023-11-28T11:57:56.373 INFO:teuthology.orchestra.run.smithi057.stdout:(120/149): python3-tempora-1.14.1-5.el8.noarch. 291 kB/s | 29 kB 00:00 2023-11-28T11:57:56.389 INFO:teuthology.orchestra.run.smithi057.stdout:(121/149): python3-trustme-0.6.0-4.el8.noarch.r 365 kB/s | 27 kB 00:00 2023-11-28T11:57:56.490 INFO:teuthology.orchestra.run.smithi057.stdout:(122/149): python3-webob-1.8.5-1.el8.1.noarch.r 2.5 MB/s | 251 kB 00:00 2023-11-28T11:57:56.515 INFO:teuthology.orchestra.run.smithi057.stdout:(123/149): python3-waitress-1.4.3-1.el8.noarch. 1.8 MB/s | 255 kB 00:00 2023-11-28T11:57:56.565 INFO:teuthology.orchestra.run.smithi057.stdout:(124/149): python3-websocket-client-0.56.0-5.el 813 kB/s | 61 kB 00:00 2023-11-28T11:57:56.599 INFO:teuthology.orchestra.run.smithi057.stdout:(125/149): python3-webtest-2.0.33-1.el8.noarch. 1.0 MB/s | 85 kB 00:00 2023-11-28T11:57:56.641 INFO:teuthology.orchestra.run.smithi057.stdout:(126/149): python3-zc-lockfile-2.0-2.el8.noarch 306 kB/s | 23 kB 00:00 2023-11-28T11:57:56.875 INFO:teuthology.orchestra.run.smithi057.stdout:(127/149): thrift-0.13.0-2.el8.x86_64.rpm 6.3 MB/s | 1.7 MB 00:00 2023-11-28T11:57:57.200 INFO:teuthology.orchestra.run.smithi057.stdout:(128/149): librados2-18.0.0-2697.g2f5f6028.el8. 6.5 MB/s | 3.7 MB 00:00 2023-11-28T11:57:57.317 INFO:teuthology.orchestra.run.smithi057.stdout:(129/149): NetworkManager-initscripts-updown-1. 1.2 MB/s | 144 kB 00:00 2023-11-28T11:57:57.434 INFO:teuthology.orchestra.run.smithi057.stdout:(130/149): librbd1-18.0.0-2697.g2f5f6028.el8.x8 6.4 MB/s | 3.6 MB 00:00 2023-11-28T11:57:57.459 INFO:teuthology.orchestra.run.smithi057.stdout:(131/149): acl-2.2.53-3.el8.x86_64.rpm 583 kB/s | 83 kB 00:00 2023-11-28T11:57:57.543 INFO:teuthology.orchestra.run.smithi057.stdout:(132/149): elfutils-debuginfod-client-0.190-1.e 706 kB/s | 77 kB 00:00 2023-11-28T11:57:57.585 INFO:teuthology.orchestra.run.smithi057.stdout:(133/149): elfutils-libelf-0.190-1.el8.x86_64.r 1.8 MB/s | 232 kB 00:00 2023-11-28T11:57:57.652 INFO:teuthology.orchestra.run.smithi057.stdout:(134/149): elfutils-libs-0.190-1.el8.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2023-11-28T11:57:57.785 INFO:teuthology.orchestra.run.smithi057.stdout:(135/149): iproute-6.2.0-5.el8.x86_64.rpm 4.2 MB/s | 854 kB 00:00 2023-11-28T11:57:57.811 INFO:teuthology.orchestra.run.smithi057.stdout:(136/149): libacl-2.2.53-3.el8.x86_64.rpm 219 kB/s | 35 kB 00:00 2023-11-28T11:57:57.911 INFO:teuthology.orchestra.run.smithi057.stdout:(137/149): libacl-devel-2.2.53-3.el8.x86_64.rpm 678 kB/s | 85 kB 00:00 2023-11-28T11:57:57.936 INFO:teuthology.orchestra.run.smithi057.stdout:(138/149): libblkid-2.32.1-43.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-11-28T11:57:58.053 INFO:teuthology.orchestra.run.smithi057.stdout:(139/149): libfdisk-2.32.1-43.el8.x86_64.rpm 1.7 MB/s | 254 kB 00:00 2023-11-28T11:57:58.078 INFO:teuthology.orchestra.run.smithi057.stdout:(140/149): libmount-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 236 kB 00:00 2023-11-28T11:57:58.187 INFO:teuthology.orchestra.run.smithi057.stdout:(141/149): libsmartcols-2.32.1-43.el8.x86_64.rp 1.3 MB/s | 179 kB 00:00 2023-11-28T11:57:58.212 INFO:teuthology.orchestra.run.smithi057.stdout:(142/149): libuuid-2.32.1-43.el8.x86_64.rpm 741 kB/s | 99 kB 00:00 2023-11-28T11:57:58.379 INFO:teuthology.orchestra.run.smithi057.stdout:(143/149): openssl-1.1.1k-11.el8.x86_64.rpm 3.8 MB/s | 737 kB 00:00 2023-11-28T11:57:58.421 INFO:teuthology.orchestra.run.smithi057.stdout:(144/149): openssl-libs-1.1.1k-11.el8.x86_64.rp 7.1 MB/s | 1.5 MB 00:00 2023-11-28T11:57:58.546 INFO:teuthology.orchestra.run.smithi057.stdout:(145/149): util-linux-user-2.32.1-43.el8.x86_64 820 kB/s | 103 kB 00:00 2023-11-28T11:57:58.672 INFO:teuthology.orchestra.run.smithi057.stdout:(146/149): xfsdump-3.1.8-7.el8.x86_64.rpm 2.6 MB/s | 334 kB 00:00 2023-11-28T11:57:58.722 INFO:teuthology.orchestra.run.smithi057.stdout:(147/149): util-linux-2.32.1-43.el8.x86_64.rpm 7.2 MB/s | 2.5 MB 00:00 2023-11-28T11:57:58.814 INFO:teuthology.orchestra.run.smithi057.stdout:(148/149): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2023-11-28T11:58:37.867 INFO:teuthology.orchestra.run.smithi057.stdout:(149/149): python3-scipy-1.0.0-21.module_el8.5. 267 kB/s | 14 MB 00:52 2023-11-28T11:58:37.872 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:58:37.872 INFO:teuthology.orchestra.run.smithi057.stdout:Total 2.1 MB/s | 212 MB 01:41 2023-11-28T11:58:38.880 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2023-11-28T11:58:39.405 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2023-11-28T11:58:39.406 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2023-11-28T11:58:40.989 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2023-11-28T11:58:40.992 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2023-11-28T11:58:46.832 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2023-11-28T11:58:47.112 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: openssl-libs-1:1.1.1k-11.el8.x86_64 1/1 2023-11-28T11:58:47.350 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : openssl-libs-1:1.1.1k-11.el8.x86_64 1/170 2023-11-28T11:58:47.592 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: openssl-libs-1:1.1.1k-11.el8.x86_64 1/170 2023-11-28T11:58:47.857 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/170 2023-11-28T11:58:47.989 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/170 2023-11-28T11:58:48.072 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/170 2023-11-28T11:58:48.281 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/170 2023-11-28T11:58:48.356 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/170 2023-11-28T11:58:48.550 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/170 2023-11-28T11:58:48.706 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/170 2023-11-28T11:58:48.773 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/170 2023-11-28T11:58:48.982 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/170 2023-11-28T11:58:49.190 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : elfutils-libelf-0.190-1.el8.x86_64 8/170 2023-11-28T11:58:49.364 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 9/170 2023-11-28T11:58:49.815 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 9/170 2023-11-28T11:58:49.972 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/170 2023-11-28T11:58:50.414 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/170 2023-11-28T11:58:50.614 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/170 2023-11-28T11:58:50.753 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/170 2023-11-28T11:58:50.939 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/170 2023-11-28T11:58:51.031 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/170 2023-11-28T11:58:51.373 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/170 2023-11-28T11:58:51.664 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/170 2023-11-28T11:58:51.757 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/170 2023-11-28T11:58:51.940 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libpq-13.5-1.el8.x86_64 15/170 2023-11-28T11:58:52.154 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 16/170 2023-11-28T11:58:52.754 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 17/170 2023-11-28T11:58:52.953 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-10.23-2.module_el8+572+929c87ac.x86_6 18/170 2023-11-28T11:58:53.122 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/170 2023-11-28T11:58:53.230 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/170 2023-11-28T11:58:53.388 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 20/170 2023-11-28T11:58:53.438 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 20/170 2023-11-28T11:58:53.648 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libacl-2.2.53-3.el8.x86_64 21/170 2023-11-28T11:58:53.879 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 22/170 2023-11-28T11:58:54.050 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 23/170 2023-11-28T11:58:54.247 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 24/170 2023-11-28T11:58:54.392 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 24/170 2023-11-28T11:58:54.572 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 25/170 2023-11-28T11:58:55.232 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 25/170 2023-11-28T11:58:55.430 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-0.3.15-4.el8.x86_64 26/170 2023-11-28T11:58:56.111 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 26/170 2023-11-28T11:58:56.365 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 27/170 2023-11-28T11:58:57.140 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 27/170 2023-11-28T11:58:57.469 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/170 2023-11-28T11:58:57.689 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/170 2023-11-28T11:58:57.945 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 30/170 2023-11-28T11:58:58.147 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 31/170 2023-11-28T11:58:58.353 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 32/170 2023-11-28T11:58:58.501 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 33/170 2023-11-28T11:58:58.728 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 34/170 2023-11-28T11:58:59.781 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 35/170 2023-11-28T11:59:00.188 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 36/170 2023-11-28T11:59:00.504 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 37/170 2023-11-28T11:59:00.781 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/170 2023-11-28T11:59:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/170 2023-11-28T11:59:03.050 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 39/170 2023-11-28T11:59:03.228 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/170 2023-11-28T11:59:03.451 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rsa-4.9-2.el8.noarch 41/170 2023-11-28T11:59:03.620 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 42/170 2023-11-28T11:59:03.829 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 43/170 2023-11-28T11:59:04.005 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : elfutils-libs-0.190-1.el8.x86_64 44/170 2023-11-28T11:59:04.197 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el8.x86_64 45/170 2023-11-28T11:59:04.445 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 46/170 2023-11-28T11:59:04.510 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 46/170 2023-11-28T11:59:04.687 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 47/170 2023-11-28T11:59:04.761 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 47/170 2023-11-28T11:59:04.931 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 48/170 2023-11-28T11:59:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 49/170 2023-11-28T11:59:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 50/170 2023-11-28T11:59:05.780 INFO:teuthology.orchestra.run.smithi181.stdout:(149/149): python3-scipy-1.0.0-21.module_el8.5. 122 kB/s | 14 MB 01:55 2023-11-28T11:59:05.783 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:59:05.784 INFO:teuthology.orchestra.run.smithi181.stdout:Total 1.4 MB/s | 212 MB 02:31 2023-11-28T11:59:05.799 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 50/170 2023-11-28T11:59:06.037 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 51/170 2023-11-28T11:59:06.056 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 51/170 2023-11-28T11:59:06.236 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 52/170 2023-11-28T11:59:06.517 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 53/170 2023-11-28T11:59:06.737 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-11-28T11:59:06.803 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 54/170 2023-11-28T11:59:06.975 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 55/170 2023-11-28T11:59:07.149 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 56/170 2023-11-28T11:59:07.201 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-11-28T11:59:07.201 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-11-28T11:59:07.350 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 57/170 2023-11-28T11:59:07.577 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 58/170 2023-11-28T11:59:07.817 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 59/170 2023-11-28T11:59:07.973 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 60/170 2023-11-28T11:59:08.126 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 61/170 2023-11-28T11:59:08.316 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 62/170 2023-11-28T11:59:08.476 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 63/170 2023-11-28T11:59:08.640 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-portend-2.6-1.el8.noarch 64/170 2023-11-28T11:59:08.654 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-11-28T11:59:08.658 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-11-28T11:59:08.812 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 65/170 2023-11-28T11:59:09.060 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 66/170 2023-11-28T11:59:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 67/170 2023-11-28T11:59:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 68/170 2023-11-28T11:59:09.875 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 69/170 2023-11-28T11:59:10.139 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 70/170 2023-11-28T11:59:10.316 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 71/170 2023-11-28T11:59:11.149 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 72/170 2023-11-28T11:59:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 73/170 2023-11-28T11:59:11.889 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 73/170 2023-11-28T11:59:12.113 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 74/170 2023-11-28T11:59:12.355 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 75/170 2023-11-28T11:59:12.772 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 76/170 2023-11-28T11:59:13.018 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 77/170 2023-11-28T11:59:13.267 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 78/170 2023-11-28T11:59:13.503 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 79/170 2023-11-28T11:59:13.729 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 80/170 2023-11-28T11:59:14.621 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-11-28T11:59:14.780 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openssl-libs-1:1.1.1k-11.el8.x86_64 1/1 2023-11-28T11:59:14.934 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 81/170 2023-11-28T11:59:15.048 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-libs-1:1.1.1k-11.el8.x86_64 1/170 2023-11-28T11:59:15.269 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openssl-libs-1:1.1.1k-11.el8.x86_64 1/170 2023-11-28T11:59:15.288 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 82/170 2023-11-28T11:59:15.478 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 83/170 2023-11-28T11:59:15.499 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/170 2023-11-28T11:59:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/170 2023-11-28T11:59:15.708 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/170 2023-11-28T11:59:15.714 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 84/170 2023-11-28T11:59:15.951 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/170 2023-11-28T11:59:16.019 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/170 2023-11-28T11:59:16.031 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 85/170 2023-11-28T11:59:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/170 2023-11-28T11:59:16.280 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 86/170 2023-11-28T11:59:16.451 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/170 2023-11-28T11:59:16.502 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/170 2023-11-28T11:59:16.523 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 87/170 2023-11-28T11:59:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/170 2023-11-28T11:59:16.761 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 88/170 2023-11-28T11:59:16.983 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-libelf-0.190-1.el8.x86_64 8/170 2023-11-28T11:59:17.095 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 89/170 2023-11-28T11:59:17.200 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 9/170 2023-11-28T11:59:17.274 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 90/170 2023-11-28T11:59:17.437 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 91/170 2023-11-28T11:59:17.607 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 9/170 2023-11-28T11:59:17.607 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 92/170 2023-11-28T11:59:17.808 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/170 2023-11-28T11:59:17.905 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 93/170 2023-11-28T11:59:18.140 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 94/170 2023-11-28T11:59:18.208 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/170 2023-11-28T11:59:18.291 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 95/170 2023-11-28T11:59:18.392 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/170 2023-11-28T11:59:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 96/170 2023-11-28T11:59:18.517 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/170 2023-11-28T11:59:18.692 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/170 2023-11-28T11:59:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 97/170 2023-11-28T11:59:18.790 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/170 2023-11-28T11:59:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : fuse-2.9.7-17.el8.x86_64 98/170 2023-11-28T11:59:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 99/170 2023-11-28T11:59:19.159 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/170 2023-11-28T11:59:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : uuid-1.6.2-43.el8.x86_64 100/170 2023-11-28T11:59:19.330 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 100/170 2023-11-28T11:59:19.483 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/170 2023-11-28T11:59:19.527 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 101/170 2023-11-28T11:59:19.548 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/170 2023-11-28T11:59:19.786 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpq-13.5-1.el8.x86_64 15/170 2023-11-28T11:59:19.991 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 16/170 2023-11-28T11:59:20.478 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 17/170 2023-11-28T11:59:20.745 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-10.23-2.module_el8+572+929c87ac.x86_6 18/170 2023-11-28T11:59:20.900 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/170 2023-11-28T11:59:21.041 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/170 2023-11-28T11:59:21.208 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 20/170 2023-11-28T11:59:21.318 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 20/170 2023-11-28T11:59:21.568 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libacl-2.2.53-3.el8.x86_64 21/170 2023-11-28T11:59:21.806 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 22/170 2023-11-28T11:59:22.037 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 23/170 2023-11-28T11:59:22.233 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 24/170 2023-11-28T11:59:22.251 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 102/170 2023-11-28T11:59:22.368 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 24/170 2023-11-28T11:59:22.534 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 102/170 2023-11-28T11:59:22.583 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 25/170 2023-11-28T11:59:23.181 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 25/170 2023-11-28T11:59:23.191 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 102/170 2023-11-28T11:59:23.358 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.15-4.el8.x86_64 26/170 2023-11-28T11:59:23.512 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 103/170 2023-11-28T11:59:23.614 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 103/170 2023-11-28T11:59:23.792 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 104/170 2023-11-28T11:59:23.973 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 26/170 2023-11-28T11:59:24.166 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 27/170 2023-11-28T11:59:24.898 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 27/170 2023-11-28T11:59:25.281 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/170 2023-11-28T11:59:25.497 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/170 2023-11-28T11:59:25.704 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 30/170 2023-11-28T11:59:25.904 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 31/170 2023-11-28T11:59:26.097 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 32/170 2023-11-28T11:59:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 33/170 2023-11-28T11:59:26.593 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 34/170 2023-11-28T11:59:27.636 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 35/170 2023-11-28T11:59:28.006 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 36/170 2023-11-28T11:59:28.350 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 37/170 2023-11-28T11:59:28.549 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/170 2023-11-28T11:59:30.417 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/170 2023-11-28T11:59:30.763 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 39/170 2023-11-28T11:59:30.976 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/170 2023-11-28T11:59:31.660 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el8.noarch 41/170 2023-11-28T11:59:31.880 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 42/170 2023-11-28T11:59:32.080 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 43/170 2023-11-28T11:59:32.302 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-libs-0.190-1.el8.x86_64 44/170 2023-11-28T11:59:32.535 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el8.x86_64 45/170 2023-11-28T11:59:32.773 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 46/170 2023-11-28T11:59:32.852 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 46/170 2023-11-28T11:59:33.015 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 47/170 2023-11-28T11:59:33.112 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 47/170 2023-11-28T11:59:33.297 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 48/170 2023-11-28T11:59:36.615 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 49/170 2023-11-28T11:59:36.616 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 50/170 2023-11-28T11:59:36.616 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 50/170 2023-11-28T11:59:36.616 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 51/170 2023-11-28T11:59:36.616 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 51/170 2023-11-28T11:59:36.616 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 52/170 2023-11-28T11:59:36.616 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 53/170 2023-11-28T11:59:36.616 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 54/170 2023-11-28T11:59:36.616 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 55/170 2023-11-28T11:59:36.616 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 56/170 2023-11-28T11:59:36.616 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 57/170 2023-11-28T11:59:36.616 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 58/170 2023-11-28T11:59:36.616 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 59/170 2023-11-28T11:59:36.617 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 60/170 2023-11-28T11:59:36.812 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 61/170 2023-11-28T11:59:36.986 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 62/170 2023-11-28T11:59:37.178 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 63/170 2023-11-28T11:59:37.347 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-2.6-1.el8.noarch 64/170 2023-11-28T11:59:37.556 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 65/170 2023-11-28T11:59:37.825 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 66/170 2023-11-28T11:59:38.098 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 67/170 2023-11-28T11:59:38.319 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 68/170 2023-11-28T11:59:38.683 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 69/170 2023-11-28T11:59:38.965 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 70/170 2023-11-28T11:59:39.174 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 71/170 2023-11-28T11:59:39.975 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 72/170 2023-11-28T11:59:40.302 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 73/170 2023-11-28T11:59:40.373 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 73/170 2023-11-28T11:59:40.636 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 74/170 2023-11-28T11:59:40.911 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 75/170 2023-11-28T11:59:41.361 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 76/170 2023-11-28T11:59:41.676 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 77/170 2023-11-28T11:59:41.925 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 78/170 2023-11-28T11:59:42.216 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 79/170 2023-11-28T11:59:42.503 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 80/170 2023-11-28T11:59:43.659 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 81/170 2023-11-28T11:59:44.021 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 82/170 2023-11-28T11:59:44.279 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 83/170 2023-11-28T11:59:44.507 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 84/170 2023-11-28T11:59:44.892 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 85/170 2023-11-28T11:59:45.114 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 86/170 2023-11-28T11:59:45.415 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 87/170 2023-11-28T11:59:45.495 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 104/170 2023-11-28T11:59:45.495 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /sys 2023-11-28T11:59:45.495 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /proc 2023-11-28T11:59:45.496 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /dev 2023-11-28T11:59:45.496 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /run 2023-11-28T11:59:45.496 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /mnt 2023-11-28T11:59:45.496 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /var/tmp 2023-11-28T11:59:45.496 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /home 2023-11-28T11:59:45.496 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /tmp 2023-11-28T11:59:45.496 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /dev 2023-11-28T11:59:45.496 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T11:59:45.665 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 88/170 2023-11-28T11:59:45.881 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 105/170 2023-11-28T11:59:46.022 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 89/170 2023-11-28T11:59:46.251 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 90/170 2023-11-28T11:59:46.459 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 91/170 2023-11-28T11:59:46.629 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 92/170 2023-11-28T11:59:46.975 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 93/170 2023-11-28T11:59:47.250 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 94/170 2023-11-28T11:59:47.427 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 95/170 2023-11-28T11:59:47.612 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 105/170 2023-11-28T11:59:47.649 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 96/170 2023-11-28T11:59:47.841 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 106/170 2023-11-28T11:59:47.950 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 97/170 2023-11-28T11:59:48.085 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 106/170 2023-11-28T11:59:48.131 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.7-17.el8.x86_64 98/170 2023-11-28T11:59:48.255 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 107/170 2023-11-28T11:59:48.340 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 99/170 2023-11-28T11:59:48.503 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-43.el8.x86_64 100/170 2023-11-28T11:59:48.604 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 100/170 2023-11-28T11:59:48.754 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 107/170 2023-11-28T11:59:48.762 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 101/170 2023-11-28T11:59:48.930 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/170 2023-11-28T11:59:48.959 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/170 2023-11-28T11:59:49.210 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 109/170 2023-11-28T11:59:49.410 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 110/170 2023-11-28T11:59:49.538 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 111/170 2023-11-28T11:59:49.607 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 111/170 2023-11-28T11:59:50.030 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : jq-1.6-8.el8.x86_64 112/170 2023-11-28T11:59:50.247 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : automake-1.16.1-8.el8.noarch 113/170 2023-11-28T11:59:50.786 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 114/170 2023-11-28T11:59:50.960 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 114/170 2023-11-28T11:59:51.082 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 115/170 2023-11-28T11:59:51.292 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 116/170 2023-11-28T11:59:51.293 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 102/170 2023-11-28T11:59:51.554 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 117/170 2023-11-28T11:59:51.562 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 102/170 2023-11-28T11:59:51.794 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 117/170 2023-11-28T11:59:51.998 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 118/170 2023-11-28T11:59:52.176 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 102/170 2023-11-28T11:59:52.776 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 103/170 2023-11-28T11:59:52.885 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 103/170 2023-11-28T11:59:53.074 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 118/170 2023-11-28T11:59:53.095 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 104/170 2023-11-28T11:59:53.262 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 119/170 2023-11-28T11:59:53.369 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 119/170 2023-11-28T11:59:53.715 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 120/170 2023-11-28T11:59:53.966 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 121/170 2023-11-28T11:59:54.009 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 121/170 2023-11-28T11:59:54.166 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 122/170 2023-11-28T11:59:54.180 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 122/170 2023-11-28T11:59:54.375 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 123/170 2023-11-28T11:59:54.520 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libtool-2.4.6-25.el8.x86_64 124/170 2023-11-28T11:59:59.384 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 124/170 2023-11-28T12:00:00.927 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/170 2023-11-28T12:00:01.156 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/170 2023-11-28T12:00:01.200 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/170 2023-11-28T12:00:01.379 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 127/170 2023-11-28T12:00:01.429 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 127/170 2023-11-28T12:00:01.603 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 128/170 2023-11-28T12:00:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 128/170 2023-11-28T12:00:02.190 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/170 2023-11-28T12:00:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/170 2023-11-28T12:00:03.691 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-contrib-10.23-2.module_el8+572+929c87 130/170 2023-11-28T12:00:03.714 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/170 2023-11-28T12:00:04.025 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openssl-devel-1:1.1.1k-11.el8.x86_64 132/170 2023-11-28T12:00:04.210 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : elfutils-libelf-devel-0.190-1.el8.x86_64 133/170 2023-11-28T12:00:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 134/170 2023-11-28T12:00:04.624 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 135/170 2023-11-28T12:00:04.809 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 136/170 2023-11-28T12:00:05.088 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : acl-2.2.53-3.el8.x86_64 137/170 2023-11-28T12:00:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libacl-devel-2.2.53-3.el8.x86_64 138/170 2023-11-28T12:00:06.361 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: postgresql-server-10.23-2.module_el8+572+929c87a 139/170 2023-11-28T12:00:06.673 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-server-10.23-2.module_el8+572+929c87a 139/170 2023-11-28T12:00:06.744 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: postgresql-server-10.23-2.module_el8+572+929c87a 139/170 2023-11-28T12:00:06.995 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : xfsdump-3.1.8-7.el8.x86_64 140/170 2023-11-28T12:00:07.187 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 141/170 2023-11-28T12:00:07.438 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 142/170 2023-11-28T12:00:07.783 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 143/170 2023-11-28T12:00:08.193 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 144/170 2023-11-28T12:00:08.458 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : openssl-1:1.1.1k-11.el8.x86_64 145/170 2023-11-28T12:00:08.597 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-11.e 146/170 2023-11-28T12:00:08.614 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-11.e 146/170 2023-11-28T12:00:08.819 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 147/170 2023-11-28T12:00:09.010 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flex-2.6.1-9.el8.x86_64 148/170 2023-11-28T12:00:09.102 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 148/170 2023-11-28T12:00:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : bison-3.0.4-10.el8.x86_64 149/170 2023-11-28T12:00:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 149/170 2023-11-28T12:00:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 150/170 2023-11-28T12:00:09.743 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 151/170 2023-11-28T12:00:09.970 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : openssl-1:1.1.1k-9.el8.x86_64 152/170 2023-11-28T12:00:10.146 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 153/170 2023-11-28T12:00:10.146 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 154/170 2023-11-28T12:00:10.409 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 154/170 2023-11-28T12:00:10.409 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 155/170 2023-11-28T12:00:10.667 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 155/170 2023-11-28T12:00:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libacl-devel-2.2.53-1.el8.x86_64 156/170 2023-11-28T12:00:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 157/170 2023-11-28T12:00:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 157/170 2023-11-28T12:00:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 158/170 2023-11-28T12:00:11.105 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 158/170 2023-11-28T12:00:11.266 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : acl-2.2.53-1.el8.x86_64 159/170 2023-11-28T12:00:11.478 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 160/170 2023-11-28T12:00:11.652 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 161/170 2023-11-28T12:00:11.682 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 104/170 2023-11-28T12:00:11.683 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2023-11-28T12:00:11.683 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2023-11-28T12:00:11.683 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-11-28T12:00:11.683 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /run 2023-11-28T12:00:11.684 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2023-11-28T12:00:11.684 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2023-11-28T12:00:11.684 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2023-11-28T12:00:11.684 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2023-11-28T12:00:11.684 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-11-28T12:00:11.685 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:11.862 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 162/170 2023-11-28T12:00:11.862 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 163/170 2023-11-28T12:00:12.088 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 163/170 2023-11-28T12:00:12.088 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 164/170 2023-11-28T12:00:12.092 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 105/170 2023-11-28T12:00:12.254 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 164/170 2023-11-28T12:00:12.255 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 165/170 2023-11-28T12:00:12.446 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 165/170 2023-11-28T12:00:12.446 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 166/170 2023-11-28T12:00:12.638 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 166/170 2023-11-28T12:00:12.638 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 167/170 2023-11-28T12:00:12.842 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 167/170 2023-11-28T12:00:12.981 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 168/170 2023-11-28T12:00:13.096 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libacl-2.2.53-1.el8.x86_64 169/170 2023-11-28T12:00:13.096 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : openssl-libs-1:1.1.1k-9.el8.x86_64 170/170 2023-11-28T12:00:13.667 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 105/170 2023-11-28T12:00:13.914 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 106/170 2023-11-28T12:00:14.003 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 170/170 2023-11-28T12:00:14.003 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/170 2023-11-28T12:00:14.003 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/170 2023-11-28T12:00:14.003 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/170 2023-11-28T12:00:14.003 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/170 2023-11-28T12:00:14.003 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/170 2023-11-28T12:00:14.004 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/170 2023-11-28T12:00:14.004 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/170 2023-11-28T12:00:14.004 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/170 2023-11-28T12:00:14.004 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/170 2023-11-28T12:00:14.004 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/170 2023-11-28T12:00:14.004 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/170 2023-11-28T12:00:14.004 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/170 2023-11-28T12:00:14.004 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/170 2023-11-28T12:00:14.004 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/170 2023-11-28T12:00:14.004 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/170 2023-11-28T12:00:14.004 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/170 2023-11-28T12:00:14.004 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/170 2023-11-28T12:00:14.005 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/170 2023-11-28T12:00:14.005 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/170 2023-11-28T12:00:14.005 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/170 2023-11-28T12:00:14.005 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/170 2023-11-28T12:00:14.005 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/170 2023-11-28T12:00:14.005 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/170 2023-11-28T12:00:14.005 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/170 2023-11-28T12:00:14.005 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/170 2023-11-28T12:00:14.019 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/170 2023-11-28T12:00:14.019 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/170 2023-11-28T12:00:14.019 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/170 2023-11-28T12:00:14.019 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/170 2023-11-28T12:00:14.019 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/170 2023-11-28T12:00:14.020 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/170 2023-11-28T12:00:14.020 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/170 2023-11-28T12:00:14.020 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/170 2023-11-28T12:00:14.020 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/170 2023-11-28T12:00:14.020 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/170 2023-11-28T12:00:14.020 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/170 2023-11-28T12:00:14.020 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/170 2023-11-28T12:00:14.020 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/170 2023-11-28T12:00:14.020 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : automake-1.16.1-8.el8.noarch 39/170 2023-11-28T12:00:14.021 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/170 2023-11-28T12:00:14.021 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/170 2023-11-28T12:00:14.021 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : jq-1.6-8.el8.x86_64 42/170 2023-11-28T12:00:14.021 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/170 2023-11-28T12:00:14.021 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/170 2023-11-28T12:00:14.021 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/170 2023-11-28T12:00:14.021 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/170 2023-11-28T12:00:14.021 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/170 2023-11-28T12:00:14.021 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/170 2023-11-28T12:00:14.021 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/170 2023-11-28T12:00:14.021 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-10.23-2.module_el8+572+929c87ac.x86_6 50/170 2023-11-28T12:00:14.022 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-contrib-10.23-2.module_el8+572+929c87 51/170 2023-11-28T12:00:14.022 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-server-10.23-2.module_el8+572+929c87a 52/170 2023-11-28T12:00:14.022 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/170 2023-11-28T12:00:14.022 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/170 2023-11-28T12:00:14.022 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/170 2023-11-28T12:00:14.022 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/170 2023-11-28T12:00:14.022 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/170 2023-11-28T12:00:14.022 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/170 2023-11-28T12:00:14.022 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/170 2023-11-28T12:00:14.022 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/170 2023-11-28T12:00:14.022 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/170 2023-11-28T12:00:14.023 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/170 2023-11-28T12:00:14.023 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/170 2023-11-28T12:00:14.023 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/170 2023-11-28T12:00:14.023 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 65/170 2023-11-28T12:00:14.023 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/170 2023-11-28T12:00:14.023 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/170 2023-11-28T12:00:14.023 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/170 2023-11-28T12:00:14.023 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/170 2023-11-28T12:00:14.023 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/170 2023-11-28T12:00:14.024 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-libelf-devel-0.190-1.el8.x86_64 71/170 2023-11-28T12:00:14.024 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 72/170 2023-11-28T12:00:14.024 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/170 2023-11-28T12:00:14.024 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/170 2023-11-28T12:00:14.024 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/170 2023-11-28T12:00:14.024 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/170 2023-11-28T12:00:14.024 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 77/170 2023-11-28T12:00:14.024 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 78/170 2023-11-28T12:00:14.024 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 79/170 2023-11-28T12:00:14.025 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 80/170 2023-11-28T12:00:14.025 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/170 2023-11-28T12:00:14.025 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 82/170 2023-11-28T12:00:14.025 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/170 2023-11-28T12:00:14.025 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 84/170 2023-11-28T12:00:14.025 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-devel-1:1.1.1k-11.el8.x86_64 85/170 2023-11-28T12:00:14.025 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 86/170 2023-11-28T12:00:14.025 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 87/170 2023-11-28T12:00:14.025 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 88/170 2023-11-28T12:00:14.025 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 89/170 2023-11-28T12:00:14.026 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 90/170 2023-11-28T12:00:14.026 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 91/170 2023-11-28T12:00:14.026 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 92/170 2023-11-28T12:00:14.026 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 93/170 2023-11-28T12:00:14.026 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 94/170 2023-11-28T12:00:14.026 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 95/170 2023-11-28T12:00:14.026 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 96/170 2023-11-28T12:00:14.026 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 97/170 2023-11-28T12:00:14.026 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 98/170 2023-11-28T12:00:14.026 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 99/170 2023-11-28T12:00:14.026 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 100/170 2023-11-28T12:00:14.027 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 101/170 2023-11-28T12:00:14.027 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 102/170 2023-11-28T12:00:14.027 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 103/170 2023-11-28T12:00:14.027 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 104/170 2023-11-28T12:00:14.027 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 105/170 2023-11-28T12:00:14.027 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 106/170 2023-11-28T12:00:14.027 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 107/170 2023-11-28T12:00:14.027 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 108/170 2023-11-28T12:00:14.027 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 109/170 2023-11-28T12:00:14.027 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 110/170 2023-11-28T12:00:14.027 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 111/170 2023-11-28T12:00:14.027 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 112/170 2023-11-28T12:00:14.028 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 113/170 2023-11-28T12:00:14.028 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 114/170 2023-11-28T12:00:14.028 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-portend-2.6-1.el8.noarch 115/170 2023-11-28T12:00:14.028 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 116/170 2023-11-28T12:00:14.028 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 117/170 2023-11-28T12:00:14.028 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 118/170 2023-11-28T12:00:14.028 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 119/170 2023-11-28T12:00:14.028 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 120/170 2023-11-28T12:00:14.028 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 121/170 2023-11-28T12:00:14.028 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 122/170 2023-11-28T12:00:14.028 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 123/170 2023-11-28T12:00:14.028 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 124/170 2023-11-28T12:00:14.029 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 125/170 2023-11-28T12:00:14.029 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 126/170 2023-11-28T12:00:14.029 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 127/170 2023-11-28T12:00:14.029 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 128/170 2023-11-28T12:00:14.029 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/170 2023-11-28T12:00:14.029 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/170 2023-11-28T12:00:14.029 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/170 2023-11-28T12:00:14.029 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/170 2023-11-28T12:00:14.029 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-11.e 133/170 2023-11-28T12:00:14.029 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 134/170 2023-11-28T12:00:14.029 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : acl-2.2.53-3.el8.x86_64 135/170 2023-11-28T12:00:14.029 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : acl-2.2.53-1.el8.x86_64 136/170 2023-11-28T12:00:14.030 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el8.x86_64 137/170 2023-11-28T12:00:14.030 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 138/170 2023-11-28T12:00:14.030 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-libelf-0.190-1.el8.x86_64 139/170 2023-11-28T12:00:14.030 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 140/170 2023-11-28T12:00:14.030 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-libs-0.190-1.el8.x86_64 141/170 2023-11-28T12:00:14.030 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 142/170 2023-11-28T12:00:14.030 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 143/170 2023-11-28T12:00:14.030 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 144/170 2023-11-28T12:00:14.030 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libacl-2.2.53-3.el8.x86_64 145/170 2023-11-28T12:00:14.030 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libacl-2.2.53-1.el8.x86_64 146/170 2023-11-28T12:00:14.031 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libacl-devel-2.2.53-3.el8.x86_64 147/170 2023-11-28T12:00:14.031 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libacl-devel-2.2.53-1.el8.x86_64 148/170 2023-11-28T12:00:14.031 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 149/170 2023-11-28T12:00:14.031 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 150/170 2023-11-28T12:00:14.031 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 151/170 2023-11-28T12:00:14.031 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 152/170 2023-11-28T12:00:14.031 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 153/170 2023-11-28T12:00:14.031 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 154/170 2023-11-28T12:00:14.031 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 155/170 2023-11-28T12:00:14.031 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 156/170 2023-11-28T12:00:14.032 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 157/170 2023-11-28T12:00:14.032 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 158/170 2023-11-28T12:00:14.032 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-1:1.1.1k-11.el8.x86_64 159/170 2023-11-28T12:00:14.032 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-1:1.1.1k-9.el8.x86_64 160/170 2023-11-28T12:00:14.032 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-libs-1:1.1.1k-11.el8.x86_64 161/170 2023-11-28T12:00:14.032 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8.x86_64 162/170 2023-11-28T12:00:14.032 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 163/170 2023-11-28T12:00:14.032 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 164/170 2023-11-28T12:00:14.032 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 165/170 2023-11-28T12:00:14.032 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 166/170 2023-11-28T12:00:14.032 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xfsdump-3.1.8-7.el8.x86_64 167/170 2023-11-28T12:00:14.032 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 168/170 2023-11-28T12:00:14.033 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 169/170 2023-11-28T12:00:14.133 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 106/170 2023-11-28T12:00:14.325 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 107/170 2023-11-28T12:00:14.777 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 107/170 2023-11-28T12:00:15.008 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/170 2023-11-28T12:00:15.033 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/170 2023-11-28T12:00:15.257 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 109/170 2023-11-28T12:00:15.495 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 110/170 2023-11-28T12:00:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 111/170 2023-11-28T12:00:15.747 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 111/170 2023-11-28T12:00:16.093 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : jq-1.6-8.el8.x86_64 112/170 2023-11-28T12:00:16.358 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.1-8.el8.noarch 113/170 2023-11-28T12:00:16.444 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 170/170 2023-11-28T12:00:16.445 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:16.445 INFO:teuthology.orchestra.run.smithi057.stdout:Upgraded: 2023-11-28T12:00:16.445 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-initscripts-updown-1:1.40.16-11.el8.noarch 2023-11-28T12:00:16.445 INFO:teuthology.orchestra.run.smithi057.stdout: acl-2.2.53-3.el8.x86_64 2023-11-28T12:00:16.445 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-debuginfod-client-0.190-1.el8.x86_64 2023-11-28T12:00:16.445 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf-0.190-1.el8.x86_64 2023-11-28T12:00:16.445 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libs-0.190-1.el8.x86_64 2023-11-28T12:00:16.445 INFO:teuthology.orchestra.run.smithi057.stdout: iproute-6.2.0-5.el8.x86_64 2023-11-28T12:00:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: libacl-2.2.53-3.el8.x86_64 2023-11-28T12:00:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: libacl-devel-2.2.53-3.el8.x86_64 2023-11-28T12:00:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: libblkid-2.32.1-43.el8.x86_64 2023-11-28T12:00:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-11-28T12:00:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: libmount-2.32.1-43.el8.x86_64 2023-11-28T12:00:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-11-28T12:00:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid-2.32.1-43.el8.x86_64 2023-11-28T12:00:16.446 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-1:1.1.1k-11.el8.x86_64 2023-11-28T12:00:16.447 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-libs-1:1.1.1k-11.el8.x86_64 2023-11-28T12:00:16.447 INFO:teuthology.orchestra.run.smithi057.stdout: util-linux-2.32.1-43.el8.x86_64 2023-11-28T12:00:16.447 INFO:teuthology.orchestra.run.smithi057.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-11-28T12:00:16.447 INFO:teuthology.orchestra.run.smithi057.stdout: xfsdump-3.1.8-7.el8.x86_64 2023-11-28T12:00:16.447 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-11-28T12:00:16.447 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2023-11-28T12:00:16.447 INFO:teuthology.orchestra.run.smithi057.stdout: automake-1.16.1-8.el8.noarch 2023-11-28T12:00:16.447 INFO:teuthology.orchestra.run.smithi057.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T12:00:16.447 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.448 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.448 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.448 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.448 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:16.448 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.448 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.448 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.448 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:16.448 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:16.448 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:16.449 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:16.449 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:16.449 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.449 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.449 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:16.449 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.450 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.450 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.450 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:16.450 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:16.450 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.450 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:16.451 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf-devel-0.190-1.el8.x86_64 2023-11-28T12:00:16.451 INFO:teuthology.orchestra.run.smithi057.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T12:00:16.451 INFO:teuthology.orchestra.run.smithi057.stdout: fuse-2.9.7-17.el8.x86_64 2023-11-28T12:00:16.451 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T12:00:16.451 INFO:teuthology.orchestra.run.smithi057.stdout: jq-1.6-8.el8.x86_64 2023-11-28T12:00:16.451 INFO:teuthology.orchestra.run.smithi057.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T12:00:16.451 INFO:teuthology.orchestra.run.smithi057.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-11-28T12:00:16.451 INFO:teuthology.orchestra.run.smithi057.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T12:00:16.451 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.452 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.452 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.452 INFO:teuthology.orchestra.run.smithi057.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-11-28T12:00:16.452 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran-8.5.0-21.el8.x86_64 2023-11-28T12:00:16.452 INFO:teuthology.orchestra.run.smithi057.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T12:00:16.452 INFO:teuthology.orchestra.run.smithi057.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T12:00:16.452 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath-8.5.0-21.el8.x86_64 2023-11-28T12:00:16.452 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-11-28T12:00:16.453 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.453 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.453 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T12:00:16.453 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.453 INFO:teuthology.orchestra.run.smithi057.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-11-28T12:00:16.453 INFO:teuthology.orchestra.run.smithi057.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T12:00:16.453 INFO:teuthology.orchestra.run.smithi057.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T12:00:16.453 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T12:00:16.453 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-11-28T12:00:16.454 INFO:teuthology.orchestra.run.smithi057.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-11-28T12:00:16.454 INFO:teuthology.orchestra.run.smithi057.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-11-28T12:00:16.454 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T12:00:16.454 INFO:teuthology.orchestra.run.smithi057.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T12:00:16.454 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-0.3.15-4.el8.x86_64 2023-11-28T12:00:16.454 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-11-28T12:00:16.454 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-devel-1:1.1.1k-11.el8.x86_64 2023-11-28T12:00:16.454 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-11-28T12:00:16.454 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-11-28T12:00:16.455 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-11-28T12:00:16.455 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-10.23-2.module_el8+572+929c87ac.x86_64 2023-11-28T12:00:16.455 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-contrib-10.23-2.module_el8+572+929c87ac.x86_64 2023-11-28T12:00:16.455 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-server-10.23-2.module_el8+572+929c87ac.x86_64 2023-11-28T12:00:16.455 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T12:00:16.455 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T12:00:16.455 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T12:00:16.455 INFO:teuthology.orchestra.run.smithi057.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T12:00:16.455 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T12:00:16.455 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.456 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.456 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.456 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T12:00:16.456 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-11-28T12:00:16.456 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T12:00:16.456 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T12:00:16.456 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-11-28T12:00:16.456 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T12:00:16.456 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T12:00:16.456 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T12:00:16.457 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T12:00:16.457 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T12:00:16.457 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T12:00:16.457 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T12:00:16.457 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T12:00:16.457 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T12:00:16.457 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako-1.0.6-14.el8.noarch 2023-11-28T12:00:16.457 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T12:00:16.457 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T12:00:16.458 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T12:00:16.458 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T12:00:16.458 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T12:00:16.458 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T12:00:16.458 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T12:00:16.458 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T12:00:16.458 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T12:00:16.458 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T12:00:16.458 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T12:00:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T12:00:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T12:00:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz-2017.2-11.el8.noarch 2023-11-28T12:00:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T12:00:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T12:00:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T12:00:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T12:00:16.460 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-11-28T12:00:16.460 INFO:teuthology.orchestra.run.smithi057.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T12:00:16.460 INFO:teuthology.orchestra.run.smithi057.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T12:00:16.460 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T12:00:16.460 INFO:teuthology.orchestra.run.smithi057.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T12:00:16.460 INFO:teuthology.orchestra.run.smithi057.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T12:00:16.460 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T12:00:16.460 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T12:00:16.460 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T12:00:16.461 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T12:00:16.461 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T12:00:16.461 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.461 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.461 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:16.461 INFO:teuthology.orchestra.run.smithi057.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T12:00:16.461 INFO:teuthology.orchestra.run.smithi057.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T12:00:16.461 INFO:teuthology.orchestra.run.smithi057.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T12:00:16.461 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-11-28T12:00:16.461 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T12:00:16.462 INFO:teuthology.orchestra.run.smithi057.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-11-28T12:00:16.462 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:16.462 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2023-11-28T12:00:16.685 DEBUG:teuthology.parallel:result is None 2023-11-28T12:00:16.895 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 114/170 2023-11-28T12:00:17.067 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 114/170 2023-11-28T12:00:17.214 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 115/170 2023-11-28T12:00:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 116/170 2023-11-28T12:00:17.649 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 117/170 2023-11-28T12:00:17.874 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 117/170 2023-11-28T12:00:18.138 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 118/170 2023-11-28T12:00:19.104 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 118/170 2023-11-28T12:00:19.340 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 119/170 2023-11-28T12:00:19.441 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 119/170 2023-11-28T12:00:19.815 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 120/170 2023-11-28T12:00:20.080 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 121/170 2023-11-28T12:00:20.117 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 121/170 2023-11-28T12:00:20.348 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 122/170 2023-11-28T12:00:20.359 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 122/170 2023-11-28T12:00:20.541 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 123/170 2023-11-28T12:00:20.740 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-25.el8.x86_64 124/170 2023-11-28T12:00:25.145 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 124/170 2023-11-28T12:00:26.585 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/170 2023-11-28T12:00:26.838 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/170 2023-11-28T12:00:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/170 2023-11-28T12:00:27.115 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 127/170 2023-11-28T12:00:27.156 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 127/170 2023-11-28T12:00:27.381 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 128/170 2023-11-28T12:00:27.735 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 128/170 2023-11-28T12:00:27.931 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/170 2023-11-28T12:00:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/170 2023-11-28T12:00:28.511 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-10.23-2.module_el8+572+929c87 130/170 2023-11-28T12:00:29.462 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/170 2023-11-28T12:00:29.852 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:1.1.1k-11.el8.x86_64 132/170 2023-11-28T12:00:30.070 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.190-1.el8.x86_64 133/170 2023-11-28T12:00:30.327 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 134/170 2023-11-28T12:00:30.502 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 135/170 2023-11-28T12:00:30.698 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 136/170 2023-11-28T12:00:31.040 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : acl-2.2.53-3.el8.x86_64 137/170 2023-11-28T12:00:31.248 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libacl-devel-2.2.53-3.el8.x86_64 138/170 2023-11-28T12:00:32.293 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.23-2.module_el8+572+929c87a 139/170 2023-11-28T12:00:32.622 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-10.23-2.module_el8+572+929c87a 139/170 2023-11-28T12:00:32.684 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.23-2.module_el8+572+929c87a 139/170 2023-11-28T12:00:32.953 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : xfsdump-3.1.8-7.el8.x86_64 140/170 2023-11-28T12:00:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 141/170 2023-11-28T12:00:33.388 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 142/170 2023-11-28T12:00:33.791 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 143/170 2023-11-28T12:00:34.241 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 144/170 2023-11-28T12:00:34.582 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-1:1.1.1k-11.el8.x86_64 145/170 2023-11-28T12:00:34.780 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-11.e 146/170 2023-11-28T12:00:34.797 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-11.e 146/170 2023-11-28T12:00:34.990 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 147/170 2023-11-28T12:00:35.246 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.1-9.el8.x86_64 148/170 2023-11-28T12:00:35.349 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 148/170 2023-11-28T12:00:35.596 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.0.4-10.el8.x86_64 149/170 2023-11-28T12:00:35.609 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 149/170 2023-11-28T12:00:35.831 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 150/170 2023-11-28T12:00:36.073 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 151/170 2023-11-28T12:00:36.281 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-1:1.1.1k-9.el8.x86_64 152/170 2023-11-28T12:00:36.465 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 153/170 2023-11-28T12:00:36.465 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 154/170 2023-11-28T12:00:36.791 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 154/170 2023-11-28T12:00:36.791 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 155/170 2023-11-28T12:00:37.091 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 155/170 2023-11-28T12:00:37.249 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libacl-devel-2.2.53-1.el8.x86_64 156/170 2023-11-28T12:00:37.253 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 157/170 2023-11-28T12:00:37.367 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 157/170 2023-11-28T12:00:37.367 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 158/170 2023-11-28T12:00:37.666 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 158/170 2023-11-28T12:00:37.885 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : acl-2.2.53-1.el8.x86_64 159/170 2023-11-28T12:00:38.115 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 160/170 2023-11-28T12:00:38.318 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 161/170 2023-11-28T12:00:38.598 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 162/170 2023-11-28T12:00:38.598 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 163/170 2023-11-28T12:00:38.840 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 163/170 2023-11-28T12:00:38.841 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 164/170 2023-11-28T12:00:39.065 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 164/170 2023-11-28T12:00:39.065 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 165/170 2023-11-28T12:00:39.276 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 165/170 2023-11-28T12:00:39.278 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 166/170 2023-11-28T12:00:39.507 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 166/170 2023-11-28T12:00:39.507 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 167/170 2023-11-28T12:00:39.756 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 167/170 2023-11-28T12:00:39.997 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 168/170 2023-11-28T12:00:40.132 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libacl-2.2.53-1.el8.x86_64 169/170 2023-11-28T12:00:40.133 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-libs-1:1.1.1k-9.el8.x86_64 170/170 2023-11-28T12:00:41.041 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 170/170 2023-11-28T12:00:41.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/170 2023-11-28T12:00:41.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/170 2023-11-28T12:00:41.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/170 2023-11-28T12:00:41.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/170 2023-11-28T12:00:41.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/170 2023-11-28T12:00:41.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/170 2023-11-28T12:00:41.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/170 2023-11-28T12:00:41.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/170 2023-11-28T12:00:41.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/170 2023-11-28T12:00:41.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/170 2023-11-28T12:00:41.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/170 2023-11-28T12:00:41.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/170 2023-11-28T12:00:41.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/170 2023-11-28T12:00:41.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/170 2023-11-28T12:00:41.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/170 2023-11-28T12:00:41.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/170 2023-11-28T12:00:41.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/170 2023-11-28T12:00:41.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/170 2023-11-28T12:00:41.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/170 2023-11-28T12:00:41.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/170 2023-11-28T12:00:41.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/170 2023-11-28T12:00:41.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/170 2023-11-28T12:00:41.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/170 2023-11-28T12:00:41.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/170 2023-11-28T12:00:41.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/170 2023-11-28T12:00:41.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/170 2023-11-28T12:00:41.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/170 2023-11-28T12:00:41.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/170 2023-11-28T12:00:41.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/170 2023-11-28T12:00:41.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/170 2023-11-28T12:00:41.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/170 2023-11-28T12:00:41.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/170 2023-11-28T12:00:41.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/170 2023-11-28T12:00:41.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/170 2023-11-28T12:00:41.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/170 2023-11-28T12:00:41.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/170 2023-11-28T12:00:41.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/170 2023-11-28T12:00:41.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/170 2023-11-28T12:00:41.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.1-8.el8.noarch 39/170 2023-11-28T12:00:41.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/170 2023-11-28T12:00:41.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/170 2023-11-28T12:00:41.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : jq-1.6-8.el8.x86_64 42/170 2023-11-28T12:00:41.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/170 2023-11-28T12:00:41.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/170 2023-11-28T12:00:41.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/170 2023-11-28T12:00:41.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/170 2023-11-28T12:00:41.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/170 2023-11-28T12:00:41.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/170 2023-11-28T12:00:41.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/170 2023-11-28T12:00:41.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-10.23-2.module_el8+572+929c87ac.x86_6 50/170 2023-11-28T12:00:41.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-10.23-2.module_el8+572+929c87 51/170 2023-11-28T12:00:41.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-10.23-2.module_el8+572+929c87a 52/170 2023-11-28T12:00:41.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/170 2023-11-28T12:00:41.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/170 2023-11-28T12:00:41.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/170 2023-11-28T12:00:41.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/170 2023-11-28T12:00:41.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/170 2023-11-28T12:00:41.049 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/170 2023-11-28T12:00:41.049 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/170 2023-11-28T12:00:41.049 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/170 2023-11-28T12:00:41.049 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/170 2023-11-28T12:00:41.050 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/170 2023-11-28T12:00:41.050 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/170 2023-11-28T12:00:41.050 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/170 2023-11-28T12:00:41.050 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 65/170 2023-11-28T12:00:41.050 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/170 2023-11-28T12:00:41.050 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/170 2023-11-28T12:00:41.050 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/170 2023-11-28T12:00:41.050 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/170 2023-11-28T12:00:41.051 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/170 2023-11-28T12:00:41.051 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.190-1.el8.x86_64 71/170 2023-11-28T12:00:41.051 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 72/170 2023-11-28T12:00:41.051 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/170 2023-11-28T12:00:41.051 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/170 2023-11-28T12:00:41.051 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/170 2023-11-28T12:00:41.051 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/170 2023-11-28T12:00:41.051 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 77/170 2023-11-28T12:00:41.051 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 78/170 2023-11-28T12:00:41.051 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 79/170 2023-11-28T12:00:41.052 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 80/170 2023-11-28T12:00:41.052 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/170 2023-11-28T12:00:41.052 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 82/170 2023-11-28T12:00:41.052 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/170 2023-11-28T12:00:41.052 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 84/170 2023-11-28T12:00:41.052 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:1.1.1k-11.el8.x86_64 85/170 2023-11-28T12:00:41.052 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 86/170 2023-11-28T12:00:41.052 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 87/170 2023-11-28T12:00:41.052 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 88/170 2023-11-28T12:00:41.052 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 89/170 2023-11-28T12:00:41.052 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 90/170 2023-11-28T12:00:41.053 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 91/170 2023-11-28T12:00:41.053 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 92/170 2023-11-28T12:00:41.053 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 93/170 2023-11-28T12:00:41.053 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 94/170 2023-11-28T12:00:41.053 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 95/170 2023-11-28T12:00:41.053 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 96/170 2023-11-28T12:00:41.053 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 97/170 2023-11-28T12:00:41.053 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 98/170 2023-11-28T12:00:41.053 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 99/170 2023-11-28T12:00:41.054 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 100/170 2023-11-28T12:00:41.054 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 101/170 2023-11-28T12:00:41.054 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 102/170 2023-11-28T12:00:41.054 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 103/170 2023-11-28T12:00:41.054 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 104/170 2023-11-28T12:00:41.054 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 105/170 2023-11-28T12:00:41.054 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 106/170 2023-11-28T12:00:41.054 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 107/170 2023-11-28T12:00:41.054 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 108/170 2023-11-28T12:00:41.054 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 109/170 2023-11-28T12:00:41.055 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 110/170 2023-11-28T12:00:41.055 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 111/170 2023-11-28T12:00:41.055 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 112/170 2023-11-28T12:00:41.055 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 113/170 2023-11-28T12:00:41.055 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 114/170 2023-11-28T12:00:41.055 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-2.6-1.el8.noarch 115/170 2023-11-28T12:00:41.055 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 116/170 2023-11-28T12:00:41.055 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 117/170 2023-11-28T12:00:41.055 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 118/170 2023-11-28T12:00:41.055 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 119/170 2023-11-28T12:00:41.056 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 120/170 2023-11-28T12:00:41.056 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 121/170 2023-11-28T12:00:41.056 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 122/170 2023-11-28T12:00:41.056 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 123/170 2023-11-28T12:00:41.057 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 124/170 2023-11-28T12:00:41.057 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 125/170 2023-11-28T12:00:41.058 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 126/170 2023-11-28T12:00:41.058 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 127/170 2023-11-28T12:00:41.058 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 128/170 2023-11-28T12:00:41.058 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/170 2023-11-28T12:00:41.058 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/170 2023-11-28T12:00:41.058 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/170 2023-11-28T12:00:41.058 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/170 2023-11-28T12:00:41.059 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-11.e 133/170 2023-11-28T12:00:41.059 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 134/170 2023-11-28T12:00:41.059 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : acl-2.2.53-3.el8.x86_64 135/170 2023-11-28T12:00:41.059 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : acl-2.2.53-1.el8.x86_64 136/170 2023-11-28T12:00:41.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el8.x86_64 137/170 2023-11-28T12:00:41.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 138/170 2023-11-28T12:00:41.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-0.190-1.el8.x86_64 139/170 2023-11-28T12:00:41.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 140/170 2023-11-28T12:00:41.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libs-0.190-1.el8.x86_64 141/170 2023-11-28T12:00:41.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 142/170 2023-11-28T12:00:41.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 143/170 2023-11-28T12:00:41.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 144/170 2023-11-28T12:00:41.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libacl-2.2.53-3.el8.x86_64 145/170 2023-11-28T12:00:41.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libacl-2.2.53-1.el8.x86_64 146/170 2023-11-28T12:00:41.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libacl-devel-2.2.53-3.el8.x86_64 147/170 2023-11-28T12:00:41.060 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libacl-devel-2.2.53-1.el8.x86_64 148/170 2023-11-28T12:00:41.061 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 149/170 2023-11-28T12:00:41.061 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 150/170 2023-11-28T12:00:41.061 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 151/170 2023-11-28T12:00:41.061 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 152/170 2023-11-28T12:00:41.061 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 153/170 2023-11-28T12:00:41.061 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 154/170 2023-11-28T12:00:41.061 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 155/170 2023-11-28T12:00:41.061 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 156/170 2023-11-28T12:00:41.062 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 157/170 2023-11-28T12:00:41.062 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 158/170 2023-11-28T12:00:41.062 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:1.1.1k-11.el8.x86_64 159/170 2023-11-28T12:00:41.062 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:1.1.1k-9.el8.x86_64 160/170 2023-11-28T12:00:41.062 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:1.1.1k-11.el8.x86_64 161/170 2023-11-28T12:00:41.062 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8.x86_64 162/170 2023-11-28T12:00:41.062 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 163/170 2023-11-28T12:00:41.063 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 164/170 2023-11-28T12:00:41.063 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 165/170 2023-11-28T12:00:41.063 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 166/170 2023-11-28T12:00:41.063 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsdump-3.1.8-7.el8.x86_64 167/170 2023-11-28T12:00:41.063 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 168/170 2023-11-28T12:00:41.064 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 169/170 2023-11-28T12:00:43.577 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 170/170 2023-11-28T12:00:43.578 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:43.578 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2023-11-28T12:00:43.578 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown-1:1.40.16-11.el8.noarch 2023-11-28T12:00:43.578 INFO:teuthology.orchestra.run.smithi181.stdout: acl-2.2.53-3.el8.x86_64 2023-11-28T12:00:43.578 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-debuginfod-client-0.190-1.el8.x86_64 2023-11-28T12:00:43.578 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-0.190-1.el8.x86_64 2023-11-28T12:00:43.578 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libs-0.190-1.el8.x86_64 2023-11-28T12:00:43.578 INFO:teuthology.orchestra.run.smithi181.stdout: iproute-6.2.0-5.el8.x86_64 2023-11-28T12:00:43.578 INFO:teuthology.orchestra.run.smithi181.stdout: libacl-2.2.53-3.el8.x86_64 2023-11-28T12:00:43.578 INFO:teuthology.orchestra.run.smithi181.stdout: libacl-devel-2.2.53-3.el8.x86_64 2023-11-28T12:00:43.579 INFO:teuthology.orchestra.run.smithi181.stdout: libblkid-2.32.1-43.el8.x86_64 2023-11-28T12:00:43.579 INFO:teuthology.orchestra.run.smithi181.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-11-28T12:00:43.579 INFO:teuthology.orchestra.run.smithi181.stdout: libmount-2.32.1-43.el8.x86_64 2023-11-28T12:00:43.579 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.579 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.579 INFO:teuthology.orchestra.run.smithi181.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-11-28T12:00:43.579 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-2.32.1-43.el8.x86_64 2023-11-28T12:00:43.579 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-1:1.1.1k-11.el8.x86_64 2023-11-28T12:00:43.579 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs-1:1.1.1k-11.el8.x86_64 2023-11-28T12:00:43.579 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-2.32.1-43.el8.x86_64 2023-11-28T12:00:43.579 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-11-28T12:00:43.579 INFO:teuthology.orchestra.run.smithi181.stdout: xfsdump-3.1.8-7.el8.x86_64 2023-11-28T12:00:43.580 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-11-28T12:00:43.580 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-11-28T12:00:43.580 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.1-8.el8.noarch 2023-11-28T12:00:43.580 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T12:00:43.580 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.580 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.580 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.580 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.580 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:43.580 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.580 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.580 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.581 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:43.581 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:43.581 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:43.581 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:43.581 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:43.581 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.581 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.581 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:43.581 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.581 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.581 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.581 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:43.582 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:43.582 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.582 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:00:43.583 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.190-1.el8.x86_64 2023-11-28T12:00:43.583 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T12:00:43.583 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.7-17.el8.x86_64 2023-11-28T12:00:43.583 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T12:00:43.583 INFO:teuthology.orchestra.run.smithi181.stdout: jq-1.6-8.el8.x86_64 2023-11-28T12:00:43.583 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T12:00:43.583 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-11-28T12:00:43.583 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T12:00:43.584 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.584 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.584 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.584 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-11-28T12:00:43.584 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-8.5.0-21.el8.x86_64 2023-11-28T12:00:43.584 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T12:00:43.584 INFO:teuthology.orchestra.run.smithi181.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T12:00:43.584 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-8.5.0-21.el8.x86_64 2023-11-28T12:00:43.584 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-11-28T12:00:43.584 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.584 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.585 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T12:00:43.585 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.585 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-11-28T12:00:43.585 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T12:00:43.585 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T12:00:43.585 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T12:00:43.585 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-11-28T12:00:43.585 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-11-28T12:00:43.585 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-11-28T12:00:43.585 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T12:00:43.585 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T12:00:43.585 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.15-4.el8.x86_64 2023-11-28T12:00:43.586 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-11-28T12:00:43.586 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:1.1.1k-11.el8.x86_64 2023-11-28T12:00:43.586 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-11-28T12:00:43.586 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-11-28T12:00:43.586 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-11-28T12:00:43.586 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-10.23-2.module_el8+572+929c87ac.x86_64 2023-11-28T12:00:43.586 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-10.23-2.module_el8+572+929c87ac.x86_64 2023-11-28T12:00:43.586 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-10.23-2.module_el8+572+929c87ac.x86_64 2023-11-28T12:00:43.586 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T12:00:43.586 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T12:00:43.586 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T12:00:43.586 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T12:00:43.587 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T12:00:43.587 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.587 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.587 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.587 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T12:00:43.587 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-11-28T12:00:43.587 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T12:00:43.587 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T12:00:43.587 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-11-28T12:00:43.587 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T12:00:43.587 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T12:00:43.587 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T12:00:43.588 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T12:00:43.588 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T12:00:43.588 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T12:00:43.588 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T12:00:43.588 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T12:00:43.588 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T12:00:43.588 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.0.6-14.el8.noarch 2023-11-28T12:00:43.588 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T12:00:43.588 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T12:00:43.588 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T12:00:43.588 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T12:00:43.588 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T12:00:43.589 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T12:00:43.589 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T12:00:43.589 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T12:00:43.589 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T12:00:43.589 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T12:00:43.589 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T12:00:43.589 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T12:00:43.589 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T12:00:43.589 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2017.2-11.el8.noarch 2023-11-28T12:00:43.589 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.589 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.589 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T12:00:43.590 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T12:00:43.590 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.590 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T12:00:43.590 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T12:00:43.590 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-11-28T12:00:43.590 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T12:00:43.590 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T12:00:43.590 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T12:00:43.590 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T12:00:43.590 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T12:00:43.590 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T12:00:43.590 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T12:00:43.591 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T12:00:43.591 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T12:00:43.591 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T12:00:43.591 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.591 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.591 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:00:43.591 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T12:00:43.591 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T12:00:43.591 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T12:00:43.591 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-11-28T12:00:43.591 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T12:00:43.592 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-11-28T12:00:43.592 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:43.592 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-11-28T12:00:43.868 DEBUG:teuthology.parallel:result is None 2023-11-28T12:00:43.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T12:00:44.194 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T12:00:44.225 INFO:teuthology.orchestra.run.smithi057.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T12:00:44.227 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T12:00:44.227 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T12:00:44.227 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T12:00:44.380 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T12:00:44.403 INFO:teuthology.orchestra.run.smithi181.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T12:00:44.404 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T12:00:44.404 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T12:00:44.404 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-28T12:00:44.404 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:00:44.404 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T12:00:44.444 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:00:44.444 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T12:00:44.475 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-28T12:00:44.475 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:00:44.475 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T12:00:44.519 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T12:00:44.594 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:00:44.594 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T12:00:44.620 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T12:00:44.688 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-28T12:00:44.688 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:00:44.688 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T12:00:44.723 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T12:00:44.797 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:00:44.798 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T12:00:44.824 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T12:00:44.892 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-28T12:00:44.892 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:00:44.892 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T12:00:44.926 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T12:00:45.002 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:00:45.002 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T12:00:45.030 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T12:00:45.099 INFO:teuthology.run_tasks:Running task ceph... 2023-11-28T12:00:45.173 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-28T12:00:45.174 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 777 /var/log/ceph 2023-11-28T12:00:45.176 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /var/log/ceph 2023-11-28T12:00:45.208 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-28T12:00:45.208 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T12:00:45.252 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T12:00:45.284 INFO:tasks.ceph:Creating extra log directories... 2023-11-28T12:00:45.284 DEBUG:teuthology.orchestra.run.smithi057:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T12:00:45.327 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T12:00:45.366 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-28T12:00:45.366 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-28T12:00:45.366 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-28T12:00:45.367 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:00:45.367 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T12:00:45.423 DEBUG:teuthology.orchestra.run.smithi057:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T12:00:45.549 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T12:00:45.583 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:00:45.583 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T12:00:45.600 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T12:00:45.712 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T12:00:45.744 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-28T12:00:45.744 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-28T12:00:45.744 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962', 'branch': 'wip-neesingh-testing-231123', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7469962', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-231123'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'issue_54557', 'sha1': '17c45c858c4cb3105f973d12c6c74f8d5037aa89'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4140, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'issue_54557', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': '17c45c858c4cb3105f973d12c6c74f8d5037aa89', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi057.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA6+4W7Ok7NoF6bVdQc/c/LOeeQAcwpH+6HftP+t3kY+m8ErJj7jYo3zzEqgHMOHWK+VUvA6Fl7wmHXamG87gGs=', 'smithi181.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBrSn8IPPHQCmw++kQtET3aogXGzcc8RD5mq/rRSwIrRlek5htPlT3joY4zEeml9nV4qK/jmu/p0jz1EVyYBsYI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '5ae09a5cc7064ca2bf994df19f3006b79794789a', 'timestamp': '2023-11-28_11:27:19', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-28T12:00:45.745 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T12:00:45.748 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T12:00:45.797 DEBUG:teuthology.orchestra.run.smithi057:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T12:00:45.811 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T12:00:45.863 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:00:45.863 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T12:00:45.906 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T12:00:45.906 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_1 2023-11-28T12:00:45.969 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T12:00:45.969 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:00:45.969 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 391759 Links: 1 2023-11-28T12:00:45.969 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:00:45.969 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:00:45.970 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-11-28 12:00:13.599360654 +0000 2023-11-28T12:00:45.970 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-11-28 11:50:16.510503075 +0000 2023-11-28T12:00:45.970 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-11-28 11:50:16.510503075 +0000 2023-11-28T12:00:45.970 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-11-28T12:00:45.970 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T12:00:46.043 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-11-28T12:00:46.043 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-11-28T12:00:46.044 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000414308 s, 1.2 MB/s 2023-11-28T12:00:46.045 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T12:00:46.110 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_2 2023-11-28T12:00:46.172 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T12:00:46.172 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:00:46.173 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 391854 Links: 1 2023-11-28T12:00:46.173 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:00:46.173 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:00:46.173 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-11-28 12:00:13.599360654 +0000 2023-11-28T12:00:46.173 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-11-28 11:50:17.102494020 +0000 2023-11-28T12:00:46.173 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-11-28 11:50:17.102494020 +0000 2023-11-28T12:00:46.173 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-11-28T12:00:46.173 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T12:00:46.246 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-11-28T12:00:46.246 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-11-28T12:00:46.246 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000290473 s, 1.8 MB/s 2023-11-28T12:00:46.247 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T12:00:46.311 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_3 2023-11-28T12:00:46.375 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T12:00:46.375 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:00:46.375 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 396593 Links: 1 2023-11-28T12:00:46.375 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:00:46.375 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:00:46.375 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-11-28 12:00:13.599360654 +0000 2023-11-28T12:00:46.375 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-11-28 11:50:17.680485179 +0000 2023-11-28T12:00:46.375 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-11-28 11:50:17.680485179 +0000 2023-11-28T12:00:46.375 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-11-28T12:00:46.376 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T12:00:46.447 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-11-28T12:00:46.447 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-11-28T12:00:46.447 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000289753 s, 1.8 MB/s 2023-11-28T12:00:46.448 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T12:00:46.513 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_4 2023-11-28T12:00:46.575 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T12:00:46.576 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:00:46.576 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 397656 Links: 1 2023-11-28T12:00:46.576 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:00:46.576 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:00:46.576 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-11-28 12:00:13.599360654 +0000 2023-11-28T12:00:46.576 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-11-28 11:50:18.280476001 +0000 2023-11-28T12:00:46.576 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-11-28 11:50:18.280476001 +0000 2023-11-28T12:00:46.576 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-11-28T12:00:46.576 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T12:00:46.648 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-11-28T12:00:46.648 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-11-28T12:00:46.649 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000298912 s, 1.7 MB/s 2023-11-28T12:00:46.649 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T12:00:46.714 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:00:46.715 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:00:46.715 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T12:00:46.737 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T12:00:46.737 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2023-11-28T12:00:46.798 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T12:00:46.798 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:00:46.799 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 394595 Links: 1 2023-11-28T12:00:46.799 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:00:46.799 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:00:46.799 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-11-28 12:00:40.631205737 +0000 2023-11-28T12:00:46.799 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-11-28 11:49:24.709519370 +0000 2023-11-28T12:00:46.799 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-11-28 11:49:24.709519370 +0000 2023-11-28T12:00:46.799 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-11-28T12:00:46.799 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T12:00:46.867 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-11-28T12:00:46.867 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-11-28T12:00:46.867 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000269297 s, 1.9 MB/s 2023-11-28T12:00:46.868 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T12:00:46.929 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2023-11-28T12:00:46.989 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T12:00:46.990 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:00:46.990 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 388059 Links: 1 2023-11-28T12:00:46.990 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:00:46.990 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:00:46.990 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-11-28 12:00:40.631205737 +0000 2023-11-28T12:00:46.990 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-11-28 11:49:25.216509523 +0000 2023-11-28T12:00:46.990 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-11-28 11:49:25.216509523 +0000 2023-11-28T12:00:46.990 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-11-28T12:00:46.990 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T12:00:47.058 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-11-28T12:00:47.058 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-11-28T12:00:47.058 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000218722 s, 2.3 MB/s 2023-11-28T12:00:47.059 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T12:00:47.119 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2023-11-28T12:00:47.179 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T12:00:47.179 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:00:47.179 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 397385 Links: 1 2023-11-28T12:00:47.179 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:00:47.180 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:00:47.180 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-11-28 12:00:40.631205737 +0000 2023-11-28T12:00:47.180 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-11-28 11:49:25.711499908 +0000 2023-11-28T12:00:47.180 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-11-28 11:49:25.711499908 +0000 2023-11-28T12:00:47.180 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-11-28T12:00:47.180 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T12:00:47.248 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-11-28T12:00:47.248 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-11-28T12:00:47.248 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000213967 s, 2.4 MB/s 2023-11-28T12:00:47.249 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T12:00:47.309 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2023-11-28T12:00:47.367 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T12:00:47.368 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:00:47.368 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 390941 Links: 1 2023-11-28T12:00:47.368 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:00:47.368 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:00:47.368 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-11-28 12:00:40.631205737 +0000 2023-11-28T12:00:47.368 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-11-28 11:49:26.228489866 +0000 2023-11-28T12:00:47.368 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-11-28 11:49:26.228489866 +0000 2023-11-28T12:00:47.368 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-11-28T12:00:47.368 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T12:00:47.435 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-11-28T12:00:47.436 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-11-28T12:00:47.436 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000233369 s, 2.2 MB/s 2023-11-28T12:00:47.437 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T12:00:47.498 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:00:47.498 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi057.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T12:00:47.498 INFO:tasks.ceph:Generating config... 2023-11-28T12:00:47.500 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-28T12:00:47.500 INFO:tasks.ceph:[client] debug client = 20 2023-11-28T12:00:47.500 INFO:tasks.ceph:[client] debug ms = 1 2023-11-28T12:00:47.501 INFO:tasks.ceph:[client] pid file = 2023-11-28T12:00:47.501 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-28T12:00:47.501 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-28T12:00:47.501 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-28T12:00:47.501 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-28T12:00:47.501 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-28T12:00:47.501 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-28T12:00:47.501 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-28T12:00:47.501 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-28T12:00:47.502 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-28T12:00:47.502 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-28T12:00:47.502 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-28T12:00:47.502 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-28T12:00:47.502 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-28T12:00:47.505 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-28T12:00:47.505 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-28T12:00:47.505 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-28T12:00:47.505 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-28T12:00:47.506 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-28T12:00:47.506 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-28T12:00:47.506 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-28T12:00:47.506 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-28T12:00:47.506 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-28T12:00:47.506 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-28T12:00:47.507 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-28T12:00:47.507 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-28T12:00:47.507 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-28T12:00:47.507 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-28T12:00:47.507 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-28T12:00:47.507 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-28T12:00:47.507 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-28T12:00:47.507 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-28T12:00:47.507 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-28T12:00:47.507 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-28T12:00:47.507 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-28T12:00:47.508 INFO:tasks.ceph:Setting up mon.a... 2023-11-28T12:00:47.508 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-28T12:00:47.569 INFO:teuthology.orchestra.run.smithi057.stdout:creating /etc/ceph/ceph.keyring 2023-11-28T12:00:47.572 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-28T12:00:47.626 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T12:00:47.657 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.57'), ('mon.b', '172.21.15.181'), ('mon.c', '[v2:172.21.15.181:3301,v1:172.21.15.181:6790]')] 2023-11-28T12:00:47.657 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.57,172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-28T12:00:47.658 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:00:47.658 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T12:00:47.719 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.57 --add b 172.21.15.181 --addv c '[v2:172.21.15.181:3301,v1:172.21.15.181:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:00:47.817 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:00:47.817 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool: generated fsid 0309b83f-6570-4eee-96fa-279822cd7766 2023-11-28T12:00:47.817 INFO:teuthology.orchestra.run.smithi057.stdout:setting min_mon_release = pacific 2023-11-28T12:00:47.817 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 0 2023-11-28T12:00:47.817 INFO:teuthology.orchestra.run.smithi057.stdout:fsid 0309b83f-6570-4eee-96fa-279822cd7766 2023-11-28T12:00:47.817 INFO:teuthology.orchestra.run.smithi057.stdout:last_changed 2023-11-28T12:00:47.817128+0000 2023-11-28T12:00:47.817 INFO:teuthology.orchestra.run.smithi057.stdout:created 2023-11-28T12:00:47.817128+0000 2023-11-28T12:00:47.817 INFO:teuthology.orchestra.run.smithi057.stdout:min_mon_release 16 (pacific) 2023-11-28T12:00:47.817 INFO:teuthology.orchestra.run.smithi057.stdout:election_strategy: 1 2023-11-28T12:00:47.818 INFO:teuthology.orchestra.run.smithi057.stdout:0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2023-11-28T12:00:47.818 INFO:teuthology.orchestra.run.smithi057.stdout:1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2023-11-28T12:00:47.818 INFO:teuthology.orchestra.run.smithi057.stdout:2: [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] mon.c 2023-11-28T12:00:47.818 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-28T12:00:47.819 DEBUG:teuthology.orchestra.run.smithi057:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T12:00:47.841 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0309b83f-6570-4eee-96fa-279822cd7766... 2023-11-28T12:00:47.842 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T12:00:47.883 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T12:00:47.933 INFO:teuthology.orchestra.run.smithi181.stdout:[global] 2023-11-28T12:00:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: chdir = "" 2023-11-28T12:00:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T12:00:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: auth supported = cephx 2023-11-28T12:00:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: filestore xattr use omap = true 2023-11-28T12:00:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: mon clock drift allowed = 1.000 2023-11-28T12:00:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: osd crush chooseleaf type = 0 2023-11-28T12:00:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: auth debug = true 2023-11-28T12:00:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.934 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on old message = true 2023-11-28T12:00:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on bug = true 2023-11-28T12:00:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T12:00:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: mon pg warn max object skew = 0 2023-11-28T12:00:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T12:00:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T12:00:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default size = 2 2023-11-28T12:00:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow primary affinity = true 2023-11-28T12:00:47.935 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow pg remap = true 2023-11-28T12:00:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on legacy crush tunables = false 2023-11-28T12:00:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on crush straw calc version zero = false 2023-11-28T12:00:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on no sortbitwise = false 2023-11-28T12:00:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on osd down out interval zero = false 2023-11-28T12:00:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on too few osds = false 2023-11-28T12:00:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T12:00:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T12:00:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: mon_allow_pool_size_one = true 2023-11-28T12:00:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T12:00:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.936 INFO:teuthology.orchestra.run.smithi181.stdout: osd default data pool replay window = 5 2023-11-28T12:00:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: mon allow pool delete = true 2023-11-28T12:00:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: mon cluster log file level = debug 2023-11-28T12:00:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: debug asserts on shutdown = true 2023-11-28T12:00:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: mon health detail to clog = false 2023-11-28T12:00:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: mon host = "172.21.15.57,172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790]" 2023-11-28T12:00:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: fsid = 0309b83f-6570-4eee-96fa-279822cd7766 2023-11-28T12:00:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.937 INFO:teuthology.orchestra.run.smithi181.stdout:[osd] 2023-11-28T12:00:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: osd journal size = 100 2023-11-28T12:00:47.937 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub load threshold = 5.0 2023-11-28T12:00:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub max interval = 600 2023-11-28T12:00:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: osd recover clone overlap = true 2023-11-28T12:00:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: osd recovery max chunk = 1048576 2023-11-28T12:00:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug shutdown = true 2023-11-28T12:00:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug op order = true 2023-11-28T12:00:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug verify stray on activate = true 2023-11-28T12:00:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: osd open classes on start = true 2023-11-28T12:00:47.938 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug pg log writeout = true 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: osd deep scrub update digest min age = 30 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: osd map max advance = 10 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: journal zero on create = true 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: filestore ondisk finisher threads = 3 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: filestore apply finisher threads = 3 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: bdev debug aio = true 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug misdirected ops = true 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: bdev async discard = True 2023-11-28T12:00:47.939 INFO:teuthology.orchestra.run.smithi181.stdout: bdev enable discard = True 2023-11-28T12:00:47.940 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore allocator = bitmap 2023-11-28T12:00:47.940 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore block size = 96636764160 2023-11-28T12:00:47.940 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore fsck on mount = True 2023-11-28T12:00:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluefs = 1/20 2023-11-28T12:00:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluestore = 1/20 2023-11-28T12:00:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-11-28T12:00:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: debug osd = 20 2023-11-28T12:00:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: debug rocksdb = 4/10 2023-11-28T12:00:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T12:00:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd full ratio = 0.9 2023-11-28T12:00:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd nearfull ratio = 0.8 2023-11-28T12:00:47.941 INFO:teuthology.orchestra.run.smithi181.stdout: osd failsafe full ratio = 0.95 2023-11-28T12:00:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: osd objectstore = bluestore 2023-11-28T12:00:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: osd op complaint time = 180 2023-11-28T12:00:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.942 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr] 2023-11-28T12:00:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-11-28T12:00:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: debug mgr = 20 2023-11-28T12:00:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2023-11-28T12:00:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2023-11-28T12:00:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2023-11-28T12:00:47.942 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2023-11-28T12:00:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: mgr/telemetry/nag = false 2023-11-28T12:00:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.943 INFO:teuthology.orchestra.run.smithi181.stdout:[mon] 2023-11-28T12:00:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-11-28T12:00:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2023-11-28T12:00:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: debug paxos = 20 2023-11-28T12:00:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2023-11-28T12:00:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: mon data avail warn = 5 2023-11-28T12:00:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: mon mgr mkfs grace = 240 2023-11-28T12:00:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2023-11-28T12:00:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd reporter subtree level = osd 2023-11-28T12:00:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd prime pg temp = true 2023-11-28T12:00:47.943 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2023-11-28T12:00:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T12:00:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T12:00:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T12:00:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T12:00:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T12:00:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T12:00:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: # 1m isn't quite enough 2023-11-28T12:00:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: mon_down_mkfs_grace = 2m 2023-11-28T12:00:47.944 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_filestore_osds = false 2023-11-28T12:00:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: mon op complaint time = 120 2023-11-28T12:00:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-11-28T12:00:47.945 INFO:teuthology.orchestra.run.smithi181.stdout:[client] 2023-11-28T12:00:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: rgw cache enabled = true 2023-11-28T12:00:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable ops log = true 2023-11-28T12:00:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable usage log = true 2023-11-28T12:00:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T12:00:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T12:00:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: client mount timeout = 600 2023-11-28T12:00:47.945 INFO:teuthology.orchestra.run.smithi181.stdout: debug client = 20 2023-11-28T12:00:47.946 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-11-28T12:00:47.946 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = "" 2023-11-28T12:00:47.946 INFO:teuthology.orchestra.run.smithi181.stdout: rados mon op timeout = 900 2023-11-28T12:00:47.947 INFO:teuthology.orchestra.run.smithi181.stdout: rados osd op timeout = 900 2023-11-28T12:00:47.947 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.a] 2023-11-28T12:00:47.947 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.b] 2023-11-28T12:00:47.947 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.c] 2023-11-28T12:00:47.947 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2023-11-28T12:00:47.948 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2023-11-28T12:00:47.948 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2023-11-28T12:00:47.948 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2023-11-28T12:00:47.948 INFO:teuthology.orchestra.run.smithi181.stdout:[mds] 2023-11-28T12:00:47.948 INFO:teuthology.orchestra.run.smithi181.stdout: debug mds = 20 2023-11-28T12:00:47.948 INFO:teuthology.orchestra.run.smithi181.stdout: debug mds balancer = 20 2023-11-28T12:00:47.948 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-11-28T12:00:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: mds debug frag = True 2023-11-28T12:00:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: mds debug scatterstat = True 2023-11-28T12:00:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: mds op complaint time = 180 2023-11-28T12:00:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: mds verify scatter = True 2023-11-28T12:00:47.949 INFO:teuthology.orchestra.run.smithi181.stdout: osd op complaint time = 180 2023-11-28T12:00:47.950 INFO:teuthology.orchestra.run.smithi181.stdout: rados mon op timeout = 900 2023-11-28T12:00:47.950 INFO:teuthology.orchestra.run.smithi181.stdout: rados osd op timeout = 900 2023-11-28T12:00:47.952 INFO:teuthology.orchestra.run.smithi057.stdout:[global] 2023-11-28T12:00:47.952 INFO:teuthology.orchestra.run.smithi057.stdout: chdir = "" 2023-11-28T12:00:47.952 INFO:teuthology.orchestra.run.smithi057.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T12:00:47.953 INFO:teuthology.orchestra.run.smithi057.stdout: auth supported = cephx 2023-11-28T12:00:47.953 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.953 INFO:teuthology.orchestra.run.smithi057.stdout: filestore xattr use omap = true 2023-11-28T12:00:47.953 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.953 INFO:teuthology.orchestra.run.smithi057.stdout: mon clock drift allowed = 1.000 2023-11-28T12:00:47.953 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.953 INFO:teuthology.orchestra.run.smithi057.stdout: osd crush chooseleaf type = 0 2023-11-28T12:00:47.953 INFO:teuthology.orchestra.run.smithi057.stdout: auth debug = true 2023-11-28T12:00:47.954 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.954 INFO:teuthology.orchestra.run.smithi057.stdout: ms die on old message = true 2023-11-28T12:00:47.954 INFO:teuthology.orchestra.run.smithi057.stdout: ms die on bug = true 2023-11-28T12:00:47.954 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.954 INFO:teuthology.orchestra.run.smithi057.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T12:00:47.954 INFO:teuthology.orchestra.run.smithi057.stdout: mon pg warn max object skew = 0 2023-11-28T12:00:47.955 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.955 INFO:teuthology.orchestra.run.smithi057.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T12:00:47.955 INFO:teuthology.orchestra.run.smithi057.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T12:00:47.955 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.955 INFO:teuthology.orchestra.run.smithi057.stdout: osd pool default size = 2 2023-11-28T12:00:47.955 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.955 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd allow primary affinity = true 2023-11-28T12:00:47.955 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd allow pg remap = true 2023-11-28T12:00:47.955 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on legacy crush tunables = false 2023-11-28T12:00:47.955 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on crush straw calc version zero = false 2023-11-28T12:00:47.955 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on no sortbitwise = false 2023-11-28T12:00:47.955 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on osd down out interval zero = false 2023-11-28T12:00:47.956 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on too few osds = false 2023-11-28T12:00:47.956 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T12:00:47.956 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T12:00:47.956 INFO:teuthology.orchestra.run.smithi057.stdout: mon_allow_pool_size_one = true 2023-11-28T12:00:47.956 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.956 INFO:teuthology.orchestra.run.smithi057.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T12:00:47.956 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.956 INFO:teuthology.orchestra.run.smithi057.stdout: osd default data pool replay window = 5 2023-11-28T12:00:47.956 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.956 INFO:teuthology.orchestra.run.smithi057.stdout: mon allow pool delete = true 2023-11-28T12:00:47.956 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.956 INFO:teuthology.orchestra.run.smithi057.stdout: mon cluster log file level = debug 2023-11-28T12:00:47.957 INFO:teuthology.orchestra.run.smithi057.stdout: debug asserts on shutdown = true 2023-11-28T12:00:47.957 INFO:teuthology.orchestra.run.smithi057.stdout: mon health detail to clog = false 2023-11-28T12:00:47.957 INFO:teuthology.orchestra.run.smithi057.stdout: mon host = "172.21.15.57,172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790]" 2023-11-28T12:00:47.957 INFO:teuthology.orchestra.run.smithi057.stdout: fsid = 0309b83f-6570-4eee-96fa-279822cd7766 2023-11-28T12:00:47.957 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.957 INFO:teuthology.orchestra.run.smithi057.stdout:[osd] 2023-11-28T12:00:47.957 INFO:teuthology.orchestra.run.smithi057.stdout: osd journal size = 100 2023-11-28T12:00:47.957 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.957 INFO:teuthology.orchestra.run.smithi057.stdout: osd scrub load threshold = 5.0 2023-11-28T12:00:47.957 INFO:teuthology.orchestra.run.smithi057.stdout: osd scrub max interval = 600 2023-11-28T12:00:47.957 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.957 INFO:teuthology.orchestra.run.smithi057.stdout: osd recover clone overlap = true 2023-11-28T12:00:47.958 INFO:teuthology.orchestra.run.smithi057.stdout: osd recovery max chunk = 1048576 2023-11-28T12:00:47.958 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.958 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug shutdown = true 2023-11-28T12:00:47.958 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug op order = true 2023-11-28T12:00:47.958 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug verify stray on activate = true 2023-11-28T12:00:47.958 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.958 INFO:teuthology.orchestra.run.smithi057.stdout: osd open classes on start = true 2023-11-28T12:00:47.958 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug pg log writeout = true 2023-11-28T12:00:47.958 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.958 INFO:teuthology.orchestra.run.smithi057.stdout: osd deep scrub update digest min age = 30 2023-11-28T12:00:47.958 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.958 INFO:teuthology.orchestra.run.smithi057.stdout: osd map max advance = 10 2023-11-28T12:00:47.959 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.959 INFO:teuthology.orchestra.run.smithi057.stdout: journal zero on create = true 2023-11-28T12:00:47.959 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.959 INFO:teuthology.orchestra.run.smithi057.stdout: filestore ondisk finisher threads = 3 2023-11-28T12:00:47.959 INFO:teuthology.orchestra.run.smithi057.stdout: filestore apply finisher threads = 3 2023-11-28T12:00:47.959 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.959 INFO:teuthology.orchestra.run.smithi057.stdout: bdev debug aio = true 2023-11-28T12:00:47.959 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug misdirected ops = true 2023-11-28T12:00:47.959 INFO:teuthology.orchestra.run.smithi057.stdout: bdev async discard = True 2023-11-28T12:00:47.959 INFO:teuthology.orchestra.run.smithi057.stdout: bdev enable discard = True 2023-11-28T12:00:47.959 INFO:teuthology.orchestra.run.smithi057.stdout: bluestore allocator = bitmap 2023-11-28T12:00:47.959 INFO:teuthology.orchestra.run.smithi057.stdout: bluestore block size = 96636764160 2023-11-28T12:00:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: bluestore fsck on mount = True 2023-11-28T12:00:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: debug bluefs = 1/20 2023-11-28T12:00:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: debug bluestore = 1/20 2023-11-28T12:00:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2023-11-28T12:00:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: debug osd = 20 2023-11-28T12:00:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: debug rocksdb = 4/10 2023-11-28T12:00:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T12:00:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd full ratio = 0.9 2023-11-28T12:00:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd nearfull ratio = 0.8 2023-11-28T12:00:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: osd failsafe full ratio = 0.95 2023-11-28T12:00:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: osd objectstore = bluestore 2023-11-28T12:00:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: osd op complaint time = 180 2023-11-28T12:00:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.961 INFO:teuthology.orchestra.run.smithi057.stdout:[mgr] 2023-11-28T12:00:47.961 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2023-11-28T12:00:47.961 INFO:teuthology.orchestra.run.smithi057.stdout: debug mgr = 20 2023-11-28T12:00:47.961 INFO:teuthology.orchestra.run.smithi057.stdout: debug mon = 20 2023-11-28T12:00:47.961 INFO:teuthology.orchestra.run.smithi057.stdout: debug auth = 20 2023-11-28T12:00:47.961 INFO:teuthology.orchestra.run.smithi057.stdout: mon reweight min pgs per osd = 4 2023-11-28T12:00:47.961 INFO:teuthology.orchestra.run.smithi057.stdout: mon reweight min bytes per osd = 10 2023-11-28T12:00:47.961 INFO:teuthology.orchestra.run.smithi057.stdout: mgr/telemetry/nag = false 2023-11-28T12:00:47.961 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.961 INFO:teuthology.orchestra.run.smithi057.stdout:[mon] 2023-11-28T12:00:47.961 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2023-11-28T12:00:47.961 INFO:teuthology.orchestra.run.smithi057.stdout: debug mon = 20 2023-11-28T12:00:47.962 INFO:teuthology.orchestra.run.smithi057.stdout: debug paxos = 20 2023-11-28T12:00:47.962 INFO:teuthology.orchestra.run.smithi057.stdout: debug auth = 20 2023-11-28T12:00:47.962 INFO:teuthology.orchestra.run.smithi057.stdout: mon data avail warn = 5 2023-11-28T12:00:47.962 INFO:teuthology.orchestra.run.smithi057.stdout: mon mgr mkfs grace = 240 2023-11-28T12:00:47.962 INFO:teuthology.orchestra.run.smithi057.stdout: mon reweight min pgs per osd = 4 2023-11-28T12:00:47.962 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd reporter subtree level = osd 2023-11-28T12:00:47.962 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd prime pg temp = true 2023-11-28T12:00:47.962 INFO:teuthology.orchestra.run.smithi057.stdout: mon reweight min bytes per osd = 10 2023-11-28T12:00:47.962 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.962 INFO:teuthology.orchestra.run.smithi057.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T12:00:47.962 INFO:teuthology.orchestra.run.smithi057.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T12:00:47.962 INFO:teuthology.orchestra.run.smithi057.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T12:00:47.962 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.963 INFO:teuthology.orchestra.run.smithi057.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T12:00:47.963 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T12:00:47.963 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T12:00:47.963 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.963 INFO:teuthology.orchestra.run.smithi057.stdout: # 1m isn't quite enough 2023-11-28T12:00:47.963 INFO:teuthology.orchestra.run.smithi057.stdout: mon_down_mkfs_grace = 2m 2023-11-28T12:00:47.963 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.963 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_filestore_osds = false 2023-11-28T12:00:47.963 INFO:teuthology.orchestra.run.smithi057.stdout: mon op complaint time = 120 2023-11-28T12:00:47.963 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:00:47.963 INFO:teuthology.orchestra.run.smithi057.stdout:[client] 2023-11-28T12:00:47.963 INFO:teuthology.orchestra.run.smithi057.stdout: rgw cache enabled = true 2023-11-28T12:00:47.964 INFO:teuthology.orchestra.run.smithi057.stdout: rgw enable ops log = true 2023-11-28T12:00:47.964 INFO:teuthology.orchestra.run.smithi057.stdout: rgw enable usage log = true 2023-11-28T12:00:47.964 INFO:teuthology.orchestra.run.smithi057.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T12:00:47.964 INFO:teuthology.orchestra.run.smithi057.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T12:00:47.964 INFO:teuthology.orchestra.run.smithi057.stdout: client mount timeout = 600 2023-11-28T12:00:47.964 INFO:teuthology.orchestra.run.smithi057.stdout: debug client = 20 2023-11-28T12:00:47.964 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2023-11-28T12:00:47.964 INFO:teuthology.orchestra.run.smithi057.stdout: pid file = "" 2023-11-28T12:00:47.964 INFO:teuthology.orchestra.run.smithi057.stdout: rados mon op timeout = 900 2023-11-28T12:00:47.964 INFO:teuthology.orchestra.run.smithi057.stdout: rados osd op timeout = 900 2023-11-28T12:00:47.964 INFO:teuthology.orchestra.run.smithi057.stdout:[mon.a] 2023-11-28T12:00:47.964 INFO:teuthology.orchestra.run.smithi057.stdout:[mon.b] 2023-11-28T12:00:47.965 INFO:teuthology.orchestra.run.smithi057.stdout:[mon.c] 2023-11-28T12:00:47.965 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.a] 2023-11-28T12:00:47.965 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.c] 2023-11-28T12:00:47.965 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.b] 2023-11-28T12:00:47.965 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.d] 2023-11-28T12:00:47.965 INFO:teuthology.orchestra.run.smithi057.stdout:[mds] 2023-11-28T12:00:47.965 INFO:teuthology.orchestra.run.smithi057.stdout: debug mds = 20 2023-11-28T12:00:47.965 INFO:teuthology.orchestra.run.smithi057.stdout: debug mds balancer = 20 2023-11-28T12:00:47.965 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2023-11-28T12:00:47.965 INFO:teuthology.orchestra.run.smithi057.stdout: mds debug frag = True 2023-11-28T12:00:47.965 INFO:teuthology.orchestra.run.smithi057.stdout: mds debug scatterstat = True 2023-11-28T12:00:47.965 INFO:teuthology.orchestra.run.smithi057.stdout: mds op complaint time = 180 2023-11-28T12:00:47.965 INFO:teuthology.orchestra.run.smithi057.stdout: mds verify scatter = True 2023-11-28T12:00:47.966 INFO:teuthology.orchestra.run.smithi057.stdout: osd op complaint time = 180 2023-11-28T12:00:47.966 INFO:teuthology.orchestra.run.smithi057.stdout: rados mon op timeout = 900 2023-11-28T12:00:47.966 INFO:teuthology.orchestra.run.smithi057.stdout: rados osd op timeout = 900 2023-11-28T12:00:47.966 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-28T12:00:47.966 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-28T12:00:48.061 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-28T12:00:48.061 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:00:48.062 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-28T12:00:48.083 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:00:48.083 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-28T12:00:48.146 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:00:48.146 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:00:48.146 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T12:00:48.146 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T12:00:48.233 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:00:48.233 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:00:48.296 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:00:48.296 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:00:48.296 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T12:00:48.296 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T12:00:48.338 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:00:48.339 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:00:48.398 INFO:tasks.ceph:Setting up mon nodes... 2023-11-28T12:00:48.398 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-28T12:00:48.398 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T12:00:48.468 INFO:teuthology.orchestra.run.smithi057.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T12:00:48.471 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T12:00:48.537 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T12:00:48.540 INFO:tasks.ceph:Setting up mds nodes... 2023-11-28T12:00:48.541 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T12:00:48.608 INFO:teuthology.orchestra.run.smithi057.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T12:00:48.610 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-28T12:00:48.642 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T12:00:48.749 INFO:teuthology.orchestra.run.smithi057.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T12:00:48.751 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-28T12:00:48.784 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T12:00:48.840 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T12:00:48.842 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-28T12:00:48.870 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T12:00:48.962 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T12:00:48.965 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-28T12:00:48.991 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-28T12:00:48.991 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-28T12:00:49.046 INFO:teuthology.orchestra.run.smithi057.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-28T12:00:49.060 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-28T12:00:49.111 INFO:teuthology.orchestra.run.smithi057.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-28T12:00:49.126 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-28T12:00:49.169 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-28T12:00:49.181 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-28T12:00:49.264 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-28T12:00:49.276 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-28T12:00:49.276 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi057.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T12:00:49.276 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-28T12:00:49.311 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:00:49.311 INFO:tasks.ceph:role: osd.0 2023-11-28T12:00:49.311 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:00:49.312 DEBUG:teuthology.orchestra.run.smithi057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T12:00:49.414 INFO:teuthology.orchestra.run.smithi057.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:00:49.414 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:00:49.415 INFO:teuthology.orchestra.run.smithi057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:00:49.415 INFO:teuthology.orchestra.run.smithi057.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:00:49.415 INFO:teuthology.orchestra.run.smithi057.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:00:49.415 INFO:teuthology.orchestra.run.smithi057.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:00:49.415 INFO:teuthology.orchestra.run.smithi057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:00:49.415 INFO:teuthology.orchestra.run.smithi057.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:00:49.415 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:00:49.415 INFO:teuthology.orchestra.run.smithi057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:00:50.698 INFO:teuthology.orchestra.run.smithi057.stdout:Discarding blocks...Done. 2023-11-28T12:00:50.700 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi057.front.sepia.ceph.com -o noatime 2023-11-28T12:00:50.700 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-28T12:00:50.751 DEBUG:teuthology.orchestra.run.smithi057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-28T12:00:50.788 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-28T12:00:50.861 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:00:50.862 INFO:tasks.ceph:role: osd.1 2023-11-28T12:00:50.862 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:00:50.862 DEBUG:teuthology.orchestra.run.smithi057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T12:00:50.965 INFO:teuthology.orchestra.run.smithi057.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:00:50.965 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:00:50.965 INFO:teuthology.orchestra.run.smithi057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:00:50.965 INFO:teuthology.orchestra.run.smithi057.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:00:50.965 INFO:teuthology.orchestra.run.smithi057.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:00:50.965 INFO:teuthology.orchestra.run.smithi057.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:00:50.965 INFO:teuthology.orchestra.run.smithi057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:00:50.965 INFO:teuthology.orchestra.run.smithi057.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:00:50.965 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:00:50.966 INFO:teuthology.orchestra.run.smithi057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:00:52.243 INFO:teuthology.orchestra.run.smithi057.stdout:Discarding blocks...Done. 2023-11-28T12:00:52.245 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi057.front.sepia.ceph.com -o noatime 2023-11-28T12:00:52.245 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-28T12:00:52.293 DEBUG:teuthology.orchestra.run.smithi057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-28T12:00:52.333 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-28T12:00:52.408 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:00:52.408 INFO:tasks.ceph:role: osd.2 2023-11-28T12:00:52.408 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:00:52.408 DEBUG:teuthology.orchestra.run.smithi057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T12:00:52.510 INFO:teuthology.orchestra.run.smithi057.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:00:52.511 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:00:52.511 INFO:teuthology.orchestra.run.smithi057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:00:52.511 INFO:teuthology.orchestra.run.smithi057.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:00:52.511 INFO:teuthology.orchestra.run.smithi057.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:00:52.511 INFO:teuthology.orchestra.run.smithi057.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:00:52.511 INFO:teuthology.orchestra.run.smithi057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:00:52.511 INFO:teuthology.orchestra.run.smithi057.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:00:52.511 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:00:52.511 INFO:teuthology.orchestra.run.smithi057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:00:53.801 INFO:teuthology.orchestra.run.smithi057.stdout:Discarding blocks...Done. 2023-11-28T12:00:53.803 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi057.front.sepia.ceph.com -o noatime 2023-11-28T12:00:53.804 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-28T12:00:53.855 DEBUG:teuthology.orchestra.run.smithi057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-28T12:00:53.892 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-28T12:00:53.966 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:00:53.966 INFO:tasks.ceph:role: osd.3 2023-11-28T12:00:53.966 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:00:53.967 DEBUG:teuthology.orchestra.run.smithi057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T12:00:54.068 INFO:teuthology.orchestra.run.smithi057.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:00:54.068 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:00:54.068 INFO:teuthology.orchestra.run.smithi057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:00:54.068 INFO:teuthology.orchestra.run.smithi057.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:00:54.069 INFO:teuthology.orchestra.run.smithi057.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:00:54.069 INFO:teuthology.orchestra.run.smithi057.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:00:54.069 INFO:teuthology.orchestra.run.smithi057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:00:54.069 INFO:teuthology.orchestra.run.smithi057.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:00:54.069 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:00:54.069 INFO:teuthology.orchestra.run.smithi057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:00:55.371 INFO:teuthology.orchestra.run.smithi057.stdout:Discarding blocks...Done. 2023-11-28T12:00:55.373 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi057.front.sepia.ceph.com -o noatime 2023-11-28T12:00:55.373 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-28T12:00:55.423 DEBUG:teuthology.orchestra.run.smithi057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-28T12:00:55.460 DEBUG:teuthology.orchestra.run.smithi057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:00:55.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:00:55.563+0000 7f5abe514680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-28T12:00:55.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:00:55.563+0000 7f5abe514680 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-28T12:00:55.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:00:55.563+0000 7f5abe514680 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T12:00:55.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:00:55.563+0000 7f5abe514680 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T12:00:55.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:00:55.563+0000 7f5abe514680 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T12:00:55.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:00:55.564+0000 7f5abe514680 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-28T12:00:58.427 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-28T12:00:58.463 DEBUG:teuthology.orchestra.run.smithi057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:00:58.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:00:58.560+0000 7f51fdb61680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-28T12:00:58.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:00:58.561+0000 7f51fdb61680 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-28T12:00:58.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:00:58.561+0000 7f51fdb61680 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T12:00:58.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:00:58.561+0000 7f51fdb61680 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T12:00:58.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:00:58.561+0000 7f51fdb61680 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T12:00:58.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:00:58.561+0000 7f51fdb61680 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-28T12:01:01.437 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-28T12:01:01.476 DEBUG:teuthology.orchestra.run.smithi057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:01:01.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:01.574+0000 7f6d06728680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-28T12:01:01.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:01.574+0000 7f6d06728680 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-28T12:01:01.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:01.574+0000 7f6d06728680 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T12:01:01.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:01.574+0000 7f6d06728680 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T12:01:01.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:01.574+0000 7f6d06728680 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T12:01:01.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:01.575+0000 7f6d06728680 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-28T12:01:04.483 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-28T12:01:04.518 DEBUG:teuthology.orchestra.run.smithi057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:01:04.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:04.614+0000 7f1ccf4fe680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-28T12:01:04.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:04.615+0000 7f1ccf4fe680 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-28T12:01:04.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:04.615+0000 7f1ccf4fe680 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T12:01:04.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:04.615+0000 7f1ccf4fe680 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T12:01:04.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:04.615+0000 7f1ccf4fe680 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T12:01:04.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:04.615+0000 7f1ccf4fe680 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-28T12:01:07.483 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-28T12:01:07.518 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-28T12:01:07.554 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:01:07.554 INFO:tasks.ceph:role: osd.4 2023-11-28T12:01:07.554 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:01:07.554 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T12:01:07.632 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:01:07.632 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:01:07.632 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:01:07.632 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:01:07.632 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:01:07.633 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:01:07.633 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:01:07.633 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:01:07.633 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:01:07.633 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:01:08.047 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-11-28T12:01:08.048 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-11-28T12:01:08.048 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-28T12:01:08.237 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-28T12:01:08.276 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-28T12:01:08.345 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:01:08.345 INFO:tasks.ceph:role: osd.5 2023-11-28T12:01:08.345 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:01:08.345 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T12:01:08.420 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:01:08.420 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:01:08.420 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:01:08.420 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:01:08.420 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:01:08.420 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:01:08.420 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:01:08.420 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:01:08.420 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:01:08.420 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:01:08.852 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-11-28T12:01:08.854 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-11-28T12:01:08.854 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-28T12:01:08.970 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-28T12:01:09.002 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-28T12:01:09.071 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:01:09.071 INFO:tasks.ceph:role: osd.6 2023-11-28T12:01:09.071 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:01:09.072 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T12:01:09.147 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:01:09.147 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:01:09.147 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:01:09.147 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:01:09.147 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:01:09.147 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:01:09.147 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:01:09.148 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:01:09.148 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:01:09.148 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:01:09.615 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-11-28T12:01:09.617 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-11-28T12:01:09.617 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-28T12:01:09.785 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-28T12:01:09.819 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-28T12:01:09.888 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:01:09.888 INFO:tasks.ceph:role: osd.7 2023-11-28T12:01:09.888 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:01:09.888 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T12:01:09.962 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:01:09.962 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:01:09.962 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:01:09.962 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:01:09.962 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:01:09.962 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:01:09.962 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:01:09.962 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:01:09.962 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:01:09.963 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:01:10.396 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-11-28T12:01:10.398 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-11-28T12:01:10.398 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-28T12:01:10.571 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-28T12:01:10.604 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:01:10.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:10.718+0000 7f12d1792680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-28T12:01:10.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:10.719+0000 7f12d1792680 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-28T12:01:10.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:10.719+0000 7f12d1792680 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T12:01:10.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:10.719+0000 7f12d1792680 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T12:01:10.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:10.719+0000 7f12d1792680 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T12:01:10.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:10.719+0000 7f12d1792680 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-28T12:01:13.567 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-28T12:01:13.596 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:01:13.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:13.684+0000 7f697745e680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-28T12:01:13.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:13.684+0000 7f697745e680 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-28T12:01:13.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:13.684+0000 7f697745e680 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T12:01:13.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:13.684+0000 7f697745e680 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T12:01:13.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:13.684+0000 7f697745e680 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T12:01:13.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:13.685+0000 7f697745e680 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-28T12:01:15.746 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:01:15.749 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:01:16.539 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-28T12:01:16.571 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:01:16.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:16.662+0000 7f8911e58680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-28T12:01:16.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:16.663+0000 7f8911e58680 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-28T12:01:16.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:16.663+0000 7f8911e58680 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T12:01:16.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:16.663+0000 7f8911e58680 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T12:01:16.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:16.663+0000 7f8911e58680 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T12:01:16.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:16.663+0000 7f8911e58680 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-28T12:01:19.518 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-28T12:01:19.550 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:01:19.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:19.643+0000 7f6bf8734680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-28T12:01:19.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:19.643+0000 7f6bf8734680 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-28T12:01:19.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:19.643+0000 7f6bf8734680 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T12:01:19.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:19.643+0000 7f6bf8734680 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T12:01:19.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:19.643+0000 7f6bf8734680 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T12:01:19.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:19.644+0000 7f6bf8734680 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-28T12:01:22.485 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-28T12:01:22.518 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-28T12:01:22.518 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:01:22.518 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-28T12:01:22.552 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:01:22.552 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-28T12:01:22.628 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:01:22.628 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-28T12:01:22.703 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:01:22.703 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-28T12:01:22.777 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:01:22.778 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-28T12:01:22.852 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:01:22.853 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-28T12:01:22.928 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:01:22.929 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-28T12:01:23.003 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:01:23.003 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-28T12:01:23.035 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:01:23.035 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-28T12:01:23.106 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:01:23.106 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-28T12:01:23.175 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:01:23.176 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-28T12:01:23.244 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:01:23.245 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-28T12:01:23.313 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:01:23.313 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-28T12:01:23.380 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:01:23.380 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-28T12:01:23.447 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:01:23.447 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-28T12:01:23.469 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:01:23.469 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-28T12:01:23.532 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:01:23.532 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-28T12:01:23.550 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:01:23.550 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-28T12:01:23.607 INFO:tasks.ceph:Adding keys to all mons... 2023-11-28T12:01:23.607 DEBUG:teuthology.orchestra.run.smithi057:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T12:01:23.609 DEBUG:teuthology.orchestra.run.smithi181:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T12:01:23.650 INFO:teuthology.orchestra.run.smithi057.stdout:[mgr.y] 2023-11-28T12:01:23.650 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQBw1mVlEPXqGxAAxVwmP5CKh4D1Hv+kYm7fxg== 2023-11-28T12:01:23.650 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.a] 2023-11-28T12:01:23.650 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQBw1mVlFqRBJBAAxYPlgmjdxnRzi0h9JBB1BQ== 2023-11-28T12:01:23.650 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.c] 2023-11-28T12:01:23.650 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQBw1mVl4Z2qLBAACYGN7Glu6Bv2yLio50flgQ== 2023-11-28T12:01:23.650 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.0] 2023-11-28T12:01:23.650 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQB31mVlUtyoIRAAVGG65BAriZ4e4QHUYCJXdg== 2023-11-28T12:01:23.650 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.1] 2023-11-28T12:01:23.651 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQB61mVlNA98IRAArcXcx4glfEvQ1gxAXQYxHA== 2023-11-28T12:01:23.651 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.2] 2023-11-28T12:01:23.651 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQB91mVlzthPIhAAFgsWUTI/2kmVCVMgcZrIdA== 2023-11-28T12:01:23.651 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.3] 2023-11-28T12:01:23.651 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCA1mVlZPuzJBAAyAAUBUEitzTq7fPdxj5gzQ== 2023-11-28T12:01:23.651 INFO:teuthology.orchestra.run.smithi057.stdout:[mgr.x] 2023-11-28T12:01:23.651 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQBw1mVl+yMTIBAAUgQtxbubBwh+1DLDFEaSKQ== 2023-11-28T12:01:23.651 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.b] 2023-11-28T12:01:23.651 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQBw1mVl0d0ZMhAAykXowuVhIqRSjBr/XMvanw== 2023-11-28T12:01:23.651 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.d] 2023-11-28T12:01:23.651 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQBw1mVl2P5jORAAR7iZXovJiiCXuxlaaZJ75Q== 2023-11-28T12:01:23.651 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.4] 2023-11-28T12:01:23.652 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCG1mVlWibrKhAAzuTgZ4ayPw3imRALW/Y4FQ== 2023-11-28T12:01:23.652 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.5] 2023-11-28T12:01:23.652 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCJ1mVlnX7bKBAAYYUYhDSDTVKkrzVKw78+Ug== 2023-11-28T12:01:23.652 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.6] 2023-11-28T12:01:23.652 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCM1mVlE7aUJxAAXM3axU5A9IEXt2/pNcvJ9Q== 2023-11-28T12:01:23.652 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.7] 2023-11-28T12:01:23.652 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCP1mVlSApqJhAA93hHHLl+o8cKTgdCaG+0Bg== 2023-11-28T12:01:23.652 INFO:teuthology.orchestra.run.smithi057.stdout:[client.0] 2023-11-28T12:01:23.652 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQBx1mVlmhvFAhAAnKh5ldWFcp+P85m8HR2vug== 2023-11-28T12:01:23.652 INFO:teuthology.orchestra.run.smithi057.stdout:[client.2] 2023-11-28T12:01:23.652 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQBx1mVlqGCwBhAA6ZJBICxFbxIjYlgA80ekuA== 2023-11-28T12:01:23.652 INFO:teuthology.orchestra.run.smithi057.stdout:[client.1] 2023-11-28T12:01:23.652 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQBx1mVlMEgaChAAGCjKh3RcPIVp2UWKGqmqRA== 2023-11-28T12:01:23.653 INFO:teuthology.orchestra.run.smithi057.stdout:[client.3] 2023-11-28T12:01:23.653 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQBx1mVldQXGDxAAkY9/3er6vMwer5oWETJvfg== 2023-11-28T12:01:23.671 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.y] 2023-11-28T12:01:23.671 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBw1mVlEPXqGxAAxVwmP5CKh4D1Hv+kYm7fxg== 2023-11-28T12:01:23.672 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2023-11-28T12:01:23.672 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBw1mVlFqRBJBAAxYPlgmjdxnRzi0h9JBB1BQ== 2023-11-28T12:01:23.672 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2023-11-28T12:01:23.672 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBw1mVl4Z2qLBAACYGN7Glu6Bv2yLio50flgQ== 2023-11-28T12:01:23.672 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2023-11-28T12:01:23.672 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB31mVlUtyoIRAAVGG65BAriZ4e4QHUYCJXdg== 2023-11-28T12:01:23.672 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2023-11-28T12:01:23.672 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB61mVlNA98IRAArcXcx4glfEvQ1gxAXQYxHA== 2023-11-28T12:01:23.672 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2023-11-28T12:01:23.672 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB91mVlzthPIhAAFgsWUTI/2kmVCVMgcZrIdA== 2023-11-28T12:01:23.672 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.3] 2023-11-28T12:01:23.672 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCA1mVlZPuzJBAAyAAUBUEitzTq7fPdxj5gzQ== 2023-11-28T12:01:23.673 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.x] 2023-11-28T12:01:23.673 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBw1mVl+yMTIBAAUgQtxbubBwh+1DLDFEaSKQ== 2023-11-28T12:01:23.673 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2023-11-28T12:01:23.673 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBw1mVl0d0ZMhAAykXowuVhIqRSjBr/XMvanw== 2023-11-28T12:01:23.673 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2023-11-28T12:01:23.673 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBw1mVl2P5jORAAR7iZXovJiiCXuxlaaZJ75Q== 2023-11-28T12:01:23.673 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.4] 2023-11-28T12:01:23.673 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCG1mVlWibrKhAAzuTgZ4ayPw3imRALW/Y4FQ== 2023-11-28T12:01:23.673 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.5] 2023-11-28T12:01:23.673 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCJ1mVlnX7bKBAAYYUYhDSDTVKkrzVKw78+Ug== 2023-11-28T12:01:23.673 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.6] 2023-11-28T12:01:23.673 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCM1mVlE7aUJxAAXM3axU5A9IEXt2/pNcvJ9Q== 2023-11-28T12:01:23.673 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.7] 2023-11-28T12:01:23.674 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCP1mVlSApqJhAA93hHHLl+o8cKTgdCaG+0Bg== 2023-11-28T12:01:23.674 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-11-28T12:01:23.674 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBx1mVlmhvFAhAAnKh5ldWFcp+P85m8HR2vug== 2023-11-28T12:01:23.674 INFO:teuthology.orchestra.run.smithi181.stdout:[client.2] 2023-11-28T12:01:23.674 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBx1mVlqGCwBhAA6ZJBICxFbxIjYlgA80ekuA== 2023-11-28T12:01:23.674 INFO:teuthology.orchestra.run.smithi181.stdout:[client.1] 2023-11-28T12:01:23.674 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBx1mVlMEgaChAAGCjKh3RcPIVp2UWKGqmqRA== 2023-11-28T12:01:23.674 INFO:teuthology.orchestra.run.smithi181.stdout:[client.3] 2023-11-28T12:01:23.674 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBx1mVldQXGDxAAkY9/3er6vMwer5oWETJvfg== 2023-11-28T12:01:23.675 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T12:01:23.696 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T12:01:23.769 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:01:23.774 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:01:23.834 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:01:23.837 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:01:23.898 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:23.900 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:23.957 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:23.960 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:24.016 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:24.019 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:24.075 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:24.078 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:24.134 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T12:01:24.137 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T12:01:24.193 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:01:24.195 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:01:24.255 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:01:24.258 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:01:24.318 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:24.320 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:24.377 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:24.379 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:24.436 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:24.439 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:24.495 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:24.497 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:01:24.554 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:01:24.557 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:01:24.617 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:01:24.619 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:01:24.679 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:01:24.681 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:01:24.741 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:01:24.744 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:01:24.804 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-28T12:01:24.804 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-28T12:01:24.834 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T12:01:24.947 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-28T12:01:24.979 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-28T12:01:25.006 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T12:01:25.104 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-28T12:01:25.131 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-28T12:01:25.198 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T12:01:25.296 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-28T12:01:25.323 DEBUG:teuthology.orchestra.run.smithi057:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:01:25.325 DEBUG:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:01:25.381 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-28T12:01:25.381 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-28T12:01:25.381 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-28T12:01:25.388 INFO:tasks.ceph.mon.a:Started 2023-11-28T12:01:25.388 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-28T12:01:25.389 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-28T12:01:25.423 INFO:tasks.ceph.mon.b:Started 2023-11-28T12:01:25.424 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-28T12:01:25.424 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-28T12:01:25.426 INFO:tasks.ceph.mon.c:Started 2023-11-28T12:01:25.426 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-28T12:01:25.426 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T12:01:25.426 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T12:01:25.429 INFO:tasks.ceph.mgr.y:Started 2023-11-28T12:01:25.429 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-28T12:01:25.429 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-28T12:01:25.431 INFO:tasks.ceph.mgr.x:Started 2023-11-28T12:01:25.431 INFO:tasks.ceph:Setting crush tunables to default 2023-11-28T12:01:25.431 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --cluster ceph osd crush tunables default 2023-11-28T12:01:25.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.569+0000 7effe6933700 1 Processor -- start 2023-11-28T12:01:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.569+0000 7effe6933700 1 -- start start 2023-11-28T12:01:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.570+0000 7effe6933700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7effe01599b0 0x7effe015d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.570+0000 7effe6933700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 0x7effe015a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.570+0000 7effe6933700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe0159ec0 con 0x7effe00b1650 2023-11-28T12:01:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.570+0000 7effe6933700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe015d8a0 con 0x7effe01599b0 2023-11-28T12:01:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.570+0000 7effe6933700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe015db80 con 0x7effe00af0a0 2023-11-28T12:01:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.570+0000 7effdffff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7effe00b1650 0x7effe00b1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38258/0 (socket says 172.21.15.57:38258) 2023-11-28T12:01:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.570+0000 7effdffff700 1 -- 172.21.15.57:0/4008282863 learned_addr learned my addr 172.21.15.57:0/4008282863 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.571+0000 7effe4ed0700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 0x7effe015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.571+0000 7effdf7fe700 1 --2- 172.21.15.57:0/4008282863 >> v2:172.21.15.181:3300/0 conn(0x7effe01599b0 0x7effe015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.571+0000 7effe4ed0700 1 -- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 msgr2=0x7effe015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T12:01:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.571+0000 7effe4ed0700 1 -- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 msgr2=0x7effe015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T12:01:25.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.571+0000 7effe4ed0700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 0x7effe015a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T12:01:25.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.571+0000 7effe4ed0700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 0x7effe015a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T12:01:25.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.571+0000 7effdf7fe700 1 -- 172.21.15.57:0/4008282863 >> v2:172.21.15.181:3300/0 conn(0x7effe01599b0 msgr2=0x7effe015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T12:01:25.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.571+0000 7effdf7fe700 1 -- 172.21.15.57:0/4008282863 >> v2:172.21.15.181:3300/0 conn(0x7effe01599b0 msgr2=0x7effe015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T12:01:25.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.571+0000 7effdf7fe700 1 --2- 172.21.15.57:0/4008282863 >> v2:172.21.15.181:3300/0 conn(0x7effe01599b0 0x7effe015d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T12:01:25.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.571+0000 7effdf7fe700 1 --2- 172.21.15.57:0/4008282863 >> v2:172.21.15.181:3300/0 conn(0x7effe01599b0 0x7effe015d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T12:01:25.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.772+0000 7effdf7fe700 1 --2- 172.21.15.57:0/4008282863 >> v2:172.21.15.181:3300/0 conn(0x7effe01599b0 0x7effe015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:25.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.772+0000 7effe4ed0700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 0x7effe015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:25.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.772+0000 7effe4ed0700 1 -- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 msgr2=0x7effe015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T12:01:25.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.772+0000 7effe4ed0700 1 -- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 msgr2=0x7effe015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T12:01:25.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.772+0000 7effe4ed0700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 0x7effe015a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T12:01:25.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.772+0000 7effdf7fe700 1 -- 172.21.15.57:0/4008282863 >> v2:172.21.15.181:3300/0 conn(0x7effe01599b0 msgr2=0x7effe015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T12:01:25.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.772+0000 7effdf7fe700 1 -- 172.21.15.57:0/4008282863 >> v2:172.21.15.181:3300/0 conn(0x7effe01599b0 msgr2=0x7effe015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T12:01:25.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.772+0000 7effdf7fe700 1 --2- 172.21.15.57:0/4008282863 >> v2:172.21.15.181:3300/0 conn(0x7effe01599b0 0x7effe015d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T12:01:25.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.772+0000 7effe4ed0700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 0x7effe015a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T12:01:25.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.772+0000 7effdf7fe700 1 --2- 172.21.15.57:0/4008282863 >> v2:172.21.15.181:3300/0 conn(0x7effe01599b0 0x7effe015d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T12:01:25.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.775+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720590765 0 0) 0x7effe0159ec0 con 0x7effe00b1650 2023-11-28T12:01:25.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.775+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effcc0035b0 con 0x7effe00b1650 2023-11-28T12:01:25.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.776+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3257697803 0 0) 0x7effcc0035b0 con 0x7effe00b1650 2023-11-28T12:01:25.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.776+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 msgr2=0x7effe015a350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T12:01:25.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.776+0000 7effdeffd700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 0x7effe015a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:25.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.776+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 >> v2:172.21.15.181:3300/0 conn(0x7effe01599b0 msgr2=0x7effe015d050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T12:01:25.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.776+0000 7effdeffd700 1 --2- 172.21.15.57:0/4008282863 >> v2:172.21.15.181:3300/0 conn(0x7effe01599b0 0x7effe015d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:25.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.776+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe015de60 con 0x7effe00b1650 2023-11-28T12:01:25.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.776+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7effd0001670 con 0x7effe00b1650 2023-11-28T12:01:25.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.776+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 >> v1:172.21.15.57:6789/0 conn(0x7effe00b1650 legacy=0x7effe00b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:25.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.776+0000 7effdeffd700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 0x7effcc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:25.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.776+0000 7effdeffd700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effcc004030 0x7effcc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:25.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.776+0000 7effdeffd700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effcc004d90 0x7effcc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:25.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.777+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe015d8a0 con 0x7effcc004d90 2023-11-28T12:01:25.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.777+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe015db80 con 0x7effcc004030 2023-11-28T12:01:25.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.777+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc00bb10 con 0x7effe00af0a0 2023-11-28T12:01:25.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.777+0000 7effdffff700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 0x7effcc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:25.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.777+0000 7effe4ed0700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effcc004d90 0x7effcc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:25.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.777+0000 7effdf7fe700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effcc004030 0x7effcc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:25.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.777+0000 7effe4ed0700 1 -- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 msgr2=0x7effcc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:25.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.777+0000 7effe4ed0700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe00af0a0 0x7effcc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:25.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.777+0000 7effe4ed0700 1 -- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effcc004030 msgr2=0x7effcc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:25.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.777+0000 7effe4ed0700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effcc004030 0x7effcc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:25.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.777+0000 7effe4ed0700 1 -- 172.21.15.57:0/4008282863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc0035b0 con 0x7effcc004d90 2023-11-28T12:01:25.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.777+0000 7effdf7fe700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effcc004030 0x7effcc004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:25.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.777+0000 7effe4ed0700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effcc004d90 0x7effcc005200 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effd8009ac0 tx=0x7effcc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b261861777ea6d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:25.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.780+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8004730 con 0x7effcc004d90 2023-11-28T12:01:25.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.780+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd800f050 con 0x7effcc004d90 2023-11-28T12:01:25.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.780+0000 7effdeffd700 1 -- 172.21.15.57:0/4008282863 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8014650 con 0x7effcc004d90 2023-11-28T12:01:25.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.781+0000 7effe6933700 1 -- 172.21.15.57:0/4008282863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effcc004d90 msgr2=0x7effcc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:25.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.781+0000 7effe6933700 1 --2- 172.21.15.57:0/4008282863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effcc004d90 0x7effcc005200 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effd8009ac0 tx=0x7effcc005880 comp rx=0 tx=0).stop 2023-11-28T12:01:25.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.781+0000 7effe4ed0700 1 -- 172.21.15.57:0/4008282863 reap_dead start 2023-11-28T12:01:25.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.781+0000 7effe6933700 1 -- 172.21.15.57:0/4008282863 shutdown_connections 2023-11-28T12:01:25.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.781+0000 7effe6933700 1 -- 172.21.15.57:0/4008282863 >> 172.21.15.57:0/4008282863 conn(0x7effe006c010 msgr2=0x7effe00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:25.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.781+0000 7effe6933700 1 -- 172.21.15.57:0/4008282863 shutdown_connections 2023-11-28T12:01:25.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.781+0000 7effe6933700 1 -- 172.21.15.57:0/4008282863 wait complete. 2023-11-28T12:01:25.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.783+0000 7effe6933700 1 Processor -- start 2023-11-28T12:01:25.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.784+0000 7effe6933700 1 -- start start 2023-11-28T12:01:25.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effe6933700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe00b41a0 0x7effe015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:25.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effe6933700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe015bee0 0x7effe006a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:25.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effe6933700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe006ab30 0x7effe00b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:25.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effe6933700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe0143e90 con 0x7effe015bee0 2023-11-28T12:01:25.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effe6933700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe015c330 con 0x7effe00b41a0 2023-11-28T12:01:25.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effe6933700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe0143280 con 0x7effe006ab30 2023-11-28T12:01:25.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effdffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe00b41a0 0x7effe015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effdf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe015bee0 0x7effe006a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effe4ed0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe006ab30 0x7effe00b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effdffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe00b41a0 0x7effe015b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:55800/0 (socket says 172.21.15.57:55800) 2023-11-28T12:01:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effdffff700 1 -- 172.21.15.57:0/1887915716 learned_addr learned my addr 172.21.15.57:0/1887915716 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effdffff700 1 -- 172.21.15.57:0/1887915716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe006ab30 msgr2=0x7effe00b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effdffff700 1 --2- 172.21.15.57:0/1887915716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe006ab30 0x7effe00b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effdffff700 1 -- 172.21.15.57:0/1887915716 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe015bee0 msgr2=0x7effe006a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effdffff700 1 --2- 172.21.15.57:0/1887915716 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe015bee0 0x7effe006a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effdffff700 1 -- 172.21.15.57:0/1887915716 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe015a350 con 0x7effe00b41a0 2023-11-28T12:01:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effe4ed0700 1 --2- 172.21.15.57:0/1887915716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe006ab30 0x7effe00b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:25.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.785+0000 7effdf7fe700 1 --2- 172.21.15.57:0/1887915716 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe015bee0 0x7effe006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:25.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.786+0000 7effdffff700 1 --2- 172.21.15.57:0/1887915716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe00b41a0 0x7effe015b9a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effd00017a0 tx=0x7effd00078e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:25.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.786+0000 7effdcff9700 1 -- 172.21.15.57:0/1887915716 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0007e30 con 0x7effe00b41a0 2023-11-28T12:01:25.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.786+0000 7effdcff9700 1 -- 172.21.15.57:0/1887915716 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd000f8a0 con 0x7effe00b41a0 2023-11-28T12:01:25.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.786+0000 7effdcff9700 1 -- 172.21.15.57:0/1887915716 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd000fc50 con 0x7effe00b41a0 2023-11-28T12:01:25.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.786+0000 7effe6933700 1 -- 172.21.15.57:0/1887915716 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe015d590 con 0x7effe00b41a0 2023-11-28T12:01:25.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.786+0000 7effe6933700 1 -- 172.21.15.57:0/1887915716 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe015a4e0 con 0x7effe00b41a0 2023-11-28T12:01:25.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.786+0000 7effdcff9700 1 -- 172.21.15.57:0/1887915716 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7effd000fdb0 con 0x7effe00b41a0 2023-11-28T12:01:25.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.786+0000 7effdcff9700 1 -- 172.21.15.57:0/1887915716 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7effd001e5b0 con 0x7effe00b41a0 2023-11-28T12:01:25.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.787+0000 7effe6933700 1 -- 172.21.15.57:0/1887915716 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe0144100 con 0x7effe00b41a0 2023-11-28T12:01:25.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.790+0000 7effdcff9700 1 -- 172.21.15.57:0/1887915716 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7effd001a070 con 0x7effe00b41a0 2023-11-28T12:01:25.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.871+0000 7effe6933700 1 -- 172.21.15.57:0/1887915716 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7effe00743f0 con 0x7effe00b41a0 2023-11-28T12:01:25.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.879+0000 7effdcff9700 1 -- 172.21.15.57:0/1887915716 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7effd00148d0 con 0x7effe00b41a0 2023-11-28T12:01:25.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.880+0000 7effe6933700 1 -- 172.21.15.57:0/1887915716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe00b41a0 msgr2=0x7effe015b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:25.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.880+0000 7effe6933700 1 --2- 172.21.15.57:0/1887915716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe00b41a0 0x7effe015b9a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effd00017a0 tx=0x7effd00078e0 comp rx=0 tx=0).stop 2023-11-28T12:01:25.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.880+0000 7effe6933700 1 -- 172.21.15.57:0/1887915716 shutdown_connections 2023-11-28T12:01:25.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.880+0000 7effe6933700 1 --2- 172.21.15.57:0/1887915716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe006ab30 0x7effe00b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:25.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.880+0000 7effe6933700 1 --2- 172.21.15.57:0/1887915716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe00b41a0 0x7effe015b9a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:25.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.880+0000 7effe6933700 1 --2- 172.21.15.57:0/1887915716 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe015bee0 0x7effe006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:25.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.880+0000 7effe6933700 1 -- 172.21.15.57:0/1887915716 >> 172.21.15.57:0/1887915716 conn(0x7effe006c010 msgr2=0x7effe006d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:25.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.880+0000 7effe6933700 1 -- 172.21.15.57:0/1887915716 shutdown_connections 2023-11-28T12:01:25.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:25.881+0000 7effe6933700 1 -- 172.21.15.57:0/1887915716 wait complete. 2023-11-28T12:01:25.883 INFO:teuthology.orchestra.run.smithi057.stderr:adjusted tunables profile to default 2023-11-28T12:01:25.895 INFO:tasks.ceph:check_enable_crimson: False 2023-11-28T12:01:25.895 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-28T12:01:25.896 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:01:25.896 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-28T12:01:25.898 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:25.897+0000 7f3f7df0a240 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T12:01:25.919 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:25.918+0000 7f0c66bae240 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T12:01:25.926 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:01:25.926 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-28T12:01:25.996 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:01:25.996 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-28T12:01:26.092 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:26.091+0000 7f3f7df0a240 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T12:01:26.098 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:01:26.098 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-28T12:01:26.123 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:26.122+0000 7f0c66bae240 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T12:01:26.152 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:01:26.153 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-28T12:01:26.179 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:01:26.179 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-28T12:01:26.246 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:01:26.246 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-28T12:01:26.314 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:01:26.314 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-28T12:01:26.381 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 3fe3ca0b-57ba-412d-ba00-5085b06b65ea 0 2023-11-28T12:01:26.456 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:26.455+0000 7f3f7df0a240 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T12:01:26.503 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:26.502+0000 7f0c66bae240 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T12:01:26.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.543+0000 7fa1ae1ed700 1 Processor -- start 2023-11-28T12:01:26.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.544+0000 7fa1ae1ed700 1 -- start start 2023-11-28T12:01:26.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.544+0000 7fa1ae1ed700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa1a80cf1d0 0x7fa1a8071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:26.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.544+0000 7fa1ae1ed700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa1a815ea70 0x7fa1a80cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:26.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.544+0000 7fa1ae1ed700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a80cb010 con 0x7fa1a806a270 2023-11-28T12:01:26.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.544+0000 7fa1ae1ed700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a815a9c0 con 0x7fa1a815ea70 2023-11-28T12:01:26.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.544+0000 7fa1ae1ed700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a815ef50 con 0x7fa1a80cf1d0 2023-11-28T12:01:26.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.544+0000 7fa1a77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa1a80cf1d0 0x7fa1a8071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:26.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.544+0000 7fa1a77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa1a80cf1d0 0x7fa1a8071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42290/0 (socket says 172.21.15.181:42290) 2023-11-28T12:01:26.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.544+0000 7fa1a77fe700 1 -- 172.21.15.181:0/1418737790 learned_addr learned my addr 172.21.15.181:0/1418737790 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-11-28T12:01:26.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.545+0000 7fa1a6ffd700 1 --2- 172.21.15.181:0/1418737790 >> v2:172.21.15.57:3300/0 conn(0x7fa1a815ea70 0x7fa1a80cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:26.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.545+0000 7fa1a77fe700 1 -- 172.21.15.181:0/1418737790 >> v2:172.21.15.57:3300/0 conn(0x7fa1a815ea70 msgr2=0x7fa1a80cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:26.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.545+0000 7fa1a77fe700 1 --2- 172.21.15.181:0/1418737790 >> v2:172.21.15.57:3300/0 conn(0x7fa1a815ea70 0x7fa1a80cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.545+0000 7fa1a77fe700 1 -- 172.21.15.181:0/1418737790 >> v1:172.21.15.57:6789/0 conn(0x7fa1a806a270 legacy=0x7fa1a806a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:26.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.545+0000 7fa1a77fe700 1 -- 172.21.15.181:0/1418737790 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a80cb240 con 0x7fa1a80cf1d0 2023-11-28T12:01:26.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.545+0000 7fa1a77fe700 1 --2- 172.21.15.181:0/1418737790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1a80cf1d0 0x7fa1a8071040 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80cd300 tx=0x7fa198009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c91dc8c66060512 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:26.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.545+0000 7fa1a67fc700 1 -- 172.21.15.181:0/1418737790 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19800b870 con 0x7fa1a80cf1d0 2023-11-28T12:01:26.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.545+0000 7fa1a67fc700 1 -- 172.21.15.181:0/1418737790 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19800b9d0 con 0x7fa1a80cf1d0 2023-11-28T12:01:26.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.545+0000 7fa1a67fc700 1 -- 172.21.15.181:0/1418737790 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19800bd60 con 0x7fa1a80cf1d0 2023-11-28T12:01:26.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.546+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/1418737790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1a80cf1d0 msgr2=0x7fa1a8071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:26.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.546+0000 7fa1ae1ed700 1 --2- 172.21.15.181:0/1418737790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1a80cf1d0 0x7fa1a8071040 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80cd300 tx=0x7fa198009bd0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.546+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/1418737790 shutdown_connections 2023-11-28T12:01:26.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.546+0000 7fa1ae1ed700 1 --2- 172.21.15.181:0/1418737790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1a80cf1d0 0x7fa1a8071040 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.546+0000 7fa1ae1ed700 1 --2- 172.21.15.181:0/1418737790 >> v2:172.21.15.57:3300/0 conn(0x7fa1a815ea70 0x7fa1a80cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.546+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/1418737790 >> 172.21.15.181:0/1418737790 conn(0x7fa1a806c010 msgr2=0x7fa1a80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:26.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.546+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/1418737790 shutdown_connections 2023-11-28T12:01:26.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.546+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/1418737790 wait complete. 2023-11-28T12:01:26.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.547+0000 7fa1ae1ed700 1 Processor -- start 2023-11-28T12:01:26.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.547+0000 7fa1ae1ed700 1 -- start start 2023-11-28T12:01:26.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1ae1ed700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1a806a270 0x7fa1a8155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:26.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1ae1ed700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1a80cf1d0 0x7fa1a8153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:26.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1ae1ed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1a815ea70 0x7fa1a80b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:26.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1ae1ed700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a81f4420 con 0x7fa1a806a270 2023-11-28T12:01:26.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1ae1ed700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a814f990 con 0x7fa1a815ea70 2023-11-28T12:01:26.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1ae1ed700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a80b0560 con 0x7fa1a80cf1d0 2023-11-28T12:01:26.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1a7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1a815ea70 0x7fa1a80b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:26.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1a7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1a815ea70 0x7fa1a80b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42298/0 (socket says 172.21.15.181:42298) 2023-11-28T12:01:26.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1a6ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1a80cf1d0 0x7fa1a8153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:26.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1a7fff700 1 -- 172.21.15.181:0/2426885594 learned_addr learned my addr 172.21.15.181:0/2426885594 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-11-28T12:01:26.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1a77fe700 1 --2- 172.21.15.181:0/2426885594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1a806a270 0x7fa1a8155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:26.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1a7fff700 1 -- 172.21.15.181:0/2426885594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1a80cf1d0 msgr2=0x7fa1a8153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:26.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1a7fff700 1 --2- 172.21.15.181:0/2426885594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1a80cf1d0 0x7fa1a8153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1a7fff700 1 -- 172.21.15.181:0/2426885594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1a806a270 msgr2=0x7fa1a8155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:26.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1a7fff700 1 --2- 172.21.15.181:0/2426885594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1a806a270 0x7fa1a8155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1a7fff700 1 -- 172.21.15.181:0/2426885594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198009550 con 0x7fa1a815ea70 2023-11-28T12:01:26.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1a6ffd700 1 --2- 172.21.15.181:0/2426885594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1a80cf1d0 0x7fa1a8153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:26.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1a7fff700 1 --2- 172.21.15.181:0/2426885594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1a815ea70 0x7fa1a80b1850 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000cc80 tx=0x7fa1a000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:26.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa18ffff700 1 -- 172.21.15.181:0/2426885594 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0003c00 con 0x7fa1a815ea70 2023-11-28T12:01:26.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa18ffff700 1 -- 172.21.15.181:0/2426885594 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a0003d60 con 0x7fa1a815ea70 2023-11-28T12:01:26.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/2426885594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a80cb800 con 0x7fa1a815ea70 2023-11-28T12:01:26.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa18ffff700 1 -- 172.21.15.181:0/2426885594 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000aaa0 con 0x7fa1a815ea70 2023-11-28T12:01:26.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.548+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/2426885594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a80cb520 con 0x7fa1a815ea70 2023-11-28T12:01:26.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.549+0000 7fa18ffff700 1 -- 172.21.15.181:0/2426885594 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fa1a0010050 con 0x7fa1a815ea70 2023-11-28T12:01:26.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.549+0000 7fa18ffff700 1 -- 172.21.15.181:0/2426885594 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa1a0012760 con 0x7fa1a815ea70 2023-11-28T12:01:26.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.549+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/2426885594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a80cbae0 con 0x7fa1a815ea70 2023-11-28T12:01:26.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.551+0000 7fa18ffff700 1 -- 172.21.15.181:0/2426885594 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fa1a00050d0 con 0x7fa1a815ea70 2023-11-28T12:01:26.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.634+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/2426885594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3fe3ca0b-57ba-412d-ba00-5085b06b65ea", "id": 0} v 0) v1 -- 0x7fa1a80b5b50 con 0x7fa1a815ea70 2023-11-28T12:01:26.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.638+0000 7fa18ffff700 1 -- 172.21.15.181:0/2426885594 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3fe3ca0b-57ba-412d-ba00-5085b06b65ea", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa1a0005270 con 0x7fa1a815ea70 2023-11-28T12:01:26.640 INFO:teuthology.orchestra.run.smithi181.stdout:0 2023-11-28T12:01:26.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.639+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/2426885594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1a815ea70 msgr2=0x7fa1a80b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:26.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.639+0000 7fa1ae1ed700 1 --2- 172.21.15.181:0/2426885594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1a815ea70 0x7fa1a80b1850 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000cc80 tx=0x7fa1a000cf90 comp rx=0 tx=0).stop 2023-11-28T12:01:26.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.639+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/2426885594 shutdown_connections 2023-11-28T12:01:26.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.639+0000 7fa1ae1ed700 1 --2- 172.21.15.181:0/2426885594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1a815ea70 0x7fa1a80b1850 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.639+0000 7fa1ae1ed700 1 --2- 172.21.15.181:0/2426885594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1a806a270 0x7fa1a8155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.639+0000 7fa1ae1ed700 1 --2- 172.21.15.181:0/2426885594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1a80cf1d0 0x7fa1a8153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.639+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/2426885594 >> 172.21.15.181:0/2426885594 conn(0x7fa1a806c010 msgr2=0x7fa1a815a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:26.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.639+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/2426885594 shutdown_connections 2023-11-28T12:01:26.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.639+0000 7fa1ae1ed700 1 -- 172.21.15.181:0/2426885594 wait complete. 2023-11-28T12:01:26.652 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new f0ce5312-030d-4de4-9c05-e13a80b0a7d1 1 2023-11-28T12:01:26.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.815+0000 7fc54808e700 1 Processor -- start 2023-11-28T12:01:26.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.815+0000 7fc54808e700 1 -- start start 2023-11-28T12:01:26.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.816+0000 7fc54808e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc54015a4d0 0x7fc54006a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:26.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.816+0000 7fc54808e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5400cdf20 0x7fc5400cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:26.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.816+0000 7fc54808e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54015e0d0 con 0x7fc54006a7b0 2023-11-28T12:01:26.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.816+0000 7fc54808e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54006ac30 con 0x7fc54015a4d0 2023-11-28T12:01:26.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.816+0000 7fc54808e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5400ce310 con 0x7fc5400cdf20 2023-11-28T12:01:26.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.816+0000 7fc54662b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5400cdf20 0x7fc5400cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:26.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.816+0000 7fc54662b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5400cdf20 0x7fc5400cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42304/0 (socket says 172.21.15.181:42304) 2023-11-28T12:01:26.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.816+0000 7fc54662b700 1 -- 172.21.15.181:0/729271538 learned_addr learned my addr 172.21.15.181:0/729271538 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-11-28T12:01:26.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.816+0000 7fc545e2a700 1 --2- 172.21.15.181:0/729271538 >> v2:172.21.15.57:3300/0 conn(0x7fc54015a4d0 0x7fc54006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:26.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc54662b700 1 -- 172.21.15.181:0/729271538 >> v2:172.21.15.57:3300/0 conn(0x7fc54015a4d0 msgr2=0x7fc54006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:26.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc54662b700 1 --2- 172.21.15.181:0/729271538 >> v2:172.21.15.57:3300/0 conn(0x7fc54015a4d0 0x7fc54006a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc54662b700 1 -- 172.21.15.181:0/729271538 >> v1:172.21.15.57:6789/0 conn(0x7fc54006a7b0 legacy=0x7fc5400cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:26.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc54662b700 1 -- 172.21.15.181:0/729271538 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5400ce560 con 0x7fc5400cdf20 2023-11-28T12:01:26.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc54662b700 1 --2- 172.21.15.181:0/729271538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5400cdf20 0x7fc5400cb010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc5400cd680 tx=0x7fc53c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9957612c0a2f81c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:26.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc544e28700 1 -- 172.21.15.181:0/729271538 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00b870 con 0x7fc5400cdf20 2023-11-28T12:01:26.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc544e28700 1 -- 172.21.15.181:0/729271538 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c00b9d0 con 0x7fc5400cdf20 2023-11-28T12:01:26.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc544e28700 1 -- 172.21.15.181:0/729271538 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00bd60 con 0x7fc5400cdf20 2023-11-28T12:01:26.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc54808e700 1 -- 172.21.15.181:0/729271538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5400cdf20 msgr2=0x7fc5400cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:26.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc54808e700 1 --2- 172.21.15.181:0/729271538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5400cdf20 0x7fc5400cb010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc5400cd680 tx=0x7fc53c009bd0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc54808e700 1 -- 172.21.15.181:0/729271538 shutdown_connections 2023-11-28T12:01:26.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc54808e700 1 --2- 172.21.15.181:0/729271538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5400cdf20 0x7fc5400cb010 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc54808e700 1 --2- 172.21.15.181:0/729271538 >> v2:172.21.15.57:3300/0 conn(0x7fc54015a4d0 0x7fc54006a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.817+0000 7fc54808e700 1 -- 172.21.15.181:0/729271538 >> 172.21.15.181:0/729271538 conn(0x7fc54006c010 msgr2=0x7fc5400cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:26.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.818+0000 7fc54808e700 1 -- 172.21.15.181:0/729271538 shutdown_connections 2023-11-28T12:01:26.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.818+0000 7fc54808e700 1 -- 172.21.15.181:0/729271538 wait complete. 2023-11-28T12:01:26.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.819+0000 7fc54808e700 1 Processor -- start 2023-11-28T12:01:26.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.819+0000 7fc54808e700 1 -- start start 2023-11-28T12:01:26.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.819+0000 7fc54808e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc54006a7b0 0x7fc540157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:26.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.819+0000 7fc54808e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5400cdf20 0x7fc54015ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:26.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.819+0000 7fc54808e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc54015a4d0 0x7fc5401f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:26.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.819+0000 7fc54808e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54015ad30 con 0x7fc5400cdf20 2023-11-28T12:01:26.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.819+0000 7fc54808e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5401f6b20 con 0x7fc54006a7b0 2023-11-28T12:01:26.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.819+0000 7fc54808e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5401f6e00 con 0x7fc54015a4d0 2023-11-28T12:01:26.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc545e2a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc54006a7b0 0x7fc540157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:26.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc545e2a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc54006a7b0 0x7fc540157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42314/0 (socket says 172.21.15.181:42314) 2023-11-28T12:01:26.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc545e2a700 1 -- 172.21.15.181:0/1369470775 learned_addr learned my addr 172.21.15.181:0/1369470775 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-11-28T12:01:26.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc54662b700 1 --2- 172.21.15.181:0/1369470775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc54015a4d0 0x7fc5401f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:26.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc545629700 1 --2- 172.21.15.181:0/1369470775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5400cdf20 0x7fc54015ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:26.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc545e2a700 1 -- 172.21.15.181:0/1369470775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc54015a4d0 msgr2=0x7fc5401f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:26.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc545e2a700 1 --2- 172.21.15.181:0/1369470775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc54015a4d0 0x7fc5401f5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc545e2a700 1 -- 172.21.15.181:0/1369470775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5400cdf20 msgr2=0x7fc54015ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:26.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc545e2a700 1 --2- 172.21.15.181:0/1369470775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5400cdf20 0x7fc54015ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc545e2a700 1 -- 172.21.15.181:0/1369470775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c009550 con 0x7fc54006a7b0 2023-11-28T12:01:26.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc545e2a700 1 --2- 172.21.15.181:0/1369470775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc54006a7b0 0x7fc540157660 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc538007e40 tx=0x7fc53800e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:26.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc5367fc700 1 -- 172.21.15.181:0/1369470775 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53800ed30 con 0x7fc54006a7b0 2023-11-28T12:01:26.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc5367fc700 1 -- 172.21.15.181:0/1369470775 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53800ee90 con 0x7fc54006a7b0 2023-11-28T12:01:26.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc5367fc700 1 -- 172.21.15.181:0/1369470775 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc538014660 con 0x7fc54006a7b0 2023-11-28T12:01:26.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc54808e700 1 -- 172.21.15.181:0/1369470775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54015e600 con 0x7fc54006a7b0 2023-11-28T12:01:26.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.820+0000 7fc54808e700 1 -- 172.21.15.181:0/1369470775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54015e320 con 0x7fc54006a7b0 2023-11-28T12:01:26.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.821+0000 7fc5367fc700 1 -- 172.21.15.181:0/1369470775 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc538012050 con 0x7fc54006a7b0 2023-11-28T12:01:26.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.821+0000 7fc5367fc700 1 -- 172.21.15.181:0/1369470775 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fc53801e9d0 con 0x7fc54006a7b0 2023-11-28T12:01:26.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.821+0000 7fc54808e700 1 -- 172.21.15.181:0/1369470775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54015e8e0 con 0x7fc54006a7b0 2023-11-28T12:01:26.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.825+0000 7fc5367fc700 1 -- 172.21.15.181:0/1369470775 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fc538014ce0 con 0x7fc54006a7b0 2023-11-28T12:01:26.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.906+0000 7fc54808e700 1 -- 172.21.15.181:0/1369470775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f0ce5312-030d-4de4-9c05-e13a80b0a7d1", "id": 1} v 0) v1 -- 0x7fc54015cce0 con 0x7fc54006a7b0 2023-11-28T12:01:26.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.910+0000 7fc5367fc700 1 -- 172.21.15.181:0/1369470775 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f0ce5312-030d-4de4-9c05-e13a80b0a7d1", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc53801eb90 con 0x7fc54006a7b0 2023-11-28T12:01:26.912 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-11-28T12:01:26.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.911+0000 7fc54808e700 1 -- 172.21.15.181:0/1369470775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc54006a7b0 msgr2=0x7fc540157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:26.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.911+0000 7fc54808e700 1 --2- 172.21.15.181:0/1369470775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc54006a7b0 0x7fc540157660 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc538007e40 tx=0x7fc53800e3c0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.911+0000 7fc54808e700 1 -- 172.21.15.181:0/1369470775 shutdown_connections 2023-11-28T12:01:26.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.911+0000 7fc54808e700 1 --2- 172.21.15.181:0/1369470775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc54015a4d0 0x7fc5401f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.911+0000 7fc54808e700 1 --2- 172.21.15.181:0/1369470775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc54006a7b0 0x7fc540157660 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.911+0000 7fc54808e700 1 --2- 172.21.15.181:0/1369470775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5400cdf20 0x7fc54015ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:26.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.911+0000 7fc54808e700 1 -- 172.21.15.181:0/1369470775 >> 172.21.15.181:0/1369470775 conn(0x7fc54006c010 msgr2=0x7fc5400b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:26.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.911+0000 7fc54808e700 1 -- 172.21.15.181:0/1369470775 shutdown_connections 2023-11-28T12:01:26.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:26.912+0000 7fc54808e700 1 -- 172.21.15.181:0/1369470775 wait complete. 2023-11-28T12:01:26.924 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 8756e996-2c62-40ae-a244-6d6a8d58d275 2 2023-11-28T12:01:26.980 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:26.979+0000 7f3f7df0a240 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T12:01:27.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.045+0000 7f643ec7e700 1 Processor -- start 2023-11-28T12:01:27.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.045+0000 7f643ec7e700 1 -- start start 2023-11-28T12:01:27.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.046+0000 7f643ec7e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64380af0a0 0x7f643815a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.046+0000 7f643ec7e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6438159ec0 con 0x7f64381599b0 2023-11-28T12:01:27.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.046+0000 7f643ec7e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643815d8a0 con 0x7f64380b1650 2023-11-28T12:01:27.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.046+0000 7f643ec7e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643815db80 con 0x7f64380af0a0 2023-11-28T12:01:27.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.046+0000 7f643d21b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64380af0a0 0x7f643815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.046+0000 7f643ca1a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f64380b1650 0x7f64380b1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40946/0 (socket says 172.21.15.181:40946) 2023-11-28T12:01:27.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.046+0000 7f643ca1a700 1 -- 172.21.15.181:0/359723166 learned_addr learned my addr 172.21.15.181:0/359723166 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-11-28T12:01:27.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.046+0000 7f643d21b700 1 -- 172.21.15.181:0/359723166 >> v1:172.21.15.181:6789/0 conn(0x7f64380b1650 legacy=0x7f64380b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.046+0000 7f643d21b700 1 -- 172.21.15.181:0/359723166 >> v1:172.21.15.57:6789/0 conn(0x7f64381599b0 legacy=0x7f643815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.046+0000 7f64377fe700 1 -- 172.21.15.181:0/359723166 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204594252 0 0) 0x7f643815d8a0 con 0x7f64380b1650 2023-11-28T12:01:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.046+0000 7f643d21b700 1 -- 172.21.15.181:0/359723166 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643815de60 con 0x7f64380af0a0 2023-11-28T12:01:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.047+0000 7f643d21b700 1 --2- 172.21.15.181:0/359723166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64380af0a0 0x7f643815a350 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6430009d90 tx=0x7f643000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13eb9870511bf81b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.047+0000 7f64377fe700 1 -- 172.21.15.181:0/359723166 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643000e070 con 0x7f64380af0a0 2023-11-28T12:01:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.047+0000 7f64377fe700 1 -- 172.21.15.181:0/359723166 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643000bbd0 con 0x7f64380af0a0 2023-11-28T12:01:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.047+0000 7f64377fe700 1 -- 172.21.15.181:0/359723166 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430012430 con 0x7f64380af0a0 2023-11-28T12:01:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.047+0000 7f643ec7e700 1 -- 172.21.15.181:0/359723166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64380af0a0 msgr2=0x7f643815a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.047+0000 7f643ec7e700 1 --2- 172.21.15.181:0/359723166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64380af0a0 0x7f643815a350 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6430009d90 tx=0x7f643000b330 comp rx=0 tx=0).stop 2023-11-28T12:01:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.047+0000 7f643ec7e700 1 -- 172.21.15.181:0/359723166 shutdown_connections 2023-11-28T12:01:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.047+0000 7f643ec7e700 1 --2- 172.21.15.181:0/359723166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64380af0a0 0x7f643815a350 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.047+0000 7f643ec7e700 1 -- 172.21.15.181:0/359723166 >> 172.21.15.181:0/359723166 conn(0x7f643806c010 msgr2=0x7f64380cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:27.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.047+0000 7f643ec7e700 1 -- 172.21.15.181:0/359723166 shutdown_connections 2023-11-28T12:01:27.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.047+0000 7f643ec7e700 1 -- 172.21.15.181:0/359723166 wait complete. 2023-11-28T12:01:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.049+0000 7f643ec7e700 1 Processor -- start 2023-11-28T12:01:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.049+0000 7f643ec7e700 1 -- start start 2023-11-28T12:01:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f643ec7e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64380af0a0 0x7f6438158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f643ec7e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64380b1650 0x7f64380c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f643ec7e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64380cbb50 0x7f64380c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f643ec7e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64381e4960 con 0x7f64380cbb50 2023-11-28T12:01:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f643ec7e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64380c3920 con 0x7f64380b1650 2023-11-28T12:01:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f643ec7e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64380c3d10 con 0x7f64380af0a0 2023-11-28T12:01:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f6437fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64380b1650 0x7f64380c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f643ca1a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64380af0a0 0x7f6438158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f6437fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64380b1650 0x7f64380c5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42344/0 (socket says 172.21.15.181:42344) 2023-11-28T12:01:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f6437fff700 1 -- 172.21.15.181:0/4275300857 learned_addr learned my addr 172.21.15.181:0/4275300857 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-11-28T12:01:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f643d21b700 1 --2- 172.21.15.181:0/4275300857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64380cbb50 0x7f64380c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f643ca1a700 1 -- 172.21.15.181:0/4275300857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64380b1650 msgr2=0x7f64380c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f643ca1a700 1 --2- 172.21.15.181:0/4275300857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64380b1650 0x7f64380c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f643ca1a700 1 -- 172.21.15.181:0/4275300857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64380cbb50 msgr2=0x7f64380c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f643ca1a700 1 --2- 172.21.15.181:0/4275300857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64380cbb50 0x7f64380c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f643ca1a700 1 -- 172.21.15.181:0/4275300857 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6430009a70 con 0x7f64380af0a0 2023-11-28T12:01:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.050+0000 7f6437fff700 1 --2- 172.21.15.181:0/4275300857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64380b1650 0x7f64380c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.051+0000 7f643ca1a700 1 --2- 172.21.15.181:0/4275300857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64380af0a0 0x7f6438158fd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c470 tx=0x7f642c00c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.051+0000 7f64357fa700 1 -- 172.21.15.181:0/4275300857 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c004c80 con 0x7f64380af0a0 2023-11-28T12:01:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.051+0000 7f64357fa700 1 -- 172.21.15.181:0/4275300857 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c004de0 con 0x7f64380af0a0 2023-11-28T12:01:27.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.051+0000 7f643ec7e700 1 -- 172.21.15.181:0/4275300857 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643815aa20 con 0x7f64380af0a0 2023-11-28T12:01:27.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.051+0000 7f64357fa700 1 -- 172.21.15.181:0/4275300857 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c007310 con 0x7f64380af0a0 2023-11-28T12:01:27.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.051+0000 7f643ec7e700 1 -- 172.21.15.181:0/4275300857 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643815a890 con 0x7f64380af0a0 2023-11-28T12:01:27.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.051+0000 7f64357fa700 1 -- 172.21.15.181:0/4275300857 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f642c01a020 con 0x7f64380af0a0 2023-11-28T12:01:27.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.051+0000 7f64357fa700 1 -- 172.21.15.181:0/4275300857 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f642c0137e0 con 0x7f64380af0a0 2023-11-28T12:01:27.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.052+0000 7f643ec7e700 1 -- 172.21.15.181:0/4275300857 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64380743f0 con 0x7f64380af0a0 2023-11-28T12:01:27.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.054+0000 7f64357fa700 1 -- 172.21.15.181:0/4275300857 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f642c01d3f0 con 0x7f64380af0a0 2023-11-28T12:01:27.072 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:27.071+0000 7f0c66bae240 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T12:01:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.137+0000 7f643ec7e700 1 -- 172.21.15.181:0/4275300857 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8756e996-2c62-40ae-a244-6d6a8d58d275", "id": 2} v 0) v1 -- 0x7f643806cd10 con 0x7f64380af0a0 2023-11-28T12:01:27.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.141+0000 7f64357fa700 1 -- 172.21.15.181:0/4275300857 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8756e996-2c62-40ae-a244-6d6a8d58d275", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f642c013530 con 0x7f64380af0a0 2023-11-28T12:01:27.142 INFO:teuthology.orchestra.run.smithi181.stdout:2 2023-11-28T12:01:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.142+0000 7f643ec7e700 1 -- 172.21.15.181:0/4275300857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64380af0a0 msgr2=0x7f6438158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.142+0000 7f643ec7e700 1 --2- 172.21.15.181:0/4275300857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64380af0a0 0x7f6438158fd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c470 tx=0x7f642c00c780 comp rx=0 tx=0).stop 2023-11-28T12:01:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.142+0000 7f643ec7e700 1 -- 172.21.15.181:0/4275300857 shutdown_connections 2023-11-28T12:01:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.142+0000 7f643ec7e700 1 --2- 172.21.15.181:0/4275300857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64380cbb50 0x7f64380c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.142+0000 7f643ec7e700 1 --2- 172.21.15.181:0/4275300857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64380af0a0 0x7f6438158fd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.142+0000 7f643ec7e700 1 --2- 172.21.15.181:0/4275300857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64380b1650 0x7f64380c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.142+0000 7f643ec7e700 1 -- 172.21.15.181:0/4275300857 >> 172.21.15.181:0/4275300857 conn(0x7f643806c010 msgr2=0x7f643806d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.142+0000 7f643ec7e700 1 -- 172.21.15.181:0/4275300857 shutdown_connections 2023-11-28T12:01:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.142+0000 7f643ec7e700 1 -- 172.21.15.181:0/4275300857 wait complete. 2023-11-28T12:01:27.156 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 5cbd5261-bd50-44f1-8192-60d55157d358 3 2023-11-28T12:01:27.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.278+0000 7fc817459700 1 Processor -- start 2023-11-28T12:01:27.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.278+0000 7fc817459700 1 -- start start 2023-11-28T12:01:27.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc817459700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc81006b220 0x7fc8100d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc817459700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8100d2d30 con 0x7fc81015c8c0 2023-11-28T12:01:27.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc817459700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8100d0b40 con 0x7fc810160cb0 2023-11-28T12:01:27.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc817459700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8100d2a50 con 0x7fc81006b220 2023-11-28T12:01:27.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc8159f6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc81006b220 0x7fc8100d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc8149f4700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc810160cb0 0x7fc81006ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40954/0 (socket says 172.21.15.181:40954) 2023-11-28T12:01:27.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc8159f6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc81006b220 0x7fc8100d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42348/0 (socket says 172.21.15.181:42348) 2023-11-28T12:01:27.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc8149f4700 1 -- 172.21.15.181:0/603076527 learned_addr learned my addr 172.21.15.181:0/603076527 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-11-28T12:01:27.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc8159f6700 1 -- 172.21.15.181:0/603076527 >> v1:172.21.15.181:6789/0 conn(0x7fc810160cb0 legacy=0x7fc81006ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc8159f6700 1 -- 172.21.15.181:0/603076527 >> v1:172.21.15.57:6789/0 conn(0x7fc81015c8c0 legacy=0x7fc8100b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc807fff700 1 -- 172.21.15.181:0/603076527 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3754581195 0 0) 0x7fc8100d0b40 con 0x7fc810160cb0 2023-11-28T12:01:27.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc8159f6700 1 -- 172.21.15.181:0/603076527 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8100cd760 con 0x7fc81006b220 2023-11-28T12:01:27.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc8159f6700 1 --2- 172.21.15.181:0/603076527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc81006b220 0x7fc8100d0fd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc808009d90 tx=0x7fc80800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b37deb3d54b339 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:27.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc807fff700 1 -- 172.21.15.181:0/603076527 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800e070 con 0x7fc81006b220 2023-11-28T12:01:27.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc807fff700 1 -- 172.21.15.181:0/603076527 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80800bbd0 con 0x7fc81006b220 2023-11-28T12:01:27.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.279+0000 7fc807fff700 1 -- 172.21.15.181:0/603076527 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808012430 con 0x7fc81006b220 2023-11-28T12:01:27.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.280+0000 7fc817459700 1 -- 172.21.15.181:0/603076527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc81006b220 msgr2=0x7fc8100d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.280+0000 7fc817459700 1 --2- 172.21.15.181:0/603076527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc81006b220 0x7fc8100d0fd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc808009d90 tx=0x7fc80800b330 comp rx=0 tx=0).stop 2023-11-28T12:01:27.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.280+0000 7fc817459700 1 -- 172.21.15.181:0/603076527 shutdown_connections 2023-11-28T12:01:27.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.280+0000 7fc817459700 1 --2- 172.21.15.181:0/603076527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc81006b220 0x7fc8100d0fd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.280+0000 7fc817459700 1 -- 172.21.15.181:0/603076527 >> 172.21.15.181:0/603076527 conn(0x7fc81006c520 msgr2=0x7fc81006c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:27.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.280+0000 7fc817459700 1 -- 172.21.15.181:0/603076527 shutdown_connections 2023-11-28T12:01:27.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.280+0000 7fc817459700 1 -- 172.21.15.181:0/603076527 wait complete. 2023-11-28T12:01:27.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.281+0000 7fc817459700 1 Processor -- start 2023-11-28T12:01:27.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.281+0000 7fc817459700 1 -- start start 2023-11-28T12:01:27.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc817459700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc81006b220 0x7fc8100c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc817459700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc81015c8c0 0x7fc8100c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc817459700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810160cb0 0x7fc81014d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc817459700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8100c7af0 con 0x7fc81015c8c0 2023-11-28T12:01:27.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc817459700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81014e340 con 0x7fc810160cb0 2023-11-28T12:01:27.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc817459700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81015cd30 con 0x7fc81006b220 2023-11-28T12:01:27.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc8151f5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc81006b220 0x7fc8100c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc8159f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810160cb0 0x7fc81014d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc8151f5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc81006b220 0x7fc8100c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42206/0 (socket says 172.21.15.181:42206) 2023-11-28T12:01:27.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc8151f5700 1 -- 172.21.15.181:0/951357472 learned_addr learned my addr 172.21.15.181:0/951357472 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-11-28T12:01:27.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc8159f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810160cb0 0x7fc81014d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42360/0 (socket says 172.21.15.181:42360) 2023-11-28T12:01:27.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc8149f4700 1 --2- 172.21.15.181:0/951357472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc81015c8c0 0x7fc8100c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc8159f6700 1 -- 172.21.15.181:0/951357472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc81006b220 msgr2=0x7fc8100c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc8159f6700 1 --2- 172.21.15.181:0/951357472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc81006b220 0x7fc8100c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc8159f6700 1 -- 172.21.15.181:0/951357472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc81015c8c0 msgr2=0x7fc8100c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc8159f6700 1 --2- 172.21.15.181:0/951357472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc81015c8c0 0x7fc8100c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc8159f6700 1 -- 172.21.15.181:0/951357472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc808009a70 con 0x7fc810160cb0 2023-11-28T12:01:27.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.282+0000 7fc8151f5700 1 --2- 172.21.15.181:0/951357472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc81006b220 0x7fc8100c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:27.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.283+0000 7fc8159f6700 1 --2- 172.21.15.181:0/951357472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810160cb0 0x7fc81014d3b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc808016000 tx=0x7fc808004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:27.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.283+0000 7fc805ffb700 1 -- 172.21.15.181:0/951357472 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80801b810 con 0x7fc810160cb0 2023-11-28T12:01:27.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.283+0000 7fc805ffb700 1 -- 172.21.15.181:0/951357472 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808020070 con 0x7fc810160cb0 2023-11-28T12:01:27.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.283+0000 7fc817459700 1 -- 172.21.15.181:0/951357472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc810161120 con 0x7fc810160cb0 2023-11-28T12:01:27.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.283+0000 7fc805ffb700 1 -- 172.21.15.181:0/951357472 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80801bbf0 con 0x7fc810160cb0 2023-11-28T12:01:27.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.283+0000 7fc817459700 1 -- 172.21.15.181:0/951357472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8100cda40 con 0x7fc810160cb0 2023-11-28T12:01:27.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.283+0000 7fc805ffb700 1 -- 172.21.15.181:0/951357472 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc808027770 con 0x7fc810160cb0 2023-11-28T12:01:27.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.283+0000 7fc805ffb700 1 -- 172.21.15.181:0/951357472 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fc808022070 con 0x7fc810160cb0 2023-11-28T12:01:27.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.284+0000 7fc817459700 1 -- 172.21.15.181:0/951357472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8100d2700 con 0x7fc810160cb0 2023-11-28T12:01:27.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.286+0000 7fc805ffb700 1 -- 172.21.15.181:0/951357472 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fc808027be0 con 0x7fc810160cb0 2023-11-28T12:01:27.349 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:27.348+0000 7f3f7df0a240 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T12:01:27.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.370+0000 7fc817459700 1 -- 172.21.15.181:0/951357472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5cbd5261-bd50-44f1-8192-60d55157d358", "id": 3} v 0) v1 -- 0x7fc810067d70 con 0x7fc810160cb0 2023-11-28T12:01:27.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.374+0000 7fc805ffb700 1 -- 172.21.15.181:0/951357472 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5cbd5261-bd50-44f1-8192-60d55157d358", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc808027d80 con 0x7fc810160cb0 2023-11-28T12:01:27.375 INFO:teuthology.orchestra.run.smithi181.stdout:3 2023-11-28T12:01:27.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.375+0000 7fc817459700 1 -- 172.21.15.181:0/951357472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810160cb0 msgr2=0x7fc81014d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.375+0000 7fc817459700 1 --2- 172.21.15.181:0/951357472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810160cb0 0x7fc81014d3b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc808016000 tx=0x7fc808004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.375+0000 7fc817459700 1 -- 172.21.15.181:0/951357472 shutdown_connections 2023-11-28T12:01:27.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.375+0000 7fc817459700 1 --2- 172.21.15.181:0/951357472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc810160cb0 0x7fc81014d3b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.375+0000 7fc817459700 1 --2- 172.21.15.181:0/951357472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc81006b220 0x7fc8100c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.375+0000 7fc817459700 1 --2- 172.21.15.181:0/951357472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc81015c8c0 0x7fc8100c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.375+0000 7fc817459700 1 -- 172.21.15.181:0/951357472 >> 172.21.15.181:0/951357472 conn(0x7fc81006c520 msgr2=0x7fc8100b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:27.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.375+0000 7fc817459700 1 -- 172.21.15.181:0/951357472 shutdown_connections 2023-11-28T12:01:27.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.375+0000 7fc817459700 1 -- 172.21.15.181:0/951357472 wait complete. 2023-11-28T12:01:27.388 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 7a633c56-49e1-4d74-926d-3ea86cab85ed 4 2023-11-28T12:01:27.457 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:27.454+0000 7f0c66bae240 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T12:01:27.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.510+0000 7f9d594bd700 1 Processor -- start 2023-11-28T12:01:27.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.510+0000 7f9d594bd700 1 -- start start 2023-11-28T12:01:27.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d594bd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9d540b1650 0x7f9d540b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d594bd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d540af0a0 0x7f9d5415a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d594bd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d54159ec0 con 0x7f9d541599b0 2023-11-28T12:01:27.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d594bd700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d540b1f60 con 0x7f9d540b1650 2023-11-28T12:01:27.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d594bd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5415d8a0 con 0x7f9d540af0a0 2023-11-28T12:01:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d540af0a0 0x7f9d5415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d527fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9d541599b0 0x7f9d5415d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40966/0 (socket says 172.21.15.181:40966) 2023-11-28T12:01:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d527fc700 1 -- 172.21.15.181:0/4260727549 learned_addr learned my addr 172.21.15.181:0/4260727549 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-11-28T12:01:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d52ffd700 1 --2- 172.21.15.181:0/4260727549 >> v2:172.21.15.57:3300/0 conn(0x7f9d540b1650 0x7f9d540b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d51ffb700 1 -- 172.21.15.181:0/4260727549 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447411922 0 0) 0x7f9d54159ec0 con 0x7f9d541599b0 2023-11-28T12:01:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d51ffb700 1 -- 172.21.15.181:0/4260727549 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d3c0035b0 con 0x7f9d541599b0 2023-11-28T12:01:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d537fe700 1 -- 172.21.15.181:0/4260727549 >> v2:172.21.15.57:3300/0 conn(0x7f9d540b1650 msgr2=0x7f9d540b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d537fe700 1 --2- 172.21.15.181:0/4260727549 >> v2:172.21.15.57:3300/0 conn(0x7f9d540b1650 0x7f9d540b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d537fe700 1 -- 172.21.15.181:0/4260727549 >> v1:172.21.15.181:6789/0 conn(0x7f9d541599b0 legacy=0x7f9d5415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d537fe700 1 -- 172.21.15.181:0/4260727549 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5415db80 con 0x7f9d540af0a0 2023-11-28T12:01:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.511+0000 7f9d537fe700 1 --2- 172.21.15.181:0/4260727549 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d540af0a0 0x7f9d5415a350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009d90 tx=0x7f9d4c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a1cea529844f3e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:27.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.512+0000 7f9d51ffb700 1 -- 172.21.15.181:0/4260727549 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00e070 con 0x7f9d540af0a0 2023-11-28T12:01:27.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.512+0000 7f9d51ffb700 1 -- 172.21.15.181:0/4260727549 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c00bb20 con 0x7f9d540af0a0 2023-11-28T12:01:27.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.512+0000 7f9d51ffb700 1 -- 172.21.15.181:0/4260727549 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c012430 con 0x7f9d540af0a0 2023-11-28T12:01:27.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.512+0000 7f9d594bd700 1 -- 172.21.15.181:0/4260727549 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d540af0a0 msgr2=0x7f9d5415a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.512+0000 7f9d594bd700 1 --2- 172.21.15.181:0/4260727549 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d540af0a0 0x7f9d5415a350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009d90 tx=0x7f9d4c00b330 comp rx=0 tx=0).stop 2023-11-28T12:01:27.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.512+0000 7f9d594bd700 1 -- 172.21.15.181:0/4260727549 shutdown_connections 2023-11-28T12:01:27.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.512+0000 7f9d594bd700 1 --2- 172.21.15.181:0/4260727549 >> v2:172.21.15.57:3300/0 conn(0x7f9d540b1650 0x7f9d540b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.512+0000 7f9d594bd700 1 --2- 172.21.15.181:0/4260727549 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d540af0a0 0x7f9d5415a350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.512+0000 7f9d594bd700 1 -- 172.21.15.181:0/4260727549 >> 172.21.15.181:0/4260727549 conn(0x7f9d5406c010 msgr2=0x7f9d540cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:27.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.512+0000 7f9d594bd700 1 -- 172.21.15.181:0/4260727549 shutdown_connections 2023-11-28T12:01:27.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.512+0000 7f9d594bd700 1 -- 172.21.15.181:0/4260727549 wait complete. 2023-11-28T12:01:27.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.514+0000 7f9d594bd700 1 Processor -- start 2023-11-28T12:01:27.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.514+0000 7f9d594bd700 1 -- start start 2023-11-28T12:01:27.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.514+0000 7f9d594bd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540af0a0 0x7f9d5415c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.514+0000 7f9d594bd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d540b1650 0x7f9d5415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.514+0000 7f9d594bd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d5414dd00 0x7f9d5415b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.514+0000 7f9d594bd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5415c400 con 0x7f9d540b1650 2023-11-28T12:01:27.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.514+0000 7f9d594bd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5415b780 con 0x7f9d540af0a0 2023-11-28T12:01:27.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.514+0000 7f9d594bd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5415ba60 con 0x7f9d5414dd00 2023-11-28T12:01:27.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d52ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540af0a0 0x7f9d5415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d52ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540af0a0 0x7f9d5415c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42376/0 (socket says 172.21.15.181:42376) 2023-11-28T12:01:27.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d52ffd700 1 -- 172.21.15.181:0/2410286408 learned_addr learned my addr 172.21.15.181:0/2410286408 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-11-28T12:01:27.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d527fc700 1 --2- 172.21.15.181:0/2410286408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d540b1650 0x7f9d5415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d537fe700 1 --2- 172.21.15.181:0/2410286408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d5414dd00 0x7f9d5415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d52ffd700 1 -- 172.21.15.181:0/2410286408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d5414dd00 msgr2=0x7f9d5415b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d52ffd700 1 --2- 172.21.15.181:0/2410286408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d5414dd00 0x7f9d5415b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d52ffd700 1 -- 172.21.15.181:0/2410286408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d540b1650 msgr2=0x7f9d5415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d52ffd700 1 --2- 172.21.15.181:0/2410286408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d540b1650 0x7f9d5415ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d52ffd700 1 -- 172.21.15.181:0/2410286408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c009a70 con 0x7f9d540af0a0 2023-11-28T12:01:27.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d527fc700 1 --2- 172.21.15.181:0/2410286408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d540b1650 0x7f9d5415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:27.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d52ffd700 1 --2- 172.21.15.181:0/2410286408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540af0a0 0x7f9d5415c820 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400e870 tx=0x7f9d4400eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:27.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d3b7fe700 1 -- 172.21.15.181:0/2410286408 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44009590 con 0x7f9d540af0a0 2023-11-28T12:01:27.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d3b7fe700 1 -- 172.21.15.181:0/2410286408 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d44013680 con 0x7f9d540af0a0 2023-11-28T12:01:27.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d3b7fe700 1 -- 172.21.15.181:0/2410286408 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d440139f0 con 0x7f9d540af0a0 2023-11-28T12:01:27.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d594bd700 1 -- 172.21.15.181:0/2410286408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5415a890 con 0x7f9d540af0a0 2023-11-28T12:01:27.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d594bd700 1 -- 172.21.15.181:0/2410286408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5415de60 con 0x7f9d540af0a0 2023-11-28T12:01:27.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.515+0000 7f9d3b7fe700 1 -- 172.21.15.181:0/2410286408 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f9d44019030 con 0x7f9d540af0a0 2023-11-28T12:01:27.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.516+0000 7f9d3b7fe700 1 -- 172.21.15.181:0/2410286408 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f9d44006620 con 0x7f9d540af0a0 2023-11-28T12:01:27.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.516+0000 7f9d594bd700 1 -- 172.21.15.181:0/2410286408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5415aa20 con 0x7f9d540af0a0 2023-11-28T12:01:27.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.519+0000 7f9d3b7fe700 1 -- 172.21.15.181:0/2410286408 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f9d44021060 con 0x7f9d540af0a0 2023-11-28T12:01:27.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.602+0000 7f9d594bd700 1 -- 172.21.15.181:0/2410286408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7a633c56-49e1-4d74-926d-3ea86cab85ed", "id": 4} v 0) v1 -- 0x7f9d5415c670 con 0x7f9d540af0a0 2023-11-28T12:01:27.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.606+0000 7f9d3b7fe700 1 -- 172.21.15.181:0/2410286408 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7a633c56-49e1-4d74-926d-3ea86cab85ed", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9d44017070 con 0x7f9d540af0a0 2023-11-28T12:01:27.608 INFO:teuthology.orchestra.run.smithi181.stdout:4 2023-11-28T12:01:27.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.607+0000 7f9d594bd700 1 -- 172.21.15.181:0/2410286408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540af0a0 msgr2=0x7f9d5415c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.607+0000 7f9d594bd700 1 --2- 172.21.15.181:0/2410286408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540af0a0 0x7f9d5415c820 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400e870 tx=0x7f9d4400eb80 comp rx=0 tx=0).stop 2023-11-28T12:01:27.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.607+0000 7f9d594bd700 1 -- 172.21.15.181:0/2410286408 shutdown_connections 2023-11-28T12:01:27.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.607+0000 7f9d594bd700 1 --2- 172.21.15.181:0/2410286408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d5414dd00 0x7f9d5415b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.607+0000 7f9d594bd700 1 --2- 172.21.15.181:0/2410286408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540af0a0 0x7f9d5415c820 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.607+0000 7f9d594bd700 1 --2- 172.21.15.181:0/2410286408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d540b1650 0x7f9d5415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.607+0000 7f9d594bd700 1 -- 172.21.15.181:0/2410286408 >> 172.21.15.181:0/2410286408 conn(0x7f9d5406c010 msgr2=0x7f9d540af710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:27.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.607+0000 7f9d594bd700 1 -- 172.21.15.181:0/2410286408 shutdown_connections 2023-11-28T12:01:27.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.608+0000 7f9d594bd700 1 -- 172.21.15.181:0/2410286408 wait complete. 2023-11-28T12:01:27.620 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 25161636-254d-4a76-abac-21f688eca3a2 5 2023-11-28T12:01:27.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.739+0000 7fd4ba884700 1 Processor -- start 2023-11-28T12:01:27.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.740+0000 7fd4ba884700 1 -- start start 2023-11-28T12:01:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.741+0000 7fd4ba884700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b40d0ad0 0x7fd4b40cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.741+0000 7fd4ba884700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b40cc740 con 0x7fd4b415a5f0 2023-11-28T12:01:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.741+0000 7fd4ba884700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b415a9c0 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.741+0000 7fd4ba884700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40c7b80 con 0x7fd4b40d0ad0 2023-11-28T12:01:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.741+0000 7fd4abfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd4b40b2ee0 0x7fd4b40ced30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40976/0 (socket says 172.21.15.181:40976) 2023-11-28T12:01:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.741+0000 7fd4b8e21700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b40d0ad0 0x7fd4b40cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.741+0000 7fd4abfff700 1 -- 172.21.15.181:0/3753976398 learned_addr learned my addr 172.21.15.181:0/3753976398 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-11-28T12:01:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.741+0000 7fd4b37fe700 1 -- 172.21.15.181:0/3753976398 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173524449 0 0) 0x7fd4b415a9c0 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.741+0000 7fd4b37fe700 1 -- 172.21.15.181:0/3753976398 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4a00035b0 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.741+0000 7fd4b8e21700 1 -- 172.21.15.181:0/3753976398 >> v1:172.21.15.181:6789/0 conn(0x7fd4b40b2ee0 legacy=0x7fd4b40ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.741+0000 7fd4b8e21700 1 -- 172.21.15.181:0/3753976398 >> v1:172.21.15.57:6789/0 conn(0x7fd4b415a5f0 legacy=0x7fd4b40b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.741+0000 7fd4b8e21700 1 -- 172.21.15.181:0/3753976398 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b40cc990 con 0x7fd4b40d0ad0 2023-11-28T12:01:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.741+0000 7fd4b8e21700 1 --2- 172.21.15.181:0/3753976398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b40d0ad0 0x7fd4b40cc200 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009dc0 tx=0x7fd4ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c69c7d7e7d775e48 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:27.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.742+0000 7fd4b37fe700 1 -- 172.21.15.181:0/3753976398 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac00e070 con 0x7fd4b40d0ad0 2023-11-28T12:01:27.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.742+0000 7fd4b37fe700 1 -- 172.21.15.181:0/3753976398 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ac00bb20 con 0x7fd4b40d0ad0 2023-11-28T12:01:27.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.742+0000 7fd4b37fe700 1 -- 172.21.15.181:0/3753976398 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac00be90 con 0x7fd4b40d0ad0 2023-11-28T12:01:27.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.742+0000 7fd4ba884700 1 -- 172.21.15.181:0/3753976398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b40d0ad0 msgr2=0x7fd4b40cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.742+0000 7fd4ba884700 1 --2- 172.21.15.181:0/3753976398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b40d0ad0 0x7fd4b40cc200 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009dc0 tx=0x7fd4ac00b330 comp rx=0 tx=0).stop 2023-11-28T12:01:27.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.743+0000 7fd4ba884700 1 -- 172.21.15.181:0/3753976398 shutdown_connections 2023-11-28T12:01:27.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.743+0000 7fd4ba884700 1 --2- 172.21.15.181:0/3753976398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b40d0ad0 0x7fd4b40cc200 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.743+0000 7fd4ba884700 1 -- 172.21.15.181:0/3753976398 >> 172.21.15.181:0/3753976398 conn(0x7fd4b406ea80 msgr2=0x7fd4b406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:27.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.743+0000 7fd4ba884700 1 -- 172.21.15.181:0/3753976398 shutdown_connections 2023-11-28T12:01:27.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.743+0000 7fd4ba884700 1 -- 172.21.15.181:0/3753976398 wait complete. 2023-11-28T12:01:27.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.744+0000 7fd4ba884700 1 Processor -- start 2023-11-28T12:01:27.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.744+0000 7fd4ba884700 1 -- start start 2023-11-28T12:01:27.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4ba884700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b40b2ee0 0x7fd4b414b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4ba884700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4b40d0ad0 0x7fd4b40c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4ba884700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4b415a5f0 0x7fd4b40c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4ba884700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40b26c0 con 0x7fd4b40d0ad0 2023-11-28T12:01:27.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4ba884700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b414b7b0 con 0x7fd4b415a5f0 2023-11-28T12:01:27.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4ba884700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40b1a40 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4b3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b40b2ee0 0x7fd4b414b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4b8e21700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4b415a5f0 0x7fd4b40c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4b3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b40b2ee0 0x7fd4b414b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42234/0 (socket says 172.21.15.181:42234) 2023-11-28T12:01:27.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4b3fff700 1 -- 172.21.15.181:0/884079822 learned_addr learned my addr 172.21.15.181:0/884079822 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-11-28T12:01:27.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4abfff700 1 --2- 172.21.15.181:0/884079822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4b40d0ad0 0x7fd4b40c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4b3fff700 1 -- 172.21.15.181:0/884079822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4b415a5f0 msgr2=0x7fd4b40c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4b3fff700 1 --2- 172.21.15.181:0/884079822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4b415a5f0 0x7fd4b40c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4b3fff700 1 -- 172.21.15.181:0/884079822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4b40d0ad0 msgr2=0x7fd4b40c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4b3fff700 1 --2- 172.21.15.181:0/884079822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4b40d0ad0 0x7fd4b40c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4b3fff700 1 -- 172.21.15.181:0/884079822 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac009a70 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.745+0000 7fd4b8e21700 1 --2- 172.21.15.181:0/884079822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4b415a5f0 0x7fd4b40c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:27.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.746+0000 7fd4b3fff700 1 --2- 172.21.15.181:0/884079822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b40b2ee0 0x7fd4b414b270 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400c510 tx=0x7fd4a400c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:27.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.746+0000 7fd4b17fa700 1 -- 172.21.15.181:0/884079822 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a40052d0 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.746+0000 7fd4b17fa700 1 -- 172.21.15.181:0/884079822 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a4005430 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.746+0000 7fd4b17fa700 1 -- 172.21.15.181:0/884079822 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a40038c0 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.746+0000 7fd4ba884700 1 -- 172.21.15.181:0/884079822 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b40ccf50 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.746+0000 7fd4ba884700 1 -- 172.21.15.181:0/884079822 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b40ccc70 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.746+0000 7fd4b17fa700 1 -- 172.21.15.181:0/884079822 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fd4a4019030 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.746+0000 7fd4b17fa700 1 -- 172.21.15.181:0/884079822 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fd4a4011d40 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.747+0000 7fd4ba884700 1 -- 172.21.15.181:0/884079822 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b40cd230 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.751+0000 7fd4b17fa700 1 -- 172.21.15.181:0/884079822 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fd4a4017070 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.834+0000 7fd4ba884700 1 -- 172.21.15.181:0/884079822 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd new", "uuid": "25161636-254d-4a76-abac-21f688eca3a2", "id": 5} v 0) v1 -- 0x7fd4b40b29e0 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.838+0000 7fd4b17fa700 1 -- 172.21.15.181:0/884079822 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "25161636-254d-4a76-abac-21f688eca3a2", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd4a40117c0 con 0x7fd4b40b2ee0 2023-11-28T12:01:27.839 INFO:teuthology.orchestra.run.smithi181.stdout:5 2023-11-28T12:01:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.839+0000 7fd4ba884700 1 -- 172.21.15.181:0/884079822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b40b2ee0 msgr2=0x7fd4b414b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.839+0000 7fd4ba884700 1 --2- 172.21.15.181:0/884079822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b40b2ee0 0x7fd4b414b270 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400c510 tx=0x7fd4a400c820 comp rx=0 tx=0).stop 2023-11-28T12:01:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.839+0000 7fd4ba884700 1 -- 172.21.15.181:0/884079822 shutdown_connections 2023-11-28T12:01:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.839+0000 7fd4ba884700 1 --2- 172.21.15.181:0/884079822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4b415a5f0 0x7fd4b40c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.839+0000 7fd4ba884700 1 --2- 172.21.15.181:0/884079822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b40b2ee0 0x7fd4b414b270 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.839+0000 7fd4ba884700 1 --2- 172.21.15.181:0/884079822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4b40d0ad0 0x7fd4b40c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.839+0000 7fd4ba884700 1 -- 172.21.15.181:0/884079822 >> 172.21.15.181:0/884079822 conn(0x7fd4b406ea80 msgr2=0x7fd4b4157e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:27.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.839+0000 7fd4ba884700 1 -- 172.21.15.181:0/884079822 shutdown_connections 2023-11-28T12:01:27.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.839+0000 7fd4ba884700 1 -- 172.21.15.181:0/884079822 wait complete. 2023-11-28T12:01:27.852 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 61c58456-c1bd-479b-902f-4db772993c6b 6 2023-11-28T12:01:27.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.972+0000 7f7a7deb5700 1 Processor -- start 2023-11-28T12:01:27.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.972+0000 7f7a7deb5700 1 -- start start 2023-11-28T12:01:27.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.972+0000 7f7a7deb5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7806a980 0x7f7a780cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.972+0000 7f7a7deb5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7a780cdd50 0x7f7a780cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.972+0000 7f7a7deb5700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7815e0d0 con 0x7f7a7815a4d0 2023-11-28T12:01:27.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.972+0000 7f7a7deb5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a780ce560 con 0x7f7a780cdd50 2023-11-28T12:01:27.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.972+0000 7f7a7deb5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7815e320 con 0x7f7a7806a980 2023-11-28T12:01:27.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a76ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7806a980 0x7f7a780cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a77fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7a780cdd50 0x7f7a780cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a76ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7806a980 0x7f7a780cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42244/0 (socket says 172.21.15.181:42244) 2023-11-28T12:01:27.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a77fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7a780cdd50 0x7f7a780cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42394/0 (socket says 172.21.15.181:42394) 2023-11-28T12:01:27.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a76ffd700 1 -- 172.21.15.181:0/839623874 learned_addr learned my addr 172.21.15.181:0/839623874 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-11-28T12:01:27.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a76ffd700 1 -- 172.21.15.181:0/839623874 >> v2:172.21.15.181:3300/0 conn(0x7f7a780cdd50 msgr2=0x7f7a780cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:27.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a76ffd700 1 --2- 172.21.15.181:0/839623874 >> v2:172.21.15.181:3300/0 conn(0x7f7a780cdd50 0x7f7a780cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a76ffd700 1 -- 172.21.15.181:0/839623874 >> v1:172.21.15.57:6789/0 conn(0x7f7a7815a4d0 legacy=0x7f7a7806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a76ffd700 1 -- 172.21.15.181:0/839623874 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7815e600 con 0x7f7a7806a980 2023-11-28T12:01:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a77fff700 1 --2- 172.21.15.181:0/839623874 >> v2:172.21.15.181:3300/0 conn(0x7f7a780cdd50 0x7f7a780cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a76ffd700 1 --2- 172.21.15.181:0/839623874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7806a980 0x7f7a780cd810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a600098f0 tx=0x7f7a60009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14665d4c47557431 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a767fc700 1 -- 172.21.15.181:0/839623874 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60011830 con 0x7f7a7806a980 2023-11-28T12:01:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a767fc700 1 -- 172.21.15.181:0/839623874 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a60011990 con 0x7f7a7806a980 2023-11-28T12:01:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.973+0000 7f7a767fc700 1 -- 172.21.15.181:0/839623874 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60011d20 con 0x7f7a7806a980 2023-11-28T12:01:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.974+0000 7f7a7deb5700 1 -- 172.21.15.181:0/839623874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7806a980 msgr2=0x7f7a780cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.974+0000 7f7a7deb5700 1 --2- 172.21.15.181:0/839623874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7806a980 0x7f7a780cd810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a600098f0 tx=0x7f7a60009c00 comp rx=0 tx=0).stop 2023-11-28T12:01:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.974+0000 7f7a7deb5700 1 -- 172.21.15.181:0/839623874 shutdown_connections 2023-11-28T12:01:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.974+0000 7f7a7deb5700 1 --2- 172.21.15.181:0/839623874 >> v2:172.21.15.181:3300/0 conn(0x7f7a780cdd50 0x7f7a780cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.974+0000 7f7a7deb5700 1 --2- 172.21.15.181:0/839623874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7806a980 0x7f7a780cd810 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.974+0000 7f7a7deb5700 1 -- 172.21.15.181:0/839623874 >> 172.21.15.181:0/839623874 conn(0x7f7a7806c010 msgr2=0x7f7a780cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.974+0000 7f7a7deb5700 1 -- 172.21.15.181:0/839623874 shutdown_connections 2023-11-28T12:01:27.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.974+0000 7f7a7deb5700 1 -- 172.21.15.181:0/839623874 wait complete. 2023-11-28T12:01:27.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.975+0000 7f7a7deb5700 1 Processor -- start 2023-11-28T12:01:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.975+0000 7f7a7deb5700 1 -- start start 2023-11-28T12:01:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a7deb5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a7806a980 0x7f7a781eff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a7deb5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a780cdd50 0x7f7a780c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a7deb5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7815a4d0 0x7f7a78153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a7deb5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780c8c10 con 0x7f7a7806a980 2023-11-28T12:01:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a7deb5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780c7f90 con 0x7f7a780cdd50 2023-11-28T12:01:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a7deb5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780c8270 con 0x7f7a7815a4d0 2023-11-28T12:01:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a77fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7815a4d0 0x7f7a78153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a76ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a780cdd50 0x7f7a780c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a77fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7815a4d0 0x7f7a78153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42260/0 (socket says 172.21.15.181:42260) 2023-11-28T12:01:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a77fff700 1 -- 172.21.15.181:0/3322532440 learned_addr learned my addr 172.21.15.181:0/3322532440 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-11-28T12:01:27.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a777fe700 1 --2- 172.21.15.181:0/3322532440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a7806a980 0x7f7a781eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a77fff700 1 -- 172.21.15.181:0/3322532440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a780cdd50 msgr2=0x7f7a780c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a77fff700 1 --2- 172.21.15.181:0/3322532440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a780cdd50 0x7f7a780c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a77fff700 1 -- 172.21.15.181:0/3322532440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a7806a980 msgr2=0x7f7a781eff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a77fff700 1 --2- 172.21.15.181:0/3322532440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a7806a980 0x7f7a781eff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a77fff700 1 -- 172.21.15.181:0/3322532440 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a68009a70 con 0x7f7a7815a4d0 2023-11-28T12:01:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a777fe700 1 --2- 172.21.15.181:0/3322532440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a7806a980 0x7f7a781eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.976+0000 7f7a77fff700 1 --2- 172.21.15.181:0/3322532440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7815a4d0 0x7f7a78153e70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800efc0 tx=0x7f7a6800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.977+0000 7f7a6ffff700 1 -- 172.21.15.181:0/3322532440 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800cb40 con 0x7f7a7815a4d0 2023-11-28T12:01:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.977+0000 7f7a6ffff700 1 -- 172.21.15.181:0/3322532440 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6800f040 con 0x7f7a7815a4d0 2023-11-28T12:01:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.977+0000 7f7a7deb5700 1 -- 172.21.15.181:0/3322532440 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a60009550 con 0x7f7a7815a4d0 2023-11-28T12:01:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.977+0000 7f7a6ffff700 1 -- 172.21.15.181:0/3322532440 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68003a20 con 0x7f7a7815a4d0 2023-11-28T12:01:27.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.977+0000 7f7a7deb5700 1 -- 172.21.15.181:0/3322532440 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7815a8a0 con 0x7f7a7815a4d0 2023-11-28T12:01:27.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.977+0000 7f7a6ffff700 1 -- 172.21.15.181:0/3322532440 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f7a68014480 con 0x7f7a7815a4d0 2023-11-28T12:01:27.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.977+0000 7f7a6ffff700 1 -- 172.21.15.181:0/3322532440 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f7a6801daf0 con 0x7f7a7815a4d0 2023-11-28T12:01:27.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.978+0000 7f7a7deb5700 1 -- 172.21.15.181:0/3322532440 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7815e8e0 con 0x7f7a7815a4d0 2023-11-28T12:01:27.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:27.982+0000 7f7a6ffff700 1 -- 172.21.15.181:0/3322532440 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f7a68018030 con 0x7f7a7815a4d0 2023-11-28T12:01:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.064+0000 7f7a7deb5700 1 -- 172.21.15.181:0/3322532440 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd new", "uuid": "61c58456-c1bd-479b-902f-4db772993c6b", "id": 6} v 0) v1 -- 0x7f7a780743f0 con 0x7f7a7815a4d0 2023-11-28T12:01:28.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.069+0000 7f7a6ffff700 1 -- 172.21.15.181:0/3322532440 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "61c58456-c1bd-479b-902f-4db772993c6b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7a6801d500 con 0x7f7a7815a4d0 2023-11-28T12:01:28.070 INFO:teuthology.orchestra.run.smithi181.stdout:6 2023-11-28T12:01:28.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.070+0000 7f7a7deb5700 1 -- 172.21.15.181:0/3322532440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7815a4d0 msgr2=0x7f7a78153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:28.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.070+0000 7f7a7deb5700 1 --2- 172.21.15.181:0/3322532440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7815a4d0 0x7f7a78153e70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800efc0 tx=0x7f7a6800c3a0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.070+0000 7f7a7deb5700 1 -- 172.21.15.181:0/3322532440 shutdown_connections 2023-11-28T12:01:28.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.070+0000 7f7a7deb5700 1 --2- 172.21.15.181:0/3322532440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a7815a4d0 0x7f7a78153e70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.070+0000 7f7a7deb5700 1 --2- 172.21.15.181:0/3322532440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a7806a980 0x7f7a781eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.070+0000 7f7a7deb5700 1 --2- 172.21.15.181:0/3322532440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a780cdd50 0x7f7a780c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.070+0000 7f7a7deb5700 1 -- 172.21.15.181:0/3322532440 >> 172.21.15.181:0/3322532440 conn(0x7f7a7806c010 msgr2=0x7f7a780cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:28.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.070+0000 7f7a7deb5700 1 -- 172.21.15.181:0/3322532440 shutdown_connections 2023-11-28T12:01:28.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.070+0000 7f7a7deb5700 1 -- 172.21.15.181:0/3322532440 wait complete. 2023-11-28T12:01:28.084 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new e986e5f1-3003-4c66-9b57-d0d8f8121bd6 7 2023-11-28T12:01:28.127 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:28.126+0000 7f3f7df0a240 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T12:01:28.202 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:28.201+0000 7f3f7df0a240 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T12:01:28.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.204+0000 7f8f5a9ca700 1 Processor -- start 2023-11-28T12:01:28.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.204+0000 7f8f5a9ca700 1 -- start start 2023-11-28T12:01:28.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.204+0000 7f8f5a9ca700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8f540cf710 0x7f8f540ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:28.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.204+0000 7f8f5a9ca700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5406a980 0x7f8f54071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:28.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.204+0000 7f8f5a9ca700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f5415cc10 con 0x7f8f54071890 2023-11-28T12:01:28.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.204+0000 7f8f5a9ca700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5406c9b0 con 0x7f8f540cf710 2023-11-28T12:01:28.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f5a9ca700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5406cc90 con 0x7f8f5406a980 2023-11-28T12:01:28.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f58f67700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5406a980 0x7f8f54071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:28.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f537fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8f54071890 0x7f8f5406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40982/0 (socket says 172.21.15.181:40982) 2023-11-28T12:01:28.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f537fe700 1 -- 172.21.15.181:0/598969614 learned_addr learned my addr 172.21.15.181:0/598969614 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-11-28T12:01:28.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f53fff700 1 --2- 172.21.15.181:0/598969614 >> v2:172.21.15.57:3300/0 conn(0x7f8f540cf710 0x7f8f540ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:28.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f58f67700 1 -- 172.21.15.181:0/598969614 >> v2:172.21.15.57:3300/0 conn(0x7f8f540cf710 msgr2=0x7f8f540ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:28.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f58f67700 1 --2- 172.21.15.181:0/598969614 >> v2:172.21.15.57:3300/0 conn(0x7f8f540cf710 0x7f8f540ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f58f67700 1 -- 172.21.15.181:0/598969614 >> v1:172.21.15.181:6789/0 conn(0x7f8f54071890 legacy=0x7f8f5406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:28.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f52ffd700 1 -- 172.21.15.181:0/598969614 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248739953 0 0) 0x7f8f5415cc10 con 0x7f8f54071890 2023-11-28T12:01:28.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f58f67700 1 -- 172.21.15.181:0/598969614 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5406cf70 con 0x7f8f5406a980 2023-11-28T12:01:28.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f58f67700 1 --2- 172.21.15.181:0/598969614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5406a980 0x7f8f54071220 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009dc0 tx=0x7f8f4c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a02479def4eac745 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:28.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f52ffd700 1 -- 172.21.15.181:0/598969614 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c00e070 con 0x7f8f5406a980 2023-11-28T12:01:28.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f52ffd700 1 -- 172.21.15.181:0/598969614 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4c00bb90 con 0x7f8f5406a980 2023-11-28T12:01:28.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.205+0000 7f8f52ffd700 1 -- 172.21.15.181:0/598969614 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c012430 con 0x7f8f5406a980 2023-11-28T12:01:28.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.206+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/598969614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5406a980 msgr2=0x7f8f54071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:28.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.206+0000 7f8f5a9ca700 1 --2- 172.21.15.181:0/598969614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5406a980 0x7f8f54071220 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009dc0 tx=0x7f8f4c00b330 comp rx=0 tx=0).stop 2023-11-28T12:01:28.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.206+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/598969614 shutdown_connections 2023-11-28T12:01:28.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.206+0000 7f8f5a9ca700 1 --2- 172.21.15.181:0/598969614 >> v2:172.21.15.57:3300/0 conn(0x7f8f540cf710 0x7f8f540ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.206+0000 7f8f5a9ca700 1 --2- 172.21.15.181:0/598969614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5406a980 0x7f8f54071220 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.206+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/598969614 >> 172.21.15.181:0/598969614 conn(0x7f8f5406c010 msgr2=0x7f8f540cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:28.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.206+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/598969614 shutdown_connections 2023-11-28T12:01:28.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.206+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/598969614 wait complete. 2023-11-28T12:01:28.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.207+0000 7f8f5a9ca700 1 Processor -- start 2023-11-28T12:01:28.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.207+0000 7f8f5a9ca700 1 -- start start 2023-11-28T12:01:28.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f5a9ca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5406a980 0x7f8f5414aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:28.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f5a9ca700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f54071890 0x7f8f540ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:28.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f5a9ca700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f540cf710 0x7f8f540ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:28.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f5a9ca700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f540cce40 con 0x7f8f540cf710 2023-11-28T12:01:28.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f5a9ca700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f540c7130 con 0x7f8f5406a980 2023-11-28T12:01:28.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f5a9ca700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f540cc880 con 0x7f8f54071890 2023-11-28T12:01:28.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f54071890 0x7f8f540ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:28.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f54071890 0x7f8f540ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:42276/0 (socket says 172.21.15.181:42276) 2023-11-28T12:01:28.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f537fe700 1 -- 172.21.15.181:0/893137091 learned_addr learned my addr 172.21.15.181:0/893137091 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-11-28T12:01:28.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f53fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5406a980 0x7f8f5414aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:28.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f58f67700 1 --2- 172.21.15.181:0/893137091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f540cf710 0x7f8f540ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:28.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f53fff700 1 -- 172.21.15.181:0/893137091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f54071890 msgr2=0x7f8f540ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:28.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f53fff700 1 --2- 172.21.15.181:0/893137091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f54071890 0x7f8f540ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f53fff700 1 -- 172.21.15.181:0/893137091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f540cf710 msgr2=0x7f8f540ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:28.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f53fff700 1 --2- 172.21.15.181:0/893137091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f540cf710 0x7f8f540ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f53fff700 1 -- 172.21.15.181:0/893137091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c009a70 con 0x7f8f5406a980 2023-11-28T12:01:28.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f537fe700 1 --2- 172.21.15.181:0/893137091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f54071890 0x7f8f540ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:28.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.208+0000 7f8f53fff700 1 --2- 172.21.15.181:0/893137091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5406a980 0x7f8f5414aef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400d5c0 tx=0x7f8f4400d8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:28.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.209+0000 7f8f50ff9700 1 -- 172.21.15.181:0/893137091 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f440125f0 con 0x7f8f5406a980 2023-11-28T12:01:28.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.209+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/893137091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5406d530 con 0x7f8f5406a980 2023-11-28T12:01:28.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.209+0000 7f8f50ff9700 1 -- 172.21.15.181:0/893137091 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f44012750 con 0x7f8f5406a980 2023-11-28T12:01:28.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.209+0000 7f8f50ff9700 1 -- 172.21.15.181:0/893137091 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44012ae0 con 0x7f8f5406a980 2023-11-28T12:01:28.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.209+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/893137091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5406d250 con 0x7f8f5406a980 2023-11-28T12:01:28.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.209+0000 7f8f50ff9700 1 -- 172.21.15.181:0/893137091 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f8f44013480 con 0x7f8f5406a980 2023-11-28T12:01:28.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.209+0000 7f8f50ff9700 1 -- 172.21.15.181:0/893137091 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f8f4401dbb0 con 0x7f8f5406a980 2023-11-28T12:01:28.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.209+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/893137091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f54160480 con 0x7f8f5406a980 2023-11-28T12:01:28.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.212+0000 7f8f50ff9700 1 -- 172.21.15.181:0/893137091 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f8f4401d550 con 0x7f8f5406a980 2023-11-28T12:01:28.285 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:28.283+0000 7f0c66bae240 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T12:01:28.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.295+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/893137091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e986e5f1-3003-4c66-9b57-d0d8f8121bd6", "id": 7} v 0) v1 -- 0x7f8f540743f0 con 0x7f8f5406a980 2023-11-28T12:01:28.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.299+0000 7f8f50ff9700 1 -- 172.21.15.181:0/893137091 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e986e5f1-3003-4c66-9b57-d0d8f8121bd6", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8f4402c0b0 con 0x7f8f5406a980 2023-11-28T12:01:28.300 INFO:teuthology.orchestra.run.smithi181.stdout:7 2023-11-28T12:01:28.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.300+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/893137091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5406a980 msgr2=0x7f8f5414aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:28.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.300+0000 7f8f5a9ca700 1 --2- 172.21.15.181:0/893137091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5406a980 0x7f8f5414aef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400d5c0 tx=0x7f8f4400d8d0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.300+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/893137091 shutdown_connections 2023-11-28T12:01:28.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.300+0000 7f8f5a9ca700 1 --2- 172.21.15.181:0/893137091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f540cf710 0x7f8f540ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.300+0000 7f8f5a9ca700 1 --2- 172.21.15.181:0/893137091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5406a980 0x7f8f5414aef0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.300+0000 7f8f5a9ca700 1 --2- 172.21.15.181:0/893137091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f54071890 0x7f8f540ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.300+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/893137091 >> 172.21.15.181:0/893137091 conn(0x7f8f5406c010 msgr2=0x7f8f540b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:28.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.300+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/893137091 shutdown_connections 2023-11-28T12:01:28.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-11-28T12:01:28.300+0000 7f8f5a9ca700 1 -- 172.21.15.181:0/893137091 wait complete. 2023-11-28T12:01:28.313 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-28T12:01:28.313 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-28T12:01:28.315 INFO:tasks.ceph.osd.0:Started 2023-11-28T12:01:28.315 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-28T12:01:28.316 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-28T12:01:28.319 INFO:tasks.ceph.osd.1:Started 2023-11-28T12:01:28.320 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-28T12:01:28.320 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-28T12:01:28.322 INFO:tasks.ceph.osd.2:Started 2023-11-28T12:01:28.322 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-28T12:01:28.323 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-28T12:01:28.325 INFO:tasks.ceph.osd.3:Started 2023-11-28T12:01:28.325 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-28T12:01:28.325 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-28T12:01:28.327 INFO:tasks.ceph.osd.4:Started 2023-11-28T12:01:28.327 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-28T12:01:28.327 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-28T12:01:28.329 INFO:tasks.ceph.osd.5:Started 2023-11-28T12:01:28.330 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-28T12:01:28.330 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-28T12:01:28.332 INFO:tasks.ceph.osd.6:Started 2023-11-28T12:01:28.332 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-28T12:01:28.332 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-28T12:01:28.334 INFO:tasks.ceph.osd.7:Started 2023-11-28T12:01:28.334 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:01:28.351 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:28.350+0000 7f3f7df0a240 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T12:01:28.384 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:28.383+0000 7f0c66bae240 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T12:01:28.435 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:28.434+0000 7f3f7df0a240 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T12:01:28.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.502+0000 7f43bdfb9700 1 Processor -- start 2023-11-28T12:01:28.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.502+0000 7f43bdfb9700 1 -- start start 2023-11-28T12:01:28.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.503+0000 7f43bdfb9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b806a270 0x7f43b806a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:28.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.503+0000 7f43bdfb9700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b806ab80 con 0x7f43b815cc80 2023-11-28T12:01:28.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.503+0000 7f43bdfb9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b80b1810 con 0x7f43b806e570 2023-11-28T12:01:28.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.503+0000 7f43bdfb9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80b2490 con 0x7f43b806a270 2023-11-28T12:01:28.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.503+0000 7f43b6ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f43b815cc80 0x7f43b806de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38284/0 (socket says 172.21.15.57:38284) 2023-11-28T12:01:28.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.503+0000 7f43b6ffd700 1 -- 172.21.15.57:0/1641977517 learned_addr learned my addr 172.21.15.57:0/1641977517 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:28.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.503+0000 7f43b77fe700 1 --2- 172.21.15.57:0/1641977517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b806a270 0x7f43b806a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:28.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.503+0000 7f43b67fc700 1 -- 172.21.15.57:0/1641977517 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264860415 0 0) 0x7f43b806ab80 con 0x7f43b815cc80 2023-11-28T12:01:28.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.503+0000 7f43b67fc700 1 -- 172.21.15.57:0/1641977517 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43a00035b0 con 0x7f43b815cc80 2023-11-28T12:01:28.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.503+0000 7f43b67fc700 1 -- 172.21.15.57:0/1641977517 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691956328 0 0) 0x7f43b80b1810 con 0x7f43b806e570 2023-11-28T12:01:28.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.503+0000 7f43b67fc700 1 -- 172.21.15.57:0/1641977517 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43b806ab80 con 0x7f43b806e570 2023-11-28T12:01:28.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.503+0000 7f43b77fe700 1 -- 172.21.15.57:0/1641977517 >> v1:172.21.15.181:6789/0 conn(0x7f43b806e570 legacy=0x7f43b8070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:28.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.503+0000 7f43b77fe700 1 -- 172.21.15.57:0/1641977517 >> v1:172.21.15.57:6789/0 conn(0x7f43b815cc80 legacy=0x7f43b806de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:28.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.504+0000 7f43b77fe700 1 -- 172.21.15.57:0/1641977517 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b80b1af0 con 0x7f43b806a270 2023-11-28T12:01:28.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.504+0000 7f43b77fe700 1 --2- 172.21.15.57:0/1641977517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b806a270 0x7f43b806a640 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009dc0 tx=0x7f43a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b8208707dec7a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:28.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.504+0000 7f43b67fc700 1 -- 172.21.15.57:0/1641977517 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800e070 con 0x7f43b806a270 2023-11-28T12:01:28.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.504+0000 7f43b67fc700 1 -- 172.21.15.57:0/1641977517 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a800bb20 con 0x7f43b806a270 2023-11-28T12:01:28.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.504+0000 7f43b67fc700 1 -- 172.21.15.57:0/1641977517 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8012430 con 0x7f43b806a270 2023-11-28T12:01:28.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.505+0000 7f43bdfb9700 1 -- 172.21.15.57:0/1641977517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b806a270 msgr2=0x7f43b806a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:28.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.505+0000 7f43bdfb9700 1 --2- 172.21.15.57:0/1641977517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b806a270 0x7f43b806a640 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009dc0 tx=0x7f43a800b330 comp rx=0 tx=0).stop 2023-11-28T12:01:28.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.505+0000 7f43bdfb9700 1 -- 172.21.15.57:0/1641977517 shutdown_connections 2023-11-28T12:01:28.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.505+0000 7f43bdfb9700 1 --2- 172.21.15.57:0/1641977517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b806a270 0x7f43b806a640 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.505+0000 7f43bdfb9700 1 -- 172.21.15.57:0/1641977517 >> 172.21.15.57:0/1641977517 conn(0x7f43b806c010 msgr2=0x7f43b80cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:28.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.505+0000 7f43bdfb9700 1 -- 172.21.15.57:0/1641977517 shutdown_connections 2023-11-28T12:01:28.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.505+0000 7f43bdfb9700 1 -- 172.21.15.57:0/1641977517 wait complete. 2023-11-28T12:01:28.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.506+0000 7f43bdfb9700 1 Processor -- start 2023-11-28T12:01:28.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.507+0000 7f43bdfb9700 1 -- start start 2023-11-28T12:01:28.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43bdfb9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b806a270 0x7f43b80b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:28.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43bdfb9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b806e570 0x7f43b80ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:28.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43bdfb9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b815cc80 0x7f43b8152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:28.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43bdfb9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b806e9b0 con 0x7f43b806a270 2023-11-28T12:01:28.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43bdfb9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80cb980 con 0x7f43b806e570 2023-11-28T12:01:28.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43bdfb9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80cad70 con 0x7f43b815cc80 2023-11-28T12:01:28.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43b77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b806a270 0x7f43b80b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:28.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43b77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b806a270 0x7f43b80b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51992/0 (socket says 172.21.15.57:51992) 2023-11-28T12:01:28.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43b77fe700 1 -- 172.21.15.57:0/4277269976 learned_addr learned my addr 172.21.15.57:0/4277269976 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43b6ffd700 1 --2- 172.21.15.57:0/4277269976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b806e570 0x7f43b80ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43b7fff700 1 --2- 172.21.15.57:0/4277269976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b815cc80 0x7f43b8152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43b77fe700 1 -- 172.21.15.57:0/4277269976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b815cc80 msgr2=0x7f43b8152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43b77fe700 1 --2- 172.21.15.57:0/4277269976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b815cc80 0x7f43b8152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43b77fe700 1 -- 172.21.15.57:0/4277269976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b806e570 msgr2=0x7f43b80ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43b77fe700 1 --2- 172.21.15.57:0/4277269976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b806e570 0x7f43b80ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43b77fe700 1 -- 172.21.15.57:0/4277269976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a8009a70 con 0x7f43b806a270 2023-11-28T12:01:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43b6ffd700 1 --2- 172.21.15.57:0/4277269976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b806e570 0x7f43b80ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43b77fe700 1 --2- 172.21.15.57:0/4277269976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b806a270 0x7f43b80b3610 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f43a8014040 tx=0x7f43a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f439ffff700 1 -- 172.21.15.57:0/4277269976 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8012430 con 0x7f43b806a270 2023-11-28T12:01:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f439ffff700 1 -- 172.21.15.57:0/4277269976 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a8020070 con 0x7f43b806a270 2023-11-28T12:01:28.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f43bdfb9700 1 -- 172.21.15.57:0/4277269976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b80b20b0 con 0x7f43b806a270 2023-11-28T12:01:28.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.508+0000 7f439ffff700 1 -- 172.21.15.57:0/4277269976 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a801ba70 con 0x7f43b806a270 2023-11-28T12:01:28.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.509+0000 7f43bdfb9700 1 -- 172.21.15.57:0/4277269976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b80b1dd0 con 0x7f43b806a270 2023-11-28T12:01:28.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.509+0000 7f439ffff700 1 -- 172.21.15.57:0/4277269976 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f43a8027480 con 0x7f43b806a270 2023-11-28T12:01:28.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.509+0000 7f439ffff700 1 -- 172.21.15.57:0/4277269976 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f43a8022070 con 0x7f43b806a270 2023-11-28T12:01:28.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.510+0000 7f43bdfb9700 1 -- 172.21.15.57:0/4277269976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b815daa0 con 0x7f43b806a270 2023-11-28T12:01:28.515 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:28.510+0000 7f3f7df0a240 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T12:01:28.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.512+0000 7f439ffff700 1 -- 172.21.15.57:0/4277269976 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f43a8027a50 con 0x7f43b806a270 2023-11-28T12:01:28.556 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:28.555+0000 7f0c66bae240 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T12:01:28.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.586+0000 7f43bdfb9700 1 -- 172.21.15.57:0/4277269976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43b80743f0 con 0x7f43b806a270 2023-11-28T12:01:28.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.587+0000 7f439ffff700 1 -- 172.21.15.57:0/4277269976 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f43a8027c50 con 0x7f43b806a270 2023-11-28T12:01:28.588 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:28.589 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":10,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:28.297052+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:01:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.588+0000 7f43bdfb9700 1 -- 172.21.15.57:0/4277269976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b806a270 msgr2=0x7f43b80b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.588+0000 7f43bdfb9700 1 --2- 172.21.15.57:0/4277269976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b806a270 0x7f43b80b3610 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f43a8014040 tx=0x7f43a8004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.588+0000 7f43bdfb9700 1 -- 172.21.15.57:0/4277269976 shutdown_connections 2023-11-28T12:01:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.588+0000 7f43bdfb9700 1 --2- 172.21.15.57:0/4277269976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b815cc80 0x7f43b8152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.588+0000 7f43bdfb9700 1 --2- 172.21.15.57:0/4277269976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b806a270 0x7f43b80b3610 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.588+0000 7f43bdfb9700 1 --2- 172.21.15.57:0/4277269976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b806e570 0x7f43b80ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:28.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.588+0000 7f43bdfb9700 1 -- 172.21.15.57:0/4277269976 >> 172.21.15.57:0/4277269976 conn(0x7f43b806c010 msgr2=0x7f43b80af870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:28.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.588+0000 7f43bdfb9700 1 -- 172.21.15.57:0/4277269976 shutdown_connections 2023-11-28T12:01:28.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:28.588+0000 7f43bdfb9700 1 -- 172.21.15.57:0/4277269976 wait complete. 2023-11-28T12:01:28.602 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-28T12:01:28.603 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-28T12:01:28.636 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:28.635+0000 7f0c66bae240 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T12:01:28.712 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:28.711+0000 7f3f7df0a240 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T12:01:28.718 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:28.717+0000 7f0c66bae240 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T12:01:28.791 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:28.790+0000 7f3f7df0a240 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T12:01:28.883 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:28.882+0000 7f3f7df0a240 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T12:01:28.929 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:28.928+0000 7f0c66bae240 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T12:01:28.932 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-11-28T12:01:28.930+0000 7f95b62c5680 -1 Falling back to public interface 2023-11-28T12:01:28.935 INFO:tasks.ceph.osd.0.smithi057.stderr:2023-11-28T12:01:28.934+0000 7f3c61eee680 -1 Falling back to public interface 2023-11-28T12:01:28.939 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-11-28T12:01:28.938+0000 7f798ae5b680 -1 Falling back to public interface 2023-11-28T12:01:28.951 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-11-28T12:01:28.950+0000 7f5314c78680 -1 Falling back to public interface 2023-11-28T12:01:28.956 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-11-28T12:01:28.956+0000 7fd3ca656680 -1 Falling back to public interface 2023-11-28T12:01:28.963 INFO:tasks.ceph.osd.1.smithi057.stderr:2023-11-28T12:01:28.962+0000 7f7ea2d47680 -1 Falling back to public interface 2023-11-28T12:01:28.971 INFO:tasks.ceph.osd.3.smithi057.stderr:2023-11-28T12:01:28.970+0000 7fbab1699680 -1 Falling back to public interface 2023-11-28T12:01:28.972 INFO:tasks.ceph.osd.2.smithi057.stderr:2023-11-28T12:01:28.970+0000 7fe2dfdaf680 -1 Falling back to public interface 2023-11-28T12:01:28.972 INFO:tasks.ceph.osd.3.smithi057.stderr:2023-11-28T12:01:28.971+0000 7fbaa8a40700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6819/0: (98) Address already in use 2023-11-28T12:01:29.019 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:29.018+0000 7f0c66bae240 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T12:01:29.069 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:29.067+0000 7f3f7df0a240 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T12:01:29.121 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:29.120+0000 7f0c66bae240 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T12:01:29.149 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:29.148+0000 7f3f7df0a240 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T12:01:29.321 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:29.320+0000 7f0c66bae240 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T12:01:29.409 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T12:01:29.413 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:29.411+0000 7f0c66bae240 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T12:01:29.438 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:29.437+0000 7f3f7df0a240 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T12:01:29.509 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:29.508+0000 7f3f7df0a240 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T12:01:29.538 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.536+0000 7f4fb29c3700 1 Processor -- start 2023-11-28T12:01:29.538 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.536+0000 7f4fb29c3700 1 -- start start 2023-11-28T12:01:29.540 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4fb29c3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4fac0b2ee0 0x7f4fac0d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:29.540 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4fb29c3700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fac15a9c0 con 0x7f4fac0cc200 2023-11-28T12:01:29.541 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4fb29c3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fac0c7b80 con 0x7f4fac15a5f0 2023-11-28T12:01:29.541 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4fb29c3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac0cc990 con 0x7f4fac0b2ee0 2023-11-28T12:01:29.541 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4fb0f60700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4fac0cc200 0x7f4fac0ced30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38292/0 (socket says 172.21.15.57:38292) 2023-11-28T12:01:29.541 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4fb0f60700 1 -- 172.21.15.57:0/1533802515 learned_addr learned my addr 172.21.15.57:0/1533802515 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:29.541 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4fab7fe700 1 --2- 172.21.15.57:0/1533802515 >> v2:172.21.15.181:3300/0 conn(0x7f4fac0b2ee0 0x7f4fac0d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:29.541 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4faaffd700 1 -- 172.21.15.57:0/1533802515 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174499801 0 0) 0x7f4fac15a9c0 con 0x7f4fac0cc200 2023-11-28T12:01:29.541 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4faaffd700 1 -- 172.21.15.57:0/1533802515 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f940035b0 con 0x7f4fac0cc200 2023-11-28T12:01:29.541 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4faaffd700 1 -- 172.21.15.57:0/1533802515 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673167367 0 0) 0x7f4fac0c7b80 con 0x7f4fac15a5f0 2023-11-28T12:01:29.541 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4faaffd700 1 -- 172.21.15.57:0/1533802515 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fac15a9c0 con 0x7f4fac15a5f0 2023-11-28T12:01:29.541 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4fab7fe700 1 -- 172.21.15.57:0/1533802515 >> v1:172.21.15.181:6789/0 conn(0x7f4fac15a5f0 legacy=0x7f4fac0b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:29.542 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4fab7fe700 1 -- 172.21.15.57:0/1533802515 >> v1:172.21.15.57:6789/0 conn(0x7f4fac0cc200 legacy=0x7f4fac0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:29.542 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.538+0000 7f4fab7fe700 1 -- 172.21.15.57:0/1533802515 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac0ccc70 con 0x7f4fac0b2ee0 2023-11-28T12:01:29.542 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.539+0000 7f4fab7fe700 1 --2- 172.21.15.57:0/1533802515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fac0b2ee0 0x7f4fac0d0ad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009840 tx=0x7f4fa0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97b8095687fc5dac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:29.542 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.539+0000 7f4faaffd700 1 -- 172.21.15.57:0/1533802515 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0011830 con 0x7f4fac0b2ee0 2023-11-28T12:01:29.542 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.539+0000 7f4faaffd700 1 -- 172.21.15.57:0/1533802515 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa0011990 con 0x7f4fac0b2ee0 2023-11-28T12:01:29.542 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.539+0000 7f4faaffd700 1 -- 172.21.15.57:0/1533802515 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0011d20 con 0x7f4fac0b2ee0 2023-11-28T12:01:29.542 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.539+0000 7f4fb29c3700 1 -- 172.21.15.57:0/1533802515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fac0b2ee0 msgr2=0x7f4fac0d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:29.542 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.539+0000 7f4fb29c3700 1 --2- 172.21.15.57:0/1533802515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fac0b2ee0 0x7f4fac0d0ad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009840 tx=0x7f4fa0009b50 comp rx=0 tx=0).stop 2023-11-28T12:01:29.542 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.539+0000 7f4fb29c3700 1 -- 172.21.15.57:0/1533802515 shutdown_connections 2023-11-28T12:01:29.542 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.539+0000 7f4fb29c3700 1 --2- 172.21.15.57:0/1533802515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fac0b2ee0 0x7f4fac0d0ad0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:29.542 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.539+0000 7f4fb29c3700 1 -- 172.21.15.57:0/1533802515 >> 172.21.15.57:0/1533802515 conn(0x7f4fac06ea80 msgr2=0x7f4fac06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:29.543 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.539+0000 7f4fb29c3700 1 -- 172.21.15.57:0/1533802515 shutdown_connections 2023-11-28T12:01:29.543 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.540+0000 7f4fb29c3700 1 -- 172.21.15.57:0/1533802515 wait complete. 2023-11-28T12:01:29.545 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.541+0000 7f4fb29c3700 1 Processor -- start 2023-11-28T12:01:29.545 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.542+0000 7f4fb29c3700 1 -- start start 2023-11-28T12:01:29.545 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.542+0000 7f4fb29c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fac0b2ee0 0x7f4fac154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:29.545 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.542+0000 7f4fb29c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fac0cc200 0x7f4fac0b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:29.545 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.542+0000 7f4fb29c3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fac15a5f0 0x7f4fac156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:29.546 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.542+0000 7f4fb29c3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0b59c0 con 0x7f4fac15a5f0 2023-11-28T12:01:29.546 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.542+0000 7f4fb29c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0b4db0 con 0x7f4fac0cc200 2023-11-28T12:01:29.546 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.542+0000 7f4fb29c3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0b5090 con 0x7f4fac0b2ee0 2023-11-28T12:01:29.546 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fb0f60700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fac15a5f0 0x7f4fac156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:29.546 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fb0f60700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fac15a5f0 0x7f4fac156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52008/0 (socket says 172.21.15.57:52008) 2023-11-28T12:01:29.546 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fb0f60700 1 -- 172.21.15.57:0/2563335100 learned_addr learned my addr 172.21.15.57:0/2563335100 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:29.546 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fab7fe700 1 --2- 172.21.15.57:0/2563335100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fac0cc200 0x7f4fac0b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:29.546 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fabfff700 1 --2- 172.21.15.57:0/2563335100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fac0b2ee0 0x7f4fac154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:29.546 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fab7fe700 1 -- 172.21.15.57:0/2563335100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fac0b2ee0 msgr2=0x7f4fac154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:29.546 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fab7fe700 1 --2- 172.21.15.57:0/2563335100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fac0b2ee0 0x7f4fac154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:29.546 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fab7fe700 1 -- 172.21.15.57:0/2563335100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fac15a5f0 msgr2=0x7f4fac156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:29.547 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fab7fe700 1 --2- 172.21.15.57:0/2563335100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fac15a5f0 0x7f4fac156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:29.547 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fab7fe700 1 -- 172.21.15.57:0/2563335100 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa00094a0 con 0x7f4fac0cc200 2023-11-28T12:01:29.547 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fb0f60700 1 --2- 172.21.15.57:0/2563335100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fac15a5f0 0x7f4fac156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:29.547 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fab7fe700 1 --2- 172.21.15.57:0/2563335100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fac0cc200 0x7f4fac0b5e50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0016000 tx=0x7f4fa0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:29.547 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fa8ff9700 1 -- 172.21.15.57:0/2563335100 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa001a6e0 con 0x7f4fac0cc200 2023-11-28T12:01:29.547 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fa8ff9700 1 -- 172.21.15.57:0/2563335100 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa001a840 con 0x7f4fac0cc200 2023-11-28T12:01:29.547 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fb29c3700 1 -- 172.21.15.57:0/2563335100 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac0cd230 con 0x7f4fac0cc200 2023-11-28T12:01:29.547 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fa8ff9700 1 -- 172.21.15.57:0/2563335100 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa001abb0 con 0x7f4fac0cc200 2023-11-28T12:01:29.547 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.543+0000 7f4fb29c3700 1 -- 172.21.15.57:0/2563335100 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac0ccf50 con 0x7f4fac0cc200 2023-11-28T12:01:29.547 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.544+0000 7f4fa8ff9700 1 -- 172.21.15.57:0/2563335100 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f4fa002b030 con 0x7f4fac0cc200 2023-11-28T12:01:29.547 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.544+0000 7f4fa8ff9700 1 -- 172.21.15.57:0/2563335100 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4fa0024af0 con 0x7f4fac0cc200 2023-11-28T12:01:29.547 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.544+0000 7f4fb29c3700 1 -- 172.21.15.57:0/2563335100 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fac0cd510 con 0x7f4fac0cc200 2023-11-28T12:01:29.548 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.547+0000 7f4fa8ff9700 1 -- 172.21.15.57:0/2563335100 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f4fa0029070 con 0x7f4fac0cc200 2023-11-28T12:01:29.622 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.620+0000 7f4fb29c3700 1 -- 172.21.15.57:0/2563335100 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fac0b5b30 con 0x7f4fac0cc200 2023-11-28T12:01:29.622 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.621+0000 7f4fa8ff9700 1 -- 172.21.15.57:0/2563335100 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4fa0015020 con 0x7f4fac0cc200 2023-11-28T12:01:29.622 INFO:teuthology.misc.health.smithi057.stdout: 2023-11-28T12:01:29.623 INFO:teuthology.misc.health.smithi057.stdout:{"epoch":10,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:28.297052+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:01:29.623 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.622+0000 7f4fb29c3700 1 -- 172.21.15.57:0/2563335100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fac0cc200 msgr2=0x7f4fac0b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:29.624 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.622+0000 7f4fb29c3700 1 --2- 172.21.15.57:0/2563335100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fac0cc200 0x7f4fac0b5e50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0016000 tx=0x7f4fa0004750 comp rx=0 tx=0).stop 2023-11-28T12:01:29.624 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.622+0000 7f4fb29c3700 1 -- 172.21.15.57:0/2563335100 shutdown_connections 2023-11-28T12:01:29.624 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.622+0000 7f4fb29c3700 1 --2- 172.21.15.57:0/2563335100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fac15a5f0 0x7f4fac156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:29.624 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.622+0000 7f4fb29c3700 1 --2- 172.21.15.57:0/2563335100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fac0b2ee0 0x7f4fac154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:29.624 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.622+0000 7f4fb29c3700 1 --2- 172.21.15.57:0/2563335100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fac0cc200 0x7f4fac0b5e50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:29.624 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.622+0000 7f4fb29c3700 1 -- 172.21.15.57:0/2563335100 >> 172.21.15.57:0/2563335100 conn(0x7f4fac06ea80 msgr2=0x7f4fac0cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:29.624 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.623+0000 7f4fb29c3700 1 -- 172.21.15.57:0/2563335100 shutdown_connections 2023-11-28T12:01:29.624 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:29.623+0000 7f4fb29c3700 1 -- 172.21.15.57:0/2563335100 wait complete. 2023-11-28T12:01:29.636 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T12:01:29.744 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:29.743+0000 7f0c66bae240 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T12:01:29.826 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:29.825+0000 7f0c66bae240 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T12:01:30.016 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:30.015+0000 7f3f7df0a240 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T12:01:30.164 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:30.163+0000 7f3f7df0a240 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T12:01:30.397 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:30.396+0000 7f0c66bae240 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T12:01:30.476 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:30.475+0000 7f3f7df0a240 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T12:01:30.563 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:30.562+0000 7f0c66bae240 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T12:01:30.915 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:30.914+0000 7f0c66bae240 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T12:01:31.011 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:31.009+0000 7f3f7df0a240 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T12:01:31.156 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:31.155+0000 7f3f7df0a240 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T12:01:31.525 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:31.524+0000 7f3f7df0a240 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T12:01:31.531 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:31.530+0000 7f0c66bae240 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T12:01:31.625 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:31.624+0000 7f3f7df0a240 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T12:01:31.702 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:31.701+0000 7f0c66bae240 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T12:01:32.218 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:32.217+0000 7f0c66bae240 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T12:01:32.330 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-11-28T12:01:32.329+0000 7f0c66bae240 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T12:01:32.439 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-11-28T12:01:32.438+0000 7f95b62c5680 -1 osd.4 0 log_to_monitors true 2023-11-28T12:01:32.448 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-11-28T12:01:32.447+0000 7f798ae5b680 -1 osd.7 0 log_to_monitors true 2023-11-28T12:01:32.477 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-11-28T12:01:32.475+0000 7fd3ca656680 -1 osd.5 0 log_to_monitors true 2023-11-28T12:01:32.488 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-11-28T12:01:32.487+0000 7f5314c78680 -1 osd.6 0 log_to_monitors true 2023-11-28T12:01:32.489 INFO:tasks.ceph.osd.2.smithi057.stderr:2023-11-28T12:01:32.489+0000 7fe2dfdaf680 -1 osd.2 0 log_to_monitors true 2023-11-28T12:01:32.491 INFO:tasks.ceph.osd.0.smithi057.stderr:2023-11-28T12:01:32.490+0000 7f3c61eee680 -1 osd.0 0 log_to_monitors true 2023-11-28T12:01:32.517 INFO:tasks.ceph.osd.1.smithi057.stderr:2023-11-28T12:01:32.516+0000 7f7ea2d47680 -1 osd.1 0 log_to_monitors true 2023-11-28T12:01:32.517 INFO:tasks.ceph.osd.3.smithi057.stderr:2023-11-28T12:01:32.516+0000 7fbab1699680 -1 osd.3 0 log_to_monitors true 2023-11-28T12:01:32.658 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:32.657+0000 7f3f3f329700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-28T12:01:32.659 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-11-28T12:01:32.657+0000 7f3f3f329700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-28T12:01:35.198 INFO:tasks.ceph.osd.0.smithi057.stderr:2023-11-28T12:01:35.197+0000 7f3c55a8e700 -1 osd.0 0 waiting for initial osdmap 2023-11-28T12:01:35.214 INFO:tasks.ceph.osd.0.smithi057.stderr:2023-11-28T12:01:35.213+0000 7f3c4de69700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T12:01:35.348 INFO:tasks.ceph.osd.1.smithi057.stderr:2023-11-28T12:01:35.347+0000 7f7e968e7700 -1 osd.1 0 waiting for initial osdmap 2023-11-28T12:01:35.359 INFO:tasks.ceph.osd.1.smithi057.stderr:2023-11-28T12:01:35.358+0000 7f7e8ecc2700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T12:01:35.378 INFO:tasks.ceph.osd.3.smithi057.stderr:2023-11-28T12:01:35.378+0000 7fbaa5239700 -1 osd.3 0 waiting for initial osdmap 2023-11-28T12:01:35.397 INFO:tasks.ceph.osd.3.smithi057.stderr:2023-11-28T12:01:35.396+0000 7fba9d614700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T12:01:35.421 INFO:tasks.ceph.osd.2.smithi057.stderr:2023-11-28T12:01:35.420+0000 7fe2d394f700 -1 osd.2 0 waiting for initial osdmap 2023-11-28T12:01:35.436 INFO:tasks.ceph.osd.2.smithi057.stderr:2023-11-28T12:01:35.436+0000 7fe2cbd2a700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T12:01:35.738 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-11-28T12:01:35.738+0000 7f5308818700 -1 osd.6 0 waiting for initial osdmap 2023-11-28T12:01:35.750 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-11-28T12:01:35.749+0000 7f5300bf3700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T12:01:35.770 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-11-28T12:01:35.770+0000 7f797e9fb700 -1 osd.7 0 waiting for initial osdmap 2023-11-28T12:01:35.776 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-11-28T12:01:35.776+0000 7fd3be1f6700 -1 osd.5 0 waiting for initial osdmap 2023-11-28T12:01:35.783 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-11-28T12:01:35.783+0000 7f95a9e65700 -1 osd.4 0 waiting for initial osdmap 2023-11-28T12:01:35.786 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-11-28T12:01:35.786+0000 7f7976dd6700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T12:01:35.790 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-11-28T12:01:35.790+0000 7fd3b65d1700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T12:01:35.796 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-11-28T12:01:35.796+0000 7f95a2240700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T12:01:36.442 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T12:01:36.585 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.577+0000 7f1aa0ca0700 1 Processor -- start 2023-11-28T12:01:36.585 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.577+0000 7f1aa0ca0700 1 -- start start 2023-11-28T12:01:36.585 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.578+0000 7f1aa0ca0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1a9c15a5f0 0x7f1a9c0d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.585 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.578+0000 7f1aa0ca0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1a9c0cc200 0x7f1a9c0b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.586 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.578+0000 7f1aa0ca0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0b2d10 0x7f1a9c15e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.586 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.578+0000 7f1aa0ca0700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9c0b3190 con 0x7f1a9c0cc200 2023-11-28T12:01:36.586 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.578+0000 7f1aa0ca0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9c15a9c0 con 0x7f1a9c15a5f0 2023-11-28T12:01:36.586 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.578+0000 7f1aa0ca0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c0c7b80 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.586 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.578+0000 7f1a99d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1a9c0cc200 0x7f1a9c0b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.586 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.578+0000 7f1a99d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1a9c0cc200 0x7f1a9c0b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39258/0 (socket says 172.21.15.57:39258) 2023-11-28T12:01:36.586 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.578+0000 7f1a99d9b700 1 -- 172.21.15.57:0/1572560994 learned_addr learned my addr 172.21.15.57:0/1572560994 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:36.586 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.578+0000 7f1a9a59c700 1 --2- 172.21.15.57:0/1572560994 >> v2:172.21.15.181:3300/0 conn(0x7f1a9c15a5f0 0x7f1a9c0d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.587 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.578+0000 7f1a9ad9d700 1 --2- 172.21.15.57:0/1572560994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0b2d10 0x7f1a9c15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.587 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.579+0000 7f1a99d9b700 1 -- 172.21.15.57:0/1572560994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0b2d10 msgr2=0x7f1a9c15e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:36.587 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.579+0000 7f1a99d9b700 1 --2- 172.21.15.57:0/1572560994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0b2d10 0x7f1a9c15e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.587 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.579+0000 7f1a99d9b700 1 -- 172.21.15.57:0/1572560994 >> v2:172.21.15.181:3300/0 conn(0x7f1a9c15a5f0 msgr2=0x7f1a9c0d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:36.587 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.579+0000 7f1a99d9b700 1 --2- 172.21.15.57:0/1572560994 >> v2:172.21.15.181:3300/0 conn(0x7f1a9c15a5f0 0x7f1a9c0d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.587 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.579+0000 7f1a99d9b700 1 -- 172.21.15.57:0/1572560994 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c0cc990 con 0x7f1a9c0cc200 2023-11-28T12:01:36.587 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.579+0000 7f1a9a59c700 1 --2- 172.21.15.57:0/1572560994 >> v2:172.21.15.181:3300/0 conn(0x7f1a9c15a5f0 0x7f1a9c0d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:36.587 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.579+0000 7f1a99d9b700 1 --2- 172.21.15.57:0/1572560994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0cc200 0x7f1a9c0b27d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0d1d40 tx=0x7f1a84009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd5aaf62c4301c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:36.587 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.579+0000 7f1a9959a700 1 -- 172.21.15.57:0/1572560994 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8400b870 con 0x7f1a9c0cc200 2023-11-28T12:01:36.587 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.579+0000 7f1a9959a700 1 -- 172.21.15.57:0/1572560994 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8400b9d0 con 0x7f1a9c0cc200 2023-11-28T12:01:36.587 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.579+0000 7f1a9959a700 1 -- 172.21.15.57:0/1572560994 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8400bd60 con 0x7f1a9c0cc200 2023-11-28T12:01:36.587 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.579+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/1572560994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0cc200 msgr2=0x7f1a9c0b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:36.587 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.579+0000 7f1aa0ca0700 1 --2- 172.21.15.57:0/1572560994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0cc200 0x7f1a9c0b27d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0d1d40 tx=0x7f1a84009bd0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.588 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.580+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/1572560994 shutdown_connections 2023-11-28T12:01:36.588 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.580+0000 7f1aa0ca0700 1 --2- 172.21.15.57:0/1572560994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0b2d10 0x7f1a9c15e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.588 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.580+0000 7f1aa0ca0700 1 --2- 172.21.15.57:0/1572560994 >> v2:172.21.15.181:3300/0 conn(0x7f1a9c15a5f0 0x7f1a9c0d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.588 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.580+0000 7f1aa0ca0700 1 --2- 172.21.15.57:0/1572560994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0cc200 0x7f1a9c0b27d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.588 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.580+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/1572560994 >> 172.21.15.57:0/1572560994 conn(0x7f1a9c06ea80 msgr2=0x7f1a9c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:36.588 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.580+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/1572560994 shutdown_connections 2023-11-28T12:01:36.588 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.580+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/1572560994 wait complete. 2023-11-28T12:01:36.588 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.581+0000 7f1aa0ca0700 1 Processor -- start 2023-11-28T12:01:36.588 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.581+0000 7f1aa0ca0700 1 -- start start 2023-11-28T12:01:36.588 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1aa0ca0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0b2d10 0x7f1a9c153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.588 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1aa0ca0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0cc200 0x7f1a9c14ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.588 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1aa0ca0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a9c15a5f0 0x7f1a9c14b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.589 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1aa0ca0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c14b6e0 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.589 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1aa0ca0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c0b3c20 con 0x7f1a9c15a5f0 2023-11-28T12:01:36.589 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1aa0ca0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c0ca610 con 0x7f1a9c0cc200 2023-11-28T12:01:36.589 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1a9a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0b2d10 0x7f1a9c153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.589 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1a9a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0b2d10 0x7f1a9c153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39274/0 (socket says 172.21.15.57:39274) 2023-11-28T12:01:36.589 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1a9a59c700 1 -- 172.21.15.57:0/544332265 learned_addr learned my addr 172.21.15.57:0/544332265 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:36.589 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1a99d9b700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0cc200 0x7f1a9c14ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.589 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1a9ad9d700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a9c15a5f0 0x7f1a9c14b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.589 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1a9a59c700 1 -- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0cc200 msgr2=0x7f1a9c14ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:36.589 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1a9a59c700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0cc200 0x7f1a9c14ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.590 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1a9a59c700 1 -- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a9c15a5f0 msgr2=0x7f1a9c14b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:36.590 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1a9a59c700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a9c15a5f0 0x7f1a9c14b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.590 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1a9a59c700 1 -- 172.21.15.57:0/544332265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a84009550 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.590 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1a9ad9d700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a9c15a5f0 0x7f1a9c14b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:01:36.590 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.582+0000 7f1a9a59c700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0b2d10 0x7f1a9c153770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00e810 tx=0x7f1a8c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:36.590 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.583+0000 7f1a92ffd700 1 -- 172.21.15.57:0/544332265 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c013610 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.590 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.583+0000 7f1a92ffd700 1 -- 172.21.15.57:0/544332265 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c013770 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.590 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.583+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/544332265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c0ccf50 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.590 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.583+0000 7f1a92ffd700 1 -- 172.21.15.57:0/544332265 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c013b20 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.591 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.583+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/544332265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9c0ccc70 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.591 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.584+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/544332265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9c0cd230 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.591 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.584+0000 7f1a92ffd700 1 -- 172.21.15.57:0/544332265 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f1a8c023070 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.591 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.585+0000 7f1a92ffd700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1a7c0682a0 0x7f1a7c06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.591 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.585+0000 7f1a92ffd700 1 -- 172.21.15.57:0/544332265 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f1a8c085310 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.591 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.585+0000 7f1a99d9b700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1a7c0682a0 0x7f1a7c06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.592 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.585+0000 7f1a99d9b700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1a7c0682a0 0x7f1a7c06a750 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1a84000c00 tx=0x7f1a84008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:36.592 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.589+0000 7f1a92ffd700 1 -- 172.21.15.57:0/544332265 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1a8c052100 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.793 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.793+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/544332265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a9c0743f0 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.793 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.793+0000 7f1a92ffd700 1 -- 172.21.15.57:0/544332265 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10536 (secure 0 0 0) 0x7f1a8c055720 con 0x7f1a9c0b2d10 2023-11-28T12:01:36.793 INFO:teuthology.misc.health.smithi057.stdout: 2023-11-28T12:01:36.794 INFO:teuthology.misc.health.smithi057.stdout:{"epoch":13,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:36.202469+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:01:36.796 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.796+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1a7c0682a0 msgr2=0x7f1a7c06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:36.796 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.796+0000 7f1aa0ca0700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1a7c0682a0 0x7f1a7c06a750 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1a84000c00 tx=0x7f1a84008030 comp rx=0 tx=0).stop 2023-11-28T12:01:36.796 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.796+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/544332265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0b2d10 msgr2=0x7f1a9c153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:36.796 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.796+0000 7f1aa0ca0700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0b2d10 0x7f1a9c153770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00e810 tx=0x7f1a8c00ebd0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.796 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.796+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/544332265 shutdown_connections 2023-11-28T12:01:36.796 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.796+0000 7f1aa0ca0700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1a7c0682a0 0x7f1a7c06a750 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.797 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.796+0000 7f1aa0ca0700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a9c15a5f0 0x7f1a9c14b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.797 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.796+0000 7f1aa0ca0700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0b2d10 0x7f1a9c153770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.797 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.796+0000 7f1aa0ca0700 1 --2- 172.21.15.57:0/544332265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0cc200 0x7f1a9c14ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.797 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.796+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/544332265 >> 172.21.15.57:0/544332265 conn(0x7f1a9c06ea80 msgr2=0x7f1a9c158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:36.797 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.796+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/544332265 shutdown_connections 2023-11-28T12:01:36.797 INFO:teuthology.misc.health.smithi057.stderr:2023-11-28T12:01:36.796+0000 7f1aa0ca0700 1 -- 172.21.15.57:0/544332265 wait complete. 2023-11-28T12:01:36.809 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-28T12:01:36.809 INFO:tasks.ceph:Creating RBD pool 2023-11-28T12:01:36.810 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-28T12:01:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.944+0000 7fc4f6928700 1 Processor -- start 2023-11-28T12:01:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.944+0000 7fc4f6928700 1 -- start start 2023-11-28T12:01:36.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f6928700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc4f015a4d0 0x7fc4f006a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f6928700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f006a7b0 0x7fc4f00cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f6928700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc4f00cdd50 0x7fc4f00b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f6928700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f006ac00 con 0x7fc4f00cdd50 2023-11-28T12:01:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f6928700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f015e0d0 con 0x7fc4f015a4d0 2023-11-28T12:01:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f6928700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00ce560 con 0x7fc4f006a7b0 2023-11-28T12:01:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f4ec5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc4f00cdd50 0x7fc4f00b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f4ec5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc4f00cdd50 0x7fc4f00b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39290/0 (socket says 172.21.15.57:39290) 2023-11-28T12:01:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f4ec5700 1 -- 172.21.15.57:0/3672540626 learned_addr learned my addr 172.21.15.57:0/3672540626 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4effff700 1 --2- 172.21.15.57:0/3672540626 >> v2:172.21.15.181:3300/0 conn(0x7fc4f015a4d0 0x7fc4f006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4ef7fe700 1 --2- 172.21.15.57:0/3672540626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f006a7b0 0x7fc4f00cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f4ec5700 1 -- 172.21.15.57:0/3672540626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f006a7b0 msgr2=0x7fc4f00cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f4ec5700 1 --2- 172.21.15.57:0/3672540626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f006a7b0 0x7fc4f00cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f4ec5700 1 -- 172.21.15.57:0/3672540626 >> v2:172.21.15.181:3300/0 conn(0x7fc4f015a4d0 msgr2=0x7fc4f006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f4ec5700 1 --2- 172.21.15.57:0/3672540626 >> v2:172.21.15.181:3300/0 conn(0x7fc4f015a4d0 0x7fc4f006a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.945+0000 7fc4f4ec5700 1 -- 172.21.15.57:0/3672540626 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f015e320 con 0x7fc4f00cdd50 2023-11-28T12:01:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.946+0000 7fc4f4ec5700 1 --2- 172.21.15.57:0/3672540626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4f00cdd50 0x7fc4f00b1650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc4f006b140 tx=0x7fc4e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c7676ee3941323 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.946+0000 7fc4eeffd700 1 -- 172.21.15.57:0/3672540626 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e800e070 con 0x7fc4f00cdd50 2023-11-28T12:01:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.946+0000 7fc4eeffd700 1 -- 172.21.15.57:0/3672540626 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e800b9c0 con 0x7fc4f00cdd50 2023-11-28T12:01:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.946+0000 7fc4eeffd700 1 -- 172.21.15.57:0/3672540626 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e800bd70 con 0x7fc4f00cdd50 2023-11-28T12:01:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.946+0000 7fc4f6928700 1 -- 172.21.15.57:0/3672540626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4f00cdd50 msgr2=0x7fc4f00b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.946+0000 7fc4f6928700 1 --2- 172.21.15.57:0/3672540626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4f00cdd50 0x7fc4f00b1650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc4f006b140 tx=0x7fc4e800b330 comp rx=0 tx=0).stop 2023-11-28T12:01:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.946+0000 7fc4f6928700 1 -- 172.21.15.57:0/3672540626 shutdown_connections 2023-11-28T12:01:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.946+0000 7fc4f6928700 1 --2- 172.21.15.57:0/3672540626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4f00cdd50 0x7fc4f00b1650 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.946+0000 7fc4f6928700 1 --2- 172.21.15.57:0/3672540626 >> v2:172.21.15.181:3300/0 conn(0x7fc4f015a4d0 0x7fc4f006a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.946+0000 7fc4f6928700 1 --2- 172.21.15.57:0/3672540626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f006a7b0 0x7fc4f00cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.946+0000 7fc4f6928700 1 -- 172.21.15.57:0/3672540626 >> 172.21.15.57:0/3672540626 conn(0x7fc4f006c010 msgr2=0x7fc4f00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.946+0000 7fc4f6928700 1 -- 172.21.15.57:0/3672540626 shutdown_connections 2023-11-28T12:01:36.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.946+0000 7fc4f6928700 1 -- 172.21.15.57:0/3672540626 wait complete. 2023-11-28T12:01:36.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.948+0000 7fc4f6928700 1 Processor -- start 2023-11-28T12:01:36.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.948+0000 7fc4f6928700 1 -- start start 2023-11-28T12:01:36.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.949+0000 7fc4f6928700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f006a7b0 0x7fc4f01f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.949+0000 7fc4f6928700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f00cdd50 0x7fc4f0152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.949+0000 7fc4f6928700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4f015a4d0 0x7fc4f00b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.949+0000 7fc4f6928700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f01f3a00 con 0x7fc4f015a4d0 2023-11-28T12:01:36.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.949+0000 7fc4f6928700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00c8260 con 0x7fc4f006a7b0 2023-11-28T12:01:36.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.949+0000 7fc4f6928700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00c8540 con 0x7fc4f00cdd50 2023-11-28T12:01:36.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.949+0000 7fc4f4ec5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4f015a4d0 0x7fc4f00b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.949+0000 7fc4f4ec5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4f015a4d0 0x7fc4f00b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39294/0 (socket says 172.21.15.57:39294) 2023-11-28T12:01:36.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.949+0000 7fc4f4ec5700 1 -- 172.21.15.57:0/1490013862 learned_addr learned my addr 172.21.15.57:0/1490013862 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:36.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4ef7fe700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f00cdd50 0x7fc4f0152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4effff700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f006a7b0 0x7fc4f01f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4f4ec5700 1 -- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f00cdd50 msgr2=0x7fc4f0152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:36.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4f4ec5700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f00cdd50 0x7fc4f0152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4f4ec5700 1 -- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f006a7b0 msgr2=0x7fc4f01f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:36.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4f4ec5700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f006a7b0 0x7fc4f01f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:36.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4f4ec5700 1 -- 172.21.15.57:0/1490013862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e8009b30 con 0x7fc4f015a4d0 2023-11-28T12:01:36.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4ef7fe700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f00cdd50 0x7fc4f0152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:01:36.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4effff700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f006a7b0 0x7fc4f01f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:01:36.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4f4ec5700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4f015a4d0 0x7fc4f00b3d60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0067450 tx=0x7fc4e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:36.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4ecff9700 1 -- 172.21.15.57:0/1490013862 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8004250 con 0x7fc4f015a4d0 2023-11-28T12:01:36.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4ecff9700 1 -- 172.21.15.57:0/1490013862 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e80043d0 con 0x7fc4f015a4d0 2023-11-28T12:01:36.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4f6928700 1 -- 172.21.15.57:0/1490013862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f015e8e0 con 0x7fc4f015a4d0 2023-11-28T12:01:36.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4ecff9700 1 -- 172.21.15.57:0/1490013862 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8004780 con 0x7fc4f015a4d0 2023-11-28T12:01:36.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.950+0000 7fc4f6928700 1 -- 172.21.15.57:0/1490013862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f015e600 con 0x7fc4f015a4d0 2023-11-28T12:01:36.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.951+0000 7fc4f6928700 1 -- 172.21.15.57:0/1490013862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f015a8a0 con 0x7fc4f015a4d0 2023-11-28T12:01:36.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.952+0000 7fc4ecff9700 1 -- 172.21.15.57:0/1490013862 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fc4e801c440 con 0x7fc4f015a4d0 2023-11-28T12:01:36.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.952+0000 7fc4ecff9700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc4d00682a0 0x7fc4d006a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:36.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.952+0000 7fc4ecff9700 1 -- 172.21.15.57:0/1490013862 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fc4e8086ca0 con 0x7fc4f015a4d0 2023-11-28T12:01:36.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.952+0000 7fc4effff700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc4d00682a0 0x7fc4d006a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:36.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.953+0000 7fc4effff700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc4d00682a0 0x7fc4d006a750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009820 tx=0x7fc4e0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:36.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:36.957+0000 7fc4ecff9700 1 -- 172.21.15.57:0/1490013862 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc4e808b070 con 0x7fc4f015a4d0 2023-11-28T12:01:37.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.158+0000 7fc4f6928700 1 -- 172.21.15.57:0/1490013862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fc4f00743f0 con 0x7fc4f015a4d0 2023-11-28T12:01:37.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.226+0000 7fc4ecff9700 1 -- 172.21.15.57:0/1490013862 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7fc4e80200a0 con 0x7fc4f015a4d0 2023-11-28T12:01:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.229+0000 7fc4f6928700 1 -- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc4d00682a0 msgr2=0x7fc4d006a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.229+0000 7fc4f6928700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc4d00682a0 0x7fc4d006a750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009820 tx=0x7fc4e0008040 comp rx=0 tx=0).stop 2023-11-28T12:01:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.229+0000 7fc4f6928700 1 -- 172.21.15.57:0/1490013862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4f015a4d0 msgr2=0x7fc4f00b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:37.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.229+0000 7fc4f6928700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4f015a4d0 0x7fc4f00b3d60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0067450 tx=0x7fc4e8004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:37.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.229+0000 7fc4f6928700 1 -- 172.21.15.57:0/1490013862 shutdown_connections 2023-11-28T12:01:37.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.229+0000 7fc4f6928700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc4d00682a0 0x7fc4d006a750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:37.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.229+0000 7fc4f6928700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4f015a4d0 0x7fc4f00b3d60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:37.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.229+0000 7fc4f6928700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f006a7b0 0x7fc4f01f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:37.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.229+0000 7fc4f6928700 1 --2- 172.21.15.57:0/1490013862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f00cdd50 0x7fc4f0152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:37.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.229+0000 7fc4f6928700 1 -- 172.21.15.57:0/1490013862 >> 172.21.15.57:0/1490013862 conn(0x7fc4f006c010 msgr2=0x7fc4f0155050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:37.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.229+0000 7fc4f6928700 1 -- 172.21.15.57:0/1490013862 shutdown_connections 2023-11-28T12:01:37.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.229+0000 7fc4f6928700 1 -- 172.21.15.57:0/1490013862 wait complete. 2023-11-28T12:01:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'rbd' created 2023-11-28T12:01:37.244 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-28T12:01:37.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.382+0000 7feef3871700 1 Processor -- start 2023-11-28T12:01:37.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.383+0000 7feef3871700 1 -- start start 2023-11-28T12:01:37.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.383+0000 7feef3871700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7feeec15e070 0x7feeec06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:37.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.383+0000 7feef3871700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeec0ceea0 0x7feeec06b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:37.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.383+0000 7feef3871700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feeec15a150 0x7feeec15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:37.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.383+0000 7feef3871700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec15ae80 con 0x7feeec15e070 2023-11-28T12:01:37.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.383+0000 7feef3871700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec0cea10 con 0x7feeec15a150 2023-11-28T12:01:37.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.383+0000 7feef3871700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0ce0e0 con 0x7feeec0ceea0 2023-11-28T12:01:37.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feef160d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7feeec15e070 0x7feeec06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:37.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feef160d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7feeec15e070 0x7feeec06b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39298/0 (socket says 172.21.15.57:39298) 2023-11-28T12:01:37.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feef160d700 1 -- 172.21.15.57:0/36849122 learned_addr learned my addr 172.21.15.57:0/36849122 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:37.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feef0e0c700 1 --2- 172.21.15.57:0/36849122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeec0ceea0 0x7feeec06b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:37.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feef1e0e700 1 --2- 172.21.15.57:0/36849122 >> v2:172.21.15.181:3300/0 conn(0x7feeec15a150 0x7feeec15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:37.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feef160d700 1 -- 172.21.15.57:0/36849122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeec0ceea0 msgr2=0x7feeec06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:37.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feef160d700 1 --2- 172.21.15.57:0/36849122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeec0ceea0 0x7feeec06b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:37.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feef160d700 1 -- 172.21.15.57:0/36849122 >> v2:172.21.15.181:3300/0 conn(0x7feeec15a150 msgr2=0x7feeec15a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:37.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feef160d700 1 --2- 172.21.15.57:0/36849122 >> v2:172.21.15.181:3300/0 conn(0x7feeec15a150 0x7feeec15a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:37.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feef160d700 1 -- 172.21.15.57:0/36849122 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec0ce3c0 con 0x7feeec15e070 2023-11-28T12:01:37.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feef160d700 1 --2- 172.21.15.57:0/36849122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec15e070 0x7feeec06b380 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feeec06c490 tx=0x7feedc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22651977ad6cfcf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:37.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feee3fff700 1 -- 172.21.15.57:0/36849122 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc00b870 con 0x7feeec15e070 2023-11-28T12:01:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feee3fff700 1 -- 172.21.15.57:0/36849122 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc00b9d0 con 0x7feeec15e070 2023-11-28T12:01:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.384+0000 7feee3fff700 1 -- 172.21.15.57:0/36849122 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc00bd60 con 0x7feeec15e070 2023-11-28T12:01:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.385+0000 7feef3871700 1 -- 172.21.15.57:0/36849122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec15e070 msgr2=0x7feeec06b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.385+0000 7feef3871700 1 --2- 172.21.15.57:0/36849122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec15e070 0x7feeec06b380 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feeec06c490 tx=0x7feedc009bd0 comp rx=0 tx=0).stop 2023-11-28T12:01:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.385+0000 7feef3871700 1 -- 172.21.15.57:0/36849122 shutdown_connections 2023-11-28T12:01:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.385+0000 7feef3871700 1 --2- 172.21.15.57:0/36849122 >> v2:172.21.15.181:3300/0 conn(0x7feeec15a150 0x7feeec15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.385+0000 7feef3871700 1 --2- 172.21.15.57:0/36849122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec15e070 0x7feeec06b380 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.385+0000 7feef3871700 1 --2- 172.21.15.57:0/36849122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeec0ceea0 0x7feeec06b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.385+0000 7feef3871700 1 -- 172.21.15.57:0/36849122 >> 172.21.15.57:0/36849122 conn(0x7feeec0cfc30 msgr2=0x7feeec0d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.385+0000 7feef3871700 1 -- 172.21.15.57:0/36849122 shutdown_connections 2023-11-28T12:01:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.385+0000 7feef3871700 1 -- 172.21.15.57:0/36849122 wait complete. 2023-11-28T12:01:37.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.387+0000 7feef3871700 1 Processor -- start 2023-11-28T12:01:37.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.387+0000 7feef3871700 1 -- start start 2023-11-28T12:01:37.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef3871700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeec0ceea0 0x7feeec1e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:37.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef3871700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeec15a150 0x7feeec14ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:37.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef3871700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec15e070 0x7feeec14d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:37.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef3871700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec1e40a0 con 0x7feeec15e070 2023-11-28T12:01:37.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef3871700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0b4380 con 0x7feeec15a150 2023-11-28T12:01:37.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef3871700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec1e50d0 con 0x7feeec0ceea0 2023-11-28T12:01:37.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef1e0e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec15e070 0x7feeec14d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:37.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef1e0e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec15e070 0x7feeec14d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39308/0 (socket says 172.21.15.57:39308) 2023-11-28T12:01:37.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef1e0e700 1 -- 172.21.15.57:0/264010736 learned_addr learned my addr 172.21.15.57:0/264010736 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef160d700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeec0ceea0 0x7feeec1e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef0e0c700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeec15a150 0x7feeec14ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef1e0e700 1 -- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeec0ceea0 msgr2=0x7feeec1e3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef1e0e700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeec0ceea0 0x7feeec1e3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef1e0e700 1 -- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeec15a150 msgr2=0x7feeec14ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef1e0e700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeec15a150 0x7feeec14ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef1e0e700 1 -- 172.21.15.57:0/264010736 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc009550 con 0x7feeec15e070 2023-11-28T12:01:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.388+0000 7feef1e0e700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec15e070 0x7feeec14d3f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feee4007e40 tx=0x7feee400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.389+0000 7feee1ffb700 1 -- 172.21.15.57:0/264010736 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee400ed30 con 0x7feeec15e070 2023-11-28T12:01:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.389+0000 7feee1ffb700 1 -- 172.21.15.57:0/264010736 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee4014420 con 0x7feeec15e070 2023-11-28T12:01:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.389+0000 7feef3871700 1 -- 172.21.15.57:0/264010736 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec15aae0 con 0x7feeec15e070 2023-11-28T12:01:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.389+0000 7feee1ffb700 1 -- 172.21.15.57:0/264010736 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40147d0 con 0x7feeec15e070 2023-11-28T12:01:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.389+0000 7feef3871700 1 -- 172.21.15.57:0/264010736 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec0ce6a0 con 0x7feeec15e070 2023-11-28T12:01:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.390+0000 7feef3871700 1 -- 172.21.15.57:0/264010736 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeec150080 con 0x7feeec15e070 2023-11-28T12:01:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.390+0000 7feee1ffb700 1 -- 172.21.15.57:0/264010736 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7feee4012050 con 0x7feeec15e070 2023-11-28T12:01:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.391+0000 7feee1ffb700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feed00682a0 0x7feed006a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.391+0000 7feee1ffb700 1 -- 172.21.15.57:0/264010736 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4813+0+0 (secure 0 0 0) 0x7feee4085c60 con 0x7feeec15e070 2023-11-28T12:01:37.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.391+0000 7feef160d700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feed00682a0 0x7feed006a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:37.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.392+0000 7feef160d700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feed00682a0 0x7feed006a750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feedc009840 tx=0x7feedc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:37.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.396+0000 7feee1ffb700 1 -- 172.21.15.57:0/264010736 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feee40529f0 con 0x7feeec15e070 2023-11-28T12:01:37.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:37.602+0000 7feef3871700 1 -- 172.21.15.57:0/264010736 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7feeec06e530 con 0x7feeec15e070 2023-11-28T12:01:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.233+0000 7feee1ffb700 1 -- 172.21.15.57:0/264010736 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7feee4056010 con 0x7feeec15e070 2023-11-28T12:01:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.236+0000 7feef3871700 1 -- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feed00682a0 msgr2=0x7feed006a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.237+0000 7feef3871700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feed00682a0 0x7feed006a750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feedc009840 tx=0x7feedc008030 comp rx=0 tx=0).stop 2023-11-28T12:01:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.237+0000 7feef3871700 1 -- 172.21.15.57:0/264010736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec15e070 msgr2=0x7feeec14d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.237+0000 7feef3871700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec15e070 0x7feeec14d3f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feee4007e40 tx=0x7feee400e3c0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.237+0000 7feef3871700 1 -- 172.21.15.57:0/264010736 shutdown_connections 2023-11-28T12:01:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.237+0000 7feef3871700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feed00682a0 0x7feed006a750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.237+0000 7feef3871700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeec15e070 0x7feeec14d3f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.237+0000 7feef3871700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeec0ceea0 0x7feeec1e3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.237+0000 7feef3871700 1 --2- 172.21.15.57:0/264010736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeec15a150 0x7feeec14ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.237+0000 7feef3871700 1 -- 172.21.15.57:0/264010736 >> 172.21.15.57:0/264010736 conn(0x7feeec0cfc30 msgr2=0x7feeec06b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:38.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.237+0000 7feef3871700 1 -- 172.21.15.57:0/264010736 shutdown_connections 2023-11-28T12:01:38.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.237+0000 7feef3871700 1 -- 172.21.15.57:0/264010736 wait complete. 2023-11-28T12:01:38.238 INFO:teuthology.orchestra.run.smithi057.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-28T12:01:38.253 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-28T12:01:38.253 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T12:01:38.254 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T12:01:38.256 INFO:tasks.ceph.mds.a:Started 2023-11-28T12:01:38.256 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T12:01:38.256 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T12:01:38.258 INFO:tasks.ceph.mds.c:Started 2023-11-28T12:01:38.258 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T12:01:38.258 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T12:01:38.261 INFO:tasks.ceph.mds.b:Started 2023-11-28T12:01:38.261 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T12:01:38.261 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T12:01:38.263 INFO:tasks.ceph.mds.d:Started 2023-11-28T12:01:38.263 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-28T12:01:38.263 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:01:38.350 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-11-28T12:01:38.350 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-11-28T12:01:38.350 INFO:tasks.ceph.mds.d.smithi181.stderr:2023-11-28T12:01:38.350+0000 7fcc27da5700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2023-11-28T12:01:38.351 INFO:tasks.ceph.mds.c.smithi057.stdout:starting mds.c at 2023-11-28T12:01:38.352 INFO:tasks.ceph.mds.a.smithi057.stdout:starting mds.a at 2023-11-28T12:01:38.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.411+0000 7f588e46a700 1 Processor -- start 2023-11-28T12:01:38.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.411+0000 7f588e46a700 1 -- start start 2023-11-28T12:01:38.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f588e46a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f58880d15f0 0x7f588815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:38.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f588e46a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f588815e010 0x7f58880ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:38.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f588e46a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588815af30 con 0x7f588806e400 2023-11-28T12:01:38.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f588e46a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588815a9d0 con 0x7f58880d15f0 2023-11-28T12:01:38.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f588e46a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58881584e0 con 0x7f588815e010 2023-11-28T12:01:38.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f58877fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f58880d15f0 0x7f588815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:38.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f588ca07700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f588815e010 0x7f58880ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:38.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f58877fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f58880d15f0 0x7f588815b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39346/0 (socket says 172.21.15.57:39346) 2023-11-28T12:01:38.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f5887fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f588806e400 0x7f588806e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:59056/0 (socket says 172.21.15.57:59056) 2023-11-28T12:01:38.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f58877fe700 1 -- 172.21.15.57:0/888020132 learned_addr learned my addr 172.21.15.57:0/888020132 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:38.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f588ca07700 1 -- 172.21.15.57:0/888020132 >> v2:172.21.15.57:3300/0 conn(0x7f58880d15f0 msgr2=0x7f588815b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:38.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f588ca07700 1 --2- 172.21.15.57:0/888020132 >> v2:172.21.15.57:3300/0 conn(0x7f58880d15f0 0x7f588815b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f588ca07700 1 -- 172.21.15.57:0/888020132 >> v1:172.21.15.181:6789/0 conn(0x7f588806e400 legacy=0x7f588806e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:38.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f5886ffd700 1 -- 172.21.15.57:0/888020132 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225643422 0 0) 0x7f588815af30 con 0x7f588806e400 2023-11-28T12:01:38.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f588ca07700 1 -- 172.21.15.57:0/888020132 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588815ef40 con 0x7f588815e010 2023-11-28T12:01:38.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f58877fe700 1 --2- 172.21.15.57:0/888020132 >> v2:172.21.15.57:3300/0 conn(0x7f58880d15f0 0x7f588815b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:38.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f588ca07700 1 --2- 172.21.15.57:0/888020132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f588815e010 0x7f58880ce970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f58880b60f0 tx=0x7f588000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0acb10930862e23 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:38.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.412+0000 7f5886ffd700 1 -- 172.21.15.57:0/888020132 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588000e070 con 0x7f588815e010 2023-11-28T12:01:38.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.413+0000 7f5886ffd700 1 -- 172.21.15.57:0/888020132 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588000bb20 con 0x7f588815e010 2023-11-28T12:01:38.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.413+0000 7f5886ffd700 1 -- 172.21.15.57:0/888020132 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880012430 con 0x7f588815e010 2023-11-28T12:01:38.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.413+0000 7f588e46a700 1 -- 172.21.15.57:0/888020132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f588815e010 msgr2=0x7f58880ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:38.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.413+0000 7f588e46a700 1 --2- 172.21.15.57:0/888020132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f588815e010 0x7f58880ce970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f58880b60f0 tx=0x7f588000b330 comp rx=0 tx=0).stop 2023-11-28T12:01:38.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.413+0000 7f588e46a700 1 -- 172.21.15.57:0/888020132 shutdown_connections 2023-11-28T12:01:38.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.413+0000 7f588e46a700 1 --2- 172.21.15.57:0/888020132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f588815e010 0x7f58880ce970 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.413+0000 7f588e46a700 1 --2- 172.21.15.57:0/888020132 >> v2:172.21.15.57:3300/0 conn(0x7f58880d15f0 0x7f588815b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.413+0000 7f588e46a700 1 -- 172.21.15.57:0/888020132 >> 172.21.15.57:0/888020132 conn(0x7f588806a270 msgr2=0x7f588806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:38.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.413+0000 7f588e46a700 1 -- 172.21.15.57:0/888020132 shutdown_connections 2023-11-28T12:01:38.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.413+0000 7f588e46a700 1 -- 172.21.15.57:0/888020132 wait complete. 2023-11-28T12:01:38.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.414+0000 7f588e46a700 1 Processor -- start 2023-11-28T12:01:38.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.415+0000 7f588e46a700 1 -- start start 2023-11-28T12:01:38.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.415+0000 7f588e46a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f588806e400 0x7f58881f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:38.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.415+0000 7f588e46a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58880d15f0 0x7f58881521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:38.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.415+0000 7f588e46a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f588815e010 0x7f58880b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:38.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.415+0000 7f588e46a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5888139dc0 con 0x7f588815e010 2023-11-28T12:01:38.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.415+0000 7f588e46a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880b0760 con 0x7f588806e400 2023-11-28T12:01:38.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.415+0000 7f588e46a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5888139140 con 0x7f58880d15f0 2023-11-28T12:01:38.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f588ca07700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f588815e010 0x7f58880b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:38.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f588ca07700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f588815e010 0x7f58880b4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39352/0 (socket says 172.21.15.57:39352) 2023-11-28T12:01:38.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f588ca07700 1 -- 172.21.15.57:0/3482050473 learned_addr learned my addr 172.21.15.57:0/3482050473 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:38.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f5887fff700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f588806e400 0x7f58881f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:38.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f58877fe700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58880d15f0 0x7f58881521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:38.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f588ca07700 1 -- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58880d15f0 msgr2=0x7f58881521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:38.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f588ca07700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58880d15f0 0x7f58881521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f588ca07700 1 -- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f588806e400 msgr2=0x7f58881f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:38.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f588ca07700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f588806e400 0x7f58881f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f588ca07700 1 -- 172.21.15.57:0/3482050473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c009550 con 0x7f588815e010 2023-11-28T12:01:38.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f588ca07700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f588815e010 0x7f58880b4cb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f588806eee0 tx=0x7f5880004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:38.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f5884ff9700 1 -- 172.21.15.57:0/3482050473 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880012430 con 0x7f588815e010 2023-11-28T12:01:38.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f5884ff9700 1 -- 172.21.15.57:0/3482050473 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5880020070 con 0x7f588815e010 2023-11-28T12:01:38.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f5884ff9700 1 -- 172.21.15.57:0/3482050473 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880004280 con 0x7f588815e010 2023-11-28T12:01:38.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f588e46a700 1 -- 172.21.15.57:0/3482050473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5880009a70 con 0x7f588815e010 2023-11-28T12:01:38.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.416+0000 7f588e46a700 1 -- 172.21.15.57:0/3482050473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588815ec60 con 0x7f588815e010 2023-11-28T12:01:38.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.417+0000 7f588e46a700 1 -- 172.21.15.57:0/3482050473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588815e980 con 0x7f588815e010 2023-11-28T12:01:38.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.418+0000 7f5884ff9700 1 -- 172.21.15.57:0/3482050473 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f588002e070 con 0x7f588815e010 2023-11-28T12:01:38.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.418+0000 7f5884ff9700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f58740682a0 0x7f587406a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:38.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.418+0000 7f5884ff9700 1 -- 172.21.15.57:0/3482050473 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f5880022070 con 0x7f588815e010 2023-11-28T12:01:38.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.419+0000 7f5887fff700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f58740682a0 0x7f587406a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:38.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.419+0000 7f5887fff700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f58740682a0 0x7f587406a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5878007fd0 tx=0x7f5878007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:38.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.423+0000 7f5884ff9700 1 -- 172.21.15.57:0/3482050473 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5880054150 con 0x7f588815e010 2023-11-28T12:01:38.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.626+0000 7f588e46a700 1 -- 172.21.15.57:0/3482050473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f588815abc0 con 0x7f588815e010 2023-11-28T12:01:38.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.627+0000 7f5884ff9700 1 -- 172.21.15.57:0/3482050473 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+14225 (secure 0 0 0) 0x7f5880057770 con 0x7f588815e010 2023-11-28T12:01:38.627 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:38.627 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":15,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:38.223765+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:01:38.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.629+0000 7f588e46a700 1 -- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f58740682a0 msgr2=0x7f587406a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:38.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.629+0000 7f588e46a700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f58740682a0 0x7f587406a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5878007fd0 tx=0x7f5878007de0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.629+0000 7f588e46a700 1 -- 172.21.15.57:0/3482050473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f588815e010 msgr2=0x7f58880b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:38.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.629+0000 7f588e46a700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f588815e010 0x7f58880b4cb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f588806eee0 tx=0x7f5880004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.629+0000 7f588e46a700 1 -- 172.21.15.57:0/3482050473 shutdown_connections 2023-11-28T12:01:38.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.629+0000 7f588e46a700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f58740682a0 0x7f587406a750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.629+0000 7f588e46a700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f588815e010 0x7f58880b4cb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.629+0000 7f588e46a700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f588806e400 0x7f58881f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.629+0000 7f588e46a700 1 --2- 172.21.15.57:0/3482050473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58880d15f0 0x7f58881521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.629+0000 7f588e46a700 1 -- 172.21.15.57:0/3482050473 >> 172.21.15.57:0/3482050473 conn(0x7f588806a270 msgr2=0x7f58881571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:38.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.629+0000 7f588e46a700 1 -- 172.21.15.57:0/3482050473 shutdown_connections 2023-11-28T12:01:38.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.629+0000 7f588e46a700 1 -- 172.21.15.57:0/3482050473 wait complete. 2023-11-28T12:01:38.643 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:01:38.644 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:01:38.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.782+0000 7f3ea8513700 1 Processor -- start 2023-11-28T12:01:38.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.782+0000 7f3ea8513700 1 -- start start 2023-11-28T12:01:38.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea8513700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3ea00b2ee0 0x7f3ea00d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:38.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea8513700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea00cc200 0x7f3ea00ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:38.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea8513700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea015a9c0 con 0x7f3ea015a5f0 2023-11-28T12:01:38.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea8513700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea00c7b80 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea8513700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00cc990 con 0x7f3ea00cc200 2023-11-28T12:01:38.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea5aae700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3ea00b2ee0 0x7f3ea00d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:38.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea5aae700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3ea00b2ee0 0x7f3ea00d0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39356/0 (socket says 172.21.15.57:39356) 2023-11-28T12:01:38.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea5aae700 1 -- 172.21.15.57:0/3969569912 learned_addr learned my addr 172.21.15.57:0/3969569912 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:38.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea6ab0700 1 --2- 172.21.15.57:0/3969569912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea00cc200 0x7f3ea00ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:38.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea5aae700 1 -- 172.21.15.57:0/3969569912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea00cc200 msgr2=0x7f3ea00ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:38.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea5aae700 1 --2- 172.21.15.57:0/3969569912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea00cc200 0x7f3ea00ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea5aae700 1 -- 172.21.15.57:0/3969569912 >> v1:172.21.15.181:6789/0 conn(0x7f3ea015a5f0 legacy=0x7f3ea00b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:38.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea5aae700 1 -- 172.21.15.57:0/3969569912 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea00ccc70 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea5aae700 1 --2- 172.21.15.57:0/3969569912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ea00b2ee0 0x7f3ea00d0ad0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e940098f0 tx=0x7f3e94009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd71b4f892844c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:38.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea52ad700 1 -- 172.21.15.57:0/3969569912 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94004230 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea52ad700 1 -- 172.21.15.57:0/3969569912 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e94004390 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.784+0000 7f3ea52ad700 1 -- 172.21.15.57:0/3969569912 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940036a0 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.785+0000 7f3ea8513700 1 -- 172.21.15.57:0/3969569912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ea00b2ee0 msgr2=0x7f3ea00d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:38.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.785+0000 7f3ea8513700 1 --2- 172.21.15.57:0/3969569912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ea00b2ee0 0x7f3ea00d0ad0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e940098f0 tx=0x7f3e94009c00 comp rx=0 tx=0).stop 2023-11-28T12:01:38.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.785+0000 7f3ea8513700 1 -- 172.21.15.57:0/3969569912 shutdown_connections 2023-11-28T12:01:38.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.785+0000 7f3ea8513700 1 --2- 172.21.15.57:0/3969569912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea00cc200 0x7f3ea00ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.785+0000 7f3ea8513700 1 --2- 172.21.15.57:0/3969569912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ea00b2ee0 0x7f3ea00d0ad0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.785+0000 7f3ea8513700 1 -- 172.21.15.57:0/3969569912 >> 172.21.15.57:0/3969569912 conn(0x7f3ea006ea80 msgr2=0x7f3ea006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:38.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.785+0000 7f3ea8513700 1 -- 172.21.15.57:0/3969569912 shutdown_connections 2023-11-28T12:01:38.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.785+0000 7f3ea8513700 1 -- 172.21.15.57:0/3969569912 wait complete. 2023-11-28T12:01:38.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.786+0000 7f3ea8513700 1 Processor -- start 2023-11-28T12:01:38.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.786+0000 7f3ea8513700 1 -- start start 2023-11-28T12:01:38.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.787+0000 7f3ea8513700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ea00b2ee0 0x7f3ea014b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:38.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.787+0000 7f3ea8513700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea00cc200 0x7f3ea00b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:38.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.787+0000 7f3ea8513700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea015a5f0 0x7f3ea00b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:38.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.787+0000 7f3ea8513700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01f4210 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.787+0000 7f3ea8513700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea014b740 con 0x7f3ea015a5f0 2023-11-28T12:01:38.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.787+0000 7f3ea8513700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01f3590 con 0x7f3ea00cc200 2023-11-28T12:01:38.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.787+0000 7f3ea62af700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ea00b2ee0 0x7f3ea014b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:38.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.787+0000 7f3ea62af700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ea00b2ee0 0x7f3ea014b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39368/0 (socket says 172.21.15.57:39368) 2023-11-28T12:01:38.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.787+0000 7f3ea62af700 1 -- 172.21.15.57:0/2541937822 learned_addr learned my addr 172.21.15.57:0/2541937822 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:38.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.787+0000 7f3ea6ab0700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea015a5f0 0x7f3ea00b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:38.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.787+0000 7f3ea5aae700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea00cc200 0x7f3ea00b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:38.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.788+0000 7f3ea62af700 1 -- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea00cc200 msgr2=0x7f3ea00b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:38.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.788+0000 7f3ea62af700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea00cc200 0x7f3ea00b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.788+0000 7f3ea62af700 1 -- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea015a5f0 msgr2=0x7f3ea00b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.788+0000 7f3ea62af700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea015a5f0 0x7f3ea00b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.788+0000 7f3ea62af700 1 -- 172.21.15.57:0/2541937822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e94009550 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.788+0000 7f3ea5aae700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea00cc200 0x7f3ea00b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.788+0000 7f3ea62af700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ea00b2ee0 0x7f3ea014b200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00c4d0 tx=0x7f3e9c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.788+0000 7f3e92ffd700 1 -- 172.21.15.57:0/2541937822 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c011790 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.788+0000 7f3e92ffd700 1 -- 172.21.15.57:0/2541937822 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c0118f0 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.788+0000 7f3ea8513700 1 -- 172.21.15.57:0/2541937822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea00cd230 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.788+0000 7f3e92ffd700 1 -- 172.21.15.57:0/2541937822 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c011ca0 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.788+0000 7f3ea8513700 1 -- 172.21.15.57:0/2541937822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea00ccf50 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.789+0000 7f3ea8513700 1 -- 172.21.15.57:0/2541937822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea00cd510 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.789+0000 7f3e92ffd700 1 -- 172.21.15.57:0/2541937822 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f3e9c010050 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.790+0000 7f3e92ffd700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3e840682a0 0x7f3e8406a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.790+0000 7f3e92ffd700 1 -- 172.21.15.57:0/2541937822 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f3e9c085640 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.790+0000 7f3ea5aae700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3e840682a0 0x7f3e8406a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:38.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.790+0000 7f3ea5aae700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3e840682a0 0x7f3e8406a750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00b32d0 tx=0x7f3e94008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:38.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.794+0000 7f3e92ffd700 1 -- 172.21.15.57:0/2541937822 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3e9c052250 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.993+0000 7f3ea8513700 1 -- 172.21.15.57:0/2541937822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3ea01f4350 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.994+0000 7f3e92ffd700 1 -- 172.21.15.57:0/2541937822 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3ea01f4350 con 0x7f3ea00b2ee0 2023-11-28T12:01:38.994 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:01:38.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.996+0000 7f3ea8513700 1 -- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3e840682a0 msgr2=0x7f3e8406a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:38.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.996+0000 7f3ea8513700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3e840682a0 0x7f3e8406a750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00b32d0 tx=0x7f3e94008030 comp rx=0 tx=0).stop 2023-11-28T12:01:38.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.996+0000 7f3ea8513700 1 -- 172.21.15.57:0/2541937822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ea00b2ee0 msgr2=0x7f3ea014b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:38.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.996+0000 7f3ea8513700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ea00b2ee0 0x7f3ea014b200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00c4d0 tx=0x7f3e9c00c890 comp rx=0 tx=0).stop 2023-11-28T12:01:38.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.996+0000 7f3ea8513700 1 -- 172.21.15.57:0/2541937822 shutdown_connections 2023-11-28T12:01:38.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.996+0000 7f3ea8513700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3e840682a0 0x7f3e8406a750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.996+0000 7f3ea8513700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea015a5f0 0x7f3ea00b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.996+0000 7f3ea8513700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ea00b2ee0 0x7f3ea014b200 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.996+0000 7f3ea8513700 1 --2- 172.21.15.57:0/2541937822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea00cc200 0x7f3ea00b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:38.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.996+0000 7f3ea8513700 1 -- 172.21.15.57:0/2541937822 >> 172.21.15.57:0/2541937822 conn(0x7f3ea006ea80 msgr2=0x7f3ea006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:38.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.996+0000 7f3ea8513700 1 -- 172.21.15.57:0/2541937822 shutdown_connections 2023-11-28T12:01:38.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:38.996+0000 7f3ea8513700 1 -- 172.21.15.57:0/2541937822 wait complete. 2023-11-28T12:01:39.010 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:01:39.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.148+0000 7fa769538700 1 Processor -- start 2023-11-28T12:01:39.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.148+0000 7fa769538700 1 -- start start 2023-11-28T12:01:39.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.149+0000 7fa769538700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa76415a4d0 0x7fa76406a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.149+0000 7fa769538700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa7640cdf20 0x7fa7640cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.149+0000 7fa769538700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76406ac00 con 0x7fa76406a7b0 2023-11-28T12:01:39.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.149+0000 7fa769538700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7640ce360 con 0x7fa7640cdf20 2023-11-28T12:01:39.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.149+0000 7fa769538700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76415e0d0 con 0x7fa76415a4d0 2023-11-28T12:01:39.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.149+0000 7fa7637fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa7640cdf20 0x7fa7640cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.150+0000 7fa7637fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa7640cdf20 0x7fa7640cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39376/0 (socket says 172.21.15.57:39376) 2023-11-28T12:01:39.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.150+0000 7fa7637fe700 1 -- 172.21.15.57:0/1587920341 learned_addr learned my addr 172.21.15.57:0/1587920341 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:39.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.150+0000 7fa762ffd700 1 --2- 172.21.15.57:0/1587920341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa76415a4d0 0x7fa76406a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.150+0000 7fa7637fe700 1 -- 172.21.15.57:0/1587920341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa76415a4d0 msgr2=0x7fa76406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:39.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.150+0000 7fa7637fe700 1 --2- 172.21.15.57:0/1587920341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa76415a4d0 0x7fa76406a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.150+0000 7fa7637fe700 1 -- 172.21.15.57:0/1587920341 >> v1:172.21.15.181:6789/0 conn(0x7fa76406a7b0 legacy=0x7fa7640cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.150+0000 7fa7637fe700 1 -- 172.21.15.57:0/1587920341 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7640ce560 con 0x7fa7640cdf20 2023-11-28T12:01:39.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.150+0000 7fa762ffd700 1 --2- 172.21.15.57:0/1587920341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa76415a4d0 0x7fa76406a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:39.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.150+0000 7fa7637fe700 1 --2- 172.21.15.57:0/1587920341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7640cdf20 0x7fa7640cb010 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009e50 tx=0x7fa75c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae5f710b3e8e439 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:39.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.150+0000 7fa761ffb700 1 -- 172.21.15.57:0/1587920341 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00e070 con 0x7fa7640cdf20 2023-11-28T12:01:39.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.150+0000 7fa761ffb700 1 -- 172.21.15.57:0/1587920341 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c00b9c0 con 0x7fa7640cdf20 2023-11-28T12:01:39.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.150+0000 7fa761ffb700 1 -- 172.21.15.57:0/1587920341 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00bd70 con 0x7fa7640cdf20 2023-11-28T12:01:39.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.151+0000 7fa769538700 1 -- 172.21.15.57:0/1587920341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7640cdf20 msgr2=0x7fa7640cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.151+0000 7fa769538700 1 --2- 172.21.15.57:0/1587920341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7640cdf20 0x7fa7640cb010 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009e50 tx=0x7fa75c00b330 comp rx=0 tx=0).stop 2023-11-28T12:01:39.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.151+0000 7fa769538700 1 -- 172.21.15.57:0/1587920341 shutdown_connections 2023-11-28T12:01:39.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.151+0000 7fa769538700 1 --2- 172.21.15.57:0/1587920341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7640cdf20 0x7fa7640cb010 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.151+0000 7fa769538700 1 --2- 172.21.15.57:0/1587920341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa76415a4d0 0x7fa76406a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.151+0000 7fa769538700 1 -- 172.21.15.57:0/1587920341 >> 172.21.15.57:0/1587920341 conn(0x7fa76406c010 msgr2=0x7fa7640cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:39.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.151+0000 7fa769538700 1 -- 172.21.15.57:0/1587920341 shutdown_connections 2023-11-28T12:01:39.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.151+0000 7fa769538700 1 -- 172.21.15.57:0/1587920341 wait complete. 2023-11-28T12:01:39.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.152+0000 7fa769538700 1 Processor -- start 2023-11-28T12:01:39.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.152+0000 7fa769538700 1 -- start start 2023-11-28T12:01:39.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.153+0000 7fa769538700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa76406a7b0 0x7fa7640b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.153+0000 7fa769538700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7640cdf20 0x7fa7640b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.153+0000 7fa769538700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa76415a4d0 0x7fa7640b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.153+0000 7fa769538700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7641f2810 con 0x7fa76406a7b0 2023-11-28T12:01:39.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.153+0000 7fa769538700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76406e8d0 con 0x7fa7640cdf20 2023-11-28T12:01:39.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.153+0000 7fa769538700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7641543b0 con 0x7fa76415a4d0 2023-11-28T12:01:39.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.153+0000 7fa762ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa76406a7b0 0x7fa7640b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.153+0000 7fa762ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa76406a7b0 0x7fa7640b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39388/0 (socket says 172.21.15.57:39388) 2023-11-28T12:01:39.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.153+0000 7fa762ffd700 1 -- 172.21.15.57:0/3965153298 learned_addr learned my addr 172.21.15.57:0/3965153298 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:39.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.153+0000 7fa7627fc700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7640cdf20 0x7fa7640b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.153+0000 7fa7637fe700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa76415a4d0 0x7fa7640b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.154+0000 7fa762ffd700 1 -- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa76415a4d0 msgr2=0x7fa7640b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.154+0000 7fa762ffd700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa76415a4d0 0x7fa7640b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.154+0000 7fa762ffd700 1 -- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7640cdf20 msgr2=0x7fa7640b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.154+0000 7fa762ffd700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7640cdf20 0x7fa7640b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.154+0000 7fa762ffd700 1 -- 172.21.15.57:0/3965153298 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c009b30 con 0x7fa76406a7b0 2023-11-28T12:01:39.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.154+0000 7fa762ffd700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa76406a7b0 0x7fa7640b1de0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa75400e810 tx=0x7fa75400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:39.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.154+0000 7fa74b7fe700 1 -- 172.21.15.57:0/3965153298 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75400c020 con 0x7fa76406a7b0 2023-11-28T12:01:39.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.154+0000 7fa74b7fe700 1 -- 172.21.15.57:0/3965153298 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa754004250 con 0x7fa76406a7b0 2023-11-28T12:01:39.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.154+0000 7fa769538700 1 -- 172.21.15.57:0/3965153298 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76415e600 con 0x7fa76406a7b0 2023-11-28T12:01:39.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.154+0000 7fa74b7fe700 1 -- 172.21.15.57:0/3965153298 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7540036a0 con 0x7fa76406a7b0 2023-11-28T12:01:39.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.154+0000 7fa769538700 1 -- 172.21.15.57:0/3965153298 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76415e320 con 0x7fa76406a7b0 2023-11-28T12:01:39.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.155+0000 7fa769538700 1 -- 172.21.15.57:0/3965153298 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76415e8e0 con 0x7fa76406a7b0 2023-11-28T12:01:39.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.155+0000 7fa74b7fe700 1 -- 172.21.15.57:0/3965153298 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fa754003800 con 0x7fa76406a7b0 2023-11-28T12:01:39.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.156+0000 7fa74b7fe700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa7500682a0 0x7fa75006a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.156+0000 7fa74b7fe700 1 -- 172.21.15.57:0/3965153298 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5199+0+0 (secure 0 0 0) 0x7fa754085450 con 0x7fa76406a7b0 2023-11-28T12:01:39.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.156+0000 7fa7627fc700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa7500682a0 0x7fa75006a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.156+0000 7fa7627fc700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa7500682a0 0x7fa75006a750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa7640b58d0 tx=0x7fa758009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:39.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.160+0000 7fa74b7fe700 1 -- 172.21.15.57:0/3965153298 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa754052060 con 0x7fa76406a7b0 2023-11-28T12:01:39.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.378+0000 7fa769538700 1 -- 172.21.15.57:0/3965153298 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7641f3610 con 0x7fa76406a7b0 2023-11-28T12:01:39.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.379+0000 7fa74b7fe700 1 -- 172.21.15.57:0/3965153298 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa754055680 con 0x7fa76406a7b0 2023-11-28T12:01:39.379 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:01:39.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.381+0000 7fa769538700 1 -- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa7500682a0 msgr2=0x7fa75006a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.381+0000 7fa769538700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa7500682a0 0x7fa75006a750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa7640b58d0 tx=0x7fa758009040 comp rx=0 tx=0).stop 2023-11-28T12:01:39.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.381+0000 7fa769538700 1 -- 172.21.15.57:0/3965153298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa76406a7b0 msgr2=0x7fa7640b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.381+0000 7fa769538700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa76406a7b0 0x7fa7640b1de0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa75400e810 tx=0x7fa75400eb20 comp rx=0 tx=0).stop 2023-11-28T12:01:39.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.381+0000 7fa769538700 1 -- 172.21.15.57:0/3965153298 shutdown_connections 2023-11-28T12:01:39.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.381+0000 7fa769538700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa7500682a0 0x7fa75006a750 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.381+0000 7fa769538700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa76415a4d0 0x7fa7640b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.381+0000 7fa769538700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa76406a7b0 0x7fa7640b1de0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.381+0000 7fa769538700 1 --2- 172.21.15.57:0/3965153298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7640cdf20 0x7fa7640b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.381+0000 7fa769538700 1 -- 172.21.15.57:0/3965153298 >> 172.21.15.57:0/3965153298 conn(0x7fa76406c010 msgr2=0x7fa76415cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:39.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.381+0000 7fa769538700 1 -- 172.21.15.57:0/3965153298 shutdown_connections 2023-11-28T12:01:39.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.381+0000 7fa769538700 1 -- 172.21.15.57:0/3965153298 wait complete. 2023-11-28T12:01:39.395 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:01:39.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.535+0000 7feedfb5a700 1 Processor -- start 2023-11-28T12:01:39.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.535+0000 7feedfb5a700 1 -- start start 2023-11-28T12:01:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedfb5a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7feed8142db0 0x7feed813ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedfb5a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feed80b1770 0x7feed80b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedfb5a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed815e010 con 0x7feed80ce040 2023-11-28T12:01:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedfb5a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed813fc90 con 0x7feed8142db0 2023-11-28T12:01:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedfb5a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed80b20d0 con 0x7feed80b1770 2023-11-28T12:01:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedd0f5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7feed8142db0 0x7feed813ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedd8f6700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7feed80ce040 0x7feed815d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:35728/0 (socket says 172.21.15.57:35728) 2023-11-28T12:01:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedd8f6700 1 -- 172.21.15.57:0/467413529 learned_addr learned my addr 172.21.15.57:0/467413529 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feede0f7700 1 --2- 172.21.15.57:0/467413529 >> v2:172.21.15.181:3300/0 conn(0x7feed80b1770 0x7feed80b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedc8f4700 1 -- 172.21.15.57:0/467413529 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468915484 0 0) 0x7feed4001490 con 0x7feed80ce040 2023-11-28T12:01:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedc8f4700 1 -- 172.21.15.57:0/467413529 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec40035b0 con 0x7feed80ce040 2023-11-28T12:01:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedd0f5700 1 -- 172.21.15.57:0/467413529 >> v2:172.21.15.181:3300/0 conn(0x7feed80b1770 msgr2=0x7feed80b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedd0f5700 1 --2- 172.21.15.57:0/467413529 >> v2:172.21.15.181:3300/0 conn(0x7feed80b1770 0x7feed80b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedd0f5700 1 -- 172.21.15.57:0/467413529 >> v1:172.21.15.57:6789/0 conn(0x7feed80ce040 legacy=0x7feed815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feedd0f5700 1 -- 172.21.15.57:0/467413529 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed813f6d0 con 0x7feed8142db0 2023-11-28T12:01:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.536+0000 7feede0f7700 1 --2- 172.21.15.57:0/467413529 >> v2:172.21.15.181:3300/0 conn(0x7feed80b1770 0x7feed80b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.537+0000 7feedd0f5700 1 --2- 172.21.15.57:0/467413529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feed8142db0 0x7feed813ed70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feed8142cd0 tx=0x7feec8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef56a73686d44aa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.537+0000 7feedc8f4700 1 -- 172.21.15.57:0/467413529 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8011830 con 0x7feed8142db0 2023-11-28T12:01:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.537+0000 7feedc8f4700 1 -- 172.21.15.57:0/467413529 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec8011990 con 0x7feed8142db0 2023-11-28T12:01:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.537+0000 7feedc8f4700 1 -- 172.21.15.57:0/467413529 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8011d20 con 0x7feed8142db0 2023-11-28T12:01:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.537+0000 7feedfb5a700 1 -- 172.21.15.57:0/467413529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feed8142db0 msgr2=0x7feed813ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.537+0000 7feedfb5a700 1 --2- 172.21.15.57:0/467413529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feed8142db0 0x7feed813ed70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feed8142cd0 tx=0x7feec8009b20 comp rx=0 tx=0).stop 2023-11-28T12:01:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.537+0000 7feedfb5a700 1 -- 172.21.15.57:0/467413529 shutdown_connections 2023-11-28T12:01:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.537+0000 7feedfb5a700 1 --2- 172.21.15.57:0/467413529 >> v2:172.21.15.181:3300/0 conn(0x7feed80b1770 0x7feed80b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.537+0000 7feedfb5a700 1 --2- 172.21.15.57:0/467413529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feed8142db0 0x7feed813ed70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.537+0000 7feedfb5a700 1 -- 172.21.15.57:0/467413529 >> 172.21.15.57:0/467413529 conn(0x7feed806c010 msgr2=0x7feed80cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.538+0000 7feedfb5a700 1 -- 172.21.15.57:0/467413529 shutdown_connections 2023-11-28T12:01:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.538+0000 7feedfb5a700 1 -- 172.21.15.57:0/467413529 wait complete. 2023-11-28T12:01:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.540+0000 7feedfb5a700 1 Processor -- start 2023-11-28T12:01:39.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.540+0000 7feedfb5a700 1 -- start start 2023-11-28T12:01:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.542+0000 7feedfb5a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feed80b1770 0x7feed8155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.542+0000 7feedfb5a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feed80ce040 0x7feed814abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.542+0000 7feedfb5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feed8142db0 0x7feed8159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.542+0000 7feedfb5a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed80ca320 con 0x7feed80ce040 2023-11-28T12:01:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.542+0000 7feedfb5a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed814b130 con 0x7feed8142db0 2023-11-28T12:01:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.542+0000 7feedfb5a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed80ca530 con 0x7feed80b1770 2023-11-28T12:01:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.542+0000 7feedd0f5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feed80ce040 0x7feed814abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.542+0000 7feedd0f5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feed80ce040 0x7feed814abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39412/0 (socket says 172.21.15.57:39412) 2023-11-28T12:01:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.542+0000 7feedd0f5700 1 -- 172.21.15.57:0/811565454 learned_addr learned my addr 172.21.15.57:0/811565454 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.542+0000 7feedd8f6700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feed80b1770 0x7feed8155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.543+0000 7feede0f7700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feed8142db0 0x7feed8159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.543+0000 7feedd0f5700 1 -- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feed80b1770 msgr2=0x7feed8155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.543+0000 7feedd0f5700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feed80b1770 0x7feed8155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.543+0000 7feedd0f5700 1 -- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feed8142db0 msgr2=0x7feed8159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.543+0000 7feedd0f5700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feed8142db0 0x7feed8159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.543+0000 7feedd0f5700 1 -- 172.21.15.57:0/811565454 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec80094a0 con 0x7feed80ce040 2023-11-28T12:01:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.543+0000 7feedd0f5700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feed80ce040 0x7feed814abf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feec80090b0 tx=0x7feec8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.543+0000 7feece7fc700 1 -- 172.21.15.57:0/811565454 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8020070 con 0x7feed80ce040 2023-11-28T12:01:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.543+0000 7feece7fc700 1 -- 172.21.15.57:0/811565454 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec8004370 con 0x7feed80ce040 2023-11-28T12:01:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.543+0000 7feedfb5a700 1 -- 172.21.15.57:0/811565454 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed815d100 con 0x7feed80ce040 2023-11-28T12:01:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.543+0000 7feece7fc700 1 -- 172.21.15.57:0/811565454 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec80036a0 con 0x7feed80ce040 2023-11-28T12:01:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.543+0000 7feedfb5a700 1 -- 172.21.15.57:0/811565454 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed813f9b0 con 0x7feed80ce040 2023-11-28T12:01:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.544+0000 7feedfb5a700 1 -- 172.21.15.57:0/811565454 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed815d3e0 con 0x7feed80ce040 2023-11-28T12:01:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.545+0000 7feece7fc700 1 -- 172.21.15.57:0/811565454 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7feec802d030 con 0x7feed80ce040 2023-11-28T12:01:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.545+0000 7feece7fc700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feebc0682a0 0x7feebc06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.545+0000 7feece7fc700 1 -- 172.21.15.57:0/811565454 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7feec80886c0 con 0x7feed80ce040 2023-11-28T12:01:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.545+0000 7feedd8f6700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feebc0682a0 0x7feebc06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.546+0000 7feedd8f6700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feebc0682a0 0x7feebc06a750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feed40015f0 tx=0x7feed4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:39.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.550+0000 7feece7fc700 1 -- 172.21.15.57:0/811565454 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feec80552c0 con 0x7feed80ce040 2023-11-28T12:01:39.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.800+0000 7feedfb5a700 1 -- 172.21.15.57:0/811565454 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feed80743f0 con 0x7feed80ce040 2023-11-28T12:01:39.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.801+0000 7feece7fc700 1 -- 172.21.15.57:0/811565454 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7feec802bde0 con 0x7feed80ce040 2023-11-28T12:01:39.801 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:39.801 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/890913444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":890913444},{"type":"v1","addr":"172.21.15.181:6836","nonce":890913444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/847658150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":847658150},{"type":"v1","addr":"172.21.15.181:6837","nonce":847658150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1379013929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1379013929},{"type":"v1","addr":"172.21.15.57:6835","nonce":1379013929}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/3776085595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3776085595},{"type":"v1","addr":"172.21.15.57:6833","nonce":3776085595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-28T12:01:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.803+0000 7feedfb5a700 1 -- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feebc0682a0 msgr2=0x7feebc06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.803+0000 7feedfb5a700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feebc0682a0 0x7feebc06a750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feed40015f0 tx=0x7feed4009040 comp rx=0 tx=0).stop 2023-11-28T12:01:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.803+0000 7feedfb5a700 1 -- 172.21.15.57:0/811565454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feed80ce040 msgr2=0x7feed814abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.803+0000 7feedfb5a700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feed80ce040 0x7feed814abf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feec80090b0 tx=0x7feec8004750 comp rx=0 tx=0).stop 2023-11-28T12:01:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.803+0000 7feedfb5a700 1 -- 172.21.15.57:0/811565454 shutdown_connections 2023-11-28T12:01:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.803+0000 7feedfb5a700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feebc0682a0 0x7feebc06a750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.803+0000 7feedfb5a700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feed8142db0 0x7feed8159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.803+0000 7feedfb5a700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feed80b1770 0x7feed8155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.803+0000 7feedfb5a700 1 --2- 172.21.15.57:0/811565454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feed80ce040 0x7feed814abf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.803+0000 7feedfb5a700 1 -- 172.21.15.57:0/811565454 >> 172.21.15.57:0/811565454 conn(0x7feed806c010 msgr2=0x7feed81583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.803+0000 7feedfb5a700 1 -- 172.21.15.57:0/811565454 shutdown_connections 2023-11-28T12:01:39.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.803+0000 7feedfb5a700 1 -- 172.21.15.57:0/811565454 wait complete. 2023-11-28T12:01:39.805 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 3 2023-11-28T12:01:39.817 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:01:39.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.954+0000 7f1d1d4c0700 1 Processor -- start 2023-11-28T12:01:39.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.954+0000 7f1d1d4c0700 1 -- start start 2023-11-28T12:01:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.955+0000 7f1d1d4c0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1d1806ab10 0x7f1d1806aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.955+0000 7f1d1d4c0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1d180cd5a0 0x7f1d18160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.955+0000 7f1d1d4c0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d180d1ee0 con 0x7f1d180b22a0 2023-11-28T12:01:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.955+0000 7f1d1d4c0700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1806b420 con 0x7f1d180cd5a0 2023-11-28T12:01:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.955+0000 7f1d1d4c0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d180cdb10 con 0x7f1d1806ab10 2023-11-28T12:01:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.955+0000 7f1d167fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1d180cd5a0 0x7f1d18160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.955+0000 7f1d167fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1d180cd5a0 0x7f1d18160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39428/0 (socket says 172.21.15.57:39428) 2023-11-28T12:01:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.955+0000 7f1d167fc700 1 -- 172.21.15.57:0/2312492100 learned_addr learned my addr 172.21.15.57:0/2312492100 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.955+0000 7f1d16ffd700 1 --2- 172.21.15.57:0/2312492100 >> v2:172.21.15.181:3300/0 conn(0x7f1d1806ab10 0x7f1d1806aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d167fc700 1 -- 172.21.15.57:0/2312492100 >> v2:172.21.15.181:3300/0 conn(0x7f1d1806ab10 msgr2=0x7f1d1806aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d167fc700 1 --2- 172.21.15.57:0/2312492100 >> v2:172.21.15.181:3300/0 conn(0x7f1d1806ab10 0x7f1d1806aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d167fc700 1 -- 172.21.15.57:0/2312492100 >> v1:172.21.15.181:6789/0 conn(0x7f1d180b22a0 legacy=0x7f1d18160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d15ffb700 1 -- 172.21.15.57:0/2312492100 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277228513 0 0) 0x7f1d180d1ee0 con 0x7f1d180b22a0 2023-11-28T12:01:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d167fc700 1 -- 172.21.15.57:0/2312492100 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d180d1890 con 0x7f1d180cd5a0 2023-11-28T12:01:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d167fc700 1 --2- 172.21.15.57:0/2312492100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d180cd5a0 0x7f1d18160300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1d180d0cb0 tx=0x7f1d00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6ae41c9fa12529 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d15ffb700 1 -- 172.21.15.57:0/2312492100 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00011830 con 0x7f1d180cd5a0 2023-11-28T12:01:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d15ffb700 1 -- 172.21.15.57:0/2312492100 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d00011990 con 0x7f1d180cd5a0 2023-11-28T12:01:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d15ffb700 1 -- 172.21.15.57:0/2312492100 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00011d20 con 0x7f1d180cd5a0 2023-11-28T12:01:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d1d4c0700 1 -- 172.21.15.57:0/2312492100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d180cd5a0 msgr2=0x7f1d18160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d1d4c0700 1 --2- 172.21.15.57:0/2312492100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d180cd5a0 0x7f1d18160300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1d180d0cb0 tx=0x7f1d00009bd0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d1d4c0700 1 -- 172.21.15.57:0/2312492100 shutdown_connections 2023-11-28T12:01:39.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d1d4c0700 1 --2- 172.21.15.57:0/2312492100 >> v2:172.21.15.181:3300/0 conn(0x7f1d1806ab10 0x7f1d1806aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d1d4c0700 1 --2- 172.21.15.57:0/2312492100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d180cd5a0 0x7f1d18160300 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.956+0000 7f1d1d4c0700 1 -- 172.21.15.57:0/2312492100 >> 172.21.15.57:0/2312492100 conn(0x7f1d1806c520 msgr2=0x7f1d1806c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:39.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.957+0000 7f1d1d4c0700 1 -- 172.21.15.57:0/2312492100 shutdown_connections 2023-11-28T12:01:39.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.957+0000 7f1d1d4c0700 1 -- 172.21.15.57:0/2312492100 wait complete. 2023-11-28T12:01:39.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.958+0000 7f1d1d4c0700 1 Processor -- start 2023-11-28T12:01:39.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.958+0000 7f1d1d4c0700 1 -- start start 2023-11-28T12:01:39.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d1d4c0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1806ab10 0x7f1d18158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d1d4c0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d180b22a0 0x7f1d180cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d1d4c0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180cd5a0 0x7f1d180cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d1d4c0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d18158ea0 con 0x7f1d180b22a0 2023-11-28T12:01:39.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d1d4c0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d181f6530 con 0x7f1d1806ab10 2023-11-28T12:01:39.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d1d4c0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d18158980 con 0x7f1d180cd5a0 2023-11-28T12:01:39.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d167fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d180b22a0 0x7f1d180cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d167fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d180b22a0 0x7f1d180cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39438/0 (socket says 172.21.15.57:39438) 2023-11-28T12:01:39.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d167fc700 1 -- 172.21.15.57:0/2109495644 learned_addr learned my addr 172.21.15.57:0/2109495644 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:39.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d16ffd700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1806ab10 0x7f1d18158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d177fe700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180cd5a0 0x7f1d180cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d167fc700 1 -- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180cd5a0 msgr2=0x7f1d180cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d167fc700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180cd5a0 0x7f1d180cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d167fc700 1 -- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1806ab10 msgr2=0x7f1d18158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d167fc700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1806ab10 0x7f1d18158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d167fc700 1 -- 172.21.15.57:0/2109495644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d00009550 con 0x7f1d180b22a0 2023-11-28T12:01:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d167fc700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d180b22a0 0x7f1d180cc030 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1d00000c00 tx=0x7f1d00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d0f7fe700 1 -- 172.21.15.57:0/2109495644 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00004510 con 0x7f1d180b22a0 2023-11-28T12:01:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.959+0000 7f1d0f7fe700 1 -- 172.21.15.57:0/2109495644 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d000036a0 con 0x7f1d180b22a0 2023-11-28T12:01:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.960+0000 7f1d0f7fe700 1 -- 172.21.15.57:0/2109495644 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00003a10 con 0x7f1d180b22a0 2023-11-28T12:01:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.960+0000 7f1d1d4c0700 1 -- 172.21.15.57:0/2109495644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1815cad0 con 0x7f1d180b22a0 2023-11-28T12:01:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.960+0000 7f1d1d4c0700 1 -- 172.21.15.57:0/2109495644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d180d1b70 con 0x7f1d180b22a0 2023-11-28T12:01:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.961+0000 7f1d0d7fa700 1 -- 172.21.15.57:0/2109495644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d180b1ec0 con 0x7f1d180b22a0 2023-11-28T12:01:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.961+0000 7f1d0f7fe700 1 -- 172.21.15.57:0/2109495644 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f1d0000f050 con 0x7f1d180b22a0 2023-11-28T12:01:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.961+0000 7f1d0f7fe700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1cf80682a0 0x7f1cf806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.961+0000 7f1d0f7fe700 1 -- 172.21.15.57:0/2109495644 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1d00086470 con 0x7f1d180b22a0 2023-11-28T12:01:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.962+0000 7f1d16ffd700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1cf80682a0 0x7f1cf806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.962+0000 7f1d16ffd700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1cf80682a0 0x7f1cf806a750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009b40 tx=0x7f1d08009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:39.966+0000 7f1d0f7fe700 1 -- 172.21.15.57:0/2109495644 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1d00053200 con 0x7f1d180b22a0 2023-11-28T12:01:40.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.169+0000 7f1d0d7fa700 1 -- 172.21.15.57:0/2109495644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d180743f0 con 0x7f1d180b22a0 2023-11-28T12:01:40.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.174+0000 7f1d0f7fe700 1 -- 172.21.15.57:0/2109495644 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14220 (secure 0 0 0) 0x7f1d00056740 con 0x7f1d180b22a0 2023-11-28T12:01:40.174 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:40.174 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":16,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:39.235279+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:01:40.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.176+0000 7f1d0d7fa700 1 -- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1cf80682a0 msgr2=0x7f1cf806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.176+0000 7f1d0d7fa700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1cf80682a0 0x7f1cf806a750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009b40 tx=0x7f1d08009210 comp rx=0 tx=0).stop 2023-11-28T12:01:40.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.176+0000 7f1d0d7fa700 1 -- 172.21.15.57:0/2109495644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d180b22a0 msgr2=0x7f1d180cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.176+0000 7f1d0d7fa700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d180b22a0 0x7f1d180cc030 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1d00000c00 tx=0x7f1d00004750 comp rx=0 tx=0).stop 2023-11-28T12:01:40.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.176+0000 7f1d0d7fa700 1 -- 172.21.15.57:0/2109495644 shutdown_connections 2023-11-28T12:01:40.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.176+0000 7f1d0d7fa700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1cf80682a0 0x7f1cf806a750 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.176+0000 7f1d0d7fa700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180cd5a0 0x7f1d180cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.176+0000 7f1d0d7fa700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1806ab10 0x7f1d18158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.176+0000 7f1d0d7fa700 1 --2- 172.21.15.57:0/2109495644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d180b22a0 0x7f1d180cc030 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.176+0000 7f1d0d7fa700 1 -- 172.21.15.57:0/2109495644 >> 172.21.15.57:0/2109495644 conn(0x7f1d1806c520 msgr2=0x7f1d180b6730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:40.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.177+0000 7f1d0d7fa700 1 -- 172.21.15.57:0/2109495644 shutdown_connections 2023-11-28T12:01:40.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.177+0000 7f1d0d7fa700 1 -- 172.21.15.57:0/2109495644 wait complete. 2023-11-28T12:01:40.190 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:01:40.191 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:01:40.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.331+0000 7f3f52a20700 1 Processor -- start 2023-11-28T12:01:40.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.332+0000 7f3f52a20700 1 -- start start 2023-11-28T12:01:40.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.332+0000 7f3f52a20700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3f4c15c8c0 0x7f3f4c160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:40.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.332+0000 7f3f52a20700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4c06b220 0x7f3f4c0d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:40.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.332+0000 7f3f52a20700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4c0d2d30 con 0x7f3f4c0b5dc0 2023-11-28T12:01:40.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.332+0000 7f3f52a20700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4c0d0b40 con 0x7f3f4c15c8c0 2023-11-28T12:01:40.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.332+0000 7f3f52a20700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c1611f0 con 0x7f3f4c06b220 2023-11-28T12:01:40.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.333+0000 7f3f4bfff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3f4c15c8c0 0x7f3f4c160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:40.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.333+0000 7f3f4b7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f3f4c0b5dc0 0x7f3f4c06ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:35734/0 (socket says 172.21.15.57:35734) 2023-11-28T12:01:40.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.333+0000 7f3f4b7fe700 1 -- 172.21.15.57:0/146366374 learned_addr learned my addr 172.21.15.57:0/146366374 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:40.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.333+0000 7f3f50fbd700 1 --2- 172.21.15.57:0/146366374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4c06b220 0x7f3f4c0d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:40.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.333+0000 7f3f4bfff700 1 -- 172.21.15.57:0/146366374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4c06b220 msgr2=0x7f3f4c0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:40.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.333+0000 7f3f4bfff700 1 --2- 172.21.15.57:0/146366374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4c06b220 0x7f3f4c0d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.333+0000 7f3f4bfff700 1 -- 172.21.15.57:0/146366374 >> v1:172.21.15.57:6789/0 conn(0x7f3f4c0b5dc0 legacy=0x7f3f4c06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.333+0000 7f3f4affd700 1 -- 172.21.15.57:0/146366374 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064534022 0 0) 0x7f3f40001110 con 0x7f3f4c0b5dc0 2023-11-28T12:01:40.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.333+0000 7f3f4bfff700 1 -- 172.21.15.57:0/146366374 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c0d2a50 con 0x7f3f4c15c8c0 2023-11-28T12:01:40.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.333+0000 7f3f4bfff700 1 --2- 172.21.15.57:0/146366374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f4c15c8c0 0x7f3f4c160cb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009840 tx=0x7f3f3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bdaf28ad7a6d1c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:40.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.333+0000 7f3f4affd700 1 -- 172.21.15.57:0/146366374 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c011830 con 0x7f3f4c15c8c0 2023-11-28T12:01:40.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.333+0000 7f3f4affd700 1 -- 172.21.15.57:0/146366374 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c011990 con 0x7f3f4c15c8c0 2023-11-28T12:01:40.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.333+0000 7f3f4affd700 1 -- 172.21.15.57:0/146366374 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c011d20 con 0x7f3f4c15c8c0 2023-11-28T12:01:40.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.334+0000 7f3f52a20700 1 -- 172.21.15.57:0/146366374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f4c15c8c0 msgr2=0x7f3f4c160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.334+0000 7f3f52a20700 1 --2- 172.21.15.57:0/146366374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f4c15c8c0 0x7f3f4c160cb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009840 tx=0x7f3f3c009b50 comp rx=0 tx=0).stop 2023-11-28T12:01:40.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.334+0000 7f3f52a20700 1 -- 172.21.15.57:0/146366374 shutdown_connections 2023-11-28T12:01:40.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.334+0000 7f3f52a20700 1 --2- 172.21.15.57:0/146366374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f4c15c8c0 0x7f3f4c160cb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.334+0000 7f3f52a20700 1 --2- 172.21.15.57:0/146366374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4c06b220 0x7f3f4c0d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.334+0000 7f3f52a20700 1 -- 172.21.15.57:0/146366374 >> 172.21.15.57:0/146366374 conn(0x7f3f4c06c520 msgr2=0x7f3f4c06c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:40.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.334+0000 7f3f52a20700 1 -- 172.21.15.57:0/146366374 shutdown_connections 2023-11-28T12:01:40.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.334+0000 7f3f52a20700 1 -- 172.21.15.57:0/146366374 wait complete. 2023-11-28T12:01:40.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.336+0000 7f3f52a20700 1 Processor -- start 2023-11-28T12:01:40.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.336+0000 7f3f52a20700 1 -- start start 2023-11-28T12:01:40.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f52a20700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f4c06b220 0x7f3f4c15cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:40.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f52a20700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4c0b5dc0 0x7f3f4c0b7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:40.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f52a20700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f4c15c8c0 0x7f3f4c0b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:40.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f52a20700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c0c7500 con 0x7f3f4c06b220 2023-11-28T12:01:40.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f52a20700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c15d1d0 con 0x7f3f4c15c8c0 2023-11-28T12:01:40.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f52a20700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c15d450 con 0x7f3f4c0b5dc0 2023-11-28T12:01:40.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f4bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f4c06b220 0x7f3f4c15cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:40.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f4bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f4c06b220 0x7f3f4c15cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39444/0 (socket says 172.21.15.57:39444) 2023-11-28T12:01:40.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f4bfff700 1 -- 172.21.15.57:0/2972849311 learned_addr learned my addr 172.21.15.57:0/2972849311 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:40.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f50fbd700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f4c15c8c0 0x7f3f4c0b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:40.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f4b7fe700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4c0b5dc0 0x7f3f4c0b7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:40.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f4bfff700 1 -- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4c0b5dc0 msgr2=0x7f3f4c0b7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f4bfff700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4c0b5dc0 0x7f3f4c0b7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f4bfff700 1 -- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f4c15c8c0 msgr2=0x7f3f4c0b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f4bfff700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f4c15c8c0 0x7f3f4c0b8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f4bfff700 1 -- 172.21.15.57:0/2972849311 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c0094a0 con 0x7f3f4c06b220 2023-11-28T12:01:40.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f4b7fe700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4c0b5dc0 0x7f3f4c0b7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:01:40.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.337+0000 7f3f4bfff700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f4c06b220 0x7f3f4c15cc90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009fd0 tx=0x7f3f3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:40.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.338+0000 7f3f48ff9700 1 -- 172.21.15.57:0/2972849311 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0036a0 con 0x7f3f4c06b220 2023-11-28T12:01:40.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.338+0000 7f3f48ff9700 1 -- 172.21.15.57:0/2972849311 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c01e070 con 0x7f3f4c06b220 2023-11-28T12:01:40.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.338+0000 7f3f52a20700 1 -- 172.21.15.57:0/2972849311 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4c0cda40 con 0x7f3f4c06b220 2023-11-28T12:01:40.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.338+0000 7f3f48ff9700 1 -- 172.21.15.57:0/2972849311 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c003a60 con 0x7f3f4c06b220 2023-11-28T12:01:40.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.338+0000 7f3f52a20700 1 -- 172.21.15.57:0/2972849311 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4c0cd760 con 0x7f3f4c06b220 2023-11-28T12:01:40.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.339+0000 7f3f52a20700 1 -- 172.21.15.57:0/2972849311 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4c0d2700 con 0x7f3f4c06b220 2023-11-28T12:01:40.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.339+0000 7f3f48ff9700 1 -- 172.21.15.57:0/2972849311 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f3f3c00f050 con 0x7f3f4c06b220 2023-11-28T12:01:40.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.340+0000 7f3f48ff9700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3f380682a0 0x7f3f3806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:40.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.340+0000 7f3f48ff9700 1 -- 172.21.15.57:0/2972849311 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3f3c0868d0 con 0x7f3f4c06b220 2023-11-28T12:01:40.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.340+0000 7f3f4b7fe700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3f380682a0 0x7f3f3806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:40.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.340+0000 7f3f4b7fe700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3f380682a0 0x7f3f3806a750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3f40007fd0 tx=0x7f3f40009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:40.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.344+0000 7f3f48ff9700 1 -- 172.21.15.57:0/2972849311 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f3c0534d0 con 0x7f3f4c06b220 2023-11-28T12:01:40.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.471+0000 7f3f48ff9700 1 -- 172.21.15.57:0/2972849311 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f3c0538a0 con 0x7f3f4c06b220 2023-11-28T12:01:40.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.553+0000 7f3f52a20700 1 -- 172.21.15.57:0/2972849311 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3f4c067d70 con 0x7f3f4c06b220 2023-11-28T12:01:40.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.553+0000 7f3f48ff9700 1 -- 172.21.15.57:0/2972849311 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3f3c056f00 con 0x7f3f4c06b220 2023-11-28T12:01:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:01:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.555+0000 7f3f52a20700 1 -- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3f380682a0 msgr2=0x7f3f3806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.555+0000 7f3f52a20700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3f380682a0 0x7f3f3806a750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3f40007fd0 tx=0x7f3f40009040 comp rx=0 tx=0).stop 2023-11-28T12:01:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.555+0000 7f3f52a20700 1 -- 172.21.15.57:0/2972849311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f4c06b220 msgr2=0x7f3f4c15cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.555+0000 7f3f52a20700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f4c06b220 0x7f3f4c15cc90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009fd0 tx=0x7f3f3c004750 comp rx=0 tx=0).stop 2023-11-28T12:01:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.555+0000 7f3f52a20700 1 -- 172.21.15.57:0/2972849311 shutdown_connections 2023-11-28T12:01:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.555+0000 7f3f52a20700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3f380682a0 0x7f3f3806a750 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.555+0000 7f3f52a20700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f4c15c8c0 0x7f3f4c0b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.555+0000 7f3f52a20700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f4c06b220 0x7f3f4c15cc90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.555+0000 7f3f52a20700 1 --2- 172.21.15.57:0/2972849311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4c0b5dc0 0x7f3f4c0b7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.555+0000 7f3f52a20700 1 -- 172.21.15.57:0/2972849311 >> 172.21.15.57:0/2972849311 conn(0x7f3f4c06c520 msgr2=0x7f3f4c06c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:40.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.556+0000 7f3f52a20700 1 -- 172.21.15.57:0/2972849311 shutdown_connections 2023-11-28T12:01:40.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.556+0000 7f3f52a20700 1 -- 172.21.15.57:0/2972849311 wait complete. 2023-11-28T12:01:40.571 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:01:40.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.711+0000 7f1efa125700 1 Processor -- start 2023-11-28T12:01:40.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.711+0000 7f1efa125700 1 -- start start 2023-11-28T12:01:40.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.712+0000 7f1efa125700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef41599b0 0x7f1ef415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:40.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.712+0000 7f1efa125700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1ef40af0a0 0x7f1ef415a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:40.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.712+0000 7f1efa125700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef4159ec0 con 0x7f1ef40b1650 2023-11-28T12:01:40.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.712+0000 7f1efa125700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef415d8a0 con 0x7f1ef40af0a0 2023-11-28T12:01:40.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.712+0000 7f1efa125700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef415db80 con 0x7f1ef41599b0 2023-11-28T12:01:40.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.712+0000 7f1ef3fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1ef40af0a0 0x7f1ef415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:40.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.712+0000 7f1ef3fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1ef40af0a0 0x7f1ef415a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39450/0 (socket says 172.21.15.57:39450) 2023-11-28T12:01:40.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.712+0000 7f1ef3fff700 1 -- 172.21.15.57:0/2520662713 learned_addr learned my addr 172.21.15.57:0/2520662713 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:40.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.712+0000 7f1ef2ffd700 1 --2- 172.21.15.57:0/2520662713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef41599b0 0x7f1ef415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:40.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.713+0000 7f1ef3fff700 1 -- 172.21.15.57:0/2520662713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef41599b0 msgr2=0x7f1ef415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:40.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.713+0000 7f1ef3fff700 1 --2- 172.21.15.57:0/2520662713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef41599b0 0x7f1ef415d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.713+0000 7f1ef3fff700 1 -- 172.21.15.57:0/2520662713 >> v1:172.21.15.181:6789/0 conn(0x7f1ef40b1650 legacy=0x7f1ef40b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.713+0000 7f1ef3fff700 1 -- 172.21.15.57:0/2520662713 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef415de60 con 0x7f1ef40af0a0 2023-11-28T12:01:40.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.713+0000 7f1ef3fff700 1 --2- 172.21.15.57:0/2520662713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ef40af0a0 0x7f1ef415a350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009e50 tx=0x7f1eec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9442ef2c1bbf0a57 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:40.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.713+0000 7f1ef27fc700 1 -- 172.21.15.57:0/2520662713 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec00e070 con 0x7f1ef40af0a0 2023-11-28T12:01:40.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.713+0000 7f1ef27fc700 1 -- 172.21.15.57:0/2520662713 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec00b910 con 0x7f1ef40af0a0 2023-11-28T12:01:40.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.713+0000 7f1ef27fc700 1 -- 172.21.15.57:0/2520662713 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec00bcc0 con 0x7f1ef40af0a0 2023-11-28T12:01:40.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.714+0000 7f1efa125700 1 -- 172.21.15.57:0/2520662713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ef40af0a0 msgr2=0x7f1ef415a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.714+0000 7f1efa125700 1 --2- 172.21.15.57:0/2520662713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ef40af0a0 0x7f1ef415a350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009e50 tx=0x7f1eec00b330 comp rx=0 tx=0).stop 2023-11-28T12:01:40.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.714+0000 7f1efa125700 1 -- 172.21.15.57:0/2520662713 shutdown_connections 2023-11-28T12:01:40.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.714+0000 7f1efa125700 1 --2- 172.21.15.57:0/2520662713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ef40af0a0 0x7f1ef415a350 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.714+0000 7f1efa125700 1 --2- 172.21.15.57:0/2520662713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef41599b0 0x7f1ef415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.714+0000 7f1efa125700 1 -- 172.21.15.57:0/2520662713 >> 172.21.15.57:0/2520662713 conn(0x7f1ef406c010 msgr2=0x7f1ef40cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:40.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.714+0000 7f1efa125700 1 -- 172.21.15.57:0/2520662713 shutdown_connections 2023-11-28T12:01:40.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.714+0000 7f1efa125700 1 -- 172.21.15.57:0/2520662713 wait complete. 2023-11-28T12:01:40.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.715+0000 7f1efa125700 1 Processor -- start 2023-11-28T12:01:40.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.715+0000 7f1efa125700 1 -- start start 2023-11-28T12:01:40.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1efa125700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ef40af0a0 0x7f1ef4157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:40.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1efa125700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef40b1650 0x7f1ef4151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:40.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1efa125700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef4152e70 0x7f1ef40b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:40.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1efa125700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef41543d0 con 0x7f1ef40af0a0 2023-11-28T12:01:40.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1efa125700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef4151650 con 0x7f1ef4152e70 2023-11-28T12:01:40.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1efa125700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef41547f0 con 0x7f1ef40b1650 2023-11-28T12:01:40.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1ef37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ef40af0a0 0x7f1ef4157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:40.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1ef37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ef40af0a0 0x7f1ef4157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39460/0 (socket says 172.21.15.57:39460) 2023-11-28T12:01:40.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1ef37fe700 1 -- 172.21.15.57:0/4059089681 learned_addr learned my addr 172.21.15.57:0/4059089681 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:40.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1ef3fff700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef4152e70 0x7f1ef40b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:40.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1ef2ffd700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef40b1650 0x7f1ef4151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:40.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1ef37fe700 1 -- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef40b1650 msgr2=0x7f1ef4151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1ef37fe700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef40b1650 0x7f1ef4151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1ef37fe700 1 -- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef4152e70 msgr2=0x7f1ef40b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1ef37fe700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef4152e70 0x7f1ef40b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1ef37fe700 1 -- 172.21.15.57:0/4059089681 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec009b30 con 0x7f1ef40af0a0 2023-11-28T12:01:40.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.716+0000 7f1ef3fff700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef4152e70 0x7f1ef40b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:40.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.717+0000 7f1ef37fe700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ef40af0a0 0x7f1ef4157fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c450 tx=0x7f1edc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:40.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.717+0000 7f1eebfff700 1 -- 172.21.15.57:0/4059089681 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0052d0 con 0x7f1ef40af0a0 2023-11-28T12:01:40.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.717+0000 7f1eebfff700 1 -- 172.21.15.57:0/4059089681 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc005430 con 0x7f1ef40af0a0 2023-11-28T12:01:40.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.717+0000 7f1efa125700 1 -- 172.21.15.57:0/4059089681 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef415a890 con 0x7f1ef40af0a0 2023-11-28T12:01:40.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.717+0000 7f1eebfff700 1 -- 172.21.15.57:0/4059089681 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0038c0 con 0x7f1ef40af0a0 2023-11-28T12:01:40.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.717+0000 7f1efa125700 1 -- 172.21.15.57:0/4059089681 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef415d590 con 0x7f1ef40af0a0 2023-11-28T12:01:40.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.718+0000 7f1efa125700 1 -- 172.21.15.57:0/4059089681 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef415aa20 con 0x7f1ef40af0a0 2023-11-28T12:01:40.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.719+0000 7f1eebfff700 1 -- 172.21.15.57:0/4059089681 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1edc019030 con 0x7f1ef40af0a0 2023-11-28T12:01:40.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.719+0000 7f1eebfff700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1ed40682f0 0x7f1ed406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:40.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.719+0000 7f1eebfff700 1 -- 172.21.15.57:0/4059089681 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1edc084bf0 con 0x7f1ef40af0a0 2023-11-28T12:01:40.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.719+0000 7f1ef2ffd700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1ed40682f0 0x7f1ed406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:40.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.720+0000 7f1ef2ffd700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1ed40682f0 0x7f1ed406a7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ef406a770 tx=0x7f1ee4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:40.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.724+0000 7f1eebfff700 1 -- 172.21.15.57:0/4059089681 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1edc0517a0 con 0x7f1ef40af0a0 2023-11-28T12:01:40.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.923+0000 7f1efa125700 1 -- 172.21.15.57:0/4059089681 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ef4154640 con 0x7f1ef40af0a0 2023-11-28T12:01:40.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.923+0000 7f1eebfff700 1 -- 172.21.15.57:0/4059089681 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1edc054dc0 con 0x7f1ef40af0a0 2023-11-28T12:01:40.925 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:01:40.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.926+0000 7f1efa125700 1 -- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1ed40682f0 msgr2=0x7f1ed406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.926+0000 7f1efa125700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1ed40682f0 0x7f1ed406a7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ef406a770 tx=0x7f1ee4008040 comp rx=0 tx=0).stop 2023-11-28T12:01:40.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.926+0000 7f1efa125700 1 -- 172.21.15.57:0/4059089681 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ef40af0a0 msgr2=0x7f1ef4157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:40.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.926+0000 7f1efa125700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ef40af0a0 0x7f1ef4157fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c450 tx=0x7f1edc00c810 comp rx=0 tx=0).stop 2023-11-28T12:01:40.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.926+0000 7f1efa125700 1 -- 172.21.15.57:0/4059089681 shutdown_connections 2023-11-28T12:01:40.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.926+0000 7f1efa125700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1ed40682f0 0x7f1ed406a7a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.926+0000 7f1efa125700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef4152e70 0x7f1ef40b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.926+0000 7f1efa125700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ef40af0a0 0x7f1ef4157fc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.926+0000 7f1efa125700 1 --2- 172.21.15.57:0/4059089681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef40b1650 0x7f1ef4151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:40.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.926+0000 7f1efa125700 1 -- 172.21.15.57:0/4059089681 >> 172.21.15.57:0/4059089681 conn(0x7f1ef406c010 msgr2=0x7f1ef4159060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:40.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.926+0000 7f1efa125700 1 -- 172.21.15.57:0/4059089681 shutdown_connections 2023-11-28T12:01:40.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:40.926+0000 7f1efa125700 1 -- 172.21.15.57:0/4059089681 wait complete. 2023-11-28T12:01:40.940 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T12:01:41.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.074+0000 7fc8bf892700 1 Processor -- start 2023-11-28T12:01:41.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.075+0000 7fc8bf892700 1 -- start start 2023-11-28T12:01:41.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.075+0000 7fc8bf892700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc8b806e400 0x7fc8b80b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:41.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.075+0000 7fc8bf892700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8b8160e70 0x7fc8b815c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:41.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.075+0000 7fc8bf892700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b806e900 con 0x7fc8b80d0c60 2023-11-28T12:01:41.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.075+0000 7fc8bf892700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b8160390 con 0x7fc8b806e400 2023-11-28T12:01:41.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.075+0000 7fc8bf892700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b8160670 con 0x7fc8b8160e70 2023-11-28T12:01:41.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.076+0000 7fc8bce2d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc8b806e400 0x7fc8b80b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:41.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.076+0000 7fc8bde2f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8b8160e70 0x7fc8b815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:41.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.076+0000 7fc8bd62e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc8b80d0c60 0x7fc8b80d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:59106/0 (socket says 172.21.15.57:59106) 2023-11-28T12:01:41.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.076+0000 7fc8bd62e700 1 -- 172.21.15.57:0/3816061844 learned_addr learned my addr 172.21.15.57:0/3816061844 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:41.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.076+0000 7fc8abfff700 1 -- 172.21.15.57:0/3816061844 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652588624 0 0) 0x7fc8b806e900 con 0x7fc8b80d0c60 2023-11-28T12:01:41.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.076+0000 7fc8abfff700 1 -- 172.21.15.57:0/3816061844 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8a40035b0 con 0x7fc8b80d0c60 2023-11-28T12:01:41.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.076+0000 7fc8bce2d700 1 -- 172.21.15.57:0/3816061844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8b8160e70 msgr2=0x7fc8b815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:41.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.076+0000 7fc8bce2d700 1 --2- 172.21.15.57:0/3816061844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8b8160e70 0x7fc8b815c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.076+0000 7fc8bce2d700 1 -- 172.21.15.57:0/3816061844 >> v1:172.21.15.181:6789/0 conn(0x7fc8b80d0c60 legacy=0x7fc8b80d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.076+0000 7fc8bce2d700 1 -- 172.21.15.57:0/3816061844 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b8160950 con 0x7fc8b806e400 2023-11-28T12:01:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.076+0000 7fc8bde2f700 1 --2- 172.21.15.57:0/3816061844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8b8160e70 0x7fc8b815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.076+0000 7fc8bce2d700 1 --2- 172.21.15.57:0/3816061844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b806e400 0x7fc8b80b5be0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009840 tx=0x7fc8ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=914347376ae8eda server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.077+0000 7fc8abfff700 1 -- 172.21.15.57:0/3816061844 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac011830 con 0x7fc8b806e400 2023-11-28T12:01:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.077+0000 7fc8abfff700 1 -- 172.21.15.57:0/3816061844 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ac011990 con 0x7fc8b806e400 2023-11-28T12:01:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.077+0000 7fc8abfff700 1 -- 172.21.15.57:0/3816061844 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac011d00 con 0x7fc8b806e400 2023-11-28T12:01:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.077+0000 7fc8bf892700 1 -- 172.21.15.57:0/3816061844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b806e400 msgr2=0x7fc8b80b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:41.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.077+0000 7fc8bf892700 1 --2- 172.21.15.57:0/3816061844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b806e400 0x7fc8b80b5be0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009840 tx=0x7fc8ac009b50 comp rx=0 tx=0).stop 2023-11-28T12:01:41.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.077+0000 7fc8bf892700 1 -- 172.21.15.57:0/3816061844 shutdown_connections 2023-11-28T12:01:41.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.077+0000 7fc8bf892700 1 --2- 172.21.15.57:0/3816061844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8b8160e70 0x7fc8b815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:41.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.077+0000 7fc8bf892700 1 --2- 172.21.15.57:0/3816061844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b806e400 0x7fc8b80b5be0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:41.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.077+0000 7fc8bf892700 1 -- 172.21.15.57:0/3816061844 >> 172.21.15.57:0/3816061844 conn(0x7fc8b806a270 msgr2=0x7fc8b806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:41.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.077+0000 7fc8bf892700 1 -- 172.21.15.57:0/3816061844 shutdown_connections 2023-11-28T12:01:41.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.077+0000 7fc8bf892700 1 -- 172.21.15.57:0/3816061844 wait complete. 2023-11-28T12:01:41.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.078+0000 7fc8bf892700 1 Processor -- start 2023-11-28T12:01:41.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.079+0000 7fc8bf892700 1 -- start start 2023-11-28T12:01:41.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.079+0000 7fc8bf892700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b806e400 0x7fc8b80b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:41.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.079+0000 7fc8bf892700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b80d0c60 0x7fc8b81558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:41.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.079+0000 7fc8bf892700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8b8160e70 0x7fc8b81e7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:41.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.079+0000 7fc8bf892700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b80cf9b0 con 0x7fc8b806e400 2023-11-28T12:01:41.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.079+0000 7fc8bf892700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b80b22f0 con 0x7fc8b80d0c60 2023-11-28T12:01:41.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.079+0000 7fc8bf892700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b80b25d0 con 0x7fc8b8160e70 2023-11-28T12:01:41.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.079+0000 7fc8bd62e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b806e400 0x7fc8b80b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:41.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.079+0000 7fc8bd62e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b806e400 0x7fc8b80b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39470/0 (socket says 172.21.15.57:39470) 2023-11-28T12:01:41.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.079+0000 7fc8bd62e700 1 -- 172.21.15.57:0/3903508511 learned_addr learned my addr 172.21.15.57:0/3903508511 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:41.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8bde2f700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8b8160e70 0x7fc8b81e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:41.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8bce2d700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b80d0c60 0x7fc8b81558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:41.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8bd62e700 1 -- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8b8160e70 msgr2=0x7fc8b81e7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:41.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8bd62e700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8b8160e70 0x7fc8b81e7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:41.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8bd62e700 1 -- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b80d0c60 msgr2=0x7fc8b81558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:41.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8bd62e700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b80d0c60 0x7fc8b81558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:41.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8bd62e700 1 -- 172.21.15.57:0/3903508511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b0009a70 con 0x7fc8b806e400 2023-11-28T12:01:41.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8bce2d700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b80d0c60 0x7fc8b81558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:41.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8bd62e700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b806e400 0x7fc8b80b3390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400c4d0 tx=0x7fc8b400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:41.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8a9ffb700 1 -- 172.21.15.57:0/3903508511 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4005d70 con 0x7fc8b806e400 2023-11-28T12:01:41.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8a9ffb700 1 -- 172.21.15.57:0/3903508511 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b40070d0 con 0x7fc8b806e400 2023-11-28T12:01:41.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8bf892700 1 -- 172.21.15.57:0/3903508511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ac0094a0 con 0x7fc8b806e400 2023-11-28T12:01:41.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8a9ffb700 1 -- 172.21.15.57:0/3903508511 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4007440 con 0x7fc8b806e400 2023-11-28T12:01:41.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.080+0000 7fc8bf892700 1 -- 172.21.15.57:0/3903508511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b815ceb0 con 0x7fc8b806e400 2023-11-28T12:01:41.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.082+0000 7fc8bf892700 1 -- 172.21.15.57:0/3903508511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b80d0740 con 0x7fc8b806e400 2023-11-28T12:01:41.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.082+0000 7fc8a9ffb700 1 -- 172.21.15.57:0/3903508511 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc8b4019030 con 0x7fc8b806e400 2023-11-28T12:01:41.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.082+0000 7fc8a9ffb700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc89c0682f0 0x7fc89c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:41.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.082+0000 7fc8a9ffb700 1 -- 172.21.15.57:0/3903508511 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc8b4084b60 con 0x7fc8b806e400 2023-11-28T12:01:41.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.083+0000 7fc8bce2d700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc89c0682f0 0x7fc89c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:41.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.083+0000 7fc8bce2d700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc89c0682f0 0x7fc89c06a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc8b806e830 tx=0x7fc8ac008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:41.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.087+0000 7fc8a9ffb700 1 -- 172.21.15.57:0/3903508511 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc8b404b5e0 con 0x7fc8b806e400 2023-11-28T12:01:41.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.290+0000 7fc8bf892700 1 -- 172.21.15.57:0/3903508511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc8b815d040 con 0x7fc8b806e400 2023-11-28T12:01:41.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.290+0000 7fc8a9ffb700 1 -- 172.21.15.57:0/3903508511 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc8b4017070 con 0x7fc8b806e400 2023-11-28T12:01:41.292 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:41.292 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2023-11-28T12:01:41.292 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:01:41.292 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 1, 2023-11-28T12:01:41.292 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": "rbd" 2023-11-28T12:01:41.292 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:01:41.293 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:01:41.293 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 2, 2023-11-28T12:01:41.293 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": ".mgr" 2023-11-28T12:01:41.293 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:01:41.293 INFO:teuthology.orchestra.run.smithi057.stdout:] 2023-11-28T12:01:41.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.293+0000 7fc8bf892700 1 -- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc89c0682f0 msgr2=0x7fc89c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:41.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.293+0000 7fc8bf892700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc89c0682f0 0x7fc89c06a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc8b806e830 tx=0x7fc8ac008030 comp rx=0 tx=0).stop 2023-11-28T12:01:41.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.293+0000 7fc8bf892700 1 -- 172.21.15.57:0/3903508511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b806e400 msgr2=0x7fc8b80b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:41.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.293+0000 7fc8bf892700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b806e400 0x7fc8b80b3390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400c4d0 tx=0x7fc8b400c7e0 comp rx=0 tx=0).stop 2023-11-28T12:01:41.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.293+0000 7fc8bf892700 1 -- 172.21.15.57:0/3903508511 shutdown_connections 2023-11-28T12:01:41.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.293+0000 7fc8bf892700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc89c0682f0 0x7fc89c06a7a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:41.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.293+0000 7fc8bf892700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8b8160e70 0x7fc8b81e7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:41.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.293+0000 7fc8bf892700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8b806e400 0x7fc8b80b3390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:41.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.293+0000 7fc8bf892700 1 --2- 172.21.15.57:0/3903508511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8b80d0c60 0x7fc8b81558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:41.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.293+0000 7fc8bf892700 1 -- 172.21.15.57:0/3903508511 >> 172.21.15.57:0/3903508511 conn(0x7fc8b806a270 msgr2=0x7fc8b8157560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:41.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.293+0000 7fc8bf892700 1 -- 172.21.15.57:0/3903508511 shutdown_connections 2023-11-28T12:01:41.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.293+0000 7fc8bf892700 1 -- 172.21.15.57:0/3903508511 wait complete. 2023-11-28T12:01:41.307 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T12:01:41.308 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T12:01:41.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.446+0000 7f4e9daa2700 1 Processor -- start 2023-11-28T12:01:41.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.446+0000 7f4e9daa2700 1 -- start start 2023-11-28T12:01:41.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.446+0000 7f4e9daa2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4e9806ba90 0x7f4e9815a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:41.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e9daa2700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e980cea10 con 0x7f4e9815e070 2023-11-28T12:01:41.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e9daa2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e980cbd10 con 0x7f4e980ceea0 2023-11-28T12:01:41.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e9daa2700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9815ae80 con 0x7f4e9806ba90 2023-11-28T12:01:41.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e96ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4e9806ba90 0x7f4e9815a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:41.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e977fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4e9815e070 0x7f4e9806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:35748/0 (socket says 172.21.15.57:35748) 2023-11-28T12:01:41.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e977fe700 1 -- 172.21.15.57:0/3019568933 learned_addr learned my addr 172.21.15.57:0/3019568933 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:41.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e96ffd700 1 -- 172.21.15.57:0/3019568933 >> v1:172.21.15.181:6789/0 conn(0x7f4e980ceea0 legacy=0x7f4e9815a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:41.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e96ffd700 1 -- 172.21.15.57:0/3019568933 >> v1:172.21.15.57:6789/0 conn(0x7f4e9815e070 legacy=0x7f4e9806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:41.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e96ffd700 1 -- 172.21.15.57:0/3019568933 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e980ce0e0 con 0x7f4e9806ba90 2023-11-28T12:01:41.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e967fc700 1 -- 172.21.15.57:0/3019568933 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608431100 0 0) 0x7f4e980cea10 con 0x7f4e9815e070 2023-11-28T12:01:41.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e96ffd700 1 --2- 172.21.15.57:0/3019568933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9806ba90 0x7f4e9815a150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009840 tx=0x7f4e8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7381a720bc01f144 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:41.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e967fc700 1 -- 172.21.15.57:0/3019568933 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c011830 con 0x7f4e9806ba90 2023-11-28T12:01:41.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e967fc700 1 -- 172.21.15.57:0/3019568933 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c011990 con 0x7f4e9806ba90 2023-11-28T12:01:41.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.447+0000 7f4e967fc700 1 -- 172.21.15.57:0/3019568933 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c011d20 con 0x7f4e9806ba90 2023-11-28T12:01:41.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.448+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3019568933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9806ba90 msgr2=0x7f4e9815a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:41.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.448+0000 7f4e9daa2700 1 --2- 172.21.15.57:0/3019568933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9806ba90 0x7f4e9815a150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009840 tx=0x7f4e8c009b50 comp rx=0 tx=0).stop 2023-11-28T12:01:41.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.448+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3019568933 shutdown_connections 2023-11-28T12:01:41.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.448+0000 7f4e9daa2700 1 --2- 172.21.15.57:0/3019568933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9806ba90 0x7f4e9815a150 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:41.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.448+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3019568933 >> 172.21.15.57:0/3019568933 conn(0x7f4e980cfc30 msgr2=0x7f4e980d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:41.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.448+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3019568933 shutdown_connections 2023-11-28T12:01:41.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.448+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3019568933 wait complete. 2023-11-28T12:01:41.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.449+0000 7f4e9daa2700 1 Processor -- start 2023-11-28T12:01:41.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.450+0000 7f4e9daa2700 1 -- start start 2023-11-28T12:01:41.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.450+0000 7f4e9daa2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e9806ba90 0x7f4e980b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:41.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.450+0000 7f4e9daa2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e980ceea0 0x7f4e9806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:41.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.450+0000 7f4e9daa2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9815e070 0x7f4e9814dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:41.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.450+0000 7f4e9daa2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e980b5760 con 0x7f4e9815e070 2023-11-28T12:01:41.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.450+0000 7f4e9daa2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9806dc60 con 0x7f4e980ceea0 2023-11-28T12:01:41.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.450+0000 7f4e9daa2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e980b46c0 con 0x7f4e9806ba90 2023-11-28T12:01:41.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.450+0000 7f4e97fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9815e070 0x7f4e9814dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:41.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.450+0000 7f4e97fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9815e070 0x7f4e9814dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39494/0 (socket says 172.21.15.57:39494) 2023-11-28T12:01:41.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.450+0000 7f4e97fff700 1 -- 172.21.15.57:0/3145494733 learned_addr learned my addr 172.21.15.57:0/3145494733 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:41.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.451+0000 7f4e977fe700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e9806ba90 0x7f4e980b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:41.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.451+0000 7f4e96ffd700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e980ceea0 0x7f4e9806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:41.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.451+0000 7f4e97fff700 1 -- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e9806ba90 msgr2=0x7f4e980b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:41.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.451+0000 7f4e97fff700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e9806ba90 0x7f4e980b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:41.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.451+0000 7f4e97fff700 1 -- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e980ceea0 msgr2=0x7f4e9806d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:41.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.451+0000 7f4e97fff700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e980ceea0 0x7f4e9806d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:41.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.451+0000 7f4e97fff700 1 -- 172.21.15.57:0/3145494733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c0094a0 con 0x7f4e9815e070 2023-11-28T12:01:41.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.451+0000 7f4e97fff700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9815e070 0x7f4e9814dcd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000cc50 tx=0x7f4e9000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:41.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.451+0000 7f4e7ffff700 1 -- 172.21.15.57:0/3145494733 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9000ad80 con 0x7f4e9815e070 2023-11-28T12:01:41.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.451+0000 7f4e7ffff700 1 -- 172.21.15.57:0/3145494733 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9000ce50 con 0x7f4e9815e070 2023-11-28T12:01:41.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.451+0000 7f4e7ffff700 1 -- 172.21.15.57:0/3145494733 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90003c40 con 0x7f4e9815e070 2023-11-28T12:01:41.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.451+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3145494733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e980ce6a0 con 0x7f4e9815e070 2023-11-28T12:01:41.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.451+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3145494733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e980ce3c0 con 0x7f4e9815e070 2023-11-28T12:01:41.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.452+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3145494733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e98150080 con 0x7f4e9815e070 2023-11-28T12:01:41.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.452+0000 7f4e7ffff700 1 -- 172.21.15.57:0/3145494733 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e90010050 con 0x7f4e9815e070 2023-11-28T12:01:41.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.453+0000 7f4e7ffff700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4e840682f0 0x7f4e8406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:41.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.453+0000 7f4e7ffff700 1 -- 172.21.15.57:0/3145494733 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4e900858a0 con 0x7f4e9815e070 2023-11-28T12:01:41.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.453+0000 7f4e977fe700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4e840682f0 0x7f4e8406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:41.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.454+0000 7f4e977fe700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4e840682f0 0x7f4e8406a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4e88000f60 tx=0x7f4e88007d90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:41.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.458+0000 7f4e7ffff700 1 -- 172.21.15.57:0/3145494733 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e90052450 con 0x7f4e9815e070 2023-11-28T12:01:41.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:41.655+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3145494733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4e980c5310 con 0x7f4e9815e070 2023-11-28T12:01:42.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.479+0000 7f4e7ffff700 1 -- 172.21.15.57:0/3145494733 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4e90055a70 con 0x7f4e9815e070 2023-11-28T12:01:42.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.484+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4e840682f0 msgr2=0x7f4e8406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:42.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.484+0000 7f4e9daa2700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4e840682f0 0x7f4e8406a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4e88000f60 tx=0x7f4e88007d90 comp rx=0 tx=0).stop 2023-11-28T12:01:42.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.484+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3145494733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9815e070 msgr2=0x7f4e9814dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:42.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.484+0000 7f4e9daa2700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9815e070 0x7f4e9814dcd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000cc50 tx=0x7f4e9000a340 comp rx=0 tx=0).stop 2023-11-28T12:01:42.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.485+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3145494733 shutdown_connections 2023-11-28T12:01:42.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.485+0000 7f4e9daa2700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4e840682f0 0x7f4e8406a7a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:42.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.485+0000 7f4e9daa2700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e9815e070 0x7f4e9814dcd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:42.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.485+0000 7f4e9daa2700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e9806ba90 0x7f4e980b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:42.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.485+0000 7f4e9daa2700 1 --2- 172.21.15.57:0/3145494733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e980ceea0 0x7f4e9806d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:42.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.485+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3145494733 >> 172.21.15.57:0/3145494733 conn(0x7f4e980cfc30 msgr2=0x7f4e98070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:42.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.486+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3145494733 shutdown_connections 2023-11-28T12:01:42.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.486+0000 7f4e9daa2700 1 -- 172.21.15.57:0/3145494733 wait complete. 2023-11-28T12:01:42.488 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' created 2023-11-28T12:01:42.503 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T12:01:42.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.649+0000 7f82c80e5700 1 Processor -- start 2023-11-28T12:01:42.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.649+0000 7f82c80e5700 1 -- start start 2023-11-28T12:01:42.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.650+0000 7f82c80e5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f82c006d170 0x7f82c00b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:42.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.650+0000 7f82c80e5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c00b27d0 0x7f82c00b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:42.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.650+0000 7f82c80e5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c0161370 con 0x7f82c015fe50 2023-11-28T12:01:42.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.650+0000 7f82c80e5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c006d5c0 con 0x7f82c006d170 2023-11-28T12:01:42.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.650+0000 7f82c80e5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c00b3190 con 0x7f82c00b27d0 2023-11-28T12:01:42.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.650+0000 7f82c5680700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f82c006d170 0x7f82c00b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:42.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.650+0000 7f82c6682700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c00b27d0 0x7f82c00b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:42.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.650+0000 7f82c5680700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f82c006d170 0x7f82c00b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:35768/0 (socket says 172.21.15.57:35768) 2023-11-28T12:01:42.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.650+0000 7f82c6682700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c00b27d0 0x7f82c00b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:49052/0 (socket says 172.21.15.57:49052) 2023-11-28T12:01:42.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.650+0000 7f82c5680700 1 -- 172.21.15.57:0/391270094 learned_addr learned my addr 172.21.15.57:0/391270094 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:42.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.651+0000 7f82c5680700 1 -- 172.21.15.57:0/391270094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c00b27d0 msgr2=0x7f82c00b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:42.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.651+0000 7f82c5680700 1 --2- 172.21.15.57:0/391270094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c00b27d0 0x7f82c00b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:42.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.651+0000 7f82c5680700 1 -- 172.21.15.57:0/391270094 >> v1:172.21.15.181:6789/0 conn(0x7f82c015fe50 legacy=0x7f82c0160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:42.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.651+0000 7f82c5680700 1 -- 172.21.15.57:0/391270094 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c00cf700 con 0x7f82c006d170 2023-11-28T12:01:42.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.651+0000 7f82c5680700 1 --2- 172.21.15.57:0/391270094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c006d170 0x7f82c00b3b80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82b40098f0 tx=0x7f82b4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eefdf9d90c69a30 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:42.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.651+0000 7f82c4e7f700 1 -- 172.21.15.57:0/391270094 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4011830 con 0x7f82c006d170 2023-11-28T12:01:42.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.651+0000 7f82c4e7f700 1 -- 172.21.15.57:0/391270094 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b4011990 con 0x7f82c006d170 2023-11-28T12:01:42.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.651+0000 7f82c4e7f700 1 -- 172.21.15.57:0/391270094 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4011d20 con 0x7f82c006d170 2023-11-28T12:01:42.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.652+0000 7f82c80e5700 1 -- 172.21.15.57:0/391270094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c006d170 msgr2=0x7f82c00b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:42.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.652+0000 7f82c80e5700 1 --2- 172.21.15.57:0/391270094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c006d170 0x7f82c00b3b80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82b40098f0 tx=0x7f82b4009c00 comp rx=0 tx=0).stop 2023-11-28T12:01:42.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.652+0000 7f82c80e5700 1 -- 172.21.15.57:0/391270094 shutdown_connections 2023-11-28T12:01:42.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.652+0000 7f82c80e5700 1 --2- 172.21.15.57:0/391270094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c00b27d0 0x7f82c00b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:42.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.652+0000 7f82c80e5700 1 --2- 172.21.15.57:0/391270094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c006d170 0x7f82c00b3b80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:42.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.652+0000 7f82c80e5700 1 -- 172.21.15.57:0/391270094 >> 172.21.15.57:0/391270094 conn(0x7f82c006ea80 msgr2=0x7f82c006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:42.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.652+0000 7f82c80e5700 1 -- 172.21.15.57:0/391270094 shutdown_connections 2023-11-28T12:01:42.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.652+0000 7f82c80e5700 1 -- 172.21.15.57:0/391270094 wait complete. 2023-11-28T12:01:42.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.653+0000 7f82c80e5700 1 Processor -- start 2023-11-28T12:01:42.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.653+0000 7f82c80e5700 1 -- start start 2023-11-28T12:01:42.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c80e5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c006d170 0x7f82c00ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:42.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c80e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c00b27d0 0x7f82c00c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:42.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c80e5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82c015fe50 0x7f82c0158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:42.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c80e5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c00c91d0 con 0x7f82c015fe50 2023-11-28T12:01:42.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c80e5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c00c8340 con 0x7f82c00b27d0 2023-11-28T12:01:42.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c80e5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0158ba0 con 0x7f82c006d170 2023-11-28T12:01:42.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c6682700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82c015fe50 0x7f82c0158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:42.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c5680700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c00b27d0 0x7f82c00c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:42.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c5e81700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c006d170 0x7f82c00ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:42.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c6682700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82c015fe50 0x7f82c0158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39508/0 (socket says 172.21.15.57:39508) 2023-11-28T12:01:42.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c6682700 1 -- 172.21.15.57:0/1854731009 learned_addr learned my addr 172.21.15.57:0/1854731009 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:42.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c6682700 1 -- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c006d170 msgr2=0x7f82c00ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:42.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c6682700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c006d170 0x7f82c00ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:42.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c6682700 1 -- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c00b27d0 msgr2=0x7f82c00c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:42.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c6682700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c00b27d0 0x7f82c00c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:42.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c6682700 1 -- 172.21.15.57:0/1854731009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b4009550 con 0x7f82c015fe50 2023-11-28T12:01:42.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c5680700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c00b27d0 0x7f82c00c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:42.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c5e81700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c006d170 0x7f82c00ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:42.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.654+0000 7f82c6682700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82c015fe50 0x7f82c0158660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f82b800efc0 tx=0x7f82b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:42.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.655+0000 7f82b27fc700 1 -- 172.21.15.57:0/1854731009 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8009ea0 con 0x7f82c015fe50 2023-11-28T12:01:42.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.655+0000 7f82b27fc700 1 -- 172.21.15.57:0/1854731009 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b800f040 con 0x7f82c015fe50 2023-11-28T12:01:42.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.655+0000 7f82c80e5700 1 -- 172.21.15.57:0/1854731009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c00cf0b0 con 0x7f82c015fe50 2023-11-28T12:01:42.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.655+0000 7f82b27fc700 1 -- 172.21.15.57:0/1854731009 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b800ced0 con 0x7f82c015fe50 2023-11-28T12:01:42.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.655+0000 7f82c80e5700 1 -- 172.21.15.57:0/1854731009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c00cedd0 con 0x7f82c015fe50 2023-11-28T12:01:42.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.656+0000 7f82c80e5700 1 -- 172.21.15.57:0/1854731009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c00cf390 con 0x7f82c015fe50 2023-11-28T12:01:42.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.656+0000 7f82b27fc700 1 -- 172.21.15.57:0/1854731009 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f82b8014480 con 0x7f82c015fe50 2023-11-28T12:01:42.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.656+0000 7f82b27fc700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82a4068320 0x7f82a406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:42.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.657+0000 7f82b27fc700 1 -- 172.21.15.57:0/1854731009 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f82b8086400 con 0x7f82c015fe50 2023-11-28T12:01:42.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.657+0000 7f82c5e81700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82a4068320 0x7f82a406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:42.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.657+0000 7f82c5e81700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82a4068320 0x7f82a406a7d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f82bc007fd0 tx=0x7f82bc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:42.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.661+0000 7f82b27fc700 1 -- 172.21.15.57:0/1854731009 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f82b8052de0 con 0x7f82c015fe50 2023-11-28T12:01:42.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:42.863+0000 7f82c80e5700 1 -- 172.21.15.57:0/1854731009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f82c00743f0 con 0x7f82c015fe50 2023-11-28T12:01:43.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.481+0000 7f82b27fc700 1 -- 172.21.15.57:0/1854731009 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f82b8056400 con 0x7f82c015fe50 2023-11-28T12:01:43.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.487+0000 7f82c80e5700 1 -- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82a4068320 msgr2=0x7f82a406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:43.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.487+0000 7f82c80e5700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82a4068320 0x7f82a406a7d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f82bc007fd0 tx=0x7f82bc009040 comp rx=0 tx=0).stop 2023-11-28T12:01:43.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.487+0000 7f82c80e5700 1 -- 172.21.15.57:0/1854731009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82c015fe50 msgr2=0x7f82c0158660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:43.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.487+0000 7f82c80e5700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82c015fe50 0x7f82c0158660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f82b800efc0 tx=0x7f82b800c3a0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.487+0000 7f82c80e5700 1 -- 172.21.15.57:0/1854731009 shutdown_connections 2023-11-28T12:01:43.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.487+0000 7f82c80e5700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82a4068320 0x7f82a406a7d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.487+0000 7f82c80e5700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82c015fe50 0x7f82c0158660 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.487+0000 7f82c80e5700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c006d170 0x7f82c00ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.487+0000 7f82c80e5700 1 --2- 172.21.15.57:0/1854731009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c00b27d0 0x7f82c00c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.487+0000 7f82c80e5700 1 -- 172.21.15.57:0/1854731009 >> 172.21.15.57:0/1854731009 conn(0x7f82c006ea80 msgr2=0x7f82c00cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:43.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.488+0000 7f82c80e5700 1 -- 172.21.15.57:0/1854731009 shutdown_connections 2023-11-28T12:01:43.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.488+0000 7f82c80e5700 1 -- 172.21.15.57:0/1854731009 wait complete. 2023-11-28T12:01:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' created 2023-11-28T12:01:43.516 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T12:01:43.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.656+0000 7fb7dc869700 1 Processor -- start 2023-11-28T12:01:43.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.656+0000 7fb7dc869700 1 -- start start 2023-11-28T12:01:43.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.657+0000 7fb7dc869700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7d806a980 0x7fb7d80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:43.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.657+0000 7fb7dc869700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7d8071890 0x7fb7d8071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:43.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.657+0000 7fb7dc869700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d815cc10 con 0x7fb7d80cf710 2023-11-28T12:01:43.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.657+0000 7fb7dc869700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d806c9b0 con 0x7fb7d806a980 2023-11-28T12:01:43.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.657+0000 7fb7dc869700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d806cc90 con 0x7fb7d8071890 2023-11-28T12:01:43.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.657+0000 7fb7d659c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fb7d80cf710 0x7fb7d806a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:35762/0 (socket says 172.21.15.57:35762) 2023-11-28T12:01:43.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d659c700 1 -- 172.21.15.57:0/2270501624 learned_addr learned my addr 172.21.15.57:0/2270501624 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:43.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d5d9b700 1 --2- 172.21.15.57:0/2270501624 >> v2:172.21.15.181:3300/0 conn(0x7fb7d806a980 0x7fb7d80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:43.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d6d9d700 1 --2- 172.21.15.57:0/2270501624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7d8071890 0x7fb7d8071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:43.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475523204 0 0) 0x7fb7d815cc10 con 0x7fb7d80cf710 2023-11-28T12:01:43.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7c00035b0 con 0x7fb7d80cf710 2023-11-28T12:01:43.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4195685151 0 0) 0x7fb7c00035b0 con 0x7fb7d80cf710 2023-11-28T12:01:43.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7d8071890 msgr2=0x7fb7d8071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:43.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 --2- 172.21.15.57:0/2270501624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7d8071890 0x7fb7d8071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 >> v2:172.21.15.181:3300/0 conn(0x7fb7d806a980 msgr2=0x7fb7d80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 --2- 172.21.15.57:0/2270501624 >> v2:172.21.15.181:3300/0 conn(0x7fb7d806a980 0x7fb7d80ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d806cf70 con 0x7fb7d80cf710 2023-11-28T12:01:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fb7cc001670 con 0x7fb7d80cf710 2023-11-28T12:01:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 >> v1:172.21.15.57:6789/0 conn(0x7fb7d80cf710 legacy=0x7fb7d806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 --2- 172.21.15.57:0/2270501624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0003a00 0x7fb7c0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 --2- 172.21.15.57:0/2270501624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c0004430 0x7fb7c00048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 --2- 172.21.15.57:0/2270501624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7d8071890 0x7fb7c0005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d806c9b0 con 0x7fb7c0004430 2023-11-28T12:01:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d806cc90 con 0x7fb7c0003a00 2023-11-28T12:01:43.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c000ba10 con 0x7fb7d8071890 2023-11-28T12:01:43.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d5d9b700 1 --2- 172.21.15.57:0/2270501624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c0004430 0x7fb7c00048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:43.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d6d9d700 1 --2- 172.21.15.57:0/2270501624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7d8071890 0x7fb7c0005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:43.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.658+0000 7fb7d659c700 1 --2- 172.21.15.57:0/2270501624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0003a00 0x7fb7c0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:43.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.659+0000 7fb7d5d9b700 1 -- 172.21.15.57:0/2270501624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7d8071890 msgr2=0x7fb7c0005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:43.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.659+0000 7fb7d5d9b700 1 --2- 172.21.15.57:0/2270501624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7d8071890 0x7fb7c0005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.659+0000 7fb7d5d9b700 1 -- 172.21.15.57:0/2270501624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0003a00 msgr2=0x7fb7c0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:43.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.659+0000 7fb7d5d9b700 1 --2- 172.21.15.57:0/2270501624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0003a00 0x7fb7c0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.659+0000 7fb7d5d9b700 1 -- 172.21.15.57:0/2270501624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c00035b0 con 0x7fb7c0004430 2023-11-28T12:01:43.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.659+0000 7fb7d5d9b700 1 --2- 172.21.15.57:0/2270501624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c0004430 0x7fb7c00048f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009b90 tx=0x7fb7c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7214247d48b7e62 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:43.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.659+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c800e6c0 con 0x7fb7c0004430 2023-11-28T12:01:43.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.659+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c800e820 con 0x7fb7c0004430 2023-11-28T12:01:43.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.659+0000 7fb7d559a700 1 -- 172.21.15.57:0/2270501624 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c800eb90 con 0x7fb7c0004430 2023-11-28T12:01:43.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.659+0000 7fb7dc869700 1 -- 172.21.15.57:0/2270501624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c0004430 msgr2=0x7fb7c00048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:43.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.659+0000 7fb7dc869700 1 --2- 172.21.15.57:0/2270501624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c0004430 0x7fb7c00048f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009b90 tx=0x7fb7c0005880 comp rx=0 tx=0).stop 2023-11-28T12:01:43.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.659+0000 7fb7d6d9d700 1 -- 172.21.15.57:0/2270501624 reap_dead start 2023-11-28T12:01:43.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.660+0000 7fb7dc869700 1 -- 172.21.15.57:0/2270501624 shutdown_connections 2023-11-28T12:01:43.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.660+0000 7fb7dc869700 1 -- 172.21.15.57:0/2270501624 >> 172.21.15.57:0/2270501624 conn(0x7fb7d806c010 msgr2=0x7fb7d80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:43.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.660+0000 7fb7dc869700 1 -- 172.21.15.57:0/2270501624 shutdown_connections 2023-11-28T12:01:43.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.660+0000 7fb7dc869700 1 -- 172.21.15.57:0/2270501624 wait complete. 2023-11-28T12:01:43.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.661+0000 7fb7dc869700 1 Processor -- start 2023-11-28T12:01:43.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.661+0000 7fb7dc869700 1 -- start start 2023-11-28T12:01:43.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7dc869700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0004430 0x7fb7d80b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:43.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7dc869700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7d81512c0 0x7fb7d814d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:43.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7dc869700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d815a4c0 0x7fb7d815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:43.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7dc869700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d815ae70 con 0x7fb7d815a4c0 2023-11-28T12:01:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7dc869700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d80b1e20 con 0x7fb7c0004430 2023-11-28T12:01:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7dc869700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d814db40 con 0x7fb7d81512c0 2023-11-28T12:01:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7d6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d815a4c0 0x7fb7d815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7d6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d815a4c0 0x7fb7d815a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39530/0 (socket says 172.21.15.57:39530) 2023-11-28T12:01:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7d6d9d700 1 -- 172.21.15.57:0/1220611843 learned_addr learned my addr 172.21.15.57:0/1220611843 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7d659c700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0004430 0x7fb7d80b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7d5d9b700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7d81512c0 0x7fb7d814d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7d6d9d700 1 -- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7d81512c0 msgr2=0x7fb7d814d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7d6d9d700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7d81512c0 0x7fb7d814d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7d6d9d700 1 -- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0004430 msgr2=0x7fb7d80b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7d6d9d700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0004430 0x7fb7d80b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7d6d9d700 1 -- 172.21.15.57:0/1220611843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d806d530 con 0x7fb7d815a4c0 2023-11-28T12:01:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.662+0000 7fb7d6d9d700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d815a4c0 0x7fb7d815a930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80672e0 tx=0x7fb7d0011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.663+0000 7fb7c6ffd700 1 -- 172.21.15.57:0/1220611843 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000e040 con 0x7fb7d815a4c0 2023-11-28T12:01:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.663+0000 7fb7c6ffd700 1 -- 172.21.15.57:0/1220611843 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d000f040 con 0x7fb7d815a4c0 2023-11-28T12:01:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.663+0000 7fb7dc869700 1 -- 172.21.15.57:0/1220611843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d806d250 con 0x7fb7d815a4c0 2023-11-28T12:01:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.663+0000 7fb7c6ffd700 1 -- 172.21.15.57:0/1220611843 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000fe40 con 0x7fb7d815a4c0 2023-11-28T12:01:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.663+0000 7fb7dc869700 1 -- 172.21.15.57:0/1220611843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d8071220 con 0x7fb7d815a4c0 2023-11-28T12:01:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.664+0000 7fb7dc869700 1 -- 172.21.15.57:0/1220611843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d815e470 con 0x7fb7d815a4c0 2023-11-28T12:01:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.664+0000 7fb7c6ffd700 1 -- 172.21.15.57:0/1220611843 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb7d0022460 con 0x7fb7d815a4c0 2023-11-28T12:01:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.665+0000 7fb7c6ffd700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7b80682f0 0x7fb7b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:43.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.665+0000 7fb7c6ffd700 1 -- 172.21.15.57:0/1220611843 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fb7d0086150 con 0x7fb7d815a4c0 2023-11-28T12:01:43.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.665+0000 7fb7d659c700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7b80682f0 0x7fb7b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:43.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.666+0000 7fb7d659c700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7b80682f0 0x7fb7b806a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0015d0 tx=0x7fb7cc007980 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:43.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.670+0000 7fb7c6ffd700 1 -- 172.21.15.57:0/1220611843 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb7d815e470 con 0x7fb7d815a4c0 2023-11-28T12:01:43.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.917+0000 7fb7dc869700 1 -- 172.21.15.57:0/1220611843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb7d815e470 con 0x7fb7d815a4c0 2023-11-28T12:01:43.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.930+0000 7fb7c6ffd700 1 -- 172.21.15.57:0/1220611843 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb7d815e470 con 0x7fb7d815a4c0 2023-11-28T12:01:43.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.939+0000 7fb7dc869700 1 -- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7b80682f0 msgr2=0x7fb7b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:43.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.940+0000 7fb7dc869700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7b80682f0 0x7fb7b806a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0015d0 tx=0x7fb7cc007980 comp rx=0 tx=0).stop 2023-11-28T12:01:43.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.940+0000 7fb7dc869700 1 -- 172.21.15.57:0/1220611843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d815a4c0 msgr2=0x7fb7d815a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:43.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.940+0000 7fb7dc869700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d815a4c0 0x7fb7d815a930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80672e0 tx=0x7fb7d0011f60 comp rx=0 tx=0).stop 2023-11-28T12:01:43.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.942+0000 7fb7dc869700 1 -- 172.21.15.57:0/1220611843 shutdown_connections 2023-11-28T12:01:43.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.942+0000 7fb7dc869700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7b80682f0 0x7fb7b806a7a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.942+0000 7fb7dc869700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d815a4c0 0x7fb7d815a930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.942+0000 7fb7dc869700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c0004430 0x7fb7d80b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.942+0000 7fb7dc869700 1 --2- 172.21.15.57:0/1220611843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7d81512c0 0x7fb7d814d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:43.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.942+0000 7fb7dc869700 1 -- 172.21.15.57:0/1220611843 >> 172.21.15.57:0/1220611843 conn(0x7fb7d806c010 msgr2=0x7fb7d80ad500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:43.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.943+0000 7fb7dc869700 1 -- 172.21.15.57:0/1220611843 shutdown_connections 2023-11-28T12:01:43.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:43.943+0000 7fb7dc869700 1 -- 172.21.15.57:0/1220611843 wait complete. 2023-11-28T12:01:43.949 INFO:teuthology.orchestra.run.smithi057.stderr:new fs with metadata pool 3 and data pool 4 2023-11-28T12:01:43.964 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:01:44.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.098+0000 7fe1155f1700 1 Processor -- start 2023-11-28T12:01:44.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.098+0000 7fe1155f1700 1 -- start start 2023-11-28T12:01:44.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe1155f1700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe11006ab10 0x7fe11006aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe1155f1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1100d0450 0x7fe110160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe1155f1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1100d1ee0 con 0x7fe1100cd5a0 2023-11-28T12:01:44.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe1155f1700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe11006b420 con 0x7fe11006ab10 2023-11-28T12:01:44.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe1155f1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100cdb10 con 0x7fe1100d0450 2023-11-28T12:01:44.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe10effd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe11006ab10 0x7fe11006aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe10effd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe11006ab10 0x7fe11006aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39538/0 (socket says 172.21.15.57:39538) 2023-11-28T12:01:44.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe10effd700 1 -- 172.21.15.57:0/4156978409 learned_addr learned my addr 172.21.15.57:0/4156978409 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:44.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe10f7fe700 1 --2- 172.21.15.57:0/4156978409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1100d0450 0x7fe110160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe10effd700 1 -- 172.21.15.57:0/4156978409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1100d0450 msgr2=0x7fe110160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:44.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe10effd700 1 --2- 172.21.15.57:0/4156978409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1100d0450 0x7fe110160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe10effd700 1 -- 172.21.15.57:0/4156978409 >> v1:172.21.15.181:6789/0 conn(0x7fe1100cd5a0 legacy=0x7fe110160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe10effd700 1 -- 172.21.15.57:0/4156978409 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1100d1890 con 0x7fe11006ab10 2023-11-28T12:01:44.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.099+0000 7fe10effd700 1 --2- 172.21.15.57:0/4156978409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe11006ab10 0x7fe11006aee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80098f0 tx=0x7fe0f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8367930d7c32bcbc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:44.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.100+0000 7fe10dffb700 1 -- 172.21.15.57:0/4156978409 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f800b8a0 con 0x7fe11006ab10 2023-11-28T12:01:44.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.100+0000 7fe10dffb700 1 -- 172.21.15.57:0/4156978409 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f800ba00 con 0x7fe11006ab10 2023-11-28T12:01:44.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.100+0000 7fe10dffb700 1 -- 172.21.15.57:0/4156978409 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f800bd90 con 0x7fe11006ab10 2023-11-28T12:01:44.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.100+0000 7fe1155f1700 1 -- 172.21.15.57:0/4156978409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe11006ab10 msgr2=0x7fe11006aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.100+0000 7fe1155f1700 1 --2- 172.21.15.57:0/4156978409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe11006ab10 0x7fe11006aee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80098f0 tx=0x7fe0f8009c00 comp rx=0 tx=0).stop 2023-11-28T12:01:44.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.100+0000 7fe1155f1700 1 -- 172.21.15.57:0/4156978409 shutdown_connections 2023-11-28T12:01:44.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.100+0000 7fe1155f1700 1 --2- 172.21.15.57:0/4156978409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe11006ab10 0x7fe11006aee0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.100+0000 7fe1155f1700 1 --2- 172.21.15.57:0/4156978409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1100d0450 0x7fe110160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.100+0000 7fe1155f1700 1 -- 172.21.15.57:0/4156978409 >> 172.21.15.57:0/4156978409 conn(0x7fe11006c520 msgr2=0x7fe11006c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:44.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.100+0000 7fe1155f1700 1 -- 172.21.15.57:0/4156978409 shutdown_connections 2023-11-28T12:01:44.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.100+0000 7fe1155f1700 1 -- 172.21.15.57:0/4156978409 wait complete. 2023-11-28T12:01:44.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.102+0000 7fe1155f1700 1 Processor -- start 2023-11-28T12:01:44.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.102+0000 7fe1155f1700 1 -- start start 2023-11-28T12:01:44.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe1155f1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe11006ab10 0x7fe1101f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe1155f1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1100cd5a0 0x7fe1100c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe1155f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1100d0450 0x7fe110152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe1155f1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100c7e00 con 0x7fe11006ab10 2023-11-28T12:01:44.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe1155f1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100ca1c0 con 0x7fe1100d0450 2023-11-28T12:01:44.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe1155f1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100ca4a0 con 0x7fe1100cd5a0 2023-11-28T12:01:44.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe10effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe11006ab10 0x7fe1101f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe10effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe11006ab10 0x7fe1101f2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39548/0 (socket says 172.21.15.57:39548) 2023-11-28T12:01:44.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe10effd700 1 -- 172.21.15.57:0/97493686 learned_addr learned my addr 172.21.15.57:0/97493686 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:44.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe10f7fe700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1100d0450 0x7fe110152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe10e7fc700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1100cd5a0 0x7fe1100c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe10effd700 1 -- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1100cd5a0 msgr2=0x7fe1100c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe10effd700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1100cd5a0 0x7fe1100c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe10effd700 1 -- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1100d0450 msgr2=0x7fe110152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe10effd700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1100d0450 0x7fe110152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe10effd700 1 -- 172.21.15.57:0/97493686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f8009550 con 0x7fe11006ab10 2023-11-28T12:01:44.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe10effd700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe11006ab10 0x7fe1101f2180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009c40 tx=0x7fe0f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:44.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe1077fe700 1 -- 172.21.15.57:0/97493686 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8004380 con 0x7fe11006ab10 2023-11-28T12:01:44.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe1077fe700 1 -- 172.21.15.57:0/97493686 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f801f070 con 0x7fe11006ab10 2023-11-28T12:01:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe1155f1700 1 -- 172.21.15.57:0/97493686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11015cad0 con 0x7fe11006ab10 2023-11-28T12:01:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe1077fe700 1 -- 172.21.15.57:0/97493686 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80036a0 con 0x7fe11006ab10 2023-11-28T12:01:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.103+0000 7fe1155f1700 1 -- 172.21.15.57:0/97493686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1100d1b70 con 0x7fe11006ab10 2023-11-28T12:01:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.104+0000 7fe1155f1700 1 -- 172.21.15.57:0/97493686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1100b1ec0 con 0x7fe11006ab10 2023-11-28T12:01:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.105+0000 7fe1077fe700 1 -- 172.21.15.57:0/97493686 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe0f802a020 con 0x7fe11006ab10 2023-11-28T12:01:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.105+0000 7fe1077fe700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe0f00682f0 0x7fe0f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.105+0000 7fe1077fe700 1 -- 172.21.15.57:0/97493686 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe0f8087330 con 0x7fe11006ab10 2023-11-28T12:01:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.106+0000 7fe10e7fc700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe0f00682f0 0x7fe0f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.106+0000 7fe10e7fc700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe0f00682f0 0x7fe0f006a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1100d3c80 tx=0x7fe100009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:44.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.110+0000 7fe1077fe700 1 -- 172.21.15.57:0/97493686 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe0f8053b90 con 0x7fe11006ab10 2023-11-28T12:01:44.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.311+0000 7fe1155f1700 1 -- 172.21.15.57:0/97493686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1100743f0 con 0x7fe11006ab10 2023-11-28T12:01:44.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.312+0000 7fe1077fe700 1 -- 172.21.15.57:0/97493686 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+17862 (secure 0 0 0) 0x7fe0f80571b0 con 0x7fe11006ab10 2023-11-28T12:01:44.313 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:44.313 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":20,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:43.919074+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:01:44.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.314+0000 7fe1155f1700 1 -- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe0f00682f0 msgr2=0x7fe0f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.314+0000 7fe1155f1700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe0f00682f0 0x7fe0f006a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1100d3c80 tx=0x7fe100009040 comp rx=0 tx=0).stop 2023-11-28T12:01:44.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.314+0000 7fe1155f1700 1 -- 172.21.15.57:0/97493686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe11006ab10 msgr2=0x7fe1101f2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.314+0000 7fe1155f1700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe11006ab10 0x7fe1101f2180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009c40 tx=0x7fe0f8004750 comp rx=0 tx=0).stop 2023-11-28T12:01:44.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.314+0000 7fe1155f1700 1 -- 172.21.15.57:0/97493686 shutdown_connections 2023-11-28T12:01:44.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.314+0000 7fe1155f1700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe0f00682f0 0x7fe0f006a7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.314+0000 7fe1155f1700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1100d0450 0x7fe110152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.314+0000 7fe1155f1700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe11006ab10 0x7fe1101f2180 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.314+0000 7fe1155f1700 1 --2- 172.21.15.57:0/97493686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1100cd5a0 0x7fe1100c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.314+0000 7fe1155f1700 1 -- 172.21.15.57:0/97493686 >> 172.21.15.57:0/97493686 conn(0x7fe11006c520 msgr2=0x7fe110158380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:44.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.314+0000 7fe1155f1700 1 -- 172.21.15.57:0/97493686 shutdown_connections 2023-11-28T12:01:44.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.314+0000 7fe1155f1700 1 -- 172.21.15.57:0/97493686 wait complete. 2023-11-28T12:01:44.329 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:01:44.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.463+0000 7fc5c9bb6700 1 Processor -- start 2023-11-28T12:01:44.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.463+0000 7fc5c9bb6700 1 -- start start 2023-11-28T12:01:44.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.464+0000 7fc5c9bb6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5c40b0880 0x7fc5c406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.464+0000 7fc5c9bb6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5c40ad1f0 0x7fc5c40ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.464+0000 7fc5c9bb6700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5c40b43f0 con 0x7fc5c4072180 2023-11-28T12:01:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.464+0000 7fc5c9bb6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5c40adbb0 con 0x7fc5c40ad1f0 2023-11-28T12:01:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.464+0000 7fc5c9bb6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c40b46d0 con 0x7fc5c40b0880 2023-11-28T12:01:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.464+0000 7fc5c2ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc5c4072180 0x7fc5c40aeb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36874/0 (socket says 172.21.15.57:36874) 2023-11-28T12:01:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.464+0000 7fc5c2ffd700 1 -- 172.21.15.57:0/773132963 learned_addr learned my addr 172.21.15.57:0/773132963 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.464+0000 7fc5c3fff700 1 --2- 172.21.15.57:0/773132963 >> v2:172.21.15.181:3300/0 conn(0x7fc5c40ad1f0 0x7fc5c40ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.464+0000 7fc5c37fe700 1 --2- 172.21.15.57:0/773132963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5c40b0880 0x7fc5c406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.465+0000 7fc5c27fc700 1 -- 172.21.15.57:0/773132963 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993486996 0 0) 0x7fc5c40b43f0 con 0x7fc5c4072180 2023-11-28T12:01:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.465+0000 7fc5c27fc700 1 -- 172.21.15.57:0/773132963 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b00035b0 con 0x7fc5c4072180 2023-11-28T12:01:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.465+0000 7fc5c3fff700 1 -- 172.21.15.57:0/773132963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5c40b0880 msgr2=0x7fc5c406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.465+0000 7fc5c3fff700 1 --2- 172.21.15.57:0/773132963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5c40b0880 0x7fc5c406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.465+0000 7fc5c3fff700 1 -- 172.21.15.57:0/773132963 >> v1:172.21.15.57:6789/0 conn(0x7fc5c4072180 legacy=0x7fc5c40aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.465+0000 7fc5c3fff700 1 -- 172.21.15.57:0/773132963 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c40b4d00 con 0x7fc5c40ad1f0 2023-11-28T12:01:44.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.465+0000 7fc5c3fff700 1 --2- 172.21.15.57:0/773132963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c40ad1f0 0x7fc5c40ad670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009d90 tx=0x7fc5bc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee58fee5f02d271b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:44.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.465+0000 7fc5c27fc700 1 -- 172.21.15.57:0/773132963 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc00e070 con 0x7fc5c40ad1f0 2023-11-28T12:01:44.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.465+0000 7fc5c27fc700 1 -- 172.21.15.57:0/773132963 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5bc00bb20 con 0x7fc5c40ad1f0 2023-11-28T12:01:44.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.465+0000 7fc5c27fc700 1 -- 172.21.15.57:0/773132963 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc012430 con 0x7fc5c40ad1f0 2023-11-28T12:01:44.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.466+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/773132963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c40ad1f0 msgr2=0x7fc5c40ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.466+0000 7fc5c9bb6700 1 --2- 172.21.15.57:0/773132963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c40ad1f0 0x7fc5c40ad670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009d90 tx=0x7fc5bc00b330 comp rx=0 tx=0).stop 2023-11-28T12:01:44.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.466+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/773132963 shutdown_connections 2023-11-28T12:01:44.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.466+0000 7fc5c9bb6700 1 --2- 172.21.15.57:0/773132963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c40ad1f0 0x7fc5c40ad670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.466+0000 7fc5c9bb6700 1 --2- 172.21.15.57:0/773132963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5c40b0880 0x7fc5c406b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.466+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/773132963 >> 172.21.15.57:0/773132963 conn(0x7fc5c40b2fc0 msgr2=0x7fc5c4071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:44.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.466+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/773132963 shutdown_connections 2023-11-28T12:01:44.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.466+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/773132963 wait complete. 2023-11-28T12:01:44.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.468+0000 7fc5c9bb6700 1 Processor -- start 2023-11-28T12:01:44.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.468+0000 7fc5c9bb6700 1 -- start start 2023-11-28T12:01:44.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c9bb6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c4072180 0x7fc5c415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c9bb6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5c40ad1f0 0x7fc5c40cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c9bb6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5c40b0880 0x7fc5c41f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c9bb6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c41f67b0 con 0x7fc5c40b0880 2023-11-28T12:01:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c9bb6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c40cf230 con 0x7fc5c4072180 2023-11-28T12:01:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c9bb6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c41f5ba0 con 0x7fc5c40ad1f0 2023-11-28T12:01:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5c40b0880 0x7fc5c41f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5c40b0880 0x7fc5c41f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47688/0 (socket says 172.21.15.57:47688) 2023-11-28T12:01:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c3fff700 1 -- 172.21.15.57:0/3831854971 learned_addr learned my addr 172.21.15.57:0/3831854971 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c2ffd700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5c40ad1f0 0x7fc5c40cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c37fe700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c4072180 0x7fc5c415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c3fff700 1 -- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5c40ad1f0 msgr2=0x7fc5c40cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c3fff700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5c40ad1f0 0x7fc5c40cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c3fff700 1 -- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c4072180 msgr2=0x7fc5c415d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c3fff700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c4072180 0x7fc5c415d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c3fff700 1 -- 172.21.15.57:0/3831854971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc009a70 con 0x7fc5c40b0880 2023-11-28T12:01:44.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c2ffd700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5c40ad1f0 0x7fc5c40cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:01:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.469+0000 7fc5c3fff700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5c40b0880 0x7fc5c41f2590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc014040 tx=0x7fc5bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.470+0000 7fc5bbfff700 1 -- 172.21.15.57:0/3831854971 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0129f0 con 0x7fc5c40b0880 2023-11-28T12:01:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.470+0000 7fc5bbfff700 1 -- 172.21.15.57:0/3831854971 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5bc020070 con 0x7fc5c40b0880 2023-11-28T12:01:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.470+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/3831854971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c40725f0 con 0x7fc5c40b0880 2023-11-28T12:01:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.470+0000 7fc5bbfff700 1 -- 172.21.15.57:0/3831854971 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc012db0 con 0x7fc5c40b0880 2023-11-28T12:01:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.470+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/3831854971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c40b4fe0 con 0x7fc5c40b0880 2023-11-28T12:01:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.471+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/3831854971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c40b49b0 con 0x7fc5c40b0880 2023-11-28T12:01:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.471+0000 7fc5bbfff700 1 -- 172.21.15.57:0/3831854971 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc5bc004040 con 0x7fc5c40b0880 2023-11-28T12:01:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.471+0000 7fc5bbfff700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5a40682f0 0x7fc5a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.472+0000 7fc5bbfff700 1 -- 172.21.15.57:0/3831854971 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc5bc022070 con 0x7fc5c40b0880 2023-11-28T12:01:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.472+0000 7fc5c37fe700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5a40682f0 0x7fc5a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.472+0000 7fc5c37fe700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5a40682f0 0x7fc5a406a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc5c406ba90 tx=0x7fc5b4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.476+0000 7fc5bbfff700 1 -- 172.21.15.57:0/3831854971 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc5bc054680 con 0x7fc5c40b0880 2023-11-28T12:01:44.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.685+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/3831854971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5c40743f0 con 0x7fc5c40b0880 2023-11-28T12:01:44.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.686+0000 7fc5bbfff700 1 -- 172.21.15.57:0/3831854971 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+17862 (secure 0 0 0) 0x7fc5bc057ca0 con 0x7fc5c40b0880 2023-11-28T12:01:44.688 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:44.688 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":20,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:43.919074+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:01:44.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.688+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5a40682f0 msgr2=0x7fc5a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.688+0000 7fc5c9bb6700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5a40682f0 0x7fc5a406a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc5c406ba90 tx=0x7fc5b4008040 comp rx=0 tx=0).stop 2023-11-28T12:01:44.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.688+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/3831854971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5c40b0880 msgr2=0x7fc5c41f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.688+0000 7fc5c9bb6700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5c40b0880 0x7fc5c41f2590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc014040 tx=0x7fc5bc004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.688+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/3831854971 shutdown_connections 2023-11-28T12:01:44.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.688+0000 7fc5c9bb6700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5a40682f0 0x7fc5a406a7a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.688+0000 7fc5c9bb6700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5c40b0880 0x7fc5c41f2590 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.688+0000 7fc5c9bb6700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c4072180 0x7fc5c415d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.689+0000 7fc5c9bb6700 1 --2- 172.21.15.57:0/3831854971 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5c40ad1f0 0x7fc5c40cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.689+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/3831854971 >> 172.21.15.57:0/3831854971 conn(0x7fc5c40b2fc0 msgr2=0x7fc5c40aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:44.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.689+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/3831854971 shutdown_connections 2023-11-28T12:01:44.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.689+0000 7fc5c9bb6700 1 -- 172.21.15.57:0/3831854971 wait complete. 2023-11-28T12:01:44.704 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T12:01:44.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.841+0000 7f5c55e9a700 1 Processor -- start 2023-11-28T12:01:44.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.841+0000 7f5c55e9a700 1 -- start start 2023-11-28T12:01:44.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.841+0000 7f5c55e9a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c500cac90 0x7f5c500cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.841+0000 7f5c55e9a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5015ceb0 con 0x7f5c500cec00 2023-11-28T12:01:44.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.841+0000 7f5c55e9a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c501608c0 con 0x7f5c50160300 2023-11-28T12:01:44.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.842+0000 7f5c55e9a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c500cb650 con 0x7f5c500cac90 2023-11-28T12:01:44.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.842+0000 7f5c4f7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f5c500cec00 0x7f5c50065fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36882/0 (socket says 172.21.15.57:36882) 2023-11-28T12:01:44.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.842+0000 7f5c4f7fe700 1 -- 172.21.15.57:0/2769764803 learned_addr learned my addr 172.21.15.57:0/2769764803 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:44.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.842+0000 7f5c4ffff700 1 --2- 172.21.15.57:0/2769764803 >> v2:172.21.15.181:3300/0 conn(0x7f5c500cac90 0x7f5c500cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.842+0000 7f5c4e7fc700 1 -- 172.21.15.57:0/2769764803 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828034577 0 0) 0x7f5c5015ceb0 con 0x7f5c500cec00 2023-11-28T12:01:44.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.842+0000 7f5c4e7fc700 1 -- 172.21.15.57:0/2769764803 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c380035b0 con 0x7f5c500cec00 2023-11-28T12:01:44.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.842+0000 7f5c4e7fc700 1 -- 172.21.15.57:0/2769764803 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723583147 0 0) 0x7f5c501608c0 con 0x7f5c50160300 2023-11-28T12:01:44.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.842+0000 7f5c4e7fc700 1 -- 172.21.15.57:0/2769764803 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c5015ceb0 con 0x7f5c50160300 2023-11-28T12:01:44.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.842+0000 7f5c4e7fc700 1 -- 172.21.15.57:0/2769764803 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2904844468 0 0) 0x7f5c380035b0 con 0x7f5c500cec00 2023-11-28T12:01:44.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.842+0000 7f5c4ffff700 1 -- 172.21.15.57:0/2769764803 >> v1:172.21.15.181:6789/0 conn(0x7f5c50160300 legacy=0x7f5c5015d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.842+0000 7f5c4ffff700 1 -- 172.21.15.57:0/2769764803 >> v1:172.21.15.57:6789/0 conn(0x7f5c500cec00 legacy=0x7f5c50065fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.842+0000 7f5c4ffff700 1 -- 172.21.15.57:0/2769764803 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c50160ba0 con 0x7f5c500cac90 2023-11-28T12:01:44.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.842+0000 7f5c4ffff700 1 --2- 172.21.15.57:0/2769764803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c500cac90 0x7f5c500cb110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009d90 tx=0x7f5c4800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6073864eff557d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:44.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.843+0000 7f5c4e7fc700 1 -- 172.21.15.57:0/2769764803 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4800e070 con 0x7f5c500cac90 2023-11-28T12:01:44.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.843+0000 7f5c4e7fc700 1 -- 172.21.15.57:0/2769764803 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48003a20 con 0x7f5c500cac90 2023-11-28T12:01:44.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.843+0000 7f5c4e7fc700 1 -- 172.21.15.57:0/2769764803 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40001670 con 0x7f5c500cac90 2023-11-28T12:01:44.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.843+0000 7f5c55e9a700 1 -- 172.21.15.57:0/2769764803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c500cac90 msgr2=0x7f5c500cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.843+0000 7f5c55e9a700 1 --2- 172.21.15.57:0/2769764803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c500cac90 0x7f5c500cb110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009d90 tx=0x7f5c4800b330 comp rx=0 tx=0).stop 2023-11-28T12:01:44.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.843+0000 7f5c55e9a700 1 -- 172.21.15.57:0/2769764803 shutdown_connections 2023-11-28T12:01:44.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.843+0000 7f5c55e9a700 1 --2- 172.21.15.57:0/2769764803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c500cac90 0x7f5c500cb110 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.843+0000 7f5c55e9a700 1 -- 172.21.15.57:0/2769764803 >> 172.21.15.57:0/2769764803 conn(0x7f5c500cf400 msgr2=0x7f5c500cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:44.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.843+0000 7f5c55e9a700 1 -- 172.21.15.57:0/2769764803 shutdown_connections 2023-11-28T12:01:44.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.843+0000 7f5c55e9a700 1 -- 172.21.15.57:0/2769764803 wait complete. 2023-11-28T12:01:44.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.844+0000 7f5c55e9a700 1 Processor -- start 2023-11-28T12:01:44.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.845+0000 7f5c55e9a700 1 -- start start 2023-11-28T12:01:44.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.845+0000 7f5c55e9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c500cac90 0x7f5c50154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.845+0000 7f5c55e9a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c500cec00 0x7f5c50154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c55e9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c50160300 0x7f5c500b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c55e9a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500ad7c0 con 0x7f5c50160300 2023-11-28T12:01:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c55e9a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500ac8f0 con 0x7f5c500cac90 2023-11-28T12:01:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c55e9a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500acb40 con 0x7f5c500cec00 2023-11-28T12:01:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c4ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c50160300 0x7f5c500b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c4ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c50160300 0x7f5c500b3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47700/0 (socket says 172.21.15.57:47700) 2023-11-28T12:01:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c4ffff700 1 -- 172.21.15.57:0/103291264 learned_addr learned my addr 172.21.15.57:0/103291264 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c4f7fe700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c500cac90 0x7f5c50154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c4effd700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c500cec00 0x7f5c50154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c4ffff700 1 -- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c500cec00 msgr2=0x7f5c50154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c4ffff700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c500cec00 0x7f5c50154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c4ffff700 1 -- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c500cac90 msgr2=0x7f5c50154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:44.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c4ffff700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c500cac90 0x7f5c50154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:44.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c4ffff700 1 -- 172.21.15.57:0/103291264 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c48009a70 con 0x7f5c50160300 2023-11-28T12:01:44.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c4f7fe700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c500cac90 0x7f5c50154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:44.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c4ffff700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c50160300 0x7f5c500b3b70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5c48015040 tx=0x7f5c48004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:44.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c37fff700 1 -- 172.21.15.57:0/103291264 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c480043c0 con 0x7f5c50160300 2023-11-28T12:01:44.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c37fff700 1 -- 172.21.15.57:0/103291264 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48004520 con 0x7f5c50160300 2023-11-28T12:01:44.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c55e9a700 1 -- 172.21.15.57:0/103291264 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c50161160 con 0x7f5c50160300 2023-11-28T12:01:44.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c37fff700 1 -- 172.21.15.57:0/103291264 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48004890 con 0x7f5c50160300 2023-11-28T12:01:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.846+0000 7f5c55e9a700 1 -- 172.21.15.57:0/103291264 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c50160e80 con 0x7f5c50160300 2023-11-28T12:01:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.847+0000 7f5c55e9a700 1 -- 172.21.15.57:0/103291264 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c500cefd0 con 0x7f5c50160300 2023-11-28T12:01:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.848+0000 7f5c37fff700 1 -- 172.21.15.57:0/103291264 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5c48018040 con 0x7f5c50160300 2023-11-28T12:01:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.848+0000 7f5c37fff700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5c3c0682f0 0x7f5c3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.848+0000 7f5c37fff700 1 -- 172.21.15.57:0/103291264 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c48087360 con 0x7f5c50160300 2023-11-28T12:01:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.849+0000 7f5c4f7fe700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5c3c0682f0 0x7f5c3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.849+0000 7f5c4f7fe700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5c3c0682f0 0x7f5c3c06a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5c40001380 tx=0x7f5c40007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:44.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:44.853+0000 7f5c37fff700 1 -- 172.21.15.57:0/103291264 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5c48053ad0 con 0x7f5c50160300 2023-11-28T12:01:45.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.101+0000 7f5c55e9a700 1 -- 172.21.15.57:0/103291264 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5c500743f0 con 0x7f5c50160300 2023-11-28T12:01:45.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.939+0000 7f5c37fff700 1 -- 172.21.15.57:0/103291264 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5c480570f0 con 0x7f5c50160300 2023-11-28T12:01:45.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.941+0000 7f5c55e9a700 1 -- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5c3c0682f0 msgr2=0x7f5c3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:45.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.941+0000 7f5c55e9a700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5c3c0682f0 0x7f5c3c06a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5c40001380 tx=0x7f5c40007aa0 comp rx=0 tx=0).stop 2023-11-28T12:01:45.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.941+0000 7f5c55e9a700 1 -- 172.21.15.57:0/103291264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c50160300 msgr2=0x7f5c500b3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:45.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.941+0000 7f5c55e9a700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c50160300 0x7f5c500b3b70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5c48015040 tx=0x7f5c48004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:45.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.942+0000 7f5c55e9a700 1 -- 172.21.15.57:0/103291264 shutdown_connections 2023-11-28T12:01:45.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.942+0000 7f5c55e9a700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5c3c0682f0 0x7f5c3c06a7a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:45.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.942+0000 7f5c55e9a700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c50160300 0x7f5c500b3b70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:45.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.942+0000 7f5c55e9a700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c500cac90 0x7f5c50154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:45.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.942+0000 7f5c55e9a700 1 --2- 172.21.15.57:0/103291264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c500cec00 0x7f5c50154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:45.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.942+0000 7f5c55e9a700 1 -- 172.21.15.57:0/103291264 >> 172.21.15.57:0/103291264 conn(0x7f5c500cf400 msgr2=0x7f5c501554d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:45.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.942+0000 7f5c55e9a700 1 -- 172.21.15.57:0/103291264 shutdown_connections 2023-11-28T12:01:45.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:45.942+0000 7f5c55e9a700 1 -- 172.21.15.57:0/103291264 wait complete. 2023-11-28T12:01:45.959 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T12:01:45.959 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T12:01:46.062 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:01:46.064 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:01:46.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.102+0000 7fe5a7ebb700 1 Processor -- start 2023-11-28T12:01:46.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.102+0000 7fe5a7ebb700 1 -- start start 2023-11-28T12:01:46.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.103+0000 7fe5a7ebb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5a006a980 0x7fe5a00cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:46.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.103+0000 7fe5a7ebb700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe5a00cdd50 0x7fe5a00cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:46.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.103+0000 7fe5a7ebb700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a015e0d0 con 0x7fe5a015a4d0 2023-11-28T12:01:46.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.103+0000 7fe5a7ebb700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a00ce560 con 0x7fe5a00cdd50 2023-11-28T12:01:46.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.103+0000 7fe5a7ebb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a015e320 con 0x7fe5a006a980 2023-11-28T12:01:46.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.103+0000 7fe5a6458700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe5a00cdd50 0x7fe5a00cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:46.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.103+0000 7fe5a5c57700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe5a015a4d0 0x7fe5a006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36888/0 (socket says 172.21.15.57:36888) 2023-11-28T12:01:46.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.103+0000 7fe5a5c57700 1 -- 172.21.15.57:0/1924814766 learned_addr learned my addr 172.21.15.57:0/1924814766 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:46.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.104+0000 7fe5a5456700 1 --2- 172.21.15.57:0/1924814766 >> v2:172.21.15.181:3300/0 conn(0x7fe5a006a980 0x7fe5a00cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:46.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.104+0000 7fe5a4c55700 1 -- 172.21.15.57:0/1924814766 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807116337 0 0) 0x7fe5a015e0d0 con 0x7fe5a015a4d0 2023-11-28T12:01:46.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.104+0000 7fe5a4c55700 1 -- 172.21.15.57:0/1924814766 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe58c0035b0 con 0x7fe5a015a4d0 2023-11-28T12:01:46.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.104+0000 7fe5a6458700 1 -- 172.21.15.57:0/1924814766 >> v2:172.21.15.181:3300/0 conn(0x7fe5a006a980 msgr2=0x7fe5a00cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:46.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.104+0000 7fe5a6458700 1 --2- 172.21.15.57:0/1924814766 >> v2:172.21.15.181:3300/0 conn(0x7fe5a006a980 0x7fe5a00cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:46.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.104+0000 7fe5a6458700 1 -- 172.21.15.57:0/1924814766 >> v1:172.21.15.57:6789/0 conn(0x7fe5a015a4d0 legacy=0x7fe5a006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:46.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.104+0000 7fe5a6458700 1 -- 172.21.15.57:0/1924814766 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a015e600 con 0x7fe5a00cdd50 2023-11-28T12:01:46.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.104+0000 7fe5a6458700 1 --2- 172.21.15.57:0/1924814766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a00cdd50 0x7fe5a00cb010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0070e20 tx=0x7fe59800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c3fcec6b14893a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:46.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.104+0000 7fe5a4c55700 1 -- 172.21.15.57:0/1924814766 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59800e070 con 0x7fe5a00cdd50 2023-11-28T12:01:46.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.104+0000 7fe5a4c55700 1 -- 172.21.15.57:0/1924814766 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59800bb20 con 0x7fe5a00cdd50 2023-11-28T12:01:46.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.104+0000 7fe5a4c55700 1 -- 172.21.15.57:0/1924814766 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598012430 con 0x7fe5a00cdd50 2023-11-28T12:01:46.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.105+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/1924814766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a00cdd50 msgr2=0x7fe5a00cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:46.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.105+0000 7fe5a7ebb700 1 --2- 172.21.15.57:0/1924814766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a00cdd50 0x7fe5a00cb010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0070e20 tx=0x7fe59800b330 comp rx=0 tx=0).stop 2023-11-28T12:01:46.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.105+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/1924814766 shutdown_connections 2023-11-28T12:01:46.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.105+0000 7fe5a7ebb700 1 --2- 172.21.15.57:0/1924814766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a00cdd50 0x7fe5a00cb010 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:46.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.105+0000 7fe5a7ebb700 1 --2- 172.21.15.57:0/1924814766 >> v2:172.21.15.181:3300/0 conn(0x7fe5a006a980 0x7fe5a00cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:46.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.105+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/1924814766 >> 172.21.15.57:0/1924814766 conn(0x7fe5a006c010 msgr2=0x7fe5a00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:46.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.105+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/1924814766 shutdown_connections 2023-11-28T12:01:46.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.105+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/1924814766 wait complete. 2023-11-28T12:01:46.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.107+0000 7fe5a7ebb700 1 Processor -- start 2023-11-28T12:01:46.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.107+0000 7fe5a7ebb700 1 -- start start 2023-11-28T12:01:46.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a7ebb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a006a980 0x7fe5a0153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:46.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a7ebb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00cdd50 0x7fe5a01470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:46.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a7ebb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a015a4d0 0x7fe5a01e5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:46.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a7ebb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00b0e60 con 0x7fe5a015a4d0 2023-11-28T12:01:46.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a7ebb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a01e43e0 con 0x7fe5a00cdd50 2023-11-28T12:01:46.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a7ebb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a01e4100 con 0x7fe5a006a980 2023-11-28T12:01:46.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a6458700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a015a4d0 0x7fe5a01e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:46.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a6458700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a015a4d0 0x7fe5a01e5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47724/0 (socket says 172.21.15.57:47724) 2023-11-28T12:01:46.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a6458700 1 -- 172.21.15.57:0/3050419748 learned_addr learned my addr 172.21.15.57:0/3050419748 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:46.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a5456700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00cdd50 0x7fe5a01470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:46.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a5c57700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a006a980 0x7fe5a0153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:46.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a6458700 1 -- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a006a980 msgr2=0x7fe5a0153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:46.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a6458700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a006a980 0x7fe5a0153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:46.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a6458700 1 -- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00cdd50 msgr2=0x7fe5a01470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:46.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a6458700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00cdd50 0x7fe5a01470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:46.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a6458700 1 -- 172.21.15.57:0/3050419748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe598009a70 con 0x7fe5a015a4d0 2023-11-28T12:01:46.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a6458700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a015a4d0 0x7fe5a01e5880 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe598014040 tx=0x7fe598004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:46.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5927fc700 1 -- 172.21.15.57:0/3050419748 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598012430 con 0x7fe5a015a4d0 2023-11-28T12:01:46.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5927fc700 1 -- 172.21.15.57:0/3050419748 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe598020070 con 0x7fe5a015a4d0 2023-11-28T12:01:46.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.108+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/3050419748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a00ce1b0 con 0x7fe5a015a4d0 2023-11-28T12:01:46.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.109+0000 7fe5927fc700 1 -- 172.21.15.57:0/3050419748 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5980098f0 con 0x7fe5a015a4d0 2023-11-28T12:01:46.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.109+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/3050419748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a015e8e0 con 0x7fe5a015a4d0 2023-11-28T12:01:46.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.110+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/3050419748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a015a8a0 con 0x7fe5a015a4d0 2023-11-28T12:01:46.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.110+0000 7fe5927fc700 1 -- 172.21.15.57:0/3050419748 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe598004040 con 0x7fe5a015a4d0 2023-11-28T12:01:46.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.110+0000 7fe5927fc700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe5840682f0 0x7fe58406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:46.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.110+0000 7fe5927fc700 1 -- 172.21.15.57:0/3050419748 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe598022070 con 0x7fe5a015a4d0 2023-11-28T12:01:46.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.111+0000 7fe5a5c57700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe5840682f0 0x7fe58406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:46.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.111+0000 7fe5a5c57700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe5840682f0 0x7fe58406a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe59c007fd0 tx=0x7fe59c007dc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:46.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.115+0000 7fe5927fc700 1 -- 172.21.15.57:0/3050419748 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe598054830 con 0x7fe5a015a4d0 2023-11-28T12:01:46.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.364+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/3050419748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe5a00b1620 con 0x7fe5a015a4d0 2023-11-28T12:01:46.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.943+0000 7fe5927fc700 1 -- 172.21.15.57:0/3050419748 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe5a00b1620 con 0x7fe5a015a4d0 2023-11-28T12:01:46.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.946+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe5840682f0 msgr2=0x7fe58406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:46.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.946+0000 7fe5a7ebb700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe5840682f0 0x7fe58406a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe59c007fd0 tx=0x7fe59c007dc0 comp rx=0 tx=0).stop 2023-11-28T12:01:46.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.946+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/3050419748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a015a4d0 msgr2=0x7fe5a01e5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:46.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.946+0000 7fe5a7ebb700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a015a4d0 0x7fe5a01e5880 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe598014040 tx=0x7fe598004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:46.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.947+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/3050419748 shutdown_connections 2023-11-28T12:01:46.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.947+0000 7fe5a7ebb700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe5840682f0 0x7fe58406a7a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:46.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.947+0000 7fe5a7ebb700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a015a4d0 0x7fe5a01e5880 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:46.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.947+0000 7fe5a7ebb700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a006a980 0x7fe5a0153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:46.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.947+0000 7fe5a7ebb700 1 --2- 172.21.15.57:0/3050419748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00cdd50 0x7fe5a01470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:46.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.947+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/3050419748 >> 172.21.15.57:0/3050419748 conn(0x7fe5a006c010 msgr2=0x7fe5a0154db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:46.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.947+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/3050419748 shutdown_connections 2023-11-28T12:01:46.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:46.947+0000 7fe5a7ebb700 1 -- 172.21.15.57:0/3050419748 wait complete. 2023-11-28T12:01:46.963 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T12:01:46.963 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:01:47.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.101+0000 7fd72833f700 1 Processor -- start 2023-11-28T12:01:47.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.101+0000 7fd72833f700 1 -- start start 2023-11-28T12:01:47.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.102+0000 7fd72833f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd7200ce040 0x7fd72015d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.102+0000 7fd72833f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd720142db0 0x7fd72013ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.102+0000 7fd72833f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72013fc90 con 0x7fd7200b1770 2023-11-28T12:01:47.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.102+0000 7fd72833f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd72013f6d0 con 0x7fd7200ce040 2023-11-28T12:01:47.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.102+0000 7fd72833f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72013f9b0 con 0x7fd720142db0 2023-11-28T12:01:47.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.102+0000 7fd7268dc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd7200b1770 0x7fd72013cd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36904/0 (socket says 172.21.15.57:36904) 2023-11-28T12:01:47.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.102+0000 7fd7268dc700 1 -- 172.21.15.57:0/2806076612 learned_addr learned my addr 172.21.15.57:0/2806076612 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:47.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.103+0000 7fd7258da700 1 --2- 172.21.15.57:0/2806076612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd720142db0 0x7fd72013ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.103+0000 7fd7260db700 1 --2- 172.21.15.57:0/2806076612 >> v2:172.21.15.181:3300/0 conn(0x7fd7200ce040 0x7fd72015d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.103+0000 7fd7250d9700 1 -- 172.21.15.57:0/2806076612 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178011502 0 0) 0x7fd72013fc90 con 0x7fd7200b1770 2023-11-28T12:01:47.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.103+0000 7fd7250d9700 1 -- 172.21.15.57:0/2806076612 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd70c0035b0 con 0x7fd7200b1770 2023-11-28T12:01:47.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.103+0000 7fd7258da700 1 -- 172.21.15.57:0/2806076612 >> v2:172.21.15.181:3300/0 conn(0x7fd7200ce040 msgr2=0x7fd72015d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:47.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.103+0000 7fd7258da700 1 --2- 172.21.15.57:0/2806076612 >> v2:172.21.15.181:3300/0 conn(0x7fd7200ce040 0x7fd72015d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.103+0000 7fd7258da700 1 -- 172.21.15.57:0/2806076612 >> v1:172.21.15.57:6789/0 conn(0x7fd7200b1770 legacy=0x7fd72013cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.103+0000 7fd7258da700 1 -- 172.21.15.57:0/2806076612 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72015d100 con 0x7fd720142db0 2023-11-28T12:01:47.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.103+0000 7fd7260db700 1 --2- 172.21.15.57:0/2806076612 >> v2:172.21.15.181:3300/0 conn(0x7fd7200ce040 0x7fd72015d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:47.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.103+0000 7fd7258da700 1 --2- 172.21.15.57:0/2806076612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd720142db0 0x7fd72013ed70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd714009840 tx=0x7fd714009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=689c2360233e0d4c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:47.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.103+0000 7fd7250d9700 1 -- 172.21.15.57:0/2806076612 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714011830 con 0x7fd720142db0 2023-11-28T12:01:47.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.103+0000 7fd7250d9700 1 -- 172.21.15.57:0/2806076612 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd714011990 con 0x7fd720142db0 2023-11-28T12:01:47.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.103+0000 7fd7250d9700 1 -- 172.21.15.57:0/2806076612 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714011d00 con 0x7fd720142db0 2023-11-28T12:01:47.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.104+0000 7fd72833f700 1 -- 172.21.15.57:0/2806076612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd720142db0 msgr2=0x7fd72013ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.104+0000 7fd72833f700 1 --2- 172.21.15.57:0/2806076612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd720142db0 0x7fd72013ed70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd714009840 tx=0x7fd714009b50 comp rx=0 tx=0).stop 2023-11-28T12:01:47.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.104+0000 7fd72833f700 1 -- 172.21.15.57:0/2806076612 shutdown_connections 2023-11-28T12:01:47.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.104+0000 7fd72833f700 1 --2- 172.21.15.57:0/2806076612 >> v2:172.21.15.181:3300/0 conn(0x7fd7200ce040 0x7fd72015d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.104+0000 7fd72833f700 1 --2- 172.21.15.57:0/2806076612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd720142db0 0x7fd72013ed70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.104+0000 7fd72833f700 1 -- 172.21.15.57:0/2806076612 >> 172.21.15.57:0/2806076612 conn(0x7fd72006c010 msgr2=0x7fd7200cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:47.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.104+0000 7fd72833f700 1 -- 172.21.15.57:0/2806076612 shutdown_connections 2023-11-28T12:01:47.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.104+0000 7fd72833f700 1 -- 172.21.15.57:0/2806076612 wait complete. 2023-11-28T12:01:47.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.105+0000 7fd72833f700 1 Processor -- start 2023-11-28T12:01:47.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.105+0000 7fd72833f700 1 -- start start 2023-11-28T12:01:47.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.106+0000 7fd72833f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7200b1770 0x7fd7200b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.106+0000 7fd72833f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7200ce040 0x7fd720156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.106+0000 7fd72833f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd720142db0 0x7fd720156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.106+0000 7fd72833f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200b3a80 con 0x7fd720142db0 2023-11-28T12:01:47.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.106+0000 7fd72833f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200b37a0 con 0x7fd7200b1770 2023-11-28T12:01:47.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.106+0000 7fd72833f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd720140110 con 0x7fd7200ce040 2023-11-28T12:01:47.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.106+0000 7fd7268dc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd720142db0 0x7fd720156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.106+0000 7fd7268dc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd720142db0 0x7fd720156cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47734/0 (socket says 172.21.15.57:47734) 2023-11-28T12:01:47.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.106+0000 7fd7268dc700 1 -- 172.21.15.57:0/531268724 learned_addr learned my addr 172.21.15.57:0/531268724 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:47.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd7258da700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7200ce040 0x7fd720156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd7260db700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7200b1770 0x7fd7200b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd7268dc700 1 -- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7200ce040 msgr2=0x7fd720156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd7268dc700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7200ce040 0x7fd720156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd7268dc700 1 -- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7200b1770 msgr2=0x7fd7200b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd7268dc700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7200b1770 0x7fd7200b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd7268dc700 1 -- 172.21.15.57:0/531268724 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c0094a0 con 0x7fd720142db0 2023-11-28T12:01:47.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd7258da700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7200ce040 0x7fd720156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:47.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd7268dc700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd720142db0 0x7fd720156cd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd71800caa0 tx=0x7fd71800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:47.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd712ffd700 1 -- 172.21.15.57:0/531268724 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71800aa80 con 0x7fd720142db0 2023-11-28T12:01:47.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd712ffd700 1 -- 172.21.15.57:0/531268724 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71800abe0 con 0x7fd720142db0 2023-11-28T12:01:47.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd72833f700 1 -- 172.21.15.57:0/531268724 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7140094a0 con 0x7fd720142db0 2023-11-28T12:01:47.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd712ffd700 1 -- 172.21.15.57:0/531268724 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718012400 con 0x7fd720142db0 2023-11-28T12:01:47.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.107+0000 7fd72833f700 1 -- 172.21.15.57:0/531268724 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7200ce410 con 0x7fd720142db0 2023-11-28T12:01:47.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.108+0000 7fd72833f700 1 -- 172.21.15.57:0/531268724 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72015d3e0 con 0x7fd720142db0 2023-11-28T12:01:47.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.109+0000 7fd712ffd700 1 -- 172.21.15.57:0/531268724 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd718019030 con 0x7fd720142db0 2023-11-28T12:01:47.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.109+0000 7fd712ffd700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd704068320 0x7fd70406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.109+0000 7fd712ffd700 1 -- 172.21.15.57:0/531268724 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd718085850 con 0x7fd720142db0 2023-11-28T12:01:47.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.109+0000 7fd7260db700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd704068320 0x7fd70406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.110+0000 7fd7260db700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd704068320 0x7fd70406a7d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009d00 tx=0x7fd71c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:47.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.113+0000 7fd712ffd700 1 -- 172.21.15.57:0/531268724 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd7180520b0 con 0x7fd720142db0 2023-11-28T12:01:47.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.359+0000 7fd72833f700 1 -- 172.21.15.57:0/531268724 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7200ce5a0 con 0x7fd720142db0 2023-11-28T12:01:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.360+0000 7fd712ffd700 1 -- 172.21.15.57:0/531268724 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4642 (secure 0 0 0) 0x7fd7180556d0 con 0x7fd720142db0 2023-11-28T12:01:47.361 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:47.361 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/890913444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":890913444},{"type":"v1","addr":"172.21.15.181:6836","nonce":890913444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/847658150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":847658150},{"type":"v1","addr":"172.21.15.181:6837","nonce":847658150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1379013929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1379013929},{"type":"v1","addr":"172.21.15.57:6835","nonce":1379013929}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:01:43.919415+0000","modified":"2023-11-28T12:01:46.937696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6833/3776085595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3776085595},{"type":"v1","addr":"172.21.15.57:6833","nonce":3776085595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T12:01:47.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.362+0000 7fd72833f700 1 -- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd704068320 msgr2=0x7fd70406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.362+0000 7fd72833f700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd704068320 0x7fd70406a7d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009d00 tx=0x7fd71c008040 comp rx=0 tx=0).stop 2023-11-28T12:01:47.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.362+0000 7fd72833f700 1 -- 172.21.15.57:0/531268724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd720142db0 msgr2=0x7fd720156cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.362+0000 7fd72833f700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd720142db0 0x7fd720156cd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd71800caa0 tx=0x7fd71800ce60 comp rx=0 tx=0).stop 2023-11-28T12:01:47.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.362+0000 7fd72833f700 1 -- 172.21.15.57:0/531268724 shutdown_connections 2023-11-28T12:01:47.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.362+0000 7fd72833f700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd704068320 0x7fd70406a7d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.362+0000 7fd72833f700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd720142db0 0x7fd720156cd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.362+0000 7fd72833f700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7200b1770 0x7fd7200b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.362+0000 7fd72833f700 1 --2- 172.21.15.57:0/531268724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7200ce040 0x7fd720156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.362+0000 7fd72833f700 1 -- 172.21.15.57:0/531268724 >> 172.21.15.57:0/531268724 conn(0x7fd72006c010 msgr2=0x7fd720071530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:47.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.362+0000 7fd72833f700 1 -- 172.21.15.57:0/531268724 shutdown_connections 2023-11-28T12:01:47.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.362+0000 7fd72833f700 1 -- 172.21.15.57:0/531268724 wait complete. 2023-11-28T12:01:47.364 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 8 2023-11-28T12:01:47.377 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:01:47.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.511+0000 7f7a89fba700 1 Processor -- start 2023-11-28T12:01:47.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.511+0000 7f7a89fba700 1 -- start start 2023-11-28T12:01:47.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.511+0000 7f7a89fba700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7a840ce040 0x7f7a8415d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.511+0000 7f7a89fba700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7a84142db0 0x7f7a8413ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.511+0000 7f7a89fba700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8413fc90 con 0x7f7a840b1770 2023-11-28T12:01:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.511+0000 7f7a89fba700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8413f6d0 con 0x7f7a840ce040 2023-11-28T12:01:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.511+0000 7f7a89fba700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8413f9b0 con 0x7f7a84142db0 2023-11-28T12:01:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.512+0000 7f7a837fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7a840ce040 0x7f7a8415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.512+0000 7f7a837fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7a840ce040 0x7f7a8415d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47736/0 (socket says 172.21.15.57:47736) 2023-11-28T12:01:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.512+0000 7f7a837fe700 1 -- 172.21.15.57:0/2517437683 learned_addr learned my addr 172.21.15.57:0/2517437683 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.512+0000 7f7a82ffd700 1 --2- 172.21.15.57:0/2517437683 >> v2:172.21.15.181:3300/0 conn(0x7f7a84142db0 0x7f7a8413ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.512+0000 7f7a837fe700 1 -- 172.21.15.57:0/2517437683 >> v2:172.21.15.181:3300/0 conn(0x7f7a84142db0 msgr2=0x7f7a8413ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.512+0000 7f7a837fe700 1 --2- 172.21.15.57:0/2517437683 >> v2:172.21.15.181:3300/0 conn(0x7f7a84142db0 0x7f7a8413ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.512+0000 7f7a837fe700 1 -- 172.21.15.57:0/2517437683 >> v1:172.21.15.181:6789/0 conn(0x7f7a840b1770 legacy=0x7f7a840b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:47.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.512+0000 7f7a837fe700 1 -- 172.21.15.57:0/2517437683 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8415d100 con 0x7f7a840ce040 2023-11-28T12:01:47.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.512+0000 7f7a837fe700 1 --2- 172.21.15.57:0/2517437683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a840ce040 0x7f7a8415d900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a840cfc20 tx=0x7f7a74009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba484452d3f3131 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:47.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.512+0000 7f7a827fc700 1 -- 172.21.15.57:0/2517437683 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74004230 con 0x7f7a840ce040 2023-11-28T12:01:47.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.512+0000 7f7a827fc700 1 -- 172.21.15.57:0/2517437683 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a74004390 con 0x7f7a840ce040 2023-11-28T12:01:47.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.512+0000 7f7a827fc700 1 -- 172.21.15.57:0/2517437683 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a740036a0 con 0x7f7a840ce040 2023-11-28T12:01:47.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.513+0000 7f7a89fba700 1 -- 172.21.15.57:0/2517437683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a840ce040 msgr2=0x7f7a8415d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.513+0000 7f7a89fba700 1 --2- 172.21.15.57:0/2517437683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a840ce040 0x7f7a8415d900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a840cfc20 tx=0x7f7a74009bd0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.513+0000 7f7a89fba700 1 -- 172.21.15.57:0/2517437683 shutdown_connections 2023-11-28T12:01:47.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.513+0000 7f7a89fba700 1 --2- 172.21.15.57:0/2517437683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a840ce040 0x7f7a8415d900 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.513+0000 7f7a89fba700 1 --2- 172.21.15.57:0/2517437683 >> v2:172.21.15.181:3300/0 conn(0x7f7a84142db0 0x7f7a8413ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.513+0000 7f7a89fba700 1 -- 172.21.15.57:0/2517437683 >> 172.21.15.57:0/2517437683 conn(0x7f7a8406c010 msgr2=0x7f7a840cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:47.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.513+0000 7f7a89fba700 1 -- 172.21.15.57:0/2517437683 shutdown_connections 2023-11-28T12:01:47.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.513+0000 7f7a89fba700 1 -- 172.21.15.57:0/2517437683 wait complete. 2023-11-28T12:01:47.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.514+0000 7f7a89fba700 1 Processor -- start 2023-11-28T12:01:47.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.514+0000 7f7a89fba700 1 -- start start 2023-11-28T12:01:47.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.515+0000 7f7a89fba700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a840b1770 0x7f7a84163370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.515+0000 7f7a89fba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a840ce040 0x7f7a841638b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.515+0000 7f7a89fba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a84142db0 0x7f7a841622e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.515+0000 7f7a89fba700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a841674d0 con 0x7f7a840b1770 2023-11-28T12:01:47.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.515+0000 7f7a89fba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8415a4c0 con 0x7f7a840ce040 2023-11-28T12:01:47.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.515+0000 7f7a89fba700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a841671f0 con 0x7f7a84142db0 2023-11-28T12:01:47.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.515+0000 7f7a837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a840b1770 0x7f7a84163370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.515+0000 7f7a837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a840b1770 0x7f7a84163370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47748/0 (socket says 172.21.15.57:47748) 2023-11-28T12:01:47.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.515+0000 7f7a837fe700 1 -- 172.21.15.57:0/3273388598 learned_addr learned my addr 172.21.15.57:0/3273388598 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:47.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.515+0000 7f7a82ffd700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a840ce040 0x7f7a841638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.515+0000 7f7a83fff700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a84142db0 0x7f7a841622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.516+0000 7f7a837fe700 1 -- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a84142db0 msgr2=0x7f7a841622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.516+0000 7f7a837fe700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a84142db0 0x7f7a841622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.516+0000 7f7a837fe700 1 -- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a840ce040 msgr2=0x7f7a841638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.516+0000 7f7a837fe700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a840ce040 0x7f7a841638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.516+0000 7f7a837fe700 1 -- 172.21.15.57:0/3273388598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a74009550 con 0x7f7a840b1770 2023-11-28T12:01:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.516+0000 7f7a837fe700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a840b1770 0x7f7a84163370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009130 tx=0x7f7a74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.516+0000 7f7a6bfff700 1 -- 172.21.15.57:0/3273388598 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74003b90 con 0x7f7a840b1770 2023-11-28T12:01:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.516+0000 7f7a6bfff700 1 -- 172.21.15.57:0/3273388598 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7401f070 con 0x7f7a840b1770 2023-11-28T12:01:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.516+0000 7f7a89fba700 1 -- 172.21.15.57:0/3273388598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a840ce410 con 0x7f7a840b1770 2023-11-28T12:01:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.516+0000 7f7a6bfff700 1 -- 172.21.15.57:0/3273388598 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74011630 con 0x7f7a840b1770 2023-11-28T12:01:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.516+0000 7f7a89fba700 1 -- 172.21.15.57:0/3273388598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8415d3e0 con 0x7f7a840b1770 2023-11-28T12:01:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.517+0000 7f7a89fba700 1 -- 172.21.15.57:0/3273388598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a840ce5a0 con 0x7f7a840b1770 2023-11-28T12:01:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.518+0000 7f7a6bfff700 1 -- 172.21.15.57:0/3273388598 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a74017020 con 0x7f7a840b1770 2023-11-28T12:01:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.518+0000 7f7a6bfff700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7a70068320 0x7f7a7006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.518+0000 7f7a6bfff700 1 -- 172.21.15.57:0/3273388598 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7a740872b0 con 0x7f7a840b1770 2023-11-28T12:01:47.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.518+0000 7f7a82ffd700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7a70068320 0x7f7a7006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.519+0000 7f7a82ffd700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7a70068320 0x7f7a7006a7d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7a84141d50 tx=0x7f7a78008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:47.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.522+0000 7f7a6bfff700 1 -- 172.21.15.57:0/3273388598 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7a74053a90 con 0x7f7a840b1770 2023-11-28T12:01:47.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.721+0000 7f7a89fba700 1 -- 172.21.15.57:0/3273388598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a840743f0 con 0x7f7a840b1770 2023-11-28T12:01:47.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.723+0000 7f7a6bfff700 1 -- 172.21.15.57:0/3273388598 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7f7a740570b0 con 0x7f7a840b1770 2023-11-28T12:01:47.724 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:47.724 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:01:47.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.725+0000 7f7a89fba700 1 -- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7a70068320 msgr2=0x7f7a7006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.725+0000 7f7a89fba700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7a70068320 0x7f7a7006a7d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7a84141d50 tx=0x7f7a78008040 comp rx=0 tx=0).stop 2023-11-28T12:01:47.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.725+0000 7f7a89fba700 1 -- 172.21.15.57:0/3273388598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a840b1770 msgr2=0x7f7a84163370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.725+0000 7f7a89fba700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a840b1770 0x7f7a84163370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009130 tx=0x7f7a74004750 comp rx=0 tx=0).stop 2023-11-28T12:01:47.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.725+0000 7f7a89fba700 1 -- 172.21.15.57:0/3273388598 shutdown_connections 2023-11-28T12:01:47.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.725+0000 7f7a89fba700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7a70068320 0x7f7a7006a7d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.725+0000 7f7a89fba700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a84142db0 0x7f7a841622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.725+0000 7f7a89fba700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a840b1770 0x7f7a84163370 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.725+0000 7f7a89fba700 1 --2- 172.21.15.57:0/3273388598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a840ce040 0x7f7a841638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.725+0000 7f7a89fba700 1 -- 172.21.15.57:0/3273388598 >> 172.21.15.57:0/3273388598 conn(0x7f7a8406c010 msgr2=0x7f7a840b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:47.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.725+0000 7f7a89fba700 1 -- 172.21.15.57:0/3273388598 shutdown_connections 2023-11-28T12:01:47.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.725+0000 7f7a89fba700 1 -- 172.21.15.57:0/3273388598 wait complete. 2023-11-28T12:01:47.742 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T12:01:47.742 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:01:47.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.876+0000 7ffb64638700 1 Processor -- start 2023-11-28T12:01:47.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.876+0000 7ffb64638700 1 -- start start 2023-11-28T12:01:47.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.877+0000 7ffb64638700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ffb5c0ce040 0x7ffb5c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.877+0000 7ffb64638700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb5c0b1770 0x7ffb5c0b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.877+0000 7ffb64638700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb5c13fc90 con 0x7ffb5c142db0 2023-11-28T12:01:47.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.877+0000 7ffb64638700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5c13f480 con 0x7ffb5c0ce040 2023-11-28T12:01:47.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.877+0000 7ffb64638700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5c0b20d0 con 0x7ffb5c0b1770 2023-11-28T12:01:47.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.877+0000 7ffb623d4700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ffb5c0ce040 0x7ffb5c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.877+0000 7ffb61bd3700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ffb5c142db0 0x7ffb5c13ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36912/0 (socket says 172.21.15.57:36912) 2023-11-28T12:01:47.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.877+0000 7ffb61bd3700 1 -- 172.21.15.57:0/704175204 learned_addr learned my addr 172.21.15.57:0/704175204 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:47.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.877+0000 7ffb62bd5700 1 --2- 172.21.15.57:0/704175204 >> v2:172.21.15.181:3300/0 conn(0x7ffb5c0b1770 0x7ffb5c0b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.878+0000 7ffb623d4700 1 -- 172.21.15.57:0/704175204 >> v2:172.21.15.181:3300/0 conn(0x7ffb5c0b1770 msgr2=0x7ffb5c0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:47.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.878+0000 7ffb623d4700 1 --2- 172.21.15.57:0/704175204 >> v2:172.21.15.181:3300/0 conn(0x7ffb5c0b1770 0x7ffb5c0b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.878+0000 7ffb623d4700 1 -- 172.21.15.57:0/704175204 >> v1:172.21.15.57:6789/0 conn(0x7ffb5c142db0 legacy=0x7ffb5c13ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.878+0000 7ffb613d2700 1 -- 172.21.15.57:0/704175204 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437673169 0 0) 0x7ffb5c13fc90 con 0x7ffb5c142db0 2023-11-28T12:01:47.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.878+0000 7ffb623d4700 1 -- 172.21.15.57:0/704175204 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c13f6d0 con 0x7ffb5c0ce040 2023-11-28T12:01:47.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.878+0000 7ffb623d4700 1 --2- 172.21.15.57:0/704175204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb5c0ce040 0x7ffb5c15d900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c142cd0 tx=0x7ffb58009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22347596bb8e4d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:47.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.878+0000 7ffb613d2700 1 -- 172.21.15.57:0/704175204 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58011830 con 0x7ffb5c0ce040 2023-11-28T12:01:47.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.878+0000 7ffb613d2700 1 -- 172.21.15.57:0/704175204 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb58011990 con 0x7ffb5c0ce040 2023-11-28T12:01:47.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.878+0000 7ffb613d2700 1 -- 172.21.15.57:0/704175204 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58011d20 con 0x7ffb5c0ce040 2023-11-28T12:01:47.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.879+0000 7ffb64638700 1 -- 172.21.15.57:0/704175204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb5c0ce040 msgr2=0x7ffb5c15d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.879+0000 7ffb64638700 1 --2- 172.21.15.57:0/704175204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb5c0ce040 0x7ffb5c15d900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c142cd0 tx=0x7ffb58009b20 comp rx=0 tx=0).stop 2023-11-28T12:01:47.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.879+0000 7ffb64638700 1 -- 172.21.15.57:0/704175204 shutdown_connections 2023-11-28T12:01:47.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.879+0000 7ffb64638700 1 --2- 172.21.15.57:0/704175204 >> v2:172.21.15.181:3300/0 conn(0x7ffb5c0b1770 0x7ffb5c0b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.879+0000 7ffb64638700 1 --2- 172.21.15.57:0/704175204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb5c0ce040 0x7ffb5c15d900 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.879+0000 7ffb64638700 1 -- 172.21.15.57:0/704175204 >> 172.21.15.57:0/704175204 conn(0x7ffb5c06c010 msgr2=0x7ffb5c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:47.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.879+0000 7ffb64638700 1 -- 172.21.15.57:0/704175204 shutdown_connections 2023-11-28T12:01:47.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.879+0000 7ffb64638700 1 -- 172.21.15.57:0/704175204 wait complete. 2023-11-28T12:01:47.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.880+0000 7ffb64638700 1 Processor -- start 2023-11-28T12:01:47.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.880+0000 7ffb64638700 1 -- start start 2023-11-28T12:01:47.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb64638700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb5c0b1770 0x7ffb5c1f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb64638700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb5c0ce040 0x7ffb5c0c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb64638700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb5c142db0 0x7ffb5c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb64638700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c0c6050 con 0x7ffb5c0b1770 2023-11-28T12:01:47.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb64638700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c153d70 con 0x7ffb5c0ce040 2023-11-28T12:01:47.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb64638700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c15b4b0 con 0x7ffb5c142db0 2023-11-28T12:01:47.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb623d4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb5c0b1770 0x7ffb5c1f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb623d4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb5c0b1770 0x7ffb5c1f3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47772/0 (socket says 172.21.15.57:47772) 2023-11-28T12:01:47.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb623d4700 1 -- 172.21.15.57:0/2361642221 learned_addr learned my addr 172.21.15.57:0/2361642221 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:47.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb62bd5700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb5c142db0 0x7ffb5c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb61bd3700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb5c0ce040 0x7ffb5c0c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb623d4700 1 -- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb5c142db0 msgr2=0x7ffb5c0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb623d4700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb5c142db0 0x7ffb5c0c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb623d4700 1 -- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb5c0ce040 msgr2=0x7ffb5c0c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:47.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb623d4700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb5c0ce040 0x7ffb5c0c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:47.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb623d4700 1 -- 172.21.15.57:0/2361642221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb580094a0 con 0x7ffb5c0b1770 2023-11-28T12:01:47.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb61bd3700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb5c0ce040 0x7ffb5c0c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:47.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.881+0000 7ffb62bd5700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb5c142db0 0x7ffb5c0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:47.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.882+0000 7ffb623d4700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb5c0b1770 0x7ffb5c1f3b00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffb580090b0 tx=0x7ffb58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:47.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.882+0000 7ffb52ffd700 1 -- 172.21.15.57:0/2361642221 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb580038a0 con 0x7ffb5c0b1770 2023-11-28T12:01:47.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.882+0000 7ffb52ffd700 1 -- 172.21.15.57:0/2361642221 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5801f070 con 0x7ffb5c0b1770 2023-11-28T12:01:47.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.882+0000 7ffb64638700 1 -- 172.21.15.57:0/2361642221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5c15d100 con 0x7ffb5c0b1770 2023-11-28T12:01:47.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.882+0000 7ffb52ffd700 1 -- 172.21.15.57:0/2361642221 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58003c20 con 0x7ffb5c0b1770 2023-11-28T12:01:47.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.882+0000 7ffb64638700 1 -- 172.21.15.57:0/2361642221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5c13f9b0 con 0x7ffb5c0b1770 2023-11-28T12:01:47.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.883+0000 7ffb64638700 1 -- 172.21.15.57:0/2361642221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5c15d3e0 con 0x7ffb5c0b1770 2023-11-28T12:01:47.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.883+0000 7ffb52ffd700 1 -- 172.21.15.57:0/2361642221 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffb5800f050 con 0x7ffb5c0b1770 2023-11-28T12:01:47.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.884+0000 7ffb52ffd700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ffb400682f0 0x7ffb4006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:47.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.884+0000 7ffb52ffd700 1 -- 172.21.15.57:0/2361642221 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb58021070 con 0x7ffb5c0b1770 2023-11-28T12:01:47.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.884+0000 7ffb61bd3700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ffb400682f0 0x7ffb4006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:47.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.885+0000 7ffb61bd3700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ffb400682f0 0x7ffb4006a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c1e7840 tx=0x7ffb4c007ca0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:47.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:47.888+0000 7ffb52ffd700 1 -- 172.21.15.57:0/2361642221 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffb58053ed0 con 0x7ffb5c0b1770 2023-11-28T12:01:48.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.083+0000 7ffb64638700 1 -- 172.21.15.57:0/2361642221 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb5c153bd0 con 0x7ffb400682f0 2023-11-28T12:01:48.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.092+0000 7ffb52ffd700 1 -- 172.21.15.57:0/2361642221 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286937 (secure 0 0 0) 0x7ffb5c153bd0 con 0x7ffb400682f0 2023-11-28T12:01:48.097 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:48.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.096+0000 7ffb64638700 1 -- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ffb400682f0 msgr2=0x7ffb4006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:48.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.096+0000 7ffb64638700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ffb400682f0 0x7ffb4006a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c1e7840 tx=0x7ffb4c007ca0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.096+0000 7ffb64638700 1 -- 172.21.15.57:0/2361642221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb5c0b1770 msgr2=0x7ffb5c1f3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:48.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.096+0000 7ffb64638700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb5c0b1770 0x7ffb5c1f3b00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffb580090b0 tx=0x7ffb58004750 comp rx=0 tx=0).stop 2023-11-28T12:01:48.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.096+0000 7ffb64638700 1 -- 172.21.15.57:0/2361642221 shutdown_connections 2023-11-28T12:01:48.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.096+0000 7ffb64638700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ffb400682f0 0x7ffb4006a7a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.096+0000 7ffb64638700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb5c142db0 0x7ffb5c0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.096+0000 7ffb64638700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffb5c0b1770 0x7ffb5c1f3b00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.096+0000 7ffb64638700 1 --2- 172.21.15.57:0/2361642221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb5c0ce040 0x7ffb5c0c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.096+0000 7ffb64638700 1 -- 172.21.15.57:0/2361642221 >> 172.21.15.57:0/2361642221 conn(0x7ffb5c06c010 msgr2=0x7ffb5c070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:48.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.096+0000 7ffb64638700 1 -- 172.21.15.57:0/2361642221 shutdown_connections 2023-11-28T12:01:48.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.096+0000 7ffb64638700 1 -- 172.21.15.57:0/2361642221 wait complete. 2023-11-28T12:01:48.101 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:01:48.111 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-28T12:01:47.658826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":65936,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":754908784,"statfs":{"total":773094113280,"available":773026594816,"internally_reserved":0,"allocated":2965504,"data_stored":1807770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.181731"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542856+0000","last_change":"2023-11-28T12:01:43.953143+0000","last_active":"2023-11-28T12:01:46.542856+0000","last_peered":"2023-11-28T12:01:46.542856+0000","last_clean":"2023-11-28T12:01:46.542856+0000","last_became_active":"2023-11-28T12:01:43.952673+0000","last_became_peered":"2023-11-28T12:01:43.952673+0000","last_unstale":"2023-11-28T12:01:46.542856+0000","last_undegraded":"2023-11-28T12:01:46.542856+0000","last_fullsized":"2023-11-28T12:01:46.542856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:18.949383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545399+0000","last_change":"2023-11-28T12:01:43.947886+0000","last_active":"2023-11-28T12:01:46.545399+0000","last_peered":"2023-11-28T12:01:46.545399+0000","last_clean":"2023-11-28T12:01:46.545399+0000","last_became_active":"2023-11-28T12:01:43.947468+0000","last_became_peered":"2023-11-28T12:01:43.947468+0000","last_unstale":"2023-11-28T12:01:46.545399+0000","last_undegraded":"2023-11-28T12:01:46.545399+0000","last_fullsized":"2023-11-28T12:01:46.545399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:41.736417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938403+0000","last_change":"2023-11-28T12:01:43.941069+0000","last_active":"2023-11-28T12:01:44.938403+0000","last_peered":"2023-11-28T12:01:44.938403+0000","last_clean":"2023-11-28T12:01:44.938403+0000","last_became_active":"2023-11-28T12:01:43.940366+0000","last_became_peered":"2023-11-28T12:01:43.940366+0000","last_unstale":"2023-11-28T12:01:44.938403+0000","last_undegraded":"2023-11-28T12:01:44.938403+0000","last_fullsized":"2023-11-28T12:01:44.938403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:40.988886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937690+0000","last_change":"2023-11-28T12:01:43.947292+0000","last_active":"2023-11-28T12:01:44.937690+0000","last_peered":"2023-11-28T12:01:44.937690+0000","last_clean":"2023-11-28T12:01:44.937690+0000","last_became_active":"2023-11-28T12:01:43.943027+0000","last_became_peered":"2023-11-28T12:01:43.943027+0000","last_unstale":"2023-11-28T12:01:44.937690+0000","last_undegraded":"2023-11-28T12:01:44.937690+0000","last_fullsized":"2023-11-28T12:01:44.937690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:19.706888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497843+0000","last_change":"2023-11-28T12:01:43.954299+0000","last_active":"2023-11-28T12:01:46.497843+0000","last_peered":"2023-11-28T12:01:46.497843+0000","last_clean":"2023-11-28T12:01:46.497843+0000","last_became_active":"2023-11-28T12:01:43.953967+0000","last_became_peered":"2023-11-28T12:01:43.953967+0000","last_unstale":"2023-11-28T12:01:46.497843+0000","last_undegraded":"2023-11-28T12:01:46.497843+0000","last_fullsized":"2023-11-28T12:01:46.497843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:35.755196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937402+0000","last_change":"2023-11-28T12:01:43.938748+0000","last_active":"2023-11-28T12:01:44.937402+0000","last_peered":"2023-11-28T12:01:44.937402+0000","last_clean":"2023-11-28T12:01:44.937402+0000","last_became_active":"2023-11-28T12:01:43.938430+0000","last_became_peered":"2023-11-28T12:01:43.938430+0000","last_unstale":"2023-11-28T12:01:44.937402+0000","last_undegraded":"2023-11-28T12:01:44.937402+0000","last_fullsized":"2023-11-28T12:01:44.937402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:49.922116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853429+0000","last_change":"2023-11-28T12:01:43.950997+0000","last_active":"2023-11-28T12:01:45.853429+0000","last_peered":"2023-11-28T12:01:45.853429+0000","last_clean":"2023-11-28T12:01:45.853429+0000","last_became_active":"2023-11-28T12:01:43.950532+0000","last_became_peered":"2023-11-28T12:01:43.950532+0000","last_unstale":"2023-11-28T12:01:45.853429+0000","last_undegraded":"2023-11-28T12:01:45.853429+0000","last_fullsized":"2023-11-28T12:01:45.853429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545106+0000","last_change":"2023-11-28T12:01:43.945419+0000","last_active":"2023-11-28T12:01:46.545106+0000","last_peered":"2023-11-28T12:01:46.545106+0000","last_clean":"2023-11-28T12:01:46.545106+0000","last_became_active":"2023-11-28T12:01:43.942913+0000","last_became_peered":"2023-11-28T12:01:43.942913+0000","last_unstale":"2023-11-28T12:01:46.545106+0000","last_undegraded":"2023-11-28T12:01:46.545106+0000","last_fullsized":"2023-11-28T12:01:46.545106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497879+0000","last_change":"2023-11-28T12:01:43.954900+0000","last_active":"2023-11-28T12:01:46.497879+0000","last_peered":"2023-11-28T12:01:46.497879+0000","last_clean":"2023-11-28T12:01:46.497879+0000","last_became_active":"2023-11-28T12:01:43.954524+0000","last_became_peered":"2023-11-28T12:01:43.954524+0000","last_unstale":"2023-11-28T12:01:46.497879+0000","last_undegraded":"2023-11-28T12:01:46.497879+0000","last_fullsized":"2023-11-28T12:01:46.497879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:10.298310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545527+0000","last_change":"2023-11-28T12:01:43.944809+0000","last_active":"2023-11-28T12:01:46.545527+0000","last_peered":"2023-11-28T12:01:46.545527+0000","last_clean":"2023-11-28T12:01:46.545527+0000","last_became_active":"2023-11-28T12:01:43.943120+0000","last_became_peered":"2023-11-28T12:01:43.943120+0000","last_unstale":"2023-11-28T12:01:46.545527+0000","last_undegraded":"2023-11-28T12:01:46.545527+0000","last_fullsized":"2023-11-28T12:01:46.545527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938327+0000","last_change":"2023-11-28T12:01:43.941446+0000","last_active":"2023-11-28T12:01:44.938327+0000","last_peered":"2023-11-28T12:01:44.938327+0000","last_clean":"2023-11-28T12:01:44.938327+0000","last_became_active":"2023-11-28T12:01:43.940640+0000","last_became_peered":"2023-11-28T12:01:43.940640+0000","last_unstale":"2023-11-28T12:01:44.938327+0000","last_undegraded":"2023-11-28T12:01:44.938327+0000","last_fullsized":"2023-11-28T12:01:44.938327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:41.605113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542711+0000","last_change":"2023-11-28T12:01:43.946959+0000","last_active":"2023-11-28T12:01:46.542711+0000","last_peered":"2023-11-28T12:01:46.542711+0000","last_clean":"2023-11-28T12:01:46.542711+0000","last_became_active":"2023-11-28T12:01:43.946674+0000","last_became_peered":"2023-11-28T12:01:43.946674+0000","last_unstale":"2023-11-28T12:01:46.542711+0000","last_undegraded":"2023-11-28T12:01:46.542711+0000","last_fullsized":"2023-11-28T12:01:46.542711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:18.910451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542620+0000","last_change":"2023-11-28T12:01:43.952292+0000","last_active":"2023-11-28T12:01:46.542620+0000","last_peered":"2023-11-28T12:01:46.542620+0000","last_clean":"2023-11-28T12:01:46.542620+0000","last_became_active":"2023-11-28T12:01:43.951948+0000","last_became_peered":"2023-11-28T12:01:43.951948+0000","last_unstale":"2023-11-28T12:01:46.542620+0000","last_undegraded":"2023-11-28T12:01:46.542620+0000","last_fullsized":"2023-11-28T12:01:46.542620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:30.672727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545157+0000","last_change":"2023-11-28T12:01:43.945700+0000","last_active":"2023-11-28T12:01:46.545157+0000","last_peered":"2023-11-28T12:01:46.545157+0000","last_clean":"2023-11-28T12:01:46.545157+0000","last_became_active":"2023-11-28T12:01:43.943097+0000","last_became_peered":"2023-11-28T12:01:43.943097+0000","last_unstale":"2023-11-28T12:01:46.545157+0000","last_undegraded":"2023-11-28T12:01:46.545157+0000","last_fullsized":"2023-11-28T12:01:46.545157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:36.019691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853575+0000","last_change":"2023-11-28T12:01:43.951123+0000","last_active":"2023-11-28T12:01:45.853575+0000","last_peered":"2023-11-28T12:01:45.853575+0000","last_clean":"2023-11-28T12:01:45.853575+0000","last_became_active":"2023-11-28T12:01:43.950867+0000","last_became_peered":"2023-11-28T12:01:43.950867+0000","last_unstale":"2023-11-28T12:01:45.853575+0000","last_undegraded":"2023-11-28T12:01:45.853575+0000","last_fullsized":"2023-11-28T12:01:45.853575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:32.123066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937717+0000","last_change":"2023-11-28T12:01:43.942191+0000","last_active":"2023-11-28T12:01:44.937717+0000","last_peered":"2023-11-28T12:01:44.937717+0000","last_clean":"2023-11-28T12:01:44.937717+0000","last_became_active":"2023-11-28T12:01:43.941576+0000","last_became_peered":"2023-11-28T12:01:43.941576+0000","last_unstale":"2023-11-28T12:01:44.937717+0000","last_undegraded":"2023-11-28T12:01:44.937717+0000","last_fullsized":"2023-11-28T12:01:44.937717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:43.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497887+0000","last_change":"2023-11-28T12:01:43.954787+0000","last_active":"2023-11-28T12:01:46.497887+0000","last_peered":"2023-11-28T12:01:46.497887+0000","last_clean":"2023-11-28T12:01:46.497887+0000","last_became_active":"2023-11-28T12:01:43.954463+0000","last_became_peered":"2023-11-28T12:01:43.954463+0000","last_unstale":"2023-11-28T12:01:46.497887+0000","last_undegraded":"2023-11-28T12:01:46.497887+0000","last_fullsized":"2023-11-28T12:01:46.497887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:23.540287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938451+0000","last_change":"2023-11-28T12:01:43.939593+0000","last_active":"2023-11-28T12:01:44.938451+0000","last_peered":"2023-11-28T12:01:44.938451+0000","last_clean":"2023-11-28T12:01:44.938451+0000","last_became_active":"2023-11-28T12:01:43.937190+0000","last_became_peered":"2023-11-28T12:01:43.937190+0000","last_unstale":"2023-11-28T12:01:44.938451+0000","last_undegraded":"2023-11-28T12:01:44.938451+0000","last_fullsized":"2023-11-28T12:01:44.938451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:16.230349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542842+0000","last_change":"2023-11-28T12:01:43.942910+0000","last_active":"2023-11-28T12:01:46.542842+0000","last_peered":"2023-11-28T12:01:46.542842+0000","last_clean":"2023-11-28T12:01:46.542842+0000","last_became_active":"2023-11-28T12:01:43.942609+0000","last_became_peered":"2023-11-28T12:01:43.942609+0000","last_unstale":"2023-11-28T12:01:46.542842+0000","last_undegraded":"2023-11-28T12:01:46.542842+0000","last_fullsized":"2023-11-28T12:01:46.542842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:37.541651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938215+0000","last_change":"2023-11-28T12:01:43.945798+0000","last_active":"2023-11-28T12:01:44.938215+0000","last_peered":"2023-11-28T12:01:44.938215+0000","last_clean":"2023-11-28T12:01:44.938215+0000","last_became_active":"2023-11-28T12:01:43.945473+0000","last_became_peered":"2023-11-28T12:01:43.945473+0000","last_unstale":"2023-11-28T12:01:44.938215+0000","last_undegraded":"2023-11-28T12:01:44.938215+0000","last_fullsized":"2023-11-28T12:01:44.938215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:29.033626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853662+0000","last_change":"2023-11-28T12:01:43.954303+0000","last_active":"2023-11-28T12:01:45.853662+0000","last_peered":"2023-11-28T12:01:45.853662+0000","last_clean":"2023-11-28T12:01:45.853662+0000","last_became_active":"2023-11-28T12:01:43.954076+0000","last_became_peered":"2023-11-28T12:01:43.954076+0000","last_unstale":"2023-11-28T12:01:45.853662+0000","last_undegraded":"2023-11-28T12:01:45.853662+0000","last_fullsized":"2023-11-28T12:01:45.853662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:41.736417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497578+0000","last_change":"2023-11-28T12:01:43.946172+0000","last_active":"2023-11-28T12:01:46.497578+0000","last_peered":"2023-11-28T12:01:46.497578+0000","last_clean":"2023-11-28T12:01:46.497578+0000","last_became_active":"2023-11-28T12:01:43.945881+0000","last_became_peered":"2023-11-28T12:01:43.945881+0000","last_unstale":"2023-11-28T12:01:46.497578+0000","last_undegraded":"2023-11-28T12:01:46.497578+0000","last_fullsized":"2023-11-28T12:01:46.497578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:43.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938826+0000","last_change":"2023-11-28T12:01:43.951156+0000","last_active":"2023-11-28T12:01:44.938826+0000","last_peered":"2023-11-28T12:01:44.938826+0000","last_clean":"2023-11-28T12:01:44.938826+0000","last_became_active":"2023-11-28T12:01:43.950873+0000","last_became_peered":"2023-11-28T12:01:43.950873+0000","last_unstale":"2023-11-28T12:01:44.938826+0000","last_undegraded":"2023-11-28T12:01:44.938826+0000","last_fullsized":"2023-11-28T12:01:44.938826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:32.718654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853513+0000","last_change":"2023-11-28T12:01:43.953938+0000","last_active":"2023-11-28T12:01:45.853513+0000","last_peered":"2023-11-28T12:01:45.853513+0000","last_clean":"2023-11-28T12:01:45.853513+0000","last_became_active":"2023-11-28T12:01:43.953649+0000","last_became_peered":"2023-11-28T12:01:43.953649+0000","last_unstale":"2023-11-28T12:01:45.853513+0000","last_undegraded":"2023-11-28T12:01:45.853513+0000","last_fullsized":"2023-11-28T12:01:45.853513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923145+0000","last_change":"2023-11-28T12:01:43.941856+0000","last_active":"2023-11-28T12:01:45.923145+0000","last_peered":"2023-11-28T12:01:45.923145+0000","last_clean":"2023-11-28T12:01:45.923145+0000","last_became_active":"2023-11-28T12:01:43.941555+0000","last_became_peered":"2023-11-28T12:01:43.941555+0000","last_unstale":"2023-11-28T12:01:45.923145+0000","last_undegraded":"2023-11-28T12:01:45.923145+0000","last_fullsized":"2023-11-28T12:01:45.923145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:01.126989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923247+0000","last_change":"2023-11-28T12:01:43.940084+0000","last_active":"2023-11-28T12:01:45.923247+0000","last_peered":"2023-11-28T12:01:45.923247+0000","last_clean":"2023-11-28T12:01:45.923247+0000","last_became_active":"2023-11-28T12:01:43.939856+0000","last_became_peered":"2023-11-28T12:01:43.939856+0000","last_unstale":"2023-11-28T12:01:45.923247+0000","last_undegraded":"2023-11-28T12:01:45.923247+0000","last_fullsized":"2023-11-28T12:01:45.923247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:50.516316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545353+0000","last_change":"2023-11-28T12:01:43.943624+0000","last_active":"2023-11-28T12:01:46.545353+0000","last_peered":"2023-11-28T12:01:46.545353+0000","last_clean":"2023-11-28T12:01:46.545353+0000","last_became_active":"2023-11-28T12:01:43.943313+0000","last_became_peered":"2023-11-28T12:01:43.943313+0000","last_unstale":"2023-11-28T12:01:46.545353+0000","last_undegraded":"2023-11-28T12:01:46.545353+0000","last_fullsized":"2023-11-28T12:01:46.545353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:10.298310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938040+0000","last_change":"2023-11-28T12:01:43.500890+0000","last_active":"2023-11-28T12:01:44.938040+0000","last_peered":"2023-11-28T12:01:44.938040+0000","last_clean":"2023-11-28T12:01:44.938040+0000","last_became_active":"2023-11-28T12:01:43.500621+0000","last_became_peered":"2023-11-28T12:01:43.500621+0000","last_unstale":"2023-11-28T12:01:44.938040+0000","last_undegraded":"2023-11-28T12:01:44.938040+0000","last_fullsized":"2023-11-28T12:01:44.938040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939156+0000","last_change":"2023-11-28T12:01:43.951003+0000","last_active":"2023-11-28T12:01:44.939156+0000","last_peered":"2023-11-28T12:01:44.939156+0000","last_clean":"2023-11-28T12:01:44.939156+0000","last_became_active":"2023-11-28T12:01:43.950717+0000","last_became_peered":"2023-11-28T12:01:43.950717+0000","last_unstale":"2023-11-28T12:01:44.939156+0000","last_undegraded":"2023-11-28T12:01:44.939156+0000","last_fullsized":"2023-11-28T12:01:44.939156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:32.123066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922884+0000","last_change":"2023-11-28T12:01:43.500086+0000","last_active":"2023-11-28T12:01:45.922884+0000","last_peered":"2023-11-28T12:01:45.922884+0000","last_clean":"2023-11-28T12:01:45.922884+0000","last_became_active":"2023-11-28T12:01:43.499317+0000","last_became_peered":"2023-11-28T12:01:43.499317+0000","last_unstale":"2023-11-28T12:01:45.922884+0000","last_undegraded":"2023-11-28T12:01:45.922884+0000","last_fullsized":"2023-11-28T12:01:45.922884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.498026+0000","last_change":"2023-11-28T12:01:43.952329+0000","last_active":"2023-11-28T12:01:46.498026+0000","last_peered":"2023-11-28T12:01:46.498026+0000","last_clean":"2023-11-28T12:01:46.498026+0000","last_became_active":"2023-11-28T12:01:43.952003+0000","last_became_peered":"2023-11-28T12:01:43.952003+0000","last_unstale":"2023-11-28T12:01:46.498026+0000","last_undegraded":"2023-11-28T12:01:46.498026+0000","last_fullsized":"2023-11-28T12:01:46.498026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938921+0000","last_change":"2023-11-28T12:01:43.499958+0000","last_active":"2023-11-28T12:01:44.938921+0000","last_peered":"2023-11-28T12:01:44.938921+0000","last_clean":"2023-11-28T12:01:44.938921+0000","last_became_active":"2023-11-28T12:01:43.499623+0000","last_became_peered":"2023-11-28T12:01:43.499623+0000","last_unstale":"2023-11-28T12:01:44.938921+0000","last_undegraded":"2023-11-28T12:01:44.938921+0000","last_fullsized":"2023-11-28T12:01:44.938921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:46.276522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937707+0000","last_change":"2023-11-28T12:01:43.945390+0000","last_active":"2023-11-28T12:01:44.937707+0000","last_peered":"2023-11-28T12:01:44.937707+0000","last_clean":"2023-11-28T12:01:44.937707+0000","last_became_active":"2023-11-28T12:01:43.945125+0000","last_became_peered":"2023-11-28T12:01:43.945125+0000","last_unstale":"2023-11-28T12:01:44.937707+0000","last_undegraded":"2023-11-28T12:01:44.937707+0000","last_fullsized":"2023-11-28T12:01:44.937707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:01.126989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497989+0000","last_change":"2023-11-28T12:01:43.497913+0000","last_active":"2023-11-28T12:01:46.497989+0000","last_peered":"2023-11-28T12:01:46.497989+0000","last_clean":"2023-11-28T12:01:46.497989+0000","last_became_active":"2023-11-28T12:01:43.497597+0000","last_became_peered":"2023-11-28T12:01:43.497597+0000","last_unstale":"2023-11-28T12:01:46.497989+0000","last_undegraded":"2023-11-28T12:01:46.497989+0000","last_fullsized":"2023-11-28T12:01:46.497989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938856+0000","last_change":"2023-11-28T12:01:43.938776+0000","last_active":"2023-11-28T12:01:44.938856+0000","last_peered":"2023-11-28T12:01:44.938856+0000","last_clean":"2023-11-28T12:01:44.938856+0000","last_became_active":"2023-11-28T12:01:43.938258+0000","last_became_peered":"2023-11-28T12:01:43.938258+0000","last_unstale":"2023-11-28T12:01:44.938856+0000","last_undegraded":"2023-11-28T12:01:44.938856+0000","last_fullsized":"2023-11-28T12:01:44.938856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:10.443506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545384+0000","last_change":"2023-11-28T12:01:43.494977+0000","last_active":"2023-11-28T12:01:46.545384+0000","last_peered":"2023-11-28T12:01:46.545384+0000","last_clean":"2023-11-28T12:01:46.545384+0000","last_became_active":"2023-11-28T12:01:43.494659+0000","last_became_peered":"2023-11-28T12:01:43.494659+0000","last_unstale":"2023-11-28T12:01:46.545384+0000","last_undegraded":"2023-11-28T12:01:46.545384+0000","last_fullsized":"2023-11-28T12:01:46.545384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:29.667067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938938+0000","last_change":"2023-11-28T12:01:43.949219+0000","last_active":"2023-11-28T12:01:44.938938+0000","last_peered":"2023-11-28T12:01:44.938938+0000","last_clean":"2023-11-28T12:01:44.938938+0000","last_became_active":"2023-11-28T12:01:43.948599+0000","last_became_peered":"2023-11-28T12:01:43.948599+0000","last_unstale":"2023-11-28T12:01:44.938938+0000","last_undegraded":"2023-11-28T12:01:44.938938+0000","last_fullsized":"2023-11-28T12:01:44.938938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:43.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853768+0000","last_change":"2023-11-28T12:01:43.504068+0000","last_active":"2023-11-28T12:01:45.853768+0000","last_peered":"2023-11-28T12:01:45.853768+0000","last_clean":"2023-11-28T12:01:45.853768+0000","last_became_active":"2023-11-28T12:01:43.503819+0000","last_became_peered":"2023-11-28T12:01:43.503819+0000","last_unstale":"2023-11-28T12:01:45.853768+0000","last_undegraded":"2023-11-28T12:01:45.853768+0000","last_fullsized":"2023-11-28T12:01:45.853768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:49.510656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922795+0000","last_change":"2023-11-28T12:01:43.941737+0000","last_active":"2023-11-28T12:01:45.922795+0000","last_peered":"2023-11-28T12:01:45.922795+0000","last_clean":"2023-11-28T12:01:45.922795+0000","last_became_active":"2023-11-28T12:01:43.941456+0000","last_became_peered":"2023-11-28T12:01:43.941456+0000","last_unstale":"2023-11-28T12:01:45.922795+0000","last_undegraded":"2023-11-28T12:01:45.922795+0000","last_fullsized":"2023-11-28T12:01:45.922795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:36.019691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938329+0000","last_change":"2023-11-28T12:01:43.492482+0000","last_active":"2023-11-28T12:01:44.938329+0000","last_peered":"2023-11-28T12:01:44.938329+0000","last_clean":"2023-11-28T12:01:44.938329+0000","last_became_active":"2023-11-28T12:01:43.492180+0000","last_became_peered":"2023-11-28T12:01:43.492180+0000","last_unstale":"2023-11-28T12:01:44.938329+0000","last_undegraded":"2023-11-28T12:01:44.938329+0000","last_fullsized":"2023-11-28T12:01:44.938329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938187+0000","last_change":"2023-11-28T12:01:43.944780+0000","last_active":"2023-11-28T12:01:44.938187+0000","last_peered":"2023-11-28T12:01:44.938187+0000","last_clean":"2023-11-28T12:01:44.938187+0000","last_became_active":"2023-11-28T12:01:43.944467+0000","last_became_peered":"2023-11-28T12:01:43.944467+0000","last_unstale":"2023-11-28T12:01:44.938187+0000","last_undegraded":"2023-11-28T12:01:44.938187+0000","last_fullsized":"2023-11-28T12:01:44.938187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:57.245235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853599+0000","last_change":"2023-11-28T12:01:43.499400+0000","last_active":"2023-11-28T12:01:45.853599+0000","last_peered":"2023-11-28T12:01:45.853599+0000","last_clean":"2023-11-28T12:01:45.853599+0000","last_became_active":"2023-11-28T12:01:43.499116+0000","last_became_peered":"2023-11-28T12:01:43.499116+0000","last_unstale":"2023-11-28T12:01:45.853599+0000","last_undegraded":"2023-11-28T12:01:45.853599+0000","last_fullsized":"2023-11-28T12:01:45.853599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:00.731296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939194+0000","last_change":"2023-11-28T12:01:43.954953+0000","last_active":"2023-11-28T12:01:44.939194+0000","last_peered":"2023-11-28T12:01:44.939194+0000","last_clean":"2023-11-28T12:01:44.939194+0000","last_became_active":"2023-11-28T12:01:43.952565+0000","last_became_peered":"2023-11-28T12:01:43.952565+0000","last_unstale":"2023-11-28T12:01:44.939194+0000","last_undegraded":"2023-11-28T12:01:44.939194+0000","last_fullsized":"2023-11-28T12:01:44.939194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:29.033626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497689+0000","last_change":"2023-11-28T12:01:43.496294+0000","last_active":"2023-11-28T12:01:46.497689+0000","last_peered":"2023-11-28T12:01:46.497689+0000","last_clean":"2023-11-28T12:01:46.497689+0000","last_became_active":"2023-11-28T12:01:43.493574+0000","last_became_peered":"2023-11-28T12:01:43.493574+0000","last_unstale":"2023-11-28T12:01:46.497689+0000","last_undegraded":"2023-11-28T12:01:46.497689+0000","last_fullsized":"2023-11-28T12:01:46.497689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545401+0000","last_change":"2023-11-28T12:01:43.942727+0000","last_active":"2023-11-28T12:01:46.545401+0000","last_peered":"2023-11-28T12:01:46.545401+0000","last_clean":"2023-11-28T12:01:46.545401+0000","last_became_active":"2023-11-28T12:01:43.942041+0000","last_became_peered":"2023-11-28T12:01:43.942041+0000","last_unstale":"2023-11-28T12:01:46.545401+0000","last_undegraded":"2023-11-28T12:01:46.545401+0000","last_fullsized":"2023-11-28T12:01:46.545401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:23.540287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853448+0000","last_change":"2023-11-28T12:01:43.499858+0000","last_active":"2023-11-28T12:01:45.853448+0000","last_peered":"2023-11-28T12:01:45.853448+0000","last_clean":"2023-11-28T12:01:45.853448+0000","last_became_active":"2023-11-28T12:01:43.499006+0000","last_became_peered":"2023-11-28T12:01:43.499006+0000","last_unstale":"2023-11-28T12:01:45.853448+0000","last_undegraded":"2023-11-28T12:01:45.853448+0000","last_fullsized":"2023-11-28T12:01:45.853448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:41:02.952529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497880+0000","last_change":"2023-11-28T12:01:43.950190+0000","last_active":"2023-11-28T12:01:46.497880+0000","last_peered":"2023-11-28T12:01:46.497880+0000","last_clean":"2023-11-28T12:01:46.497880+0000","last_became_active":"2023-11-28T12:01:43.949923+0000","last_became_peered":"2023-11-28T12:01:43.949923+0000","last_unstale":"2023-11-28T12:01:46.497880+0000","last_undegraded":"2023-11-28T12:01:46.497880+0000","last_fullsized":"2023-11-28T12:01:46.497880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:32.123066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937355+0000","last_change":"2023-11-28T12:01:43.493841+0000","last_active":"2023-11-28T12:01:44.937355+0000","last_peered":"2023-11-28T12:01:44.937355+0000","last_clean":"2023-11-28T12:01:44.937355+0000","last_became_active":"2023-11-28T12:01:43.493559+0000","last_became_peered":"2023-11-28T12:01:43.493559+0000","last_unstale":"2023-11-28T12:01:44.937355+0000","last_undegraded":"2023-11-28T12:01:44.937355+0000","last_fullsized":"2023-11-28T12:01:44.937355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939401+0000","last_change":"2023-11-28T12:01:43.954120+0000","last_active":"2023-11-28T12:01:44.939401+0000","last_peered":"2023-11-28T12:01:44.939401+0000","last_clean":"2023-11-28T12:01:44.939401+0000","last_became_active":"2023-11-28T12:01:43.953634+0000","last_became_peered":"2023-11-28T12:01:43.953634+0000","last_unstale":"2023-11-28T12:01:44.939401+0000","last_undegraded":"2023-11-28T12:01:44.939401+0000","last_fullsized":"2023-11-28T12:01:44.939401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:46.302273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853794+0000","last_change":"2023-11-28T12:01:43.499686+0000","last_active":"2023-11-28T12:01:45.853794+0000","last_peered":"2023-11-28T12:01:45.853794+0000","last_clean":"2023-11-28T12:01:45.853794+0000","last_became_active":"2023-11-28T12:01:43.499379+0000","last_became_peered":"2023-11-28T12:01:43.499379+0000","last_unstale":"2023-11-28T12:01:45.853794+0000","last_undegraded":"2023-11-28T12:01:45.853794+0000","last_fullsized":"2023-11-28T12:01:45.853794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:07.361707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545042+0000","last_change":"2023-11-28T12:01:43.943016+0000","last_active":"2023-11-28T12:01:46.545042+0000","last_peered":"2023-11-28T12:01:46.545042+0000","last_clean":"2023-11-28T12:01:46.545042+0000","last_became_active":"2023-11-28T12:01:43.941928+0000","last_became_peered":"2023-11-28T12:01:43.941928+0000","last_unstale":"2023-11-28T12:01:46.545042+0000","last_undegraded":"2023-11-28T12:01:46.545042+0000","last_fullsized":"2023-11-28T12:01:46.545042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:01.126989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853782+0000","last_change":"2023-11-28T12:01:43.499257+0000","last_active":"2023-11-28T12:01:45.853782+0000","last_peered":"2023-11-28T12:01:45.853782+0000","last_clean":"2023-11-28T12:01:45.853782+0000","last_became_active":"2023-11-28T12:01:43.498973+0000","last_became_peered":"2023-11-28T12:01:43.498973+0000","last_unstale":"2023-11-28T12:01:45.853782+0000","last_undegraded":"2023-11-28T12:01:45.853782+0000","last_fullsized":"2023-11-28T12:01:45.853782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:46:41.932790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937789+0000","last_change":"2023-11-28T12:01:43.937615+0000","last_active":"2023-11-28T12:01:44.937789+0000","last_peered":"2023-11-28T12:01:44.937789+0000","last_clean":"2023-11-28T12:01:44.937789+0000","last_became_active":"2023-11-28T12:01:43.937115+0000","last_became_peered":"2023-11-28T12:01:43.937115+0000","last_unstale":"2023-11-28T12:01:44.937789+0000","last_undegraded":"2023-11-28T12:01:44.937789+0000","last_fullsized":"2023-11-28T12:01:44.937789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:51.664836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853717+0000","last_change":"2023-11-28T12:01:43.496645+0000","last_active":"2023-11-28T12:01:45.853717+0000","last_peered":"2023-11-28T12:01:45.853717+0000","last_clean":"2023-11-28T12:01:45.853717+0000","last_became_active":"2023-11-28T12:01:43.496393+0000","last_became_peered":"2023-11-28T12:01:43.496393+0000","last_unstale":"2023-11-28T12:01:45.853717+0000","last_undegraded":"2023-11-28T12:01:45.853717+0000","last_fullsized":"2023-11-28T12:01:45.853717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:18.701227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937935+0000","last_change":"2023-11-28T12:01:43.946845+0000","last_active":"2023-11-28T12:01:44.937935+0000","last_peered":"2023-11-28T12:01:44.937935+0000","last_clean":"2023-11-28T12:01:44.937935+0000","last_became_active":"2023-11-28T12:01:43.946572+0000","last_became_peered":"2023-11-28T12:01:43.946572+0000","last_unstale":"2023-11-28T12:01:44.937935+0000","last_undegraded":"2023-11-28T12:01:44.937935+0000","last_fullsized":"2023-11-28T12:01:44.937935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545284+0000","last_change":"2023-11-28T12:01:43.500124+0000","last_active":"2023-11-28T12:01:46.545284+0000","last_peered":"2023-11-28T12:01:46.545284+0000","last_clean":"2023-11-28T12:01:46.545284+0000","last_became_active":"2023-11-28T12:01:43.499857+0000","last_became_peered":"2023-11-28T12:01:43.499857+0000","last_unstale":"2023-11-28T12:01:46.545284+0000","last_undegraded":"2023-11-28T12:01:46.545284+0000","last_fullsized":"2023-11-28T12:01:46.545284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497627+0000","last_change":"2023-11-28T12:01:43.945003+0000","last_active":"2023-11-28T12:01:46.497627+0000","last_peered":"2023-11-28T12:01:46.497627+0000","last_clean":"2023-11-28T12:01:46.497627+0000","last_became_active":"2023-11-28T12:01:43.944725+0000","last_became_peered":"2023-11-28T12:01:43.944725+0000","last_unstale":"2023-11-28T12:01:46.497627+0000","last_undegraded":"2023-11-28T12:01:46.497627+0000","last_fullsized":"2023-11-28T12:01:46.497627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:46.302273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939198+0000","last_change":"2023-11-28T12:01:43.500073+0000","last_active":"2023-11-28T12:01:44.939198+0000","last_peered":"2023-11-28T12:01:44.939198+0000","last_clean":"2023-11-28T12:01:44.939198+0000","last_became_active":"2023-11-28T12:01:43.499770+0000","last_became_peered":"2023-11-28T12:01:43.499770+0000","last_unstale":"2023-11-28T12:01:44.939198+0000","last_undegraded":"2023-11-28T12:01:44.939198+0000","last_fullsized":"2023-11-28T12:01:44.939198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938256+0000","last_change":"2023-11-28T12:01:43.954024+0000","last_active":"2023-11-28T12:01:44.938256+0000","last_peered":"2023-11-28T12:01:44.938256+0000","last_clean":"2023-11-28T12:01:44.938256+0000","last_became_active":"2023-11-28T12:01:43.953743+0000","last_became_peered":"2023-11-28T12:01:43.953743+0000","last_unstale":"2023-11-28T12:01:44.938256+0000","last_undegraded":"2023-11-28T12:01:44.938256+0000","last_fullsized":"2023-11-28T12:01:44.938256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:35.755196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937423+0000","last_change":"2023-11-28T12:01:43.491888+0000","last_active":"2023-11-28T12:01:44.937423+0000","last_peered":"2023-11-28T12:01:44.937423+0000","last_clean":"2023-11-28T12:01:44.937423+0000","last_became_active":"2023-11-28T12:01:43.491585+0000","last_became_peered":"2023-11-28T12:01:43.491585+0000","last_unstale":"2023-11-28T12:01:44.937423+0000","last_undegraded":"2023-11-28T12:01:44.937423+0000","last_fullsized":"2023-11-28T12:01:44.937423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938535+0000","last_change":"2023-11-28T12:01:43.940969+0000","last_active":"2023-11-28T12:01:44.938535+0000","last_peered":"2023-11-28T12:01:44.938535+0000","last_clean":"2023-11-28T12:01:44.938535+0000","last_became_active":"2023-11-28T12:01:43.940012+0000","last_became_peered":"2023-11-28T12:01:43.940012+0000","last_unstale":"2023-11-28T12:01:44.938535+0000","last_undegraded":"2023-11-28T12:01:44.938535+0000","last_fullsized":"2023-11-28T12:01:44.938535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:20:45.411292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937934+0000","last_change":"2023-11-28T12:01:43.493475+0000","last_active":"2023-11-28T12:01:44.937934+0000","last_peered":"2023-11-28T12:01:44.937934+0000","last_clean":"2023-11-28T12:01:44.937934+0000","last_became_active":"2023-11-28T12:01:43.493206+0000","last_became_peered":"2023-11-28T12:01:43.493206+0000","last_unstale":"2023-11-28T12:01:44.937934+0000","last_undegraded":"2023-11-28T12:01:44.937934+0000","last_fullsized":"2023-11-28T12:01:44.937934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938568+0000","last_change":"2023-11-28T12:01:43.944744+0000","last_active":"2023-11-28T12:01:44.938568+0000","last_peered":"2023-11-28T12:01:44.938568+0000","last_clean":"2023-11-28T12:01:44.938568+0000","last_became_active":"2023-11-28T12:01:43.943939+0000","last_became_peered":"2023-11-28T12:01:43.943939+0000","last_unstale":"2023-11-28T12:01:44.938568+0000","last_undegraded":"2023-11-28T12:01:44.938568+0000","last_fullsized":"2023-11-28T12:01:44.938568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:10.443506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938813+0000","last_change":"2023-11-28T12:01:43.497932+0000","last_active":"2023-11-28T12:01:44.938813+0000","last_peered":"2023-11-28T12:01:44.938813+0000","last_clean":"2023-11-28T12:01:44.938813+0000","last_became_active":"2023-11-28T12:01:43.497623+0000","last_became_peered":"2023-11-28T12:01:43.497623+0000","last_unstale":"2023-11-28T12:01:44.938813+0000","last_undegraded":"2023-11-28T12:01:44.938813+0000","last_fullsized":"2023-11-28T12:01:44.938813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:46:41.932790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923287+0000","last_change":"2023-11-28T12:01:43.943517+0000","last_active":"2023-11-28T12:01:45.923287+0000","last_peered":"2023-11-28T12:01:45.923287+0000","last_clean":"2023-11-28T12:01:45.923287+0000","last_became_active":"2023-11-28T12:01:43.943220+0000","last_became_peered":"2023-11-28T12:01:43.943220+0000","last_unstale":"2023-11-28T12:01:45.923287+0000","last_undegraded":"2023-11-28T12:01:45.923287+0000","last_fullsized":"2023-11-28T12:01:45.923287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939358+0000","last_change":"2023-11-28T12:01:43.495923+0000","last_active":"2023-11-28T12:01:44.939358+0000","last_peered":"2023-11-28T12:01:44.939358+0000","last_clean":"2023-11-28T12:01:44.939358+0000","last_became_active":"2023-11-28T12:01:43.494593+0000","last_became_peered":"2023-11-28T12:01:43.494593+0000","last_unstale":"2023-11-28T12:01:44.939358+0000","last_undegraded":"2023-11-28T12:01:44.939358+0000","last_fullsized":"2023-11-28T12:01:44.939358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.928332+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:01:43.928332+0000","last_peered":"2023-11-28T12:01:43.928332+0000","last_clean":"2023-11-28T12:01:43.928332+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:01:43.928332+0000","last_undegraded":"2023-11-28T12:01:43.928332+0000","last_fullsized":"2023-11-28T12:01:43.928332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938743+0000","last_change":"2023-11-28T12:01:43.938577+0000","last_active":"2023-11-28T12:01:44.938743+0000","last_peered":"2023-11-28T12:01:44.938743+0000","last_clean":"2023-11-28T12:01:44.938743+0000","last_became_active":"2023-11-28T12:01:43.937923+0000","last_became_peered":"2023-11-28T12:01:43.937923+0000","last_unstale":"2023-11-28T12:01:44.938743+0000","last_undegraded":"2023-11-28T12:01:44.938743+0000","last_fullsized":"2023-11-28T12:01:44.938743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:40.988886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.928705+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:01:43.928705+0000","last_peered":"2023-11-28T12:01:43.928705+0000","last_clean":"2023-11-28T12:01:43.928705+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:01:43.928705+0000","last_undegraded":"2023-11-28T12:01:43.928705+0000","last_fullsized":"2023-11-28T12:01:43.928705+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.498025+0000","last_change":"2023-11-28T12:01:43.500640+0000","last_active":"2023-11-28T12:01:46.498025+0000","last_peered":"2023-11-28T12:01:46.498025+0000","last_clean":"2023-11-28T12:01:46.498025+0000","last_became_active":"2023-11-28T12:01:43.500348+0000","last_became_peered":"2023-11-28T12:01:43.500348+0000","last_unstale":"2023-11-28T12:01:46.498025+0000","last_undegraded":"2023-11-28T12:01:46.498025+0000","last_fullsized":"2023-11-28T12:01:46.498025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:40.727625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937843+0000","last_change":"2023-11-28T12:01:43.944208+0000","last_active":"2023-11-28T12:01:44.937843+0000","last_peered":"2023-11-28T12:01:44.937843+0000","last_clean":"2023-11-28T12:01:44.937843+0000","last_became_active":"2023-11-28T12:01:43.943957+0000","last_became_peered":"2023-11-28T12:01:43.943957+0000","last_unstale":"2023-11-28T12:01:44.937843+0000","last_undegraded":"2023-11-28T12:01:44.937843+0000","last_fullsized":"2023-11-28T12:01:44.937843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:54:35.260564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.929563+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:01:43.929563+0000","last_peered":"2023-11-28T12:01:43.929563+0000","last_clean":"2023-11-28T12:01:43.929563+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:01:43.929563+0000","last_undegraded":"2023-11-28T12:01:43.929563+0000","last_fullsized":"2023-11-28T12:01:43.929563+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.934971+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:01:43.934971+0000","last_peered":"2023-11-28T12:01:43.934971+0000","last_clean":"2023-11-28T12:01:43.934971+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:01:43.934971+0000","last_undegraded":"2023-11-28T12:01:43.934971+0000","last_fullsized":"2023-11-28T12:01:43.934971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497590+0000","last_change":"2023-11-28T12:01:43.498540+0000","last_active":"2023-11-28T12:01:46.497590+0000","last_peered":"2023-11-28T12:01:46.497590+0000","last_clean":"2023-11-28T12:01:46.497590+0000","last_became_active":"2023-11-28T12:01:43.498040+0000","last_became_peered":"2023-11-28T12:01:43.498040+0000","last_unstale":"2023-11-28T12:01:46.497590+0000","last_undegraded":"2023-11-28T12:01:46.497590+0000","last_fullsized":"2023-11-28T12:01:46.497590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:29.667067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497902+0000","last_change":"2023-11-28T12:01:43.946239+0000","last_active":"2023-11-28T12:01:46.497902+0000","last_peered":"2023-11-28T12:01:46.497902+0000","last_clean":"2023-11-28T12:01:46.497902+0000","last_became_active":"2023-11-28T12:01:43.945937+0000","last_became_peered":"2023-11-28T12:01:43.945937+0000","last_unstale":"2023-11-28T12:01:46.497902+0000","last_undegraded":"2023-11-28T12:01:46.497902+0000","last_fullsized":"2023-11-28T12:01:46.497902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:40.988886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.935299+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:01:43.935299+0000","last_peered":"2023-11-28T12:01:43.935299+0000","last_clean":"2023-11-28T12:01:43.935299+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:01:43.935299+0000","last_undegraded":"2023-11-28T12:01:43.935299+0000","last_fullsized":"2023-11-28T12:01:43.935299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922707+0000","last_change":"2023-11-28T12:01:43.495719+0000","last_active":"2023-11-28T12:01:45.922707+0000","last_peered":"2023-11-28T12:01:45.922707+0000","last_clean":"2023-11-28T12:01:45.922707+0000","last_became_active":"2023-11-28T12:01:43.494865+0000","last_became_peered":"2023-11-28T12:01:43.494865+0000","last_unstale":"2023-11-28T12:01:45.922707+0000","last_undegraded":"2023-11-28T12:01:45.922707+0000","last_fullsized":"2023-11-28T12:01:45.922707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:29.667067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923227+0000","last_change":"2023-11-28T12:01:43.946289+0000","last_active":"2023-11-28T12:01:45.923227+0000","last_peered":"2023-11-28T12:01:45.923227+0000","last_clean":"2023-11-28T12:01:45.923227+0000","last_became_active":"2023-11-28T12:01:43.945759+0000","last_became_peered":"2023-11-28T12:01:43.945759+0000","last_unstale":"2023-11-28T12:01:45.923227+0000","last_undegraded":"2023-11-28T12:01:45.923227+0000","last_fullsized":"2023-11-28T12:01:45.923227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:41.736417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.934867+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:01:43.934867+0000","last_peered":"2023-11-28T12:01:43.934867+0000","last_clean":"2023-11-28T12:01:43.934867+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:01:43.934867+0000","last_undegraded":"2023-11-28T12:01:43.934867+0000","last_fullsized":"2023-11-28T12:01:43.934867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938960+0000","last_change":"2023-11-28T12:01:43.496920+0000","last_active":"2023-11-28T12:01:44.938960+0000","last_peered":"2023-11-28T12:01:44.938960+0000","last_clean":"2023-11-28T12:01:44.938960+0000","last_became_active":"2023-11-28T12:01:43.495644+0000","last_became_peered":"2023-11-28T12:01:43.495644+0000","last_unstale":"2023-11-28T12:01:44.938960+0000","last_undegraded":"2023-11-28T12:01:44.938960+0000","last_fullsized":"2023-11-28T12:01:44.938960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937511+0000","last_change":"2023-11-28T12:01:43.950061+0000","last_active":"2023-11-28T12:01:44.937511+0000","last_peered":"2023-11-28T12:01:44.937511+0000","last_clean":"2023-11-28T12:01:44.937511+0000","last_became_active":"2023-11-28T12:01:43.949778+0000","last_became_peered":"2023-11-28T12:01:43.949778+0000","last_unstale":"2023-11-28T12:01:44.937511+0000","last_undegraded":"2023-11-28T12:01:44.937511+0000","last_fullsized":"2023-11-28T12:01:44.937511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:36.019691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.932921+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:01:43.932921+0000","last_peered":"2023-11-28T12:01:43.932921+0000","last_clean":"2023-11-28T12:01:43.932921+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:01:43.932921+0000","last_undegraded":"2023-11-28T12:01:43.932921+0000","last_fullsized":"2023-11-28T12:01:43.932921+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938196+0000","last_change":"2023-11-28T12:01:43.499396+0000","last_active":"2023-11-28T12:01:44.938196+0000","last_peered":"2023-11-28T12:01:44.938196+0000","last_clean":"2023-11-28T12:01:44.938196+0000","last_became_active":"2023-11-28T12:01:43.499103+0000","last_became_peered":"2023-11-28T12:01:43.499103+0000","last_unstale":"2023-11-28T12:01:44.938196+0000","last_undegraded":"2023-11-28T12:01:44.938196+0000","last_fullsized":"2023-11-28T12:01:44.938196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923174+0000","last_change":"2023-11-28T12:01:43.941086+0000","last_active":"2023-11-28T12:01:45.923174+0000","last_peered":"2023-11-28T12:01:45.923174+0000","last_clean":"2023-11-28T12:01:45.923174+0000","last_became_active":"2023-11-28T12:01:43.940781+0000","last_became_peered":"2023-11-28T12:01:43.940781+0000","last_unstale":"2023-11-28T12:01:45.923174+0000","last_undegraded":"2023-11-28T12:01:45.923174+0000","last_fullsized":"2023-11-28T12:01:45.923174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:19.706888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.935713+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:01:43.935713+0000","last_peered":"2023-11-28T12:01:43.935713+0000","last_clean":"2023-11-28T12:01:43.935713+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:01:43.935713+0000","last_undegraded":"2023-11-28T12:01:43.935713+0000","last_fullsized":"2023-11-28T12:01:43.935713+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923245+0000","last_change":"2023-11-28T12:01:43.501876+0000","last_active":"2023-11-28T12:01:45.923245+0000","last_peered":"2023-11-28T12:01:45.923245+0000","last_clean":"2023-11-28T12:01:45.923245+0000","last_became_active":"2023-11-28T12:01:43.501509+0000","last_became_peered":"2023-11-28T12:01:43.501509+0000","last_unstale":"2023-11-28T12:01:45.923245+0000","last_undegraded":"2023-11-28T12:01:45.923245+0000","last_fullsized":"2023-11-28T12:01:45.923245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497880+0000","last_change":"2023-11-28T12:01:43.947364+0000","last_active":"2023-11-28T12:01:46.497880+0000","last_peered":"2023-11-28T12:01:46.497880+0000","last_clean":"2023-11-28T12:01:46.497880+0000","last_became_active":"2023-11-28T12:01:43.947043+0000","last_became_peered":"2023-11-28T12:01:43.947043+0000","last_unstale":"2023-11-28T12:01:46.497880+0000","last_undegraded":"2023-11-28T12:01:46.497880+0000","last_fullsized":"2023-11-28T12:01:46.497880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.936792+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:01:43.936792+0000","last_peered":"2023-11-28T12:01:43.936792+0000","last_clean":"2023-11-28T12:01:43.936792+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:01:43.936792+0000","last_undegraded":"2023-11-28T12:01:43.936792+0000","last_fullsized":"2023-11-28T12:01:43.936792+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545482+0000","last_change":"2023-11-28T12:01:43.497088+0000","last_active":"2023-11-28T12:01:46.545482+0000","last_peered":"2023-11-28T12:01:46.545482+0000","last_clean":"2023-11-28T12:01:46.545482+0000","last_became_active":"2023-11-28T12:01:43.496456+0000","last_became_peered":"2023-11-28T12:01:43.496456+0000","last_unstale":"2023-11-28T12:01:46.545482+0000","last_undegraded":"2023-11-28T12:01:46.545482+0000","last_fullsized":"2023-11-28T12:01:46.545482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923037+0000","last_change":"2023-11-28T12:01:43.946582+0000","last_active":"2023-11-28T12:01:45.923037+0000","last_peered":"2023-11-28T12:01:45.923037+0000","last_clean":"2023-11-28T12:01:45.923037+0000","last_became_active":"2023-11-28T12:01:43.946281+0000","last_became_peered":"2023-11-28T12:01:43.946281+0000","last_unstale":"2023-11-28T12:01:45.923037+0000","last_undegraded":"2023-11-28T12:01:45.923037+0000","last_fullsized":"2023-11-28T12:01:45.923037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:10.298310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497709+0000","last_change":"2023-11-28T12:01:43.499312+0000","last_active":"2023-11-28T12:01:46.497709+0000","last_peered":"2023-11-28T12:01:46.497709+0000","last_clean":"2023-11-28T12:01:46.497709+0000","last_became_active":"2023-11-28T12:01:43.499020+0000","last_became_peered":"2023-11-28T12:01:43.499020+0000","last_unstale":"2023-11-28T12:01:46.497709+0000","last_undegraded":"2023-11-28T12:01:46.497709+0000","last_fullsized":"2023-11-28T12:01:46.497709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542759+0000","last_change":"2023-11-28T12:01:43.945612+0000","last_active":"2023-11-28T12:01:46.542759+0000","last_peered":"2023-11-28T12:01:46.542759+0000","last_clean":"2023-11-28T12:01:46.542759+0000","last_became_active":"2023-11-28T12:01:43.945304+0000","last_became_peered":"2023-11-28T12:01:43.945304+0000","last_unstale":"2023-11-28T12:01:46.542759+0000","last_undegraded":"2023-11-28T12:01:46.542759+0000","last_fullsized":"2023-11-28T12:01:46.542759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:20.571945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545110+0000","last_change":"2023-11-28T12:01:43.496842+0000","last_active":"2023-11-28T12:01:46.545110+0000","last_peered":"2023-11-28T12:01:46.545110+0000","last_clean":"2023-11-28T12:01:46.545110+0000","last_became_active":"2023-11-28T12:01:43.496567+0000","last_became_peered":"2023-11-28T12:01:43.496567+0000","last_unstale":"2023-11-28T12:01:46.545110+0000","last_undegraded":"2023-11-28T12:01:46.545110+0000","last_fullsized":"2023-11-28T12:01:46.545110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938343+0000","last_change":"2023-11-28T12:01:43.941160+0000","last_active":"2023-11-28T12:01:44.938343+0000","last_peered":"2023-11-28T12:01:44.938343+0000","last_clean":"2023-11-28T12:01:44.938343+0000","last_became_active":"2023-11-28T12:01:43.940593+0000","last_became_peered":"2023-11-28T12:01:43.940593+0000","last_unstale":"2023-11-28T12:01:44.938343+0000","last_undegraded":"2023-11-28T12:01:44.938343+0000","last_fullsized":"2023-11-28T12:01:44.938343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:46.302273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545507+0000","last_change":"2023-11-28T12:01:43.497657+0000","last_active":"2023-11-28T12:01:46.545507+0000","last_peered":"2023-11-28T12:01:46.545507+0000","last_clean":"2023-11-28T12:01:46.545507+0000","last_became_active":"2023-11-28T12:01:43.496814+0000","last_became_peered":"2023-11-28T12:01:43.496814+0000","last_unstale":"2023-11-28T12:01:46.545507+0000","last_undegraded":"2023-11-28T12:01:46.545507+0000","last_fullsized":"2023-11-28T12:01:46.545507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938078+0000","last_change":"2023-11-28T12:01:43.938233+0000","last_active":"2023-11-28T12:01:44.938078+0000","last_peered":"2023-11-28T12:01:44.938078+0000","last_clean":"2023-11-28T12:01:44.938078+0000","last_became_active":"2023-11-28T12:01:43.937825+0000","last_became_peered":"2023-11-28T12:01:43.937825+0000","last_unstale":"2023-11-28T12:01:44.938078+0000","last_undegraded":"2023-11-28T12:01:44.938078+0000","last_fullsized":"2023-11-28T12:01:44.938078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:04.509152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939417+0000","last_change":"2023-11-28T12:01:43.485835+0000","last_active":"2023-11-28T12:01:44.939417+0000","last_peered":"2023-11-28T12:01:44.939417+0000","last_clean":"2023-11-28T12:01:44.939417+0000","last_became_active":"2023-11-28T12:01:43.484610+0000","last_became_peered":"2023-11-28T12:01:43.484610+0000","last_unstale":"2023-11-28T12:01:44.939417+0000","last_undegraded":"2023-11-28T12:01:44.939417+0000","last_fullsized":"2023-11-28T12:01:44.939417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:40.727625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939028+0000","last_change":"2023-11-28T12:01:43.949382+0000","last_active":"2023-11-28T12:01:44.939028+0000","last_peered":"2023-11-28T12:01:44.939028+0000","last_clean":"2023-11-28T12:01:44.939028+0000","last_became_active":"2023-11-28T12:01:43.948949+0000","last_became_peered":"2023-11-28T12:01:43.948949+0000","last_unstale":"2023-11-28T12:01:44.939028+0000","last_undegraded":"2023-11-28T12:01:44.939028+0000","last_fullsized":"2023-11-28T12:01:44.939028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:35.755196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497980+0000","last_change":"2023-11-28T12:01:43.498156+0000","last_active":"2023-11-28T12:01:46.497980+0000","last_peered":"2023-11-28T12:01:46.497980+0000","last_clean":"2023-11-28T12:01:46.497980+0000","last_became_active":"2023-11-28T12:01:43.496510+0000","last_became_peered":"2023-11-28T12:01:43.496510+0000","last_unstale":"2023-11-28T12:01:46.497980+0000","last_undegraded":"2023-11-28T12:01:46.497980+0000","last_fullsized":"2023-11-28T12:01:46.497980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:46.276522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937629+0000","last_change":"2023-11-28T12:01:43.944985+0000","last_active":"2023-11-28T12:01:44.937629+0000","last_peered":"2023-11-28T12:01:44.937629+0000","last_clean":"2023-11-28T12:01:44.937629+0000","last_became_active":"2023-11-28T12:01:43.943580+0000","last_became_peered":"2023-11-28T12:01:43.943580+0000","last_unstale":"2023-11-28T12:01:44.937629+0000","last_undegraded":"2023-11-28T12:01:44.937629+0000","last_fullsized":"2023-11-28T12:01:44.937629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937476+0000","last_change":"2023-11-28T12:01:43.941917+0000","last_active":"2023-11-28T12:01:44.937476+0000","last_peered":"2023-11-28T12:01:44.937476+0000","last_clean":"2023-11-28T12:01:44.937476+0000","last_became_active":"2023-11-28T12:01:43.941625+0000","last_became_peered":"2023-11-28T12:01:43.941625+0000","last_unstale":"2023-11-28T12:01:44.937476+0000","last_undegraded":"2023-11-28T12:01:44.937476+0000","last_fullsized":"2023-11-28T12:01:44.937476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:23.540287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938902+0000","last_change":"2023-11-28T12:01:43.943568+0000","last_active":"2023-11-28T12:01:44.938902+0000","last_peered":"2023-11-28T12:01:44.938902+0000","last_clean":"2023-11-28T12:01:44.938902+0000","last_became_active":"2023-11-28T12:01:43.943204+0000","last_became_peered":"2023-11-28T12:01:43.943204+0000","last_unstale":"2023-11-28T12:01:44.938902+0000","last_undegraded":"2023-11-28T12:01:44.938902+0000","last_fullsized":"2023-11-28T12:01:44.938902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937986+0000","last_change":"2023-11-28T12:01:43.943816+0000","last_active":"2023-11-28T12:01:44.937986+0000","last_peered":"2023-11-28T12:01:44.937986+0000","last_clean":"2023-11-28T12:01:44.937986+0000","last_became_active":"2023-11-28T12:01:43.943506+0000","last_became_peered":"2023-11-28T12:01:43.943506+0000","last_unstale":"2023-11-28T12:01:44.937986+0000","last_undegraded":"2023-11-28T12:01:44.937986+0000","last_fullsized":"2023-11-28T12:01:44.937986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:28.088904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939262+0000","last_change":"2023-11-28T12:01:43.950413+0000","last_active":"2023-11-28T12:01:44.939262+0000","last_peered":"2023-11-28T12:01:44.939262+0000","last_clean":"2023-11-28T12:01:44.939262+0000","last_became_active":"2023-11-28T12:01:43.950110+0000","last_became_peered":"2023-11-28T12:01:43.950110+0000","last_unstale":"2023-11-28T12:01:44.939262+0000","last_undegraded":"2023-11-28T12:01:44.939262+0000","last_fullsized":"2023-11-28T12:01:44.939262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:57.245235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922987+0000","last_change":"2023-11-28T12:01:43.938293+0000","last_active":"2023-11-28T12:01:45.922987+0000","last_peered":"2023-11-28T12:01:45.922987+0000","last_clean":"2023-11-28T12:01:45.922987+0000","last_became_active":"2023-11-28T12:01:43.937951+0000","last_became_peered":"2023-11-28T12:01:43.937951+0000","last_unstale":"2023-11-28T12:01:45.922987+0000","last_undegraded":"2023-11-28T12:01:45.922987+0000","last_fullsized":"2023-11-28T12:01:45.922987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9084,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362756,"statfs":{"total":96636764160,"available":96627462144,"internally_reserved":0,"allocated":839680,"data_stored":671592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7900,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363940,"statfs":{"total":96636764160,"available":96628674560,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9056,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362784,"statfs":{"total":96636764160,"available":96627490816,"internally_reserved":0,"allocated":811008,"data_stored":669828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7956,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363884,"statfs":{"total":96636764160,"available":96628617216,"internally_reserved":0,"allocated":208896,"data_stored":73373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7888,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363952,"statfs":{"total":96636764160,"available":96628686848,"internally_reserved":0,"allocated":204800,"data_stored":72807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8032,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363808,"statfs":{"total":96636764160,"available":96628539392,"internally_reserved":0,"allocated":221184,"data_stored":79550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8048,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363792,"statfs":{"total":96636764160,"available":96628523008,"internally_reserved":0,"allocated":237568,"data_stored":81064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7972,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363868,"statfs":{"total":96636764160,"available":96628600832,"internally_reserved":0,"allocated":225280,"data_stored":80096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:01:48.117 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:01:48.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.287+0000 7f1aac1bd700 1 Processor -- start 2023-11-28T12:01:48.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.288+0000 7f1aac1bd700 1 -- start start 2023-11-28T12:01:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.288+0000 7f1aac1bd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aa40d0c60 0x7f1aa40b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.288+0000 7f1aac1bd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1aa4160e70 0x7f1aa415c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.288+0000 7f1aac1bd700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa406e900 con 0x7f1aa406e400 2023-11-28T12:01:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.288+0000 7f1aac1bd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa4160390 con 0x7f1aa4160e70 2023-11-28T12:01:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.288+0000 7f1aac1bd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa4160670 con 0x7f1aa40d0c60 2023-11-28T12:01:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.289+0000 7f1aa9758700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f1aa406e400 0x7f1aa40d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36914/0 (socket says 172.21.15.57:36914) 2023-11-28T12:01:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.289+0000 7f1aa9758700 1 -- 172.21.15.57:0/1342114139 learned_addr learned my addr 172.21.15.57:0/1342114139 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.289+0000 7f1aaa75a700 1 --2- 172.21.15.57:0/1342114139 >> v2:172.21.15.181:3300/0 conn(0x7f1aa4160e70 0x7f1aa415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.289+0000 7f1aa9f59700 1 --2- 172.21.15.57:0/1342114139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aa40d0c60 0x7f1aa40b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.289+0000 7f1aa8f57700 1 -- 172.21.15.57:0/1342114139 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124486812 0 0) 0x7f1aa406e900 con 0x7f1aa406e400 2023-11-28T12:01:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.289+0000 7f1aa8f57700 1 -- 172.21.15.57:0/1342114139 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a900035b0 con 0x7f1aa406e400 2023-11-28T12:01:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.289+0000 7f1aaa75a700 1 -- 172.21.15.57:0/1342114139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aa40d0c60 msgr2=0x7f1aa40b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.289+0000 7f1aaa75a700 1 --2- 172.21.15.57:0/1342114139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aa40d0c60 0x7f1aa40b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.289+0000 7f1aaa75a700 1 -- 172.21.15.57:0/1342114139 >> v1:172.21.15.57:6789/0 conn(0x7f1aa406e400 legacy=0x7f1aa40d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:48.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.289+0000 7f1aaa75a700 1 -- 172.21.15.57:0/1342114139 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4160950 con 0x7f1aa4160e70 2023-11-28T12:01:48.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.289+0000 7f1aaa75a700 1 --2- 172.21.15.57:0/1342114139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa4160e70 0x7f1aa415c970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009d90 tx=0x7f1a9c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c7255693ac3126c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:48.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.289+0000 7f1aa8f57700 1 -- 172.21.15.57:0/1342114139 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c00e070 con 0x7f1aa4160e70 2023-11-28T12:01:48.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.290+0000 7f1aa8f57700 1 -- 172.21.15.57:0/1342114139 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c00bb20 con 0x7f1aa4160e70 2023-11-28T12:01:48.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.290+0000 7f1aa8f57700 1 -- 172.21.15.57:0/1342114139 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c012430 con 0x7f1aa4160e70 2023-11-28T12:01:48.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.290+0000 7f1aac1bd700 1 -- 172.21.15.57:0/1342114139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa4160e70 msgr2=0x7f1aa415c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:48.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.290+0000 7f1aac1bd700 1 --2- 172.21.15.57:0/1342114139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa4160e70 0x7f1aa415c970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009d90 tx=0x7f1a9c00b330 comp rx=0 tx=0).stop 2023-11-28T12:01:48.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.290+0000 7f1aac1bd700 1 -- 172.21.15.57:0/1342114139 shutdown_connections 2023-11-28T12:01:48.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.290+0000 7f1aac1bd700 1 --2- 172.21.15.57:0/1342114139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa4160e70 0x7f1aa415c970 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.290+0000 7f1aac1bd700 1 --2- 172.21.15.57:0/1342114139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aa40d0c60 0x7f1aa40b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.290+0000 7f1aac1bd700 1 -- 172.21.15.57:0/1342114139 >> 172.21.15.57:0/1342114139 conn(0x7f1aa406a270 msgr2=0x7f1aa406a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:48.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.290+0000 7f1aac1bd700 1 -- 172.21.15.57:0/1342114139 shutdown_connections 2023-11-28T12:01:48.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.290+0000 7f1aac1bd700 1 -- 172.21.15.57:0/1342114139 wait complete. 2023-11-28T12:01:48.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.291+0000 7f1aac1bd700 1 Processor -- start 2023-11-28T12:01:48.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.292+0000 7f1aac1bd700 1 -- start start 2023-11-28T12:01:48.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.292+0000 7f1aac1bd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa406e400 0x7f1aa414dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:48.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.292+0000 7f1aac1bd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1aa40d0c60 0x7f1aa40b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:48.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.292+0000 7f1aac1bd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aa4160e70 0x7f1aa4154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:48.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.292+0000 7f1aac1bd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40b19d0 con 0x7f1aa40d0c60 2023-11-28T12:01:48.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.292+0000 7f1aac1bd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa41e64a0 con 0x7f1aa406e400 2023-11-28T12:01:48.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.292+0000 7f1aac1bd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40b1c20 con 0x7f1aa4160e70 2023-11-28T12:01:48.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.292+0000 7f1aa9758700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1aa40d0c60 0x7f1aa40b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:48.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.292+0000 7f1aa9758700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1aa40d0c60 0x7f1aa40b2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47784/0 (socket says 172.21.15.57:47784) 2023-11-28T12:01:48.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.292+0000 7f1aa9758700 1 -- 172.21.15.57:0/2335441684 learned_addr learned my addr 172.21.15.57:0/2335441684 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:48.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1aaa75a700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aa4160e70 0x7f1aa4154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:48.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1aa9f59700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa406e400 0x7f1aa414dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:48.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1aa9758700 1 -- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aa4160e70 msgr2=0x7f1aa4154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:48.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1aa9758700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aa4160e70 0x7f1aa4154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1aa9758700 1 -- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa406e400 msgr2=0x7f1aa414dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:48.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1aa9758700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa406e400 0x7f1aa414dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1aa9758700 1 -- 172.21.15.57:0/2335441684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c009a70 con 0x7f1aa40d0c60 2023-11-28T12:01:48.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1aaa75a700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aa4160e70 0x7f1aa4154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:48.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1aa9758700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1aa40d0c60 0x7f1aa40b2700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a94005910 tx=0x7f1a9400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:48.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1a9a7fc700 1 -- 172.21.15.57:0/2335441684 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a94005d70 con 0x7f1aa40d0c60 2023-11-28T12:01:48.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1a9a7fc700 1 -- 172.21.15.57:0/2335441684 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a940070d0 con 0x7f1aa40d0c60 2023-11-28T12:01:48.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1aac1bd700 1 -- 172.21.15.57:0/2335441684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa415ceb0 con 0x7f1aa40d0c60 2023-11-28T12:01:48.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1a9a7fc700 1 -- 172.21.15.57:0/2335441684 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a94007440 con 0x7f1aa40d0c60 2023-11-28T12:01:48.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.293+0000 7f1aac1bd700 1 -- 172.21.15.57:0/2335441684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa40d0740 con 0x7f1aa40d0c60 2023-11-28T12:01:48.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.294+0000 7f1aac1bd700 1 -- 172.21.15.57:0/2335441684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa415d040 con 0x7f1aa40d0c60 2023-11-28T12:01:48.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.295+0000 7f1a9a7fc700 1 -- 172.21.15.57:0/2335441684 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1a94019030 con 0x7f1aa40d0c60 2023-11-28T12:01:48.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.295+0000 7f1a9a7fc700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1a880682f0 0x7f1a8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:48.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.295+0000 7f1a9a7fc700 1 -- 172.21.15.57:0/2335441684 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a94084ec0 con 0x7f1aa40d0c60 2023-11-28T12:01:48.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.295+0000 7f1aa9f59700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1a880682f0 0x7f1a8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:48.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.296+0000 7f1aa9f59700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1a880682f0 0x7f1a8806a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009be0 tx=0x7f1aa0009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:48.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.299+0000 7f1a9a7fc700 1 -- 172.21.15.57:0/2335441684 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1a9404b5e0 con 0x7f1aa40d0c60 2023-11-28T12:01:48.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.493+0000 7f1aac1bd700 1 -- 172.21.15.57:0/2335441684 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1aa40743f0 con 0x7f1a880682f0 2023-11-28T12:01:48.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.501+0000 7f1a9a7fc700 1 -- 172.21.15.57:0/2335441684 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286937 (secure 0 0 0) 0x7f1aa40743f0 con 0x7f1a880682f0 2023-11-28T12:01:48.506 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:48.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.504+0000 7f1aac1bd700 1 -- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1a880682f0 msgr2=0x7f1a8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:48.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.504+0000 7f1aac1bd700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1a880682f0 0x7f1a8806a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009be0 tx=0x7f1aa0009210 comp rx=0 tx=0).stop 2023-11-28T12:01:48.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.505+0000 7f1aac1bd700 1 -- 172.21.15.57:0/2335441684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1aa40d0c60 msgr2=0x7f1aa40b2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:48.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.505+0000 7f1aac1bd700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1aa40d0c60 0x7f1aa40b2700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a94005910 tx=0x7f1a9400c9e0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.505+0000 7f1aac1bd700 1 -- 172.21.15.57:0/2335441684 shutdown_connections 2023-11-28T12:01:48.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.505+0000 7f1aac1bd700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1a880682f0 0x7f1a8806a7a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.505+0000 7f1aac1bd700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aa4160e70 0x7f1aa4154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.505+0000 7f1aac1bd700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aa406e400 0x7f1aa414dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.505+0000 7f1aac1bd700 1 --2- 172.21.15.57:0/2335441684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1aa40d0c60 0x7f1aa40b2700 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.505+0000 7f1aac1bd700 1 -- 172.21.15.57:0/2335441684 >> 172.21.15.57:0/2335441684 conn(0x7f1aa406a270 msgr2=0x7f1aa415b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:48.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.505+0000 7f1aac1bd700 1 -- 172.21.15.57:0/2335441684 shutdown_connections 2023-11-28T12:01:48.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.505+0000 7f1aac1bd700 1 -- 172.21.15.57:0/2335441684 wait complete. 2023-11-28T12:01:48.509 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:01:48.520 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-28T12:01:47.658826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":65936,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":754908784,"statfs":{"total":773094113280,"available":773026594816,"internally_reserved":0,"allocated":2965504,"data_stored":1807770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.181731"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542856+0000","last_change":"2023-11-28T12:01:43.953143+0000","last_active":"2023-11-28T12:01:46.542856+0000","last_peered":"2023-11-28T12:01:46.542856+0000","last_clean":"2023-11-28T12:01:46.542856+0000","last_became_active":"2023-11-28T12:01:43.952673+0000","last_became_peered":"2023-11-28T12:01:43.952673+0000","last_unstale":"2023-11-28T12:01:46.542856+0000","last_undegraded":"2023-11-28T12:01:46.542856+0000","last_fullsized":"2023-11-28T12:01:46.542856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:18.949383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545399+0000","last_change":"2023-11-28T12:01:43.947886+0000","last_active":"2023-11-28T12:01:46.545399+0000","last_peered":"2023-11-28T12:01:46.545399+0000","last_clean":"2023-11-28T12:01:46.545399+0000","last_became_active":"2023-11-28T12:01:43.947468+0000","last_became_peered":"2023-11-28T12:01:43.947468+0000","last_unstale":"2023-11-28T12:01:46.545399+0000","last_undegraded":"2023-11-28T12:01:46.545399+0000","last_fullsized":"2023-11-28T12:01:46.545399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:41.736417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938403+0000","last_change":"2023-11-28T12:01:43.941069+0000","last_active":"2023-11-28T12:01:44.938403+0000","last_peered":"2023-11-28T12:01:44.938403+0000","last_clean":"2023-11-28T12:01:44.938403+0000","last_became_active":"2023-11-28T12:01:43.940366+0000","last_became_peered":"2023-11-28T12:01:43.940366+0000","last_unstale":"2023-11-28T12:01:44.938403+0000","last_undegraded":"2023-11-28T12:01:44.938403+0000","last_fullsized":"2023-11-28T12:01:44.938403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:40.988886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937690+0000","last_change":"2023-11-28T12:01:43.947292+0000","last_active":"2023-11-28T12:01:44.937690+0000","last_peered":"2023-11-28T12:01:44.937690+0000","last_clean":"2023-11-28T12:01:44.937690+0000","last_became_active":"2023-11-28T12:01:43.943027+0000","last_became_peered":"2023-11-28T12:01:43.943027+0000","last_unstale":"2023-11-28T12:01:44.937690+0000","last_undegraded":"2023-11-28T12:01:44.937690+0000","last_fullsized":"2023-11-28T12:01:44.937690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:19.706888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497843+0000","last_change":"2023-11-28T12:01:43.954299+0000","last_active":"2023-11-28T12:01:46.497843+0000","last_peered":"2023-11-28T12:01:46.497843+0000","last_clean":"2023-11-28T12:01:46.497843+0000","last_became_active":"2023-11-28T12:01:43.953967+0000","last_became_peered":"2023-11-28T12:01:43.953967+0000","last_unstale":"2023-11-28T12:01:46.497843+0000","last_undegraded":"2023-11-28T12:01:46.497843+0000","last_fullsized":"2023-11-28T12:01:46.497843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:35.755196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937402+0000","last_change":"2023-11-28T12:01:43.938748+0000","last_active":"2023-11-28T12:01:44.937402+0000","last_peered":"2023-11-28T12:01:44.937402+0000","last_clean":"2023-11-28T12:01:44.937402+0000","last_became_active":"2023-11-28T12:01:43.938430+0000","last_became_peered":"2023-11-28T12:01:43.938430+0000","last_unstale":"2023-11-28T12:01:44.937402+0000","last_undegraded":"2023-11-28T12:01:44.937402+0000","last_fullsized":"2023-11-28T12:01:44.937402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:49.922116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853429+0000","last_change":"2023-11-28T12:01:43.950997+0000","last_active":"2023-11-28T12:01:45.853429+0000","last_peered":"2023-11-28T12:01:45.853429+0000","last_clean":"2023-11-28T12:01:45.853429+0000","last_became_active":"2023-11-28T12:01:43.950532+0000","last_became_peered":"2023-11-28T12:01:43.950532+0000","last_unstale":"2023-11-28T12:01:45.853429+0000","last_undegraded":"2023-11-28T12:01:45.853429+0000","last_fullsized":"2023-11-28T12:01:45.853429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545106+0000","last_change":"2023-11-28T12:01:43.945419+0000","last_active":"2023-11-28T12:01:46.545106+0000","last_peered":"2023-11-28T12:01:46.545106+0000","last_clean":"2023-11-28T12:01:46.545106+0000","last_became_active":"2023-11-28T12:01:43.942913+0000","last_became_peered":"2023-11-28T12:01:43.942913+0000","last_unstale":"2023-11-28T12:01:46.545106+0000","last_undegraded":"2023-11-28T12:01:46.545106+0000","last_fullsized":"2023-11-28T12:01:46.545106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497879+0000","last_change":"2023-11-28T12:01:43.954900+0000","last_active":"2023-11-28T12:01:46.497879+0000","last_peered":"2023-11-28T12:01:46.497879+0000","last_clean":"2023-11-28T12:01:46.497879+0000","last_became_active":"2023-11-28T12:01:43.954524+0000","last_became_peered":"2023-11-28T12:01:43.954524+0000","last_unstale":"2023-11-28T12:01:46.497879+0000","last_undegraded":"2023-11-28T12:01:46.497879+0000","last_fullsized":"2023-11-28T12:01:46.497879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:10.298310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545527+0000","last_change":"2023-11-28T12:01:43.944809+0000","last_active":"2023-11-28T12:01:46.545527+0000","last_peered":"2023-11-28T12:01:46.545527+0000","last_clean":"2023-11-28T12:01:46.545527+0000","last_became_active":"2023-11-28T12:01:43.943120+0000","last_became_peered":"2023-11-28T12:01:43.943120+0000","last_unstale":"2023-11-28T12:01:46.545527+0000","last_undegraded":"2023-11-28T12:01:46.545527+0000","last_fullsized":"2023-11-28T12:01:46.545527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938327+0000","last_change":"2023-11-28T12:01:43.941446+0000","last_active":"2023-11-28T12:01:44.938327+0000","last_peered":"2023-11-28T12:01:44.938327+0000","last_clean":"2023-11-28T12:01:44.938327+0000","last_became_active":"2023-11-28T12:01:43.940640+0000","last_became_peered":"2023-11-28T12:01:43.940640+0000","last_unstale":"2023-11-28T12:01:44.938327+0000","last_undegraded":"2023-11-28T12:01:44.938327+0000","last_fullsized":"2023-11-28T12:01:44.938327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:41.605113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542711+0000","last_change":"2023-11-28T12:01:43.946959+0000","last_active":"2023-11-28T12:01:46.542711+0000","last_peered":"2023-11-28T12:01:46.542711+0000","last_clean":"2023-11-28T12:01:46.542711+0000","last_became_active":"2023-11-28T12:01:43.946674+0000","last_became_peered":"2023-11-28T12:01:43.946674+0000","last_unstale":"2023-11-28T12:01:46.542711+0000","last_undegraded":"2023-11-28T12:01:46.542711+0000","last_fullsized":"2023-11-28T12:01:46.542711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:18.910451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542620+0000","last_change":"2023-11-28T12:01:43.952292+0000","last_active":"2023-11-28T12:01:46.542620+0000","last_peered":"2023-11-28T12:01:46.542620+0000","last_clean":"2023-11-28T12:01:46.542620+0000","last_became_active":"2023-11-28T12:01:43.951948+0000","last_became_peered":"2023-11-28T12:01:43.951948+0000","last_unstale":"2023-11-28T12:01:46.542620+0000","last_undegraded":"2023-11-28T12:01:46.542620+0000","last_fullsized":"2023-11-28T12:01:46.542620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:30.672727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545157+0000","last_change":"2023-11-28T12:01:43.945700+0000","last_active":"2023-11-28T12:01:46.545157+0000","last_peered":"2023-11-28T12:01:46.545157+0000","last_clean":"2023-11-28T12:01:46.545157+0000","last_became_active":"2023-11-28T12:01:43.943097+0000","last_became_peered":"2023-11-28T12:01:43.943097+0000","last_unstale":"2023-11-28T12:01:46.545157+0000","last_undegraded":"2023-11-28T12:01:46.545157+0000","last_fullsized":"2023-11-28T12:01:46.545157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:36.019691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853575+0000","last_change":"2023-11-28T12:01:43.951123+0000","last_active":"2023-11-28T12:01:45.853575+0000","last_peered":"2023-11-28T12:01:45.853575+0000","last_clean":"2023-11-28T12:01:45.853575+0000","last_became_active":"2023-11-28T12:01:43.950867+0000","last_became_peered":"2023-11-28T12:01:43.950867+0000","last_unstale":"2023-11-28T12:01:45.853575+0000","last_undegraded":"2023-11-28T12:01:45.853575+0000","last_fullsized":"2023-11-28T12:01:45.853575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:32.123066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937717+0000","last_change":"2023-11-28T12:01:43.942191+0000","last_active":"2023-11-28T12:01:44.937717+0000","last_peered":"2023-11-28T12:01:44.937717+0000","last_clean":"2023-11-28T12:01:44.937717+0000","last_became_active":"2023-11-28T12:01:43.941576+0000","last_became_peered":"2023-11-28T12:01:43.941576+0000","last_unstale":"2023-11-28T12:01:44.937717+0000","last_undegraded":"2023-11-28T12:01:44.937717+0000","last_fullsized":"2023-11-28T12:01:44.937717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:43.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497887+0000","last_change":"2023-11-28T12:01:43.954787+0000","last_active":"2023-11-28T12:01:46.497887+0000","last_peered":"2023-11-28T12:01:46.497887+0000","last_clean":"2023-11-28T12:01:46.497887+0000","last_became_active":"2023-11-28T12:01:43.954463+0000","last_became_peered":"2023-11-28T12:01:43.954463+0000","last_unstale":"2023-11-28T12:01:46.497887+0000","last_undegraded":"2023-11-28T12:01:46.497887+0000","last_fullsized":"2023-11-28T12:01:46.497887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:23.540287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938451+0000","last_change":"2023-11-28T12:01:43.939593+0000","last_active":"2023-11-28T12:01:44.938451+0000","last_peered":"2023-11-28T12:01:44.938451+0000","last_clean":"2023-11-28T12:01:44.938451+0000","last_became_active":"2023-11-28T12:01:43.937190+0000","last_became_peered":"2023-11-28T12:01:43.937190+0000","last_unstale":"2023-11-28T12:01:44.938451+0000","last_undegraded":"2023-11-28T12:01:44.938451+0000","last_fullsized":"2023-11-28T12:01:44.938451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:16.230349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542842+0000","last_change":"2023-11-28T12:01:43.942910+0000","last_active":"2023-11-28T12:01:46.542842+0000","last_peered":"2023-11-28T12:01:46.542842+0000","last_clean":"2023-11-28T12:01:46.542842+0000","last_became_active":"2023-11-28T12:01:43.942609+0000","last_became_peered":"2023-11-28T12:01:43.942609+0000","last_unstale":"2023-11-28T12:01:46.542842+0000","last_undegraded":"2023-11-28T12:01:46.542842+0000","last_fullsized":"2023-11-28T12:01:46.542842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:37.541651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938215+0000","last_change":"2023-11-28T12:01:43.945798+0000","last_active":"2023-11-28T12:01:44.938215+0000","last_peered":"2023-11-28T12:01:44.938215+0000","last_clean":"2023-11-28T12:01:44.938215+0000","last_became_active":"2023-11-28T12:01:43.945473+0000","last_became_peered":"2023-11-28T12:01:43.945473+0000","last_unstale":"2023-11-28T12:01:44.938215+0000","last_undegraded":"2023-11-28T12:01:44.938215+0000","last_fullsized":"2023-11-28T12:01:44.938215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:29.033626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853662+0000","last_change":"2023-11-28T12:01:43.954303+0000","last_active":"2023-11-28T12:01:45.853662+0000","last_peered":"2023-11-28T12:01:45.853662+0000","last_clean":"2023-11-28T12:01:45.853662+0000","last_became_active":"2023-11-28T12:01:43.954076+0000","last_became_peered":"2023-11-28T12:01:43.954076+0000","last_unstale":"2023-11-28T12:01:45.853662+0000","last_undegraded":"2023-11-28T12:01:45.853662+0000","last_fullsized":"2023-11-28T12:01:45.853662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:41.736417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497578+0000","last_change":"2023-11-28T12:01:43.946172+0000","last_active":"2023-11-28T12:01:46.497578+0000","last_peered":"2023-11-28T12:01:46.497578+0000","last_clean":"2023-11-28T12:01:46.497578+0000","last_became_active":"2023-11-28T12:01:43.945881+0000","last_became_peered":"2023-11-28T12:01:43.945881+0000","last_unstale":"2023-11-28T12:01:46.497578+0000","last_undegraded":"2023-11-28T12:01:46.497578+0000","last_fullsized":"2023-11-28T12:01:46.497578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:43.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938826+0000","last_change":"2023-11-28T12:01:43.951156+0000","last_active":"2023-11-28T12:01:44.938826+0000","last_peered":"2023-11-28T12:01:44.938826+0000","last_clean":"2023-11-28T12:01:44.938826+0000","last_became_active":"2023-11-28T12:01:43.950873+0000","last_became_peered":"2023-11-28T12:01:43.950873+0000","last_unstale":"2023-11-28T12:01:44.938826+0000","last_undegraded":"2023-11-28T12:01:44.938826+0000","last_fullsized":"2023-11-28T12:01:44.938826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:32.718654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853513+0000","last_change":"2023-11-28T12:01:43.953938+0000","last_active":"2023-11-28T12:01:45.853513+0000","last_peered":"2023-11-28T12:01:45.853513+0000","last_clean":"2023-11-28T12:01:45.853513+0000","last_became_active":"2023-11-28T12:01:43.953649+0000","last_became_peered":"2023-11-28T12:01:43.953649+0000","last_unstale":"2023-11-28T12:01:45.853513+0000","last_undegraded":"2023-11-28T12:01:45.853513+0000","last_fullsized":"2023-11-28T12:01:45.853513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923145+0000","last_change":"2023-11-28T12:01:43.941856+0000","last_active":"2023-11-28T12:01:45.923145+0000","last_peered":"2023-11-28T12:01:45.923145+0000","last_clean":"2023-11-28T12:01:45.923145+0000","last_became_active":"2023-11-28T12:01:43.941555+0000","last_became_peered":"2023-11-28T12:01:43.941555+0000","last_unstale":"2023-11-28T12:01:45.923145+0000","last_undegraded":"2023-11-28T12:01:45.923145+0000","last_fullsized":"2023-11-28T12:01:45.923145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:01.126989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923247+0000","last_change":"2023-11-28T12:01:43.940084+0000","last_active":"2023-11-28T12:01:45.923247+0000","last_peered":"2023-11-28T12:01:45.923247+0000","last_clean":"2023-11-28T12:01:45.923247+0000","last_became_active":"2023-11-28T12:01:43.939856+0000","last_became_peered":"2023-11-28T12:01:43.939856+0000","last_unstale":"2023-11-28T12:01:45.923247+0000","last_undegraded":"2023-11-28T12:01:45.923247+0000","last_fullsized":"2023-11-28T12:01:45.923247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:50.516316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545353+0000","last_change":"2023-11-28T12:01:43.943624+0000","last_active":"2023-11-28T12:01:46.545353+0000","last_peered":"2023-11-28T12:01:46.545353+0000","last_clean":"2023-11-28T12:01:46.545353+0000","last_became_active":"2023-11-28T12:01:43.943313+0000","last_became_peered":"2023-11-28T12:01:43.943313+0000","last_unstale":"2023-11-28T12:01:46.545353+0000","last_undegraded":"2023-11-28T12:01:46.545353+0000","last_fullsized":"2023-11-28T12:01:46.545353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:10.298310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938040+0000","last_change":"2023-11-28T12:01:43.500890+0000","last_active":"2023-11-28T12:01:44.938040+0000","last_peered":"2023-11-28T12:01:44.938040+0000","last_clean":"2023-11-28T12:01:44.938040+0000","last_became_active":"2023-11-28T12:01:43.500621+0000","last_became_peered":"2023-11-28T12:01:43.500621+0000","last_unstale":"2023-11-28T12:01:44.938040+0000","last_undegraded":"2023-11-28T12:01:44.938040+0000","last_fullsized":"2023-11-28T12:01:44.938040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939156+0000","last_change":"2023-11-28T12:01:43.951003+0000","last_active":"2023-11-28T12:01:44.939156+0000","last_peered":"2023-11-28T12:01:44.939156+0000","last_clean":"2023-11-28T12:01:44.939156+0000","last_became_active":"2023-11-28T12:01:43.950717+0000","last_became_peered":"2023-11-28T12:01:43.950717+0000","last_unstale":"2023-11-28T12:01:44.939156+0000","last_undegraded":"2023-11-28T12:01:44.939156+0000","last_fullsized":"2023-11-28T12:01:44.939156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:32.123066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922884+0000","last_change":"2023-11-28T12:01:43.500086+0000","last_active":"2023-11-28T12:01:45.922884+0000","last_peered":"2023-11-28T12:01:45.922884+0000","last_clean":"2023-11-28T12:01:45.922884+0000","last_became_active":"2023-11-28T12:01:43.499317+0000","last_became_peered":"2023-11-28T12:01:43.499317+0000","last_unstale":"2023-11-28T12:01:45.922884+0000","last_undegraded":"2023-11-28T12:01:45.922884+0000","last_fullsized":"2023-11-28T12:01:45.922884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.498026+0000","last_change":"2023-11-28T12:01:43.952329+0000","last_active":"2023-11-28T12:01:46.498026+0000","last_peered":"2023-11-28T12:01:46.498026+0000","last_clean":"2023-11-28T12:01:46.498026+0000","last_became_active":"2023-11-28T12:01:43.952003+0000","last_became_peered":"2023-11-28T12:01:43.952003+0000","last_unstale":"2023-11-28T12:01:46.498026+0000","last_undegraded":"2023-11-28T12:01:46.498026+0000","last_fullsized":"2023-11-28T12:01:46.498026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938921+0000","last_change":"2023-11-28T12:01:43.499958+0000","last_active":"2023-11-28T12:01:44.938921+0000","last_peered":"2023-11-28T12:01:44.938921+0000","last_clean":"2023-11-28T12:01:44.938921+0000","last_became_active":"2023-11-28T12:01:43.499623+0000","last_became_peered":"2023-11-28T12:01:43.499623+0000","last_unstale":"2023-11-28T12:01:44.938921+0000","last_undegraded":"2023-11-28T12:01:44.938921+0000","last_fullsized":"2023-11-28T12:01:44.938921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:46.276522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937707+0000","last_change":"2023-11-28T12:01:43.945390+0000","last_active":"2023-11-28T12:01:44.937707+0000","last_peered":"2023-11-28T12:01:44.937707+0000","last_clean":"2023-11-28T12:01:44.937707+0000","last_became_active":"2023-11-28T12:01:43.945125+0000","last_became_peered":"2023-11-28T12:01:43.945125+0000","last_unstale":"2023-11-28T12:01:44.937707+0000","last_undegraded":"2023-11-28T12:01:44.937707+0000","last_fullsized":"2023-11-28T12:01:44.937707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:01.126989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497989+0000","last_change":"2023-11-28T12:01:43.497913+0000","last_active":"2023-11-28T12:01:46.497989+0000","last_peered":"2023-11-28T12:01:46.497989+0000","last_clean":"2023-11-28T12:01:46.497989+0000","last_became_active":"2023-11-28T12:01:43.497597+0000","last_became_peered":"2023-11-28T12:01:43.497597+0000","last_unstale":"2023-11-28T12:01:46.497989+0000","last_undegraded":"2023-11-28T12:01:46.497989+0000","last_fullsized":"2023-11-28T12:01:46.497989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938856+0000","last_change":"2023-11-28T12:01:43.938776+0000","last_active":"2023-11-28T12:01:44.938856+0000","last_peered":"2023-11-28T12:01:44.938856+0000","last_clean":"2023-11-28T12:01:44.938856+0000","last_became_active":"2023-11-28T12:01:43.938258+0000","last_became_peered":"2023-11-28T12:01:43.938258+0000","last_unstale":"2023-11-28T12:01:44.938856+0000","last_undegraded":"2023-11-28T12:01:44.938856+0000","last_fullsized":"2023-11-28T12:01:44.938856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:10.443506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545384+0000","last_change":"2023-11-28T12:01:43.494977+0000","last_active":"2023-11-28T12:01:46.545384+0000","last_peered":"2023-11-28T12:01:46.545384+0000","last_clean":"2023-11-28T12:01:46.545384+0000","last_became_active":"2023-11-28T12:01:43.494659+0000","last_became_peered":"2023-11-28T12:01:43.494659+0000","last_unstale":"2023-11-28T12:01:46.545384+0000","last_undegraded":"2023-11-28T12:01:46.545384+0000","last_fullsized":"2023-11-28T12:01:46.545384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:29.667067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938938+0000","last_change":"2023-11-28T12:01:43.949219+0000","last_active":"2023-11-28T12:01:44.938938+0000","last_peered":"2023-11-28T12:01:44.938938+0000","last_clean":"2023-11-28T12:01:44.938938+0000","last_became_active":"2023-11-28T12:01:43.948599+0000","last_became_peered":"2023-11-28T12:01:43.948599+0000","last_unstale":"2023-11-28T12:01:44.938938+0000","last_undegraded":"2023-11-28T12:01:44.938938+0000","last_fullsized":"2023-11-28T12:01:44.938938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:43.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853768+0000","last_change":"2023-11-28T12:01:43.504068+0000","last_active":"2023-11-28T12:01:45.853768+0000","last_peered":"2023-11-28T12:01:45.853768+0000","last_clean":"2023-11-28T12:01:45.853768+0000","last_became_active":"2023-11-28T12:01:43.503819+0000","last_became_peered":"2023-11-28T12:01:43.503819+0000","last_unstale":"2023-11-28T12:01:45.853768+0000","last_undegraded":"2023-11-28T12:01:45.853768+0000","last_fullsized":"2023-11-28T12:01:45.853768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:49.510656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922795+0000","last_change":"2023-11-28T12:01:43.941737+0000","last_active":"2023-11-28T12:01:45.922795+0000","last_peered":"2023-11-28T12:01:45.922795+0000","last_clean":"2023-11-28T12:01:45.922795+0000","last_became_active":"2023-11-28T12:01:43.941456+0000","last_became_peered":"2023-11-28T12:01:43.941456+0000","last_unstale":"2023-11-28T12:01:45.922795+0000","last_undegraded":"2023-11-28T12:01:45.922795+0000","last_fullsized":"2023-11-28T12:01:45.922795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:36.019691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938329+0000","last_change":"2023-11-28T12:01:43.492482+0000","last_active":"2023-11-28T12:01:44.938329+0000","last_peered":"2023-11-28T12:01:44.938329+0000","last_clean":"2023-11-28T12:01:44.938329+0000","last_became_active":"2023-11-28T12:01:43.492180+0000","last_became_peered":"2023-11-28T12:01:43.492180+0000","last_unstale":"2023-11-28T12:01:44.938329+0000","last_undegraded":"2023-11-28T12:01:44.938329+0000","last_fullsized":"2023-11-28T12:01:44.938329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938187+0000","last_change":"2023-11-28T12:01:43.944780+0000","last_active":"2023-11-28T12:01:44.938187+0000","last_peered":"2023-11-28T12:01:44.938187+0000","last_clean":"2023-11-28T12:01:44.938187+0000","last_became_active":"2023-11-28T12:01:43.944467+0000","last_became_peered":"2023-11-28T12:01:43.944467+0000","last_unstale":"2023-11-28T12:01:44.938187+0000","last_undegraded":"2023-11-28T12:01:44.938187+0000","last_fullsized":"2023-11-28T12:01:44.938187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:57.245235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853599+0000","last_change":"2023-11-28T12:01:43.499400+0000","last_active":"2023-11-28T12:01:45.853599+0000","last_peered":"2023-11-28T12:01:45.853599+0000","last_clean":"2023-11-28T12:01:45.853599+0000","last_became_active":"2023-11-28T12:01:43.499116+0000","last_became_peered":"2023-11-28T12:01:43.499116+0000","last_unstale":"2023-11-28T12:01:45.853599+0000","last_undegraded":"2023-11-28T12:01:45.853599+0000","last_fullsized":"2023-11-28T12:01:45.853599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:00.731296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939194+0000","last_change":"2023-11-28T12:01:43.954953+0000","last_active":"2023-11-28T12:01:44.939194+0000","last_peered":"2023-11-28T12:01:44.939194+0000","last_clean":"2023-11-28T12:01:44.939194+0000","last_became_active":"2023-11-28T12:01:43.952565+0000","last_became_peered":"2023-11-28T12:01:43.952565+0000","last_unstale":"2023-11-28T12:01:44.939194+0000","last_undegraded":"2023-11-28T12:01:44.939194+0000","last_fullsized":"2023-11-28T12:01:44.939194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:29.033626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497689+0000","last_change":"2023-11-28T12:01:43.496294+0000","last_active":"2023-11-28T12:01:46.497689+0000","last_peered":"2023-11-28T12:01:46.497689+0000","last_clean":"2023-11-28T12:01:46.497689+0000","last_became_active":"2023-11-28T12:01:43.493574+0000","last_became_peered":"2023-11-28T12:01:43.493574+0000","last_unstale":"2023-11-28T12:01:46.497689+0000","last_undegraded":"2023-11-28T12:01:46.497689+0000","last_fullsized":"2023-11-28T12:01:46.497689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545401+0000","last_change":"2023-11-28T12:01:43.942727+0000","last_active":"2023-11-28T12:01:46.545401+0000","last_peered":"2023-11-28T12:01:46.545401+0000","last_clean":"2023-11-28T12:01:46.545401+0000","last_became_active":"2023-11-28T12:01:43.942041+0000","last_became_peered":"2023-11-28T12:01:43.942041+0000","last_unstale":"2023-11-28T12:01:46.545401+0000","last_undegraded":"2023-11-28T12:01:46.545401+0000","last_fullsized":"2023-11-28T12:01:46.545401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:23.540287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853448+0000","last_change":"2023-11-28T12:01:43.499858+0000","last_active":"2023-11-28T12:01:45.853448+0000","last_peered":"2023-11-28T12:01:45.853448+0000","last_clean":"2023-11-28T12:01:45.853448+0000","last_became_active":"2023-11-28T12:01:43.499006+0000","last_became_peered":"2023-11-28T12:01:43.499006+0000","last_unstale":"2023-11-28T12:01:45.853448+0000","last_undegraded":"2023-11-28T12:01:45.853448+0000","last_fullsized":"2023-11-28T12:01:45.853448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:41:02.952529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497880+0000","last_change":"2023-11-28T12:01:43.950190+0000","last_active":"2023-11-28T12:01:46.497880+0000","last_peered":"2023-11-28T12:01:46.497880+0000","last_clean":"2023-11-28T12:01:46.497880+0000","last_became_active":"2023-11-28T12:01:43.949923+0000","last_became_peered":"2023-11-28T12:01:43.949923+0000","last_unstale":"2023-11-28T12:01:46.497880+0000","last_undegraded":"2023-11-28T12:01:46.497880+0000","last_fullsized":"2023-11-28T12:01:46.497880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:32.123066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937355+0000","last_change":"2023-11-28T12:01:43.493841+0000","last_active":"2023-11-28T12:01:44.937355+0000","last_peered":"2023-11-28T12:01:44.937355+0000","last_clean":"2023-11-28T12:01:44.937355+0000","last_became_active":"2023-11-28T12:01:43.493559+0000","last_became_peered":"2023-11-28T12:01:43.493559+0000","last_unstale":"2023-11-28T12:01:44.937355+0000","last_undegraded":"2023-11-28T12:01:44.937355+0000","last_fullsized":"2023-11-28T12:01:44.937355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939401+0000","last_change":"2023-11-28T12:01:43.954120+0000","last_active":"2023-11-28T12:01:44.939401+0000","last_peered":"2023-11-28T12:01:44.939401+0000","last_clean":"2023-11-28T12:01:44.939401+0000","last_became_active":"2023-11-28T12:01:43.953634+0000","last_became_peered":"2023-11-28T12:01:43.953634+0000","last_unstale":"2023-11-28T12:01:44.939401+0000","last_undegraded":"2023-11-28T12:01:44.939401+0000","last_fullsized":"2023-11-28T12:01:44.939401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:46.302273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853794+0000","last_change":"2023-11-28T12:01:43.499686+0000","last_active":"2023-11-28T12:01:45.853794+0000","last_peered":"2023-11-28T12:01:45.853794+0000","last_clean":"2023-11-28T12:01:45.853794+0000","last_became_active":"2023-11-28T12:01:43.499379+0000","last_became_peered":"2023-11-28T12:01:43.499379+0000","last_unstale":"2023-11-28T12:01:45.853794+0000","last_undegraded":"2023-11-28T12:01:45.853794+0000","last_fullsized":"2023-11-28T12:01:45.853794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:07.361707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545042+0000","last_change":"2023-11-28T12:01:43.943016+0000","last_active":"2023-11-28T12:01:46.545042+0000","last_peered":"2023-11-28T12:01:46.545042+0000","last_clean":"2023-11-28T12:01:46.545042+0000","last_became_active":"2023-11-28T12:01:43.941928+0000","last_became_peered":"2023-11-28T12:01:43.941928+0000","last_unstale":"2023-11-28T12:01:46.545042+0000","last_undegraded":"2023-11-28T12:01:46.545042+0000","last_fullsized":"2023-11-28T12:01:46.545042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:01.126989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853782+0000","last_change":"2023-11-28T12:01:43.499257+0000","last_active":"2023-11-28T12:01:45.853782+0000","last_peered":"2023-11-28T12:01:45.853782+0000","last_clean":"2023-11-28T12:01:45.853782+0000","last_became_active":"2023-11-28T12:01:43.498973+0000","last_became_peered":"2023-11-28T12:01:43.498973+0000","last_unstale":"2023-11-28T12:01:45.853782+0000","last_undegraded":"2023-11-28T12:01:45.853782+0000","last_fullsized":"2023-11-28T12:01:45.853782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:46:41.932790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937789+0000","last_change":"2023-11-28T12:01:43.937615+0000","last_active":"2023-11-28T12:01:44.937789+0000","last_peered":"2023-11-28T12:01:44.937789+0000","last_clean":"2023-11-28T12:01:44.937789+0000","last_became_active":"2023-11-28T12:01:43.937115+0000","last_became_peered":"2023-11-28T12:01:43.937115+0000","last_unstale":"2023-11-28T12:01:44.937789+0000","last_undegraded":"2023-11-28T12:01:44.937789+0000","last_fullsized":"2023-11-28T12:01:44.937789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:51.664836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853717+0000","last_change":"2023-11-28T12:01:43.496645+0000","last_active":"2023-11-28T12:01:45.853717+0000","last_peered":"2023-11-28T12:01:45.853717+0000","last_clean":"2023-11-28T12:01:45.853717+0000","last_became_active":"2023-11-28T12:01:43.496393+0000","last_became_peered":"2023-11-28T12:01:43.496393+0000","last_unstale":"2023-11-28T12:01:45.853717+0000","last_undegraded":"2023-11-28T12:01:45.853717+0000","last_fullsized":"2023-11-28T12:01:45.853717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:18.701227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937935+0000","last_change":"2023-11-28T12:01:43.946845+0000","last_active":"2023-11-28T12:01:44.937935+0000","last_peered":"2023-11-28T12:01:44.937935+0000","last_clean":"2023-11-28T12:01:44.937935+0000","last_became_active":"2023-11-28T12:01:43.946572+0000","last_became_peered":"2023-11-28T12:01:43.946572+0000","last_unstale":"2023-11-28T12:01:44.937935+0000","last_undegraded":"2023-11-28T12:01:44.937935+0000","last_fullsized":"2023-11-28T12:01:44.937935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545284+0000","last_change":"2023-11-28T12:01:43.500124+0000","last_active":"2023-11-28T12:01:46.545284+0000","last_peered":"2023-11-28T12:01:46.545284+0000","last_clean":"2023-11-28T12:01:46.545284+0000","last_became_active":"2023-11-28T12:01:43.499857+0000","last_became_peered":"2023-11-28T12:01:43.499857+0000","last_unstale":"2023-11-28T12:01:46.545284+0000","last_undegraded":"2023-11-28T12:01:46.545284+0000","last_fullsized":"2023-11-28T12:01:46.545284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497627+0000","last_change":"2023-11-28T12:01:43.945003+0000","last_active":"2023-11-28T12:01:46.497627+0000","last_peered":"2023-11-28T12:01:46.497627+0000","last_clean":"2023-11-28T12:01:46.497627+0000","last_became_active":"2023-11-28T12:01:43.944725+0000","last_became_peered":"2023-11-28T12:01:43.944725+0000","last_unstale":"2023-11-28T12:01:46.497627+0000","last_undegraded":"2023-11-28T12:01:46.497627+0000","last_fullsized":"2023-11-28T12:01:46.497627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:46.302273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939198+0000","last_change":"2023-11-28T12:01:43.500073+0000","last_active":"2023-11-28T12:01:44.939198+0000","last_peered":"2023-11-28T12:01:44.939198+0000","last_clean":"2023-11-28T12:01:44.939198+0000","last_became_active":"2023-11-28T12:01:43.499770+0000","last_became_peered":"2023-11-28T12:01:43.499770+0000","last_unstale":"2023-11-28T12:01:44.939198+0000","last_undegraded":"2023-11-28T12:01:44.939198+0000","last_fullsized":"2023-11-28T12:01:44.939198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938256+0000","last_change":"2023-11-28T12:01:43.954024+0000","last_active":"2023-11-28T12:01:44.938256+0000","last_peered":"2023-11-28T12:01:44.938256+0000","last_clean":"2023-11-28T12:01:44.938256+0000","last_became_active":"2023-11-28T12:01:43.953743+0000","last_became_peered":"2023-11-28T12:01:43.953743+0000","last_unstale":"2023-11-28T12:01:44.938256+0000","last_undegraded":"2023-11-28T12:01:44.938256+0000","last_fullsized":"2023-11-28T12:01:44.938256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:35.755196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937423+0000","last_change":"2023-11-28T12:01:43.491888+0000","last_active":"2023-11-28T12:01:44.937423+0000","last_peered":"2023-11-28T12:01:44.937423+0000","last_clean":"2023-11-28T12:01:44.937423+0000","last_became_active":"2023-11-28T12:01:43.491585+0000","last_became_peered":"2023-11-28T12:01:43.491585+0000","last_unstale":"2023-11-28T12:01:44.937423+0000","last_undegraded":"2023-11-28T12:01:44.937423+0000","last_fullsized":"2023-11-28T12:01:44.937423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938535+0000","last_change":"2023-11-28T12:01:43.940969+0000","last_active":"2023-11-28T12:01:44.938535+0000","last_peered":"2023-11-28T12:01:44.938535+0000","last_clean":"2023-11-28T12:01:44.938535+0000","last_became_active":"2023-11-28T12:01:43.940012+0000","last_became_peered":"2023-11-28T12:01:43.940012+0000","last_unstale":"2023-11-28T12:01:44.938535+0000","last_undegraded":"2023-11-28T12:01:44.938535+0000","last_fullsized":"2023-11-28T12:01:44.938535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:20:45.411292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937934+0000","last_change":"2023-11-28T12:01:43.493475+0000","last_active":"2023-11-28T12:01:44.937934+0000","last_peered":"2023-11-28T12:01:44.937934+0000","last_clean":"2023-11-28T12:01:44.937934+0000","last_became_active":"2023-11-28T12:01:43.493206+0000","last_became_peered":"2023-11-28T12:01:43.493206+0000","last_unstale":"2023-11-28T12:01:44.937934+0000","last_undegraded":"2023-11-28T12:01:44.937934+0000","last_fullsized":"2023-11-28T12:01:44.937934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938568+0000","last_change":"2023-11-28T12:01:43.944744+0000","last_active":"2023-11-28T12:01:44.938568+0000","last_peered":"2023-11-28T12:01:44.938568+0000","last_clean":"2023-11-28T12:01:44.938568+0000","last_became_active":"2023-11-28T12:01:43.943939+0000","last_became_peered":"2023-11-28T12:01:43.943939+0000","last_unstale":"2023-11-28T12:01:44.938568+0000","last_undegraded":"2023-11-28T12:01:44.938568+0000","last_fullsized":"2023-11-28T12:01:44.938568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:10.443506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938813+0000","last_change":"2023-11-28T12:01:43.497932+0000","last_active":"2023-11-28T12:01:44.938813+0000","last_peered":"2023-11-28T12:01:44.938813+0000","last_clean":"2023-11-28T12:01:44.938813+0000","last_became_active":"2023-11-28T12:01:43.497623+0000","last_became_peered":"2023-11-28T12:01:43.497623+0000","last_unstale":"2023-11-28T12:01:44.938813+0000","last_undegraded":"2023-11-28T12:01:44.938813+0000","last_fullsized":"2023-11-28T12:01:44.938813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:46:41.932790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923287+0000","last_change":"2023-11-28T12:01:43.943517+0000","last_active":"2023-11-28T12:01:45.923287+0000","last_peered":"2023-11-28T12:01:45.923287+0000","last_clean":"2023-11-28T12:01:45.923287+0000","last_became_active":"2023-11-28T12:01:43.943220+0000","last_became_peered":"2023-11-28T12:01:43.943220+0000","last_unstale":"2023-11-28T12:01:45.923287+0000","last_undegraded":"2023-11-28T12:01:45.923287+0000","last_fullsized":"2023-11-28T12:01:45.923287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939358+0000","last_change":"2023-11-28T12:01:43.495923+0000","last_active":"2023-11-28T12:01:44.939358+0000","last_peered":"2023-11-28T12:01:44.939358+0000","last_clean":"2023-11-28T12:01:44.939358+0000","last_became_active":"2023-11-28T12:01:43.494593+0000","last_became_peered":"2023-11-28T12:01:43.494593+0000","last_unstale":"2023-11-28T12:01:44.939358+0000","last_undegraded":"2023-11-28T12:01:44.939358+0000","last_fullsized":"2023-11-28T12:01:44.939358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.928332+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:01:43.928332+0000","last_peered":"2023-11-28T12:01:43.928332+0000","last_clean":"2023-11-28T12:01:43.928332+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:01:43.928332+0000","last_undegraded":"2023-11-28T12:01:43.928332+0000","last_fullsized":"2023-11-28T12:01:43.928332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938743+0000","last_change":"2023-11-28T12:01:43.938577+0000","last_active":"2023-11-28T12:01:44.938743+0000","last_peered":"2023-11-28T12:01:44.938743+0000","last_clean":"2023-11-28T12:01:44.938743+0000","last_became_active":"2023-11-28T12:01:43.937923+0000","last_became_peered":"2023-11-28T12:01:43.937923+0000","last_unstale":"2023-11-28T12:01:44.938743+0000","last_undegraded":"2023-11-28T12:01:44.938743+0000","last_fullsized":"2023-11-28T12:01:44.938743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:40.988886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.928705+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:01:43.928705+0000","last_peered":"2023-11-28T12:01:43.928705+0000","last_clean":"2023-11-28T12:01:43.928705+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:01:43.928705+0000","last_undegraded":"2023-11-28T12:01:43.928705+0000","last_fullsized":"2023-11-28T12:01:43.928705+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.498025+0000","last_change":"2023-11-28T12:01:43.500640+0000","last_active":"2023-11-28T12:01:46.498025+0000","last_peered":"2023-11-28T12:01:46.498025+0000","last_clean":"2023-11-28T12:01:46.498025+0000","last_became_active":"2023-11-28T12:01:43.500348+0000","last_became_peered":"2023-11-28T12:01:43.500348+0000","last_unstale":"2023-11-28T12:01:46.498025+0000","last_undegraded":"2023-11-28T12:01:46.498025+0000","last_fullsized":"2023-11-28T12:01:46.498025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:40.727625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937843+0000","last_change":"2023-11-28T12:01:43.944208+0000","last_active":"2023-11-28T12:01:44.937843+0000","last_peered":"2023-11-28T12:01:44.937843+0000","last_clean":"2023-11-28T12:01:44.937843+0000","last_became_active":"2023-11-28T12:01:43.943957+0000","last_became_peered":"2023-11-28T12:01:43.943957+0000","last_unstale":"2023-11-28T12:01:44.937843+0000","last_undegraded":"2023-11-28T12:01:44.937843+0000","last_fullsized":"2023-11-28T12:01:44.937843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:54:35.260564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.929563+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:01:43.929563+0000","last_peered":"2023-11-28T12:01:43.929563+0000","last_clean":"2023-11-28T12:01:43.929563+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:01:43.929563+0000","last_undegraded":"2023-11-28T12:01:43.929563+0000","last_fullsized":"2023-11-28T12:01:43.929563+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.934971+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:01:43.934971+0000","last_peered":"2023-11-28T12:01:43.934971+0000","last_clean":"2023-11-28T12:01:43.934971+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:01:43.934971+0000","last_undegraded":"2023-11-28T12:01:43.934971+0000","last_fullsized":"2023-11-28T12:01:43.934971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497590+0000","last_change":"2023-11-28T12:01:43.498540+0000","last_active":"2023-11-28T12:01:46.497590+0000","last_peered":"2023-11-28T12:01:46.497590+0000","last_clean":"2023-11-28T12:01:46.497590+0000","last_became_active":"2023-11-28T12:01:43.498040+0000","last_became_peered":"2023-11-28T12:01:43.498040+0000","last_unstale":"2023-11-28T12:01:46.497590+0000","last_undegraded":"2023-11-28T12:01:46.497590+0000","last_fullsized":"2023-11-28T12:01:46.497590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:29.667067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497902+0000","last_change":"2023-11-28T12:01:43.946239+0000","last_active":"2023-11-28T12:01:46.497902+0000","last_peered":"2023-11-28T12:01:46.497902+0000","last_clean":"2023-11-28T12:01:46.497902+0000","last_became_active":"2023-11-28T12:01:43.945937+0000","last_became_peered":"2023-11-28T12:01:43.945937+0000","last_unstale":"2023-11-28T12:01:46.497902+0000","last_undegraded":"2023-11-28T12:01:46.497902+0000","last_fullsized":"2023-11-28T12:01:46.497902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:40.988886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.935299+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:01:43.935299+0000","last_peered":"2023-11-28T12:01:43.935299+0000","last_clean":"2023-11-28T12:01:43.935299+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:01:43.935299+0000","last_undegraded":"2023-11-28T12:01:43.935299+0000","last_fullsized":"2023-11-28T12:01:43.935299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922707+0000","last_change":"2023-11-28T12:01:43.495719+0000","last_active":"2023-11-28T12:01:45.922707+0000","last_peered":"2023-11-28T12:01:45.922707+0000","last_clean":"2023-11-28T12:01:45.922707+0000","last_became_active":"2023-11-28T12:01:43.494865+0000","last_became_peered":"2023-11-28T12:01:43.494865+0000","last_unstale":"2023-11-28T12:01:45.922707+0000","last_undegraded":"2023-11-28T12:01:45.922707+0000","last_fullsized":"2023-11-28T12:01:45.922707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:29.667067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923227+0000","last_change":"2023-11-28T12:01:43.946289+0000","last_active":"2023-11-28T12:01:45.923227+0000","last_peered":"2023-11-28T12:01:45.923227+0000","last_clean":"2023-11-28T12:01:45.923227+0000","last_became_active":"2023-11-28T12:01:43.945759+0000","last_became_peered":"2023-11-28T12:01:43.945759+0000","last_unstale":"2023-11-28T12:01:45.923227+0000","last_undegraded":"2023-11-28T12:01:45.923227+0000","last_fullsized":"2023-11-28T12:01:45.923227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:41.736417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.934867+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:01:43.934867+0000","last_peered":"2023-11-28T12:01:43.934867+0000","last_clean":"2023-11-28T12:01:43.934867+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:01:43.934867+0000","last_undegraded":"2023-11-28T12:01:43.934867+0000","last_fullsized":"2023-11-28T12:01:43.934867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938960+0000","last_change":"2023-11-28T12:01:43.496920+0000","last_active":"2023-11-28T12:01:44.938960+0000","last_peered":"2023-11-28T12:01:44.938960+0000","last_clean":"2023-11-28T12:01:44.938960+0000","last_became_active":"2023-11-28T12:01:43.495644+0000","last_became_peered":"2023-11-28T12:01:43.495644+0000","last_unstale":"2023-11-28T12:01:44.938960+0000","last_undegraded":"2023-11-28T12:01:44.938960+0000","last_fullsized":"2023-11-28T12:01:44.938960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937511+0000","last_change":"2023-11-28T12:01:43.950061+0000","last_active":"2023-11-28T12:01:44.937511+0000","last_peered":"2023-11-28T12:01:44.937511+0000","last_clean":"2023-11-28T12:01:44.937511+0000","last_became_active":"2023-11-28T12:01:43.949778+0000","last_became_peered":"2023-11-28T12:01:43.949778+0000","last_unstale":"2023-11-28T12:01:44.937511+0000","last_undegraded":"2023-11-28T12:01:44.937511+0000","last_fullsized":"2023-11-28T12:01:44.937511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:36.019691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.932921+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:01:43.932921+0000","last_peered":"2023-11-28T12:01:43.932921+0000","last_clean":"2023-11-28T12:01:43.932921+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:01:43.932921+0000","last_undegraded":"2023-11-28T12:01:43.932921+0000","last_fullsized":"2023-11-28T12:01:43.932921+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938196+0000","last_change":"2023-11-28T12:01:43.499396+0000","last_active":"2023-11-28T12:01:44.938196+0000","last_peered":"2023-11-28T12:01:44.938196+0000","last_clean":"2023-11-28T12:01:44.938196+0000","last_became_active":"2023-11-28T12:01:43.499103+0000","last_became_peered":"2023-11-28T12:01:43.499103+0000","last_unstale":"2023-11-28T12:01:44.938196+0000","last_undegraded":"2023-11-28T12:01:44.938196+0000","last_fullsized":"2023-11-28T12:01:44.938196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923174+0000","last_change":"2023-11-28T12:01:43.941086+0000","last_active":"2023-11-28T12:01:45.923174+0000","last_peered":"2023-11-28T12:01:45.923174+0000","last_clean":"2023-11-28T12:01:45.923174+0000","last_became_active":"2023-11-28T12:01:43.940781+0000","last_became_peered":"2023-11-28T12:01:43.940781+0000","last_unstale":"2023-11-28T12:01:45.923174+0000","last_undegraded":"2023-11-28T12:01:45.923174+0000","last_fullsized":"2023-11-28T12:01:45.923174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:19.706888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.935713+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:01:43.935713+0000","last_peered":"2023-11-28T12:01:43.935713+0000","last_clean":"2023-11-28T12:01:43.935713+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:01:43.935713+0000","last_undegraded":"2023-11-28T12:01:43.935713+0000","last_fullsized":"2023-11-28T12:01:43.935713+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923245+0000","last_change":"2023-11-28T12:01:43.501876+0000","last_active":"2023-11-28T12:01:45.923245+0000","last_peered":"2023-11-28T12:01:45.923245+0000","last_clean":"2023-11-28T12:01:45.923245+0000","last_became_active":"2023-11-28T12:01:43.501509+0000","last_became_peered":"2023-11-28T12:01:43.501509+0000","last_unstale":"2023-11-28T12:01:45.923245+0000","last_undegraded":"2023-11-28T12:01:45.923245+0000","last_fullsized":"2023-11-28T12:01:45.923245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497880+0000","last_change":"2023-11-28T12:01:43.947364+0000","last_active":"2023-11-28T12:01:46.497880+0000","last_peered":"2023-11-28T12:01:46.497880+0000","last_clean":"2023-11-28T12:01:46.497880+0000","last_became_active":"2023-11-28T12:01:43.947043+0000","last_became_peered":"2023-11-28T12:01:43.947043+0000","last_unstale":"2023-11-28T12:01:46.497880+0000","last_undegraded":"2023-11-28T12:01:46.497880+0000","last_fullsized":"2023-11-28T12:01:46.497880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.936792+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:01:43.936792+0000","last_peered":"2023-11-28T12:01:43.936792+0000","last_clean":"2023-11-28T12:01:43.936792+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:01:43.936792+0000","last_undegraded":"2023-11-28T12:01:43.936792+0000","last_fullsized":"2023-11-28T12:01:43.936792+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545482+0000","last_change":"2023-11-28T12:01:43.497088+0000","last_active":"2023-11-28T12:01:46.545482+0000","last_peered":"2023-11-28T12:01:46.545482+0000","last_clean":"2023-11-28T12:01:46.545482+0000","last_became_active":"2023-11-28T12:01:43.496456+0000","last_became_peered":"2023-11-28T12:01:43.496456+0000","last_unstale":"2023-11-28T12:01:46.545482+0000","last_undegraded":"2023-11-28T12:01:46.545482+0000","last_fullsized":"2023-11-28T12:01:46.545482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923037+0000","last_change":"2023-11-28T12:01:43.946582+0000","last_active":"2023-11-28T12:01:45.923037+0000","last_peered":"2023-11-28T12:01:45.923037+0000","last_clean":"2023-11-28T12:01:45.923037+0000","last_became_active":"2023-11-28T12:01:43.946281+0000","last_became_peered":"2023-11-28T12:01:43.946281+0000","last_unstale":"2023-11-28T12:01:45.923037+0000","last_undegraded":"2023-11-28T12:01:45.923037+0000","last_fullsized":"2023-11-28T12:01:45.923037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:10.298310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497709+0000","last_change":"2023-11-28T12:01:43.499312+0000","last_active":"2023-11-28T12:01:46.497709+0000","last_peered":"2023-11-28T12:01:46.497709+0000","last_clean":"2023-11-28T12:01:46.497709+0000","last_became_active":"2023-11-28T12:01:43.499020+0000","last_became_peered":"2023-11-28T12:01:43.499020+0000","last_unstale":"2023-11-28T12:01:46.497709+0000","last_undegraded":"2023-11-28T12:01:46.497709+0000","last_fullsized":"2023-11-28T12:01:46.497709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542759+0000","last_change":"2023-11-28T12:01:43.945612+0000","last_active":"2023-11-28T12:01:46.542759+0000","last_peered":"2023-11-28T12:01:46.542759+0000","last_clean":"2023-11-28T12:01:46.542759+0000","last_became_active":"2023-11-28T12:01:43.945304+0000","last_became_peered":"2023-11-28T12:01:43.945304+0000","last_unstale":"2023-11-28T12:01:46.542759+0000","last_undegraded":"2023-11-28T12:01:46.542759+0000","last_fullsized":"2023-11-28T12:01:46.542759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:20.571945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545110+0000","last_change":"2023-11-28T12:01:43.496842+0000","last_active":"2023-11-28T12:01:46.545110+0000","last_peered":"2023-11-28T12:01:46.545110+0000","last_clean":"2023-11-28T12:01:46.545110+0000","last_became_active":"2023-11-28T12:01:43.496567+0000","last_became_peered":"2023-11-28T12:01:43.496567+0000","last_unstale":"2023-11-28T12:01:46.545110+0000","last_undegraded":"2023-11-28T12:01:46.545110+0000","last_fullsized":"2023-11-28T12:01:46.545110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938343+0000","last_change":"2023-11-28T12:01:43.941160+0000","last_active":"2023-11-28T12:01:44.938343+0000","last_peered":"2023-11-28T12:01:44.938343+0000","last_clean":"2023-11-28T12:01:44.938343+0000","last_became_active":"2023-11-28T12:01:43.940593+0000","last_became_peered":"2023-11-28T12:01:43.940593+0000","last_unstale":"2023-11-28T12:01:44.938343+0000","last_undegraded":"2023-11-28T12:01:44.938343+0000","last_fullsized":"2023-11-28T12:01:44.938343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:46.302273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545507+0000","last_change":"2023-11-28T12:01:43.497657+0000","last_active":"2023-11-28T12:01:46.545507+0000","last_peered":"2023-11-28T12:01:46.545507+0000","last_clean":"2023-11-28T12:01:46.545507+0000","last_became_active":"2023-11-28T12:01:43.496814+0000","last_became_peered":"2023-11-28T12:01:43.496814+0000","last_unstale":"2023-11-28T12:01:46.545507+0000","last_undegraded":"2023-11-28T12:01:46.545507+0000","last_fullsized":"2023-11-28T12:01:46.545507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938078+0000","last_change":"2023-11-28T12:01:43.938233+0000","last_active":"2023-11-28T12:01:44.938078+0000","last_peered":"2023-11-28T12:01:44.938078+0000","last_clean":"2023-11-28T12:01:44.938078+0000","last_became_active":"2023-11-28T12:01:43.937825+0000","last_became_peered":"2023-11-28T12:01:43.937825+0000","last_unstale":"2023-11-28T12:01:44.938078+0000","last_undegraded":"2023-11-28T12:01:44.938078+0000","last_fullsized":"2023-11-28T12:01:44.938078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:04.509152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939417+0000","last_change":"2023-11-28T12:01:43.485835+0000","last_active":"2023-11-28T12:01:44.939417+0000","last_peered":"2023-11-28T12:01:44.939417+0000","last_clean":"2023-11-28T12:01:44.939417+0000","last_became_active":"2023-11-28T12:01:43.484610+0000","last_became_peered":"2023-11-28T12:01:43.484610+0000","last_unstale":"2023-11-28T12:01:44.939417+0000","last_undegraded":"2023-11-28T12:01:44.939417+0000","last_fullsized":"2023-11-28T12:01:44.939417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:40.727625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939028+0000","last_change":"2023-11-28T12:01:43.949382+0000","last_active":"2023-11-28T12:01:44.939028+0000","last_peered":"2023-11-28T12:01:44.939028+0000","last_clean":"2023-11-28T12:01:44.939028+0000","last_became_active":"2023-11-28T12:01:43.948949+0000","last_became_peered":"2023-11-28T12:01:43.948949+0000","last_unstale":"2023-11-28T12:01:44.939028+0000","last_undegraded":"2023-11-28T12:01:44.939028+0000","last_fullsized":"2023-11-28T12:01:44.939028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:35.755196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497980+0000","last_change":"2023-11-28T12:01:43.498156+0000","last_active":"2023-11-28T12:01:46.497980+0000","last_peered":"2023-11-28T12:01:46.497980+0000","last_clean":"2023-11-28T12:01:46.497980+0000","last_became_active":"2023-11-28T12:01:43.496510+0000","last_became_peered":"2023-11-28T12:01:43.496510+0000","last_unstale":"2023-11-28T12:01:46.497980+0000","last_undegraded":"2023-11-28T12:01:46.497980+0000","last_fullsized":"2023-11-28T12:01:46.497980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:46.276522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937629+0000","last_change":"2023-11-28T12:01:43.944985+0000","last_active":"2023-11-28T12:01:44.937629+0000","last_peered":"2023-11-28T12:01:44.937629+0000","last_clean":"2023-11-28T12:01:44.937629+0000","last_became_active":"2023-11-28T12:01:43.943580+0000","last_became_peered":"2023-11-28T12:01:43.943580+0000","last_unstale":"2023-11-28T12:01:44.937629+0000","last_undegraded":"2023-11-28T12:01:44.937629+0000","last_fullsized":"2023-11-28T12:01:44.937629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937476+0000","last_change":"2023-11-28T12:01:43.941917+0000","last_active":"2023-11-28T12:01:44.937476+0000","last_peered":"2023-11-28T12:01:44.937476+0000","last_clean":"2023-11-28T12:01:44.937476+0000","last_became_active":"2023-11-28T12:01:43.941625+0000","last_became_peered":"2023-11-28T12:01:43.941625+0000","last_unstale":"2023-11-28T12:01:44.937476+0000","last_undegraded":"2023-11-28T12:01:44.937476+0000","last_fullsized":"2023-11-28T12:01:44.937476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:23.540287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938902+0000","last_change":"2023-11-28T12:01:43.943568+0000","last_active":"2023-11-28T12:01:44.938902+0000","last_peered":"2023-11-28T12:01:44.938902+0000","last_clean":"2023-11-28T12:01:44.938902+0000","last_became_active":"2023-11-28T12:01:43.943204+0000","last_became_peered":"2023-11-28T12:01:43.943204+0000","last_unstale":"2023-11-28T12:01:44.938902+0000","last_undegraded":"2023-11-28T12:01:44.938902+0000","last_fullsized":"2023-11-28T12:01:44.938902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937986+0000","last_change":"2023-11-28T12:01:43.943816+0000","last_active":"2023-11-28T12:01:44.937986+0000","last_peered":"2023-11-28T12:01:44.937986+0000","last_clean":"2023-11-28T12:01:44.937986+0000","last_became_active":"2023-11-28T12:01:43.943506+0000","last_became_peered":"2023-11-28T12:01:43.943506+0000","last_unstale":"2023-11-28T12:01:44.937986+0000","last_undegraded":"2023-11-28T12:01:44.937986+0000","last_fullsized":"2023-11-28T12:01:44.937986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:28.088904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939262+0000","last_change":"2023-11-28T12:01:43.950413+0000","last_active":"2023-11-28T12:01:44.939262+0000","last_peered":"2023-11-28T12:01:44.939262+0000","last_clean":"2023-11-28T12:01:44.939262+0000","last_became_active":"2023-11-28T12:01:43.950110+0000","last_became_peered":"2023-11-28T12:01:43.950110+0000","last_unstale":"2023-11-28T12:01:44.939262+0000","last_undegraded":"2023-11-28T12:01:44.939262+0000","last_fullsized":"2023-11-28T12:01:44.939262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:57.245235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922987+0000","last_change":"2023-11-28T12:01:43.938293+0000","last_active":"2023-11-28T12:01:45.922987+0000","last_peered":"2023-11-28T12:01:45.922987+0000","last_clean":"2023-11-28T12:01:45.922987+0000","last_became_active":"2023-11-28T12:01:43.937951+0000","last_became_peered":"2023-11-28T12:01:43.937951+0000","last_unstale":"2023-11-28T12:01:45.922987+0000","last_undegraded":"2023-11-28T12:01:45.922987+0000","last_fullsized":"2023-11-28T12:01:45.922987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9084,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362756,"statfs":{"total":96636764160,"available":96627462144,"internally_reserved":0,"allocated":839680,"data_stored":671592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7900,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363940,"statfs":{"total":96636764160,"available":96628674560,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9056,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362784,"statfs":{"total":96636764160,"available":96627490816,"internally_reserved":0,"allocated":811008,"data_stored":669828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7956,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363884,"statfs":{"total":96636764160,"available":96628617216,"internally_reserved":0,"allocated":208896,"data_stored":73373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7888,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363952,"statfs":{"total":96636764160,"available":96628686848,"internally_reserved":0,"allocated":204800,"data_stored":72807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8032,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363808,"statfs":{"total":96636764160,"available":96628539392,"internally_reserved":0,"allocated":221184,"data_stored":79550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8048,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363792,"statfs":{"total":96636764160,"available":96628523008,"internally_reserved":0,"allocated":237568,"data_stored":81064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7972,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363868,"statfs":{"total":96636764160,"available":96628600832,"internally_reserved":0,"allocated":225280,"data_stored":80096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:01:48.525 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T12:01:48.525 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-28T12:01:48.525 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-28T12:01:48.525 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T12:01:48.526 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-28T12:01:48.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.697+0000 7f3d59caa700 1 Processor -- start 2023-11-28T12:01:48.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.697+0000 7f3d59caa700 1 -- start start 2023-11-28T12:01:48.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.698+0000 7f3d59caa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d5406a270 0x7f3d5406a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:48.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.698+0000 7f3d59caa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d54070d90 0x7f3d540711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:48.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.698+0000 7f3d59caa700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3d5406da40 0x7f3d5406dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:48.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.698+0000 7f3d59caa700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d54151a50 con 0x7f3d5406da40 2023-11-28T12:01:48.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.698+0000 7f3d59caa700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5415b440 con 0x7f3d5406a270 2023-11-28T12:01:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.698+0000 7f3d59caa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5406ab80 con 0x7f3d54070d90 2023-11-28T12:01:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.698+0000 7f3d53fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3d5406da40 0x7f3d5406dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.698+0000 7f3d53fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3d5406da40 0x7f3d5406dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47790/0 (socket says 172.21.15.57:47790) 2023-11-28T12:01:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.698+0000 7f3d53fff700 1 -- 172.21.15.57:0/700941581 learned_addr learned my addr 172.21.15.57:0/700941581 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.699+0000 7f3d537fe700 1 --2- 172.21.15.57:0/700941581 >> v2:172.21.15.181:3300/0 conn(0x7f3d5406a270 0x7f3d5406a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.699+0000 7f3d52ffd700 1 --2- 172.21.15.57:0/700941581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d54070d90 0x7f3d540711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.699+0000 7f3d53fff700 1 -- 172.21.15.57:0/700941581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d54070d90 msgr2=0x7f3d540711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.699+0000 7f3d53fff700 1 --2- 172.21.15.57:0/700941581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d54070d90 0x7f3d540711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.699+0000 7f3d53fff700 1 -- 172.21.15.57:0/700941581 >> v2:172.21.15.181:3300/0 conn(0x7f3d5406a270 msgr2=0x7f3d5406a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.699+0000 7f3d53fff700 1 --2- 172.21.15.57:0/700941581 >> v2:172.21.15.181:3300/0 conn(0x7f3d5406a270 0x7f3d5406a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.699+0000 7f3d53fff700 1 -- 172.21.15.57:0/700941581 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d54071ac0 con 0x7f3d5406da40 2023-11-28T12:01:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.699+0000 7f3d53fff700 1 --2- 172.21.15.57:0/700941581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d5406da40 0x7f3d5406dec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d540cd250 tx=0x7f3d4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79429f6124d2bb50 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:48.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.699+0000 7f3d527fc700 1 -- 172.21.15.57:0/700941581 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c00b870 con 0x7f3d5406da40 2023-11-28T12:01:48.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.699+0000 7f3d527fc700 1 -- 172.21.15.57:0/700941581 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c00b9d0 con 0x7f3d5406da40 2023-11-28T12:01:48.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.699+0000 7f3d527fc700 1 -- 172.21.15.57:0/700941581 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c00bd60 con 0x7f3d5406da40 2023-11-28T12:01:48.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.700+0000 7f3d59caa700 1 -- 172.21.15.57:0/700941581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d5406da40 msgr2=0x7f3d5406dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:48.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.700+0000 7f3d59caa700 1 --2- 172.21.15.57:0/700941581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d5406da40 0x7f3d5406dec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d540cd250 tx=0x7f3d4c009bd0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.700+0000 7f3d59caa700 1 -- 172.21.15.57:0/700941581 shutdown_connections 2023-11-28T12:01:48.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.700+0000 7f3d59caa700 1 --2- 172.21.15.57:0/700941581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d5406da40 0x7f3d5406dec0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.700+0000 7f3d59caa700 1 --2- 172.21.15.57:0/700941581 >> v2:172.21.15.181:3300/0 conn(0x7f3d5406a270 0x7f3d5406a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.700+0000 7f3d59caa700 1 --2- 172.21.15.57:0/700941581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d54070d90 0x7f3d540711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.700+0000 7f3d59caa700 1 -- 172.21.15.57:0/700941581 >> 172.21.15.57:0/700941581 conn(0x7f3d5406c010 msgr2=0x7f3d540cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:48.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.700+0000 7f3d59caa700 1 -- 172.21.15.57:0/700941581 shutdown_connections 2023-11-28T12:01:48.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.700+0000 7f3d59caa700 1 -- 172.21.15.57:0/700941581 wait complete. 2023-11-28T12:01:48.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.701+0000 7f3d59caa700 1 Processor -- start 2023-11-28T12:01:48.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.701+0000 7f3d59caa700 1 -- start start 2023-11-28T12:01:48.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.702+0000 7f3d59caa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5406a270 0x7f3d540c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:48.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.702+0000 7f3d59caa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d5406da40 0x7f3d54150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:48.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.702+0000 7f3d59caa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d54070d90 0x7f3d54153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:48.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.702+0000 7f3d59caa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d540c6600 con 0x7f3d5406da40 2023-11-28T12:01:48.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.702+0000 7f3d59caa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d540c8090 con 0x7f3d5406a270 2023-11-28T12:01:48.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.702+0000 7f3d59caa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d540c8370 con 0x7f3d54070d90 2023-11-28T12:01:48.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.702+0000 7f3d52ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d5406da40 0x7f3d54150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:48.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.702+0000 7f3d52ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d5406da40 0x7f3d54150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47798/0 (socket says 172.21.15.57:47798) 2023-11-28T12:01:48.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.702+0000 7f3d52ffd700 1 -- 172.21.15.57:0/1893583460 learned_addr learned my addr 172.21.15.57:0/1893583460 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:48.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.702+0000 7f3d537fe700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5406a270 0x7f3d540c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:48.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.702+0000 7f3d53fff700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d54070d90 0x7f3d54153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:48.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.703+0000 7f3d52ffd700 1 -- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d54070d90 msgr2=0x7f3d54153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:48.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.703+0000 7f3d52ffd700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d54070d90 0x7f3d54153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.703+0000 7f3d52ffd700 1 -- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5406a270 msgr2=0x7f3d540c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:48.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.703+0000 7f3d52ffd700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5406a270 0x7f3d540c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.703+0000 7f3d52ffd700 1 -- 172.21.15.57:0/1893583460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c009550 con 0x7f3d5406da40 2023-11-28T12:01:48.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.703+0000 7f3d52ffd700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d5406da40 0x7f3d54150b80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00d800 tx=0x7f3d3c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:48.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.703+0000 7f3d4bfff700 1 -- 172.21.15.57:0/1893583460 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0125f0 con 0x7f3d5406da40 2023-11-28T12:01:48.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.703+0000 7f3d4bfff700 1 -- 172.21.15.57:0/1893583460 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c012750 con 0x7f3d5406da40 2023-11-28T12:01:48.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.703+0000 7f3d59caa700 1 -- 172.21.15.57:0/1893583460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5415b0d0 con 0x7f3d5406da40 2023-11-28T12:01:48.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.703+0000 7f3d4bfff700 1 -- 172.21.15.57:0/1893583460 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c012b00 con 0x7f3d5406da40 2023-11-28T12:01:48.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.703+0000 7f3d59caa700 1 -- 172.21.15.57:0/1893583460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5415adf0 con 0x7f3d5406da40 2023-11-28T12:01:48.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.704+0000 7f3d59caa700 1 -- 172.21.15.57:0/1893583460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5415aac0 con 0x7f3d5406da40 2023-11-28T12:01:48.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.704+0000 7f3d4bfff700 1 -- 172.21.15.57:0/1893583460 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3d3c012c60 con 0x7f3d5406da40 2023-11-28T12:01:48.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.705+0000 7f3d4bfff700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d340682f0 0x7f3d3406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:48.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.705+0000 7f3d4bfff700 1 -- 172.21.15.57:0/1893583460 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3d3c085e70 con 0x7f3d5406da40 2023-11-28T12:01:48.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.705+0000 7f3d537fe700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d340682f0 0x7f3d3406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:48.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.706+0000 7f3d537fe700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d340682f0 0x7f3d3406a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3d44003b30 tx=0x7f3d44009830 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:48.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.709+0000 7f3d4bfff700 1 -- 172.21.15.57:0/1893583460 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3d3c0526d0 con 0x7f3d5406da40 2023-11-28T12:01:48.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.974+0000 7f3d59caa700 1 -- 172.21.15.57:0/1893583460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3d540b1f70 con 0x7f3d5406da40 2023-11-28T12:01:48.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.977+0000 7f3d4bfff700 1 -- 172.21.15.57:0/1893583460 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+166384 (secure 0 0 0) 0x7f3d3c055cf0 con 0x7f3d5406da40 2023-11-28T12:01:48.983 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:48.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.980+0000 7f3d59caa700 1 -- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d340682f0 msgr2=0x7f3d3406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:48.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.980+0000 7f3d59caa700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d340682f0 0x7f3d3406a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3d44003b30 tx=0x7f3d44009830 comp rx=0 tx=0).stop 2023-11-28T12:01:48.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.980+0000 7f3d59caa700 1 -- 172.21.15.57:0/1893583460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d5406da40 msgr2=0x7f3d54150b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:48.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.980+0000 7f3d59caa700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d5406da40 0x7f3d54150b80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00d800 tx=0x7f3d3c00db10 comp rx=0 tx=0).stop 2023-11-28T12:01:48.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.980+0000 7f3d59caa700 1 -- 172.21.15.57:0/1893583460 shutdown_connections 2023-11-28T12:01:48.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.980+0000 7f3d59caa700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d340682f0 0x7f3d3406a7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.980+0000 7f3d59caa700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d54070d90 0x7f3d54153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.980+0000 7f3d59caa700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5406a270 0x7f3d540c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.980+0000 7f3d59caa700 1 --2- 172.21.15.57:0/1893583460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d5406da40 0x7f3d54150b80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:48.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.980+0000 7f3d59caa700 1 -- 172.21.15.57:0/1893583460 >> 172.21.15.57:0/1893583460 conn(0x7f3d5406c010 msgr2=0x7f3d5406de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:48.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.980+0000 7f3d59caa700 1 -- 172.21.15.57:0/1893583460 shutdown_connections 2023-11-28T12:01:48.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:48.980+0000 7f3d59caa700 1 -- 172.21.15.57:0/1893583460 wait complete. 2023-11-28T12:01:48.994 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":5,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":120259},{"type":"v1","addr":"172.21.15.181:6833","nonce":120259}]},"active_addr":"172.21.15.181:6833/120259","active_change":"2023-11-28T12:01:31.639204+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":870090499}]},{"addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":1324657400}]},{"addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2274040570}]},{"addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":488314645}]}]} 2023-11-28T12:01:48.998 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-28T12:01:48.998 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-28T12:01:48.999 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:01:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.132+0000 7f94bb323700 1 Processor -- start 2023-11-28T12:01:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.133+0000 7f94bb323700 1 -- start start 2023-11-28T12:01:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94bb323700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94b40d15f0 0x7f94b415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94bb323700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f94b415e010 0x7f94b40ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94bb323700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b415af30 con 0x7f94b406e400 2023-11-28T12:01:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94bb323700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b415a9d0 con 0x7f94b415e010 2023-11-28T12:01:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94bb323700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b41584e0 con 0x7f94b40d15f0 2023-11-28T12:01:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94b98c0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f94b415e010 0x7f94b40ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94b90bf700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f94b406e400 0x7f94b406e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36928/0 (socket says 172.21.15.57:36928) 2023-11-28T12:01:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94b90bf700 1 -- 172.21.15.57:0/742138310 learned_addr learned my addr 172.21.15.57:0/742138310 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94b88be700 1 --2- 172.21.15.57:0/742138310 >> v2:172.21.15.181:3300/0 conn(0x7f94b40d15f0 0x7f94b415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:49.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94b98c0700 1 -- 172.21.15.57:0/742138310 >> v2:172.21.15.181:3300/0 conn(0x7f94b40d15f0 msgr2=0x7f94b415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:49.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94b98c0700 1 --2- 172.21.15.57:0/742138310 >> v2:172.21.15.181:3300/0 conn(0x7f94b40d15f0 0x7f94b415b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94b98c0700 1 -- 172.21.15.57:0/742138310 >> v1:172.21.15.57:6789/0 conn(0x7f94b406e400 legacy=0x7f94b406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:49.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94b98c0700 1 -- 172.21.15.57:0/742138310 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b415ef40 con 0x7f94b415e010 2023-11-28T12:01:49.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94abfff700 1 -- 172.21.15.57:0/742138310 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414166919 0 0) 0x7f94b415af30 con 0x7f94b406e400 2023-11-28T12:01:49.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94b98c0700 1 --2- 172.21.15.57:0/742138310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94b415e010 0x7f94b40ce970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f94b40ce6e0 tx=0x7f94ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6214c727fbecb66a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:49.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.134+0000 7f94abfff700 1 -- 172.21.15.57:0/742138310 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac00e070 con 0x7f94b415e010 2023-11-28T12:01:49.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.135+0000 7f94abfff700 1 -- 172.21.15.57:0/742138310 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac00bbd0 con 0x7f94b415e010 2023-11-28T12:01:49.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.135+0000 7f94abfff700 1 -- 172.21.15.57:0/742138310 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac012430 con 0x7f94b415e010 2023-11-28T12:01:49.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.135+0000 7f94bb323700 1 -- 172.21.15.57:0/742138310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94b415e010 msgr2=0x7f94b40ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:49.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.135+0000 7f94bb323700 1 --2- 172.21.15.57:0/742138310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94b415e010 0x7f94b40ce970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f94b40ce6e0 tx=0x7f94ac00b330 comp rx=0 tx=0).stop 2023-11-28T12:01:49.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.135+0000 7f94bb323700 1 -- 172.21.15.57:0/742138310 shutdown_connections 2023-11-28T12:01:49.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.135+0000 7f94bb323700 1 --2- 172.21.15.57:0/742138310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94b415e010 0x7f94b40ce970 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.135+0000 7f94bb323700 1 --2- 172.21.15.57:0/742138310 >> v2:172.21.15.181:3300/0 conn(0x7f94b40d15f0 0x7f94b415b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.135+0000 7f94bb323700 1 -- 172.21.15.57:0/742138310 >> 172.21.15.57:0/742138310 conn(0x7f94b406a270 msgr2=0x7f94b406a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:49.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.135+0000 7f94bb323700 1 -- 172.21.15.57:0/742138310 shutdown_connections 2023-11-28T12:01:49.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.135+0000 7f94bb323700 1 -- 172.21.15.57:0/742138310 wait complete. 2023-11-28T12:01:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.137+0000 7f94bb323700 1 Processor -- start 2023-11-28T12:01:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.137+0000 7f94bb323700 1 -- start start 2023-11-28T12:01:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.137+0000 7f94bb323700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94b406e400 0x7f94b41f1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.137+0000 7f94bb323700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94b40d15f0 0x7f94b412e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.137+0000 7f94bb323700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94b415e010 0x7f94b415d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.137+0000 7f94bb323700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b415daa0 con 0x7f94b415e010 2023-11-28T12:01:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.137+0000 7f94bb323700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b41f0730 con 0x7f94b40d15f0 2023-11-28T12:01:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.137+0000 7f94bb323700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b41f0a10 con 0x7f94b406e400 2023-11-28T12:01:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.137+0000 7f94b98c0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94b415e010 0x7f94b415d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94b98c0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94b415e010 0x7f94b415d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47814/0 (socket says 172.21.15.57:47814) 2023-11-28T12:01:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94b98c0700 1 -- 172.21.15.57:0/1582253613 learned_addr learned my addr 172.21.15.57:0/1582253613 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94b88be700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94b40d15f0 0x7f94b412e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94b90bf700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94b406e400 0x7f94b41f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94b98c0700 1 -- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94b406e400 msgr2=0x7f94b41f1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94b98c0700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94b406e400 0x7f94b41f1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94b98c0700 1 -- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94b40d15f0 msgr2=0x7f94b412e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94b98c0700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94b40d15f0 0x7f94b412e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94b98c0700 1 -- 172.21.15.57:0/1582253613 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac009a70 con 0x7f94b415e010 2023-11-28T12:01:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94b98c0700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94b415e010 0x7f94b415d030 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94ac014040 tx=0x7f94ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94a9ffb700 1 -- 172.21.15.57:0/1582253613 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac012430 con 0x7f94b415e010 2023-11-28T12:01:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94a9ffb700 1 -- 172.21.15.57:0/1582253613 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac020070 con 0x7f94b415e010 2023-11-28T12:01:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94bb323700 1 -- 172.21.15.57:0/1582253613 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b415ec60 con 0x7f94b415e010 2023-11-28T12:01:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94a9ffb700 1 -- 172.21.15.57:0/1582253613 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac004280 con 0x7f94b415e010 2023-11-28T12:01:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.138+0000 7f94bb323700 1 -- 172.21.15.57:0/1582253613 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b415e980 con 0x7f94b415e010 2023-11-28T12:01:49.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.139+0000 7f94bb323700 1 -- 172.21.15.57:0/1582253613 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b415abc0 con 0x7f94b415e010 2023-11-28T12:01:49.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.140+0000 7f94a9ffb700 1 -- 172.21.15.57:0/1582253613 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f94ac02e070 con 0x7f94b415e010 2023-11-28T12:01:49.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.141+0000 7f94a9ffb700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f94980682f0 0x7f949806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:49.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.141+0000 7f94a9ffb700 1 -- 172.21.15.57:0/1582253613 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f94ac022070 con 0x7f94b415e010 2023-11-28T12:01:49.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.141+0000 7f94b90bf700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f94980682f0 0x7f949806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:49.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.142+0000 7f94b90bf700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f94980682f0 0x7f949806a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94a4000f60 tx=0x7f94a4007be0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.145+0000 7f94a9ffb700 1 -- 172.21.15.57:0/1582253613 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f94ac054a50 con 0x7f94b415e010 2023-11-28T12:01:49.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.345+0000 7f94bb323700 1 -- 172.21.15.57:0/1582253613 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94b41f0f00 con 0x7f94b415e010 2023-11-28T12:01:49.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.346+0000 7f94a9ffb700 1 -- 172.21.15.57:0/1582253613 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7f94ac058070 con 0x7f94b415e010 2023-11-28T12:01:49.348 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:49.348 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:01:49.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.349+0000 7f94bb323700 1 -- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f94980682f0 msgr2=0x7f949806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:49.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.349+0000 7f94bb323700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f94980682f0 0x7f949806a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94a4000f60 tx=0x7f94a4007be0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.349+0000 7f94bb323700 1 -- 172.21.15.57:0/1582253613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94b415e010 msgr2=0x7f94b415d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:49.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.349+0000 7f94bb323700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94b415e010 0x7f94b415d030 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94ac014040 tx=0x7f94ac004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.349+0000 7f94bb323700 1 -- 172.21.15.57:0/1582253613 shutdown_connections 2023-11-28T12:01:49.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.349+0000 7f94bb323700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f94980682f0 0x7f949806a7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.349+0000 7f94bb323700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94b415e010 0x7f94b415d030 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.349+0000 7f94bb323700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94b406e400 0x7f94b41f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.349+0000 7f94bb323700 1 --2- 172.21.15.57:0/1582253613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94b40d15f0 0x7f94b412e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.349+0000 7f94bb323700 1 -- 172.21.15.57:0/1582253613 >> 172.21.15.57:0/1582253613 conn(0x7f94b406a270 msgr2=0x7f94b4156960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:49.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.349+0000 7f94bb323700 1 -- 172.21.15.57:0/1582253613 shutdown_connections 2023-11-28T12:01:49.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.349+0000 7f94bb323700 1 -- 172.21.15.57:0/1582253613 wait complete. 2023-11-28T12:01:49.364 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-28T12:01:49.364 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:01:49.378 INFO:tasks.ceph.mds.c.smithi057.stderr:2023-11-28T12:01:49.377+0000 7f0635dad700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T12:01:49.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.498+0000 7fd117884700 1 Processor -- start 2023-11-28T12:01:49.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.499+0000 7fd117884700 1 -- start start 2023-11-28T12:01:49.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.500+0000 7fd117884700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd110160e70 0x7fd1100b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:49.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.500+0000 7fd117884700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd11006e900 con 0x7fd1100d0c60 2023-11-28T12:01:49.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.500+0000 7fd117884700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd110160390 con 0x7fd11006e400 2023-11-28T12:01:49.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.500+0000 7fd117884700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd110160670 con 0x7fd110160e70 2023-11-28T12:01:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.500+0000 7fd115620700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd1100d0c60 0x7fd1100d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36942/0 (socket says 172.21.15.57:36942) 2023-11-28T12:01:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.500+0000 7fd115e21700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd110160e70 0x7fd1100b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.500+0000 7fd115620700 1 -- 172.21.15.57:0/894619732 learned_addr learned my addr 172.21.15.57:0/894619732 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.500+0000 7fd115e21700 1 -- 172.21.15.57:0/894619732 >> v1:172.21.15.181:6789/0 conn(0x7fd11006e400 legacy=0x7fd11015c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.500+0000 7fd115e21700 1 -- 172.21.15.57:0/894619732 >> v1:172.21.15.57:6789/0 conn(0x7fd1100d0c60 legacy=0x7fd1100d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.500+0000 7fd115e21700 1 -- 172.21.15.57:0/894619732 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd110160950 con 0x7fd110160e70 2023-11-28T12:01:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.500+0000 7fd107fff700 1 -- 172.21.15.57:0/894619732 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583998420 0 0) 0x7fd11006e900 con 0x7fd1100d0c60 2023-11-28T12:01:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.500+0000 7fd115e21700 1 --2- 172.21.15.57:0/894619732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd110160e70 0x7fd1100b5be0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd108009d90 tx=0x7fd10800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0edd6e179ca069 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.500+0000 7fd107fff700 1 -- 172.21.15.57:0/894619732 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10800e070 con 0x7fd110160e70 2023-11-28T12:01:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.501+0000 7fd107fff700 1 -- 172.21.15.57:0/894619732 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10800bbd0 con 0x7fd110160e70 2023-11-28T12:01:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.501+0000 7fd107fff700 1 -- 172.21.15.57:0/894619732 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108012430 con 0x7fd110160e70 2023-11-28T12:01:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.501+0000 7fd117884700 1 -- 172.21.15.57:0/894619732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd110160e70 msgr2=0x7fd1100b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:49.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.501+0000 7fd117884700 1 --2- 172.21.15.57:0/894619732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd110160e70 0x7fd1100b5be0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd108009d90 tx=0x7fd10800b330 comp rx=0 tx=0).stop 2023-11-28T12:01:49.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.501+0000 7fd117884700 1 -- 172.21.15.57:0/894619732 shutdown_connections 2023-11-28T12:01:49.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.501+0000 7fd117884700 1 --2- 172.21.15.57:0/894619732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd110160e70 0x7fd1100b5be0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.501+0000 7fd117884700 1 -- 172.21.15.57:0/894619732 >> 172.21.15.57:0/894619732 conn(0x7fd11006a270 msgr2=0x7fd11006a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:49.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.501+0000 7fd117884700 1 -- 172.21.15.57:0/894619732 shutdown_connections 2023-11-28T12:01:49.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.501+0000 7fd117884700 1 -- 172.21.15.57:0/894619732 wait complete. 2023-11-28T12:01:49.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.502+0000 7fd117884700 1 Processor -- start 2023-11-28T12:01:49.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.502+0000 7fd117884700 1 -- start start 2023-11-28T12:01:49.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.503+0000 7fd117884700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd11006e400 0x7fd110149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:49.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.503+0000 7fd117884700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1100d0c60 0x7fd110153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:49.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.503+0000 7fd117884700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd110160e70 0x7fd11015f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.503+0000 7fd117884700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd110154f50 con 0x7fd110160e70 2023-11-28T12:01:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.503+0000 7fd117884700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1101551a0 con 0x7fd1100d0c60 2023-11-28T12:01:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.503+0000 7fd117884700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd110155480 con 0x7fd11006e400 2023-11-28T12:01:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.503+0000 7fd115e21700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd110160e70 0x7fd11015f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.503+0000 7fd115e21700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd110160e70 0x7fd11015f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47832/0 (socket says 172.21.15.57:47832) 2023-11-28T12:01:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.503+0000 7fd115e21700 1 -- 172.21.15.57:0/344771299 learned_addr learned my addr 172.21.15.57:0/344771299 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.503+0000 7fd114e1f700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1100d0c60 0x7fd110153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.503+0000 7fd115620700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd11006e400 0x7fd110149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.504+0000 7fd115e21700 1 -- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd11006e400 msgr2=0x7fd110149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.504+0000 7fd115e21700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd11006e400 0x7fd110149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.504+0000 7fd115e21700 1 -- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1100d0c60 msgr2=0x7fd110153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:49.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.504+0000 7fd115e21700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1100d0c60 0x7fd110153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.504+0000 7fd115e21700 1 -- 172.21.15.57:0/344771299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd108009a70 con 0x7fd110160e70 2023-11-28T12:01:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.504+0000 7fd115e21700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd110160e70 0x7fd11015f460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd108014040 tx=0x7fd108004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.504+0000 7fd105ffb700 1 -- 172.21.15.57:0/344771299 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108012430 con 0x7fd110160e70 2023-11-28T12:01:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.504+0000 7fd105ffb700 1 -- 172.21.15.57:0/344771299 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd108020070 con 0x7fd110160e70 2023-11-28T12:01:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.504+0000 7fd117884700 1 -- 172.21.15.57:0/344771299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11015d080 con 0x7fd110160e70 2023-11-28T12:01:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.504+0000 7fd105ffb700 1 -- 172.21.15.57:0/344771299 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108004280 con 0x7fd110160e70 2023-11-28T12:01:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.504+0000 7fd117884700 1 -- 172.21.15.57:0/344771299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1100d0740 con 0x7fd110160e70 2023-11-28T12:01:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.505+0000 7fd117884700 1 -- 172.21.15.57:0/344771299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11015d210 con 0x7fd110160e70 2023-11-28T12:01:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.506+0000 7fd105ffb700 1 -- 172.21.15.57:0/344771299 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd10802e070 con 0x7fd110160e70 2023-11-28T12:01:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.506+0000 7fd105ffb700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0f4068320 0x7fd0f406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.506+0000 7fd105ffb700 1 -- 172.21.15.57:0/344771299 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd108022070 con 0x7fd110160e70 2023-11-28T12:01:49.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.507+0000 7fd115620700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0f4068320 0x7fd0f406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:49.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.507+0000 7fd115620700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0f4068320 0x7fd0f406a7d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd100007b10 tx=0x7fd100007a60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:49.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.511+0000 7fd105ffb700 1 -- 172.21.15.57:0/344771299 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd1080541f0 con 0x7fd110160e70 2023-11-28T12:01:49.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.712+0000 7fd117884700 1 -- 172.21.15.57:0/344771299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1100d1560 con 0x7fd110160e70 2023-11-28T12:01:49.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.713+0000 7fd105ffb700 1 -- 172.21.15.57:0/344771299 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7fd108057810 con 0x7fd110160e70 2023-11-28T12:01:49.714 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:49.715 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:01:49.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.715+0000 7fd117884700 1 -- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0f4068320 msgr2=0x7fd0f406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:49.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.715+0000 7fd117884700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0f4068320 0x7fd0f406a7d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd100007b10 tx=0x7fd100007a60 comp rx=0 tx=0).stop 2023-11-28T12:01:49.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.715+0000 7fd117884700 1 -- 172.21.15.57:0/344771299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd110160e70 msgr2=0x7fd11015f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:49.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.715+0000 7fd117884700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd110160e70 0x7fd11015f460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd108014040 tx=0x7fd108004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.715+0000 7fd117884700 1 -- 172.21.15.57:0/344771299 shutdown_connections 2023-11-28T12:01:49.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.715+0000 7fd117884700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0f4068320 0x7fd0f406a7d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.715+0000 7fd117884700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd110160e70 0x7fd11015f460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.715+0000 7fd117884700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd11006e400 0x7fd110149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.715+0000 7fd117884700 1 --2- 172.21.15.57:0/344771299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1100d0c60 0x7fd110153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:49.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.716+0000 7fd117884700 1 -- 172.21.15.57:0/344771299 >> 172.21.15.57:0/344771299 conn(0x7fd11006a270 msgr2=0x7fd1101528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:49.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.716+0000 7fd117884700 1 -- 172.21.15.57:0/344771299 shutdown_connections 2023-11-28T12:01:49.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:49.716+0000 7fd117884700 1 -- 172.21.15.57:0/344771299 wait complete. 2023-11-28T12:01:49.731 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-28T12:01:49.731 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-28T12:01:49.732 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-28T12:01:49.732 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-28T12:01:49.732 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-28T12:01:49.733 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-28T12:01:49.733 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-28T12:01:49.733 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-28T12:01:50.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.005+0000 7f557eec4700 1 Processor -- start 2023-11-28T12:01:50.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.005+0000 7f557eec4700 1 -- start start 2023-11-28T12:01:50.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.006+0000 7f557eec4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5578158e10 0x7f55780cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.006+0000 7f557eec4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f557806cb50 0x7f5578070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.006+0000 7f557eec4700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55780ca140 con 0x7f55780ae850 2023-11-28T12:01:50.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.006+0000 7f557eec4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f557806ac30 con 0x7f5578158e10 2023-11-28T12:01:50.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.006+0000 7f557eec4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55780c9b80 con 0x7f557806cb50 2023-11-28T12:01:50.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.007+0000 7f557d461700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f557806cb50 0x7f5578070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.007+0000 7f557d461700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f557806cb50 0x7f5578070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:60696/0 (socket says 172.21.15.57:60696) 2023-11-28T12:01:50.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.007+0000 7f557d461700 1 -- 172.21.15.57:0/3600037788 learned_addr learned my addr 172.21.15.57:0/3600037788 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.007+0000 7f557d461700 1 -- 172.21.15.57:0/3600037788 >> v2:172.21.15.181:3300/0 conn(0x7f5578158e10 msgr2=0x7f55780cb5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T12:01:50.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.007+0000 7f557d461700 1 --2- 172.21.15.57:0/3600037788 >> v2:172.21.15.181:3300/0 conn(0x7f5578158e10 0x7f55780cb5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.007+0000 7f557d461700 1 -- 172.21.15.57:0/3600037788 >> v1:172.21.15.57:6789/0 conn(0x7f55780ae850 legacy=0x7f557806c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.007+0000 7f557d461700 1 -- 172.21.15.57:0/3600037788 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55780c9e60 con 0x7f557806cb50 2023-11-28T12:01:50.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.008+0000 7f557d461700 1 --2- 172.21.15.57:0/3600037788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f557806cb50 0x7f5578070370 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5570009e50 tx=0x7f557000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76529574765b92ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.008+0000 7f928bbab700 1 Processor -- start 2023-11-28T12:01:50.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.009+0000 7f928bbab700 1 -- start start 2023-11-28T12:01:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.009+0000 7f55777fe700 1 -- 172.21.15.57:0/3600037788 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557000e070 con 0x7f557806cb50 2023-11-28T12:01:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.009+0000 7f55777fe700 1 -- 172.21.15.57:0/3600037788 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5570003a20 con 0x7f557806cb50 2023-11-28T12:01:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.009+0000 7f55777fe700 1 -- 172.21.15.57:0/3600037788 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570003dd0 con 0x7f557806cb50 2023-11-28T12:01:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.009+0000 7f928bbab700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f928406d230 0x7f92840adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f928bbab700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f928406b770 0x7f92840b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f928bbab700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928406bbd0 con 0x7f92840cbc50 2023-11-28T12:01:50.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f928bbab700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928406c700 con 0x7f928406d230 2023-11-28T12:01:50.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f928bbab700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9284158e20 con 0x7f928406b770 2023-11-28T12:01:50.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f557eec4700 1 -- 172.21.15.57:0/3600037788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f557806cb50 msgr2=0x7f5578070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f557eec4700 1 --2- 172.21.15.57:0/3600037788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f557806cb50 0x7f5578070370 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5570009e50 tx=0x7f557000b330 comp rx=0 tx=0).stop 2023-11-28T12:01:50.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f928a148700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f928406b770 0x7f92840b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f928a148700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f928406b770 0x7f92840b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:49298/0 (socket says 172.21.15.57:49298) 2023-11-28T12:01:50.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f928a148700 1 -- 172.21.15.57:0/957182128 learned_addr learned my addr 172.21.15.57:0/957182128 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f557eec4700 1 -- 172.21.15.57:0/3600037788 shutdown_connections 2023-11-28T12:01:50.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f557eec4700 1 --2- 172.21.15.57:0/3600037788 >> v2:172.21.15.181:3300/0 conn(0x7f5578158e10 0x7f55780cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f557eec4700 1 --2- 172.21.15.57:0/3600037788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f557806cb50 0x7f5578070370 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f557eec4700 1 -- 172.21.15.57:0/3600037788 >> 172.21.15.57:0/3600037788 conn(0x7f55780cf400 msgr2=0x7f55780cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f9289947700 1 --2- 172.21.15.57:0/957182128 >> v2:172.21.15.57:3300/0 conn(0x7f928406d230 0x7f92840adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f928a148700 1 -- 172.21.15.57:0/957182128 >> v2:172.21.15.57:3300/0 conn(0x7f928406d230 msgr2=0x7f92840adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f928a148700 1 --2- 172.21.15.57:0/957182128 >> v2:172.21.15.57:3300/0 conn(0x7f928406d230 0x7f92840adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f928a148700 1 -- 172.21.15.57:0/957182128 >> v1:172.21.15.181:6789/0 conn(0x7f92840cbc50 legacy=0x7f9284158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f928a148700 1 -- 172.21.15.57:0/957182128 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92840ca7f0 con 0x7f928406b770 2023-11-28T12:01:50.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f557eec4700 1 -- 172.21.15.57:0/3600037788 shutdown_connections 2023-11-28T12:01:50.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.010+0000 7f557eec4700 1 -- 172.21.15.57:0/3600037788 wait complete. 2023-11-28T12:01:50.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.011+0000 7f928a148700 1 --2- 172.21.15.57:0/957182128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f928406b770 0x7f92840b0540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f92840723e0 tx=0x7f927c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7be100bbf2a45ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.011+0000 7f9288945700 1 -- 172.21.15.57:0/957182128 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00e070 con 0x7f928406b770 2023-11-28T12:01:50.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.011+0000 7f9288945700 1 -- 172.21.15.57:0/957182128 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c00b9c0 con 0x7f928406b770 2023-11-28T12:01:50.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.011+0000 7f9288945700 1 -- 172.21.15.57:0/957182128 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00bd70 con 0x7f928406b770 2023-11-28T12:01:50.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.012+0000 7f557eec4700 1 Processor -- start 2023-11-28T12:01:50.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.012+0000 7f557eec4700 1 -- start start 2023-11-28T12:01:50.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.013+0000 7f557eec4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f557806cb50 0x7f55780c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.013+0000 7f557eec4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55780ae850 0x7f5578153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.013+0000 7f557eec4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5578158e10 0x7f55780c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.013+0000 7f557eec4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55780c7b60 con 0x7f557806cb50 2023-11-28T12:01:50.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.013+0000 7f557eec4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55780c7e40 con 0x7f5578158e10 2023-11-28T12:01:50.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.013+0000 7f557eec4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55780c8120 con 0x7f55780ae850 2023-11-28T12:01:50.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.014+0000 7f5577fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55780ae850 0x7f5578153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.014+0000 7f557d461700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5578158e10 0x7f55780c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.014+0000 7f5577fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55780ae850 0x7f5578153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:60704/0 (socket says 172.21.15.57:60704) 2023-11-28T12:01:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.014+0000 7f5577fff700 1 -- 172.21.15.57:0/916540593 learned_addr learned my addr 172.21.15.57:0/916540593 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.014+0000 7f557d461700 1 -- 172.21.15.57:0/916540593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55780ae850 msgr2=0x7f5578153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.014+0000 7f557d461700 1 --2- 172.21.15.57:0/916540593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55780ae850 0x7f5578153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.014+0000 7f557d461700 1 -- 172.21.15.57:0/916540593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f557806cb50 msgr2=0x7f55780c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.014+0000 7f557d461700 1 --2- 172.21.15.57:0/916540593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f557806cb50 0x7f55780c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.014+0000 7f557d461700 1 -- 172.21.15.57:0/916540593 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5570009b30 con 0x7f5578158e10 2023-11-28T12:01:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.014+0000 7f5577fff700 1 --2- 172.21.15.57:0/916540593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55780ae850 0x7f5578153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.014+0000 7f557d461700 1 --2- 172.21.15.57:0/916540593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5578158e10 0x7f55780c6fe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5570014000 tx=0x7f5570004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.015+0000 7f55757fa700 1 -- 172.21.15.57:0/916540593 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570004440 con 0x7f5578158e10 2023-11-28T12:01:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.015+0000 7f55757fa700 1 -- 172.21.15.57:0/916540593 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55700045e0 con 0x7f5578158e10 2023-11-28T12:01:50.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.015+0000 7f557eec4700 1 -- 172.21.15.57:0/916540593 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557806b100 con 0x7f5578158e10 2023-11-28T12:01:50.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.015+0000 7f55757fa700 1 -- 172.21.15.57:0/916540593 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570012430 con 0x7f5578158e10 2023-11-28T12:01:50.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.015+0000 7f557eec4700 1 -- 172.21.15.57:0/916540593 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557806ae20 con 0x7f5578158e10 2023-11-28T12:01:50.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.012+0000 7f928bbab700 1 -- 172.21.15.57:0/957182128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f928406b770 msgr2=0x7f92840b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.012+0000 7f928bbab700 1 --2- 172.21.15.57:0/957182128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f928406b770 0x7f92840b0540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f92840723e0 tx=0x7f927c00b330 comp rx=0 tx=0).stop 2023-11-28T12:01:50.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.013+0000 7f928bbab700 1 -- 172.21.15.57:0/957182128 shutdown_connections 2023-11-28T12:01:50.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.013+0000 7f928bbab700 1 --2- 172.21.15.57:0/957182128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f928406b770 0x7f92840b0540 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.013+0000 7f928bbab700 1 --2- 172.21.15.57:0/957182128 >> v2:172.21.15.57:3300/0 conn(0x7f928406d230 0x7f92840adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.013+0000 7f928bbab700 1 -- 172.21.15.57:0/957182128 >> 172.21.15.57:0/957182128 conn(0x7f928406aed0 msgr2=0x7f92840cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.013+0000 7f928bbab700 1 -- 172.21.15.57:0/957182128 shutdown_connections 2023-11-28T12:01:50.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.013+0000 7f928bbab700 1 -- 172.21.15.57:0/957182128 wait complete. 2023-11-28T12:01:50.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.015+0000 7f928bbab700 1 Processor -- start 2023-11-28T12:01:50.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.015+0000 7fa6551a0700 1 Processor -- start 2023-11-28T12:01:50.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.015+0000 7fa6551a0700 1 -- start start 2023-11-28T12:01:50.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.017+0000 7fa6551a0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa65006cdd0 0x7fa6500b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.017+0000 7fa6551a0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6500b0a80 0x7fa65006bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.017+0000 7fa6551a0700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6500cc690 con 0x7fa6501609c0 2023-11-28T12:01:50.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.017+0000 7fa6551a0700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6500cc8c0 con 0x7fa65006cdd0 2023-11-28T12:01:50.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.017+0000 7fa6551a0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500caa30 con 0x7fa6500b0a80 2023-11-28T12:01:50.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.015+0000 7f8df6edb700 1 Processor -- start 2023-11-28T12:01:50.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.015+0000 7f8df6edb700 1 -- start start 2023-11-28T12:01:50.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.016+0000 7f8df6edb700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8df00ae020 0x7f8df006c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.017+0000 7f8df6edb700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df00cb160 con 0x7f8df006c930 2023-11-28T12:01:50.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.017+0000 7f8df6edb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df006b910 con 0x7f8df008ac20 2023-11-28T12:01:50.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.017+0000 7f8df6edb700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df006bbf0 con 0x7f8df00ae020 2023-11-28T12:01:50.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.017+0000 7f55757fa700 1 -- 172.21.15.57:0/916540593 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5570012590 con 0x7f5578158e10 2023-11-28T12:01:50.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.016+0000 7f928bbab700 1 -- start start 2023-11-28T12:01:50.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.017+0000 7f8deffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8df008ac20 0x7f8df00cc690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:43842/0 (socket says 172.21.15.57:43842) 2023-11-28T12:01:50.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.017+0000 7f8deffff700 1 -- 172.21.15.57:0/3517389408 learned_addr learned my addr 172.21.15.57:0/3517389408 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:50.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.017+0000 7fa64f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6500b0a80 0x7fa65006bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7fa64f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6500b0a80 0x7fa65006bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:60720/0 (socket says 172.21.15.57:60720) 2023-11-28T12:01:50.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7fa64f59e700 1 -- 172.21.15.57:0/3516113547 learned_addr learned my addr 172.21.15.57:0/3516113547 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f928bbab700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928406b770 0x7f92840b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f928bbab700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f928406d230 0x7f92840b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f928bbab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92840cbc50 0x7f92840b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f928bbab700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92840b4830 con 0x7f928406b770 2023-11-28T12:01:50.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f928bbab700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92840b4600 con 0x7f92840cbc50 2023-11-28T12:01:50.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f928bbab700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92840b09f0 con 0x7f928406d230 2023-11-28T12:01:50.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.017+0000 7f55757fa700 1 --2- 172.21.15.57:0/916540593 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f555c0683c0 0x7f555c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f55757fa700 1 -- 172.21.15.57:0/916540593 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5570087460 con 0x7f5578158e10 2023-11-28T12:01:50.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7fa64f59e700 1 -- 172.21.15.57:0/3516113547 >> v2:172.21.15.57:3300/0 conn(0x7fa65006cdd0 msgr2=0x7fa6500b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7fa64f59e700 1 --2- 172.21.15.57:0/3516113547 >> v2:172.21.15.57:3300/0 conn(0x7fa65006cdd0 0x7fa6500b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7fa64f59e700 1 -- 172.21.15.57:0/3516113547 >> v1:172.21.15.57:6789/0 conn(0x7fa6501609c0 legacy=0x7fa6500ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7fa64f59e700 1 -- 172.21.15.57:0/3516113547 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6500b0ee0 con 0x7fa6500b0a80 2023-11-28T12:01:50.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f928a148700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92840cbc50 0x7f92840b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f928a148700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92840cbc50 0x7f92840b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:49312/0 (socket says 172.21.15.57:49312) 2023-11-28T12:01:50.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f928a148700 1 -- 172.21.15.57:0/915514456 learned_addr learned my addr 172.21.15.57:0/915514456 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 222214746 0 0) 0x7f8df006b910 con 0x7f8df008ac20 2023-11-28T12:01:50.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8de00035b0 con 0x7f8df008ac20 2023-11-28T12:01:50.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f557cc60700 1 --2- 172.21.15.57:0/916540593 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f555c0683c0 0x7f555c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f557eec4700 1 --2- 172.21.15.57:0/916540593 >> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] conn(0x7f55580015e0 0x7f5558003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f557eec4700 1 -- 172.21.15.57:0/916540593 --> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5558006bc0 con 0x7f55580015e0 2023-11-28T12:01:50.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4065885702 0 0) 0x7f8de00035b0 con 0x7f8df008ac20 2023-11-28T12:01:50.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 >> v2:172.21.15.57:3300/0 conn(0x7f8df00ae020 msgr2=0x7f8df006c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 --2- 172.21.15.57:0/3517389408 >> v2:172.21.15.57:3300/0 conn(0x7f8df00ae020 0x7f8df006c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 >> v1:172.21.15.57:6789/0 conn(0x7f8df006c930 legacy=0x7f8df00b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f9289146700 1 --2- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f928406d230 0x7f92840b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df006bed0 con 0x7f8df008ac20 2023-11-28T12:01:50.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f8de4001410 con 0x7f8df008ac20 2023-11-28T12:01:50.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 >> v1:172.21.15.181:6789/0 conn(0x7f8df008ac20 legacy=0x7f8df00cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 --2- 172.21.15.57:0/3517389408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8de00037f0 0x7f8de0003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 --2- 172.21.15.57:0/3517389408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8de0004170 0x7f8de000b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 --2- 172.21.15.57:0/3517389408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8de000b700 0x7f8de000dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df006bbf0 con 0x7f8de000b700 2023-11-28T12:01:50.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de000e490 con 0x7f8de00037f0 2023-11-28T12:01:50.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.018+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de000e650 con 0x7f8de0004170 2023-11-28T12:01:50.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f928a148700 1 -- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f928406d230 msgr2=0x7f92840b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f928a148700 1 --2- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f928406d230 0x7f92840b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f928a148700 1 -- 172.21.15.57:0/915514456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928406b770 msgr2=0x7f92840b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f928a148700 1 --2- 172.21.15.57:0/915514456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928406b770 0x7f92840b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f928a148700 1 -- 172.21.15.57:0/915514456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c009b30 con 0x7f92840cbc50 2023-11-28T12:01:50.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f5577fff700 1 --2- 172.21.15.57:0/916540593 >> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] conn(0x7f55580015e0 0x7f5558003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f557cc60700 1 --2- 172.21.15.57:0/916540593 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f555c0683c0 0x7f555c06a870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55640096d0 tx=0x7f5564006c30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f928a148700 1 --2- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92840cbc50 0x7f92840b2060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f928406d180 tx=0x7f927c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f5577fff700 1 --2- 172.21.15.57:0/916540593 >> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] conn(0x7f55580015e0 0x7f5558003a90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7fa64f59e700 1 --2- 172.21.15.57:0/3516113547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6500b0a80 0x7fa65006bd30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa6400098f0 tx=0x7fa640009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c009422cbf8d3bd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7fa64dd9b700 1 -- 172.21.15.57:0/3516113547 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640004230 con 0x7fa6500b0a80 2023-11-28T12:01:50.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7fa64dd9b700 1 -- 172.21.15.57:0/3516113547 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa640004390 con 0x7fa6500b0a80 2023-11-28T12:01:50.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7fa64dd9b700 1 -- 172.21.15.57:0/3516113547 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6400036a0 con 0x7fa6500b0a80 2023-11-28T12:01:50.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f8deffff700 1 --2- 172.21.15.57:0/3517389408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8de0004170 0x7f8de000b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f8df4c77700 1 --2- 172.21.15.57:0/3517389408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8de00037f0 0x7f8de0003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f8df4c77700 1 -- 172.21.15.57:0/3517389408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8de0004170 msgr2=0x7f8de000b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f8df4c77700 1 --2- 172.21.15.57:0/3517389408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8de0004170 0x7f8de000b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f8df4c77700 1 -- 172.21.15.57:0/3517389408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8de000b700 msgr2=0x7f8de000dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f8df4c77700 1 --2- 172.21.15.57:0/3517389408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8de000b700 0x7f8de000dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f8df4c77700 1 -- 172.21.15.57:0/3517389408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df006bed0 con 0x7f8de00037f0 2023-11-28T12:01:50.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.019+0000 7f8df4c77700 1 --2- 172.21.15.57:0/3517389408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8de00037f0 0x7f8de0003be0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00dd30 tx=0x7f8de0004870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43befbd050118636 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.020+0000 7f8df5478700 1 -- 172.21.15.57:0/3517389408 reap_dead start 2023-11-28T12:01:50.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.020+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc018ad0 con 0x7f8de00037f0 2023-11-28T12:01:50.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.020+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ddc018c30 con 0x7f8de00037f0 2023-11-28T12:01:50.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.020+0000 7f8def7fe700 1 -- 172.21.15.57:0/3517389408 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc00de80 con 0x7f8de00037f0 2023-11-28T12:01:50.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.020+0000 7fa6551a0700 1 -- 172.21.15.57:0/3516113547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6500b0a80 msgr2=0x7fa65006bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.020+0000 7fa6551a0700 1 --2- 172.21.15.57:0/3516113547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6500b0a80 0x7fa65006bd30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa6400098f0 tx=0x7fa640009c00 comp rx=0 tx=0).stop 2023-11-28T12:01:50.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.020+0000 7fa6551a0700 1 -- 172.21.15.57:0/3516113547 shutdown_connections 2023-11-28T12:01:50.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.020+0000 7fa6551a0700 1 --2- 172.21.15.57:0/3516113547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6500b0a80 0x7fa65006bd30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.020+0000 7fa6551a0700 1 --2- 172.21.15.57:0/3516113547 >> v2:172.21.15.57:3300/0 conn(0x7fa65006cdd0 0x7fa6500b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.020+0000 7fa6551a0700 1 -- 172.21.15.57:0/3516113547 >> 172.21.15.57:0/3516113547 conn(0x7fa65006aed0 msgr2=0x7fa6500cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.021+0000 7fa6551a0700 1 -- 172.21.15.57:0/3516113547 shutdown_connections 2023-11-28T12:01:50.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.021+0000 7fa6551a0700 1 -- 172.21.15.57:0/3516113547 wait complete. 2023-11-28T12:01:50.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.022+0000 7fc2aed72700 1 Processor -- start 2023-11-28T12:01:50.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.020+0000 7f55757fa700 1 -- 172.21.15.57:0/916540593 <== osd.2 v2:172.21.15.57:6816/120771 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5558006bc0 con 0x7f55580015e0 2023-11-28T12:01:50.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.022+0000 7f927a7fc700 1 -- 172.21.15.57:0/915514456 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c004960 con 0x7f92840cbc50 2023-11-28T12:01:50.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.022+0000 7f927a7fc700 1 -- 172.21.15.57:0/915514456 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c0092e0 con 0x7f92840cbc50 2023-11-28T12:01:50.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.022+0000 7f927a7fc700 1 -- 172.21.15.57:0/915514456 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c01c640 con 0x7f92840cbc50 2023-11-28T12:01:50.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.022+0000 7f928bbab700 1 -- 172.21.15.57:0/915514456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92840cadb0 con 0x7f92840cbc50 2023-11-28T12:01:50.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.022+0000 7f928bbab700 1 -- 172.21.15.57:0/915514456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92840caad0 con 0x7f92840cbc50 2023-11-28T12:01:50.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.021+0000 7f8df6edb700 1 -- 172.21.15.57:0/3517389408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8de00037f0 msgr2=0x7f8de0003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.021+0000 7f8df6edb700 1 --2- 172.21.15.57:0/3517389408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8de00037f0 0x7f8de0003be0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00dd30 tx=0x7f8de0004870 comp rx=0 tx=0).stop 2023-11-28T12:01:50.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.022+0000 7f8df6edb700 1 -- 172.21.15.57:0/3517389408 shutdown_connections 2023-11-28T12:01:50.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.022+0000 7f8df6edb700 1 --2- 172.21.15.57:0/3517389408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8de00037f0 0x7f8de0003be0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.022+0000 7f8df6edb700 1 -- 172.21.15.57:0/3517389408 >> 172.21.15.57:0/3517389408 conn(0x7f8df006aed0 msgr2=0x7f8df00cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.023+0000 7f928bbab700 1 -- 172.21.15.57:0/915514456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f92840cb090 con 0x7f92840cbc50 2023-11-28T12:01:50.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc06b594700 1 Processor -- start 2023-11-28T12:01:50.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.022+0000 7fc2aed72700 1 -- start start 2023-11-28T12:01:50.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.023+0000 7fc2aed72700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc2a8158980 0x7fc2a80adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc2aed72700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a806d230 0x7fc2a806b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc2aed72700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a806d6c0 con 0x7fc2a80b0540 2023-11-28T12:01:50.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc2aed72700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a806c700 con 0x7fc2a8158980 2023-11-28T12:01:50.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc2aed72700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80ca7f0 con 0x7fc2a806d230 2023-11-28T12:01:50.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc29ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a806d230 0x7fc2a806b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc29ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a806d230 0x7fc2a806b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:60744/0 (socket says 172.21.15.57:60744) 2023-11-28T12:01:50.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc29ffff700 1 -- 172.21.15.57:0/429670850 learned_addr learned my addr 172.21.15.57:0/429670850 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc29ffff700 1 -- 172.21.15.57:0/429670850 >> v2:172.21.15.57:3300/0 conn(0x7fc2a8158980 msgr2=0x7fc2a80adf60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T12:01:50.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc29ffff700 1 --2- 172.21.15.57:0/429670850 >> v2:172.21.15.57:3300/0 conn(0x7fc2a8158980 0x7fc2a80adf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc29ffff700 1 -- 172.21.15.57:0/429670850 >> v1:172.21.15.181:6789/0 conn(0x7fc2a80b0540 legacy=0x7fc2a80cdc40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T12:01:50.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc29ffff700 1 -- 172.21.15.57:0/429670850 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a80caad0 con 0x7fc2a806d230 2023-11-28T12:01:50.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc29ffff700 1 --2- 172.21.15.57:0/429670850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a806d230 0x7fc2a806b770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc29800d1e0 tx=0x7fc29800d4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=598ad95eef5fd4e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.023+0000 7fa6551a0700 1 Processor -- start 2023-11-28T12:01:50.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.023+0000 7fa6551a0700 1 -- start start 2023-11-28T12:01:50.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fa6551a0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa65006cdd0 0x7fa650160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fa6551a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6500b0a80 0x7fa6500c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fa6551a0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6501609c0 0x7fa6500c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fa6551a0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa650160690 con 0x7fa6501609c0 2023-11-28T12:01:50.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fa6551a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500c3430 con 0x7fa6500b0a80 2023-11-28T12:01:50.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fa6551a0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500c2fa0 con 0x7fa65006cdd0 2023-11-28T12:01:50.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fa64e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6500b0a80 0x7fa6500c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fa64e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6500b0a80 0x7fa6500c7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:49322/0 (socket says 172.21.15.57:49322) 2023-11-28T12:01:50.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fa64e59c700 1 -- 172.21.15.57:0/1047955357 learned_addr learned my addr 172.21.15.57:0/1047955357 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fa64ed9d700 1 --2- 172.21.15.57:0/1047955357 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa65006cdd0 0x7fa650160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7f92c85ac700 1 Processor -- start 2023-11-28T12:01:50.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7f92c85ac700 1 -- start start 2023-11-28T12:01:50.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fa64e59c700 1 -- 172.21.15.57:0/1047955357 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa65006cdd0 msgr2=0x7fa650160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fa64e59c700 1 --2- 172.21.15.57:0/1047955357 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa65006cdd0 0x7fa650160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fa64e59c700 1 -- 172.21.15.57:0/1047955357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6501609c0 msgr2=0x7fa6500c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fa64e59c700 1 --2- 172.21.15.57:0/1047955357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6501609c0 0x7fa6500c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fa64e59c700 1 -- 172.21.15.57:0/1047955357 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa640009550 con 0x7fa6500b0a80 2023-11-28T12:01:50.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7fa64e59c700 1 --2- 172.21.15.57:0/1047955357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6500b0a80 0x7fa6500c7e50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa64400b4d0 tx=0x7fa64400b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7f8df6edb700 1 -- 172.21.15.57:0/3517389408 shutdown_connections 2023-11-28T12:01:50.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7fa6377fe700 1 -- 172.21.15.57:0/1047955357 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644011830 con 0x7fa6500b0a80 2023-11-28T12:01:50.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7fa6377fe700 1 -- 172.21.15.57:0/1047955357 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa644011990 con 0x7fa6500b0a80 2023-11-28T12:01:50.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7fa6377fe700 1 -- 172.21.15.57:0/1047955357 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644011d40 con 0x7fa6500b0a80 2023-11-28T12:01:50.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7fa6551a0700 1 -- 172.21.15.57:0/1047955357 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6500caff0 con 0x7fa6500b0a80 2023-11-28T12:01:50.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7fa6551a0700 1 -- 172.21.15.57:0/1047955357 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6500cad10 con 0x7fa6500b0a80 2023-11-28T12:01:50.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.028+0000 7fa6551a0700 1 -- 172.21.15.57:0/1047955357 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa630000fc0 con 0x7fa6500b0a80 2023-11-28T12:01:50.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.028+0000 7fa6377fe700 1 -- 172.21.15.57:0/1047955357 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa644021070 con 0x7fa6500b0a80 2023-11-28T12:01:50.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.028+0000 7fa6377fe700 1 --2- 172.21.15.57:0/1047955357 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa638068450 0x7fa63806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.028+0000 7fa6377fe700 1 -- 172.21.15.57:0/1047955357 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa644085ba0 con 0x7fa6500b0a80 2023-11-28T12:01:50.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.029+0000 7fa6377fe700 1 --2- 172.21.15.57:0/1047955357 >> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] conn(0x7fa638070c80 0x7fa638073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.029+0000 7fa6377fe700 1 -- 172.21.15.57:0/1047955357 --> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa638073740 con 0x7fa638070c80 2023-11-28T12:01:50.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.029+0000 7fa6377fe700 1 -- 172.21.15.57:0/1047955357 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa644011ea0 con 0x7fa6500b0a80 2023-11-28T12:01:50.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.029+0000 7fa64f59e700 1 --2- 172.21.15.57:0/1047955357 >> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] conn(0x7fa638070c80 0x7fa638073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.029+0000 7fa64f59e700 1 --2- 172.21.15.57:0/1047955357 >> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] conn(0x7fa638070c80 0x7fa638073090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7fa6377fe700 1 -- 172.21.15.57:0/1047955357 <== osd.0 v2:172.21.15.57:6800/120754 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa638073740 con 0x7fa638070c80 2023-11-28T12:01:50.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.032+0000 7fa64ed9d700 1 --2- 172.21.15.57:0/1047955357 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa638068450 0x7fa63806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7fc29f7fe700 1 -- 172.21.15.57:0/429670850 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298010070 con 0x7fc2a806d230 2023-11-28T12:01:50.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7fc29f7fe700 1 -- 172.21.15.57:0/429670850 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29800dc60 con 0x7fc2a806d230 2023-11-28T12:01:50.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7fc29f7fe700 1 -- 172.21.15.57:0/429670850 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298007b60 con 0x7fc2a806d230 2023-11-28T12:01:50.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.031+0000 7fc2aed72700 1 -- 172.21.15.57:0/429670850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a806d230 msgr2=0x7fc2a806b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.031+0000 7fc2aed72700 1 --2- 172.21.15.57:0/429670850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a806d230 0x7fc2a806b770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc29800d1e0 tx=0x7fc29800d4f0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7f92c85ac700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92c0065ef0 0x7f92c006c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7f92c85ac700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c00cac90 con 0x7f92c00cef30 2023-11-28T12:01:50.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7f92c85ac700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c006b0d0 con 0x7f92c0066360 2023-11-28T12:01:50.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7f92c85ac700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c00c2ac0 con 0x7f92c0065ef0 2023-11-28T12:01:50.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.028+0000 7f92c5b47700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92c0065ef0 0x7f92c006c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.028+0000 7f92c6b49700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f92c0066360 0x7f92c00cb100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:43884/0 (socket says 172.21.15.57:43884) 2023-11-28T12:01:50.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.028+0000 7f92c6b49700 1 -- 172.21.15.57:0/258515980 learned_addr learned my addr 172.21.15.57:0/258515980 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:50.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.029+0000 7f92c5b47700 1 -- 172.21.15.57:0/258515980 >> v1:172.21.15.181:6789/0 conn(0x7f92c0066360 legacy=0x7f92c00cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.029+0000 7f92c5b47700 1 -- 172.21.15.57:0/258515980 >> v1:172.21.15.57:6789/0 conn(0x7f92c00cef30 legacy=0x7f92c0164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.029+0000 7f92c5b47700 1 -- 172.21.15.57:0/258515980 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c006a270 con 0x7f92c0065ef0 2023-11-28T12:01:50.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.029+0000 7f92c5b47700 1 --2- 172.21.15.57:0/258515980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92c0065ef0 0x7f92c006c440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0097c0 tx=0x7f92bc009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8856153b75c1ed17 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7f927a7fc700 1 -- 172.21.15.57:0/915514456 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f927c022030 con 0x7f92840cbc50 2023-11-28T12:01:50.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7f927a7fc700 1 --2- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9268068340 0x7f926806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.027+0000 7f927a7fc700 1 -- 172.21.15.57:0/915514456 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f927c087810 con 0x7f92840cbc50 2023-11-28T12:01:50.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.027+0000 7f927a7fc700 1 --2- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] conn(0x7f9268070a80 0x7f9268072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.027+0000 7f927a7fc700 1 -- 172.21.15.57:0/915514456 --> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9268073540 con 0x7f9268070a80 2023-11-28T12:01:50.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.027+0000 7f927a7fc700 1 -- 172.21.15.57:0/915514456 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f927c087b80 con 0x7f92840cbc50 2023-11-28T12:01:50.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.027+0000 7f9289146700 1 --2- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] conn(0x7f9268070a80 0x7f9268072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.028+0000 7f9289947700 1 --2- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9268068340 0x7f926806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.029+0000 7f9289947700 1 --2- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9268068340 0x7f926806a7f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92740097f0 tx=0x7f9274008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7f9289146700 1 --2- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] conn(0x7f9268070a80 0x7f9268072e90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.031+0000 7f927a7fc700 1 -- 172.21.15.57:0/915514456 <== osd.4 v2:172.21.15.181:6800/120958 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9268073540 con 0x7f9268070a80 2023-11-28T12:01:50.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7f8df6edb700 1 -- 172.21.15.57:0/3517389408 wait complete. 2023-11-28T12:01:50.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.028+0000 7f8df6edb700 1 Processor -- start 2023-11-28T12:01:50.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.028+0000 7f8df6edb700 1 -- start start 2023-11-28T12:01:50.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7f8df6edb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8de00037f0 0x7f8df00c3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7f8df6edb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df00c37f0 0x7f8df0089dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7f8df6edb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df016bc40 0x7f8df016c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7f8df6edb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df00b46c0 con 0x7f8de00037f0 2023-11-28T12:01:50.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7f8df6edb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df016c600 con 0x7f8df00c37f0 2023-11-28T12:01:50.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7f8df6edb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df00c3c80 con 0x7f8df016bc40 2023-11-28T12:01:50.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.031+0000 7f8df4c77700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8de00037f0 0x7f8df00c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.031+0000 7f8df4c77700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8de00037f0 0x7f8df00c3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47904/0 (socket says 172.21.15.57:47904) 2023-11-28T12:01:50.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.031+0000 7f8df4c77700 1 -- 172.21.15.57:0/1300470240 learned_addr learned my addr 172.21.15.57:0/1300470240 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.031+0000 7f8df4c77700 1 -- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df016bc40 msgr2=0x7f8df016c0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T12:01:50.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.031+0000 7f8df4c77700 1 --2- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df016bc40 0x7f8df016c0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.031+0000 7f8df4c77700 1 -- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df00c37f0 msgr2=0x7f8df0089dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T12:01:50.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.031+0000 7f8df4c77700 1 --2- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df00c37f0 0x7f8df0089dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.031+0000 7f8df4c77700 1 -- 172.21.15.57:0/1300470240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df00cada0 con 0x7f8de00037f0 2023-11-28T12:01:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.024+0000 7fc06b594700 1 -- start start 2023-11-28T12:01:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fc06b594700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc064160c30 0x7fc0640ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fc06b594700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0640adda0 0x7fc0640b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fc06b594700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0640c2460 con 0x7fc0640b0d20 2023-11-28T12:01:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fc06b594700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0640ae210 con 0x7fc064160c30 2023-11-28T12:01:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fc06b594700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc064161000 con 0x7fc0640adda0 2023-11-28T12:01:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fc068b2f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0640adda0 0x7fc0640b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fc068b2f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0640adda0 0x7fc0640b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:60768/0 (socket says 172.21.15.57:60768) 2023-11-28T12:01:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fc068b2f700 1 -- 172.21.15.57:0/2202839217 learned_addr learned my addr 172.21.15.57:0/2202839217 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fc069330700 1 --2- 172.21.15.57:0/2202839217 >> v2:172.21.15.57:3300/0 conn(0x7fc064160c30 0x7fc0640ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fc068b2f700 1 -- 172.21.15.57:0/2202839217 >> v2:172.21.15.57:3300/0 conn(0x7fc064160c30 msgr2=0x7fc0640ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fc068b2f700 1 --2- 172.21.15.57:0/2202839217 >> v2:172.21.15.57:3300/0 conn(0x7fc064160c30 0x7fc0640ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fc068b2f700 1 -- 172.21.15.57:0/2202839217 >> v1:172.21.15.181:6789/0 conn(0x7fc0640b0d20 legacy=0x7fc0640cc690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T12:01:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.025+0000 7fc068b2f700 1 -- 172.21.15.57:0/2202839217 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06406b770 con 0x7fc0640adda0 2023-11-28T12:01:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7fc068b2f700 1 --2- 172.21.15.57:0/2202839217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0640adda0 0x7fc0640b07e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc0600098f0 tx=0x7fc060009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7594cca821bee16a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7fc05bfff700 1 -- 172.21.15.57:0/2202839217 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc060004230 con 0x7fc0640adda0 2023-11-28T12:01:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7fc05bfff700 1 -- 172.21.15.57:0/2202839217 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc060004390 con 0x7fc0640adda0 2023-11-28T12:01:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.026+0000 7fc05bfff700 1 -- 172.21.15.57:0/2202839217 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600036a0 con 0x7fc0640adda0 2023-11-28T12:01:50.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.028+0000 7fc06b594700 1 -- 172.21.15.57:0/2202839217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0640adda0 msgr2=0x7fc0640b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.028+0000 7fc06b594700 1 --2- 172.21.15.57:0/2202839217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0640adda0 0x7fc0640b07e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc0600098f0 tx=0x7fc060009c00 comp rx=0 tx=0).stop 2023-11-28T12:01:50.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7fc06b594700 1 -- 172.21.15.57:0/2202839217 shutdown_connections 2023-11-28T12:01:50.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7fc06b594700 1 --2- 172.21.15.57:0/2202839217 >> v2:172.21.15.57:3300/0 conn(0x7fc064160c30 0x7fc0640ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7fc06b594700 1 --2- 172.21.15.57:0/2202839217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0640adda0 0x7fc0640b07e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.030+0000 7fc06b594700 1 -- 172.21.15.57:0/2202839217 >> 172.21.15.57:0/2202839217 conn(0x7fc0640cb540 msgr2=0x7fc0640cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.032+0000 7fa64ed9d700 1 --2- 172.21.15.57:0/1047955357 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa638068450 0x7fa63806a900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6500ada30 tx=0x7fa648009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.032+0000 7f8df4c77700 1 --2- 172.21.15.57:0/1300470240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8de00037f0 0x7f8df00c3000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8de000e620 tx=0x7f8ddc004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.032+0000 7fc2aed72700 1 -- 172.21.15.57:0/429670850 shutdown_connections 2023-11-28T12:01:50.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.032+0000 7fc2aed72700 1 --2- 172.21.15.57:0/429670850 >> v2:172.21.15.57:3300/0 conn(0x7fc2a8158980 0x7fc2a80adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.032+0000 7fc2aed72700 1 --2- 172.21.15.57:0/429670850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a806d230 0x7fc2a806b770 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.032+0000 7fc2aed72700 1 -- 172.21.15.57:0/429670850 >> 172.21.15.57:0/429670850 conn(0x7fc2a806aed0 msgr2=0x7fc2a80cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.032+0000 7f8ded7fa700 1 -- 172.21.15.57:0/1300470240 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc00fa20 con 0x7f8de00037f0 2023-11-28T12:01:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.032+0000 7f8ded7fa700 1 -- 172.21.15.57:0/1300470240 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ddc00fb80 con 0x7f8de00037f0 2023-11-28T12:01:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.032+0000 7f8ded7fa700 1 -- 172.21.15.57:0/1300470240 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc00ff30 con 0x7f8de00037f0 2023-11-28T12:01:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.033+0000 7fc2aed72700 1 -- 172.21.15.57:0/429670850 shutdown_connections 2023-11-28T12:01:50.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.033+0000 7f8df6edb700 1 -- 172.21.15.57:0/1300470240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df00cac10 con 0x7f8de00037f0 2023-11-28T12:01:50.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.033+0000 7f8df6edb700 1 -- 172.21.15.57:0/1300470240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df00caf30 con 0x7f8de00037f0 2023-11-28T12:01:50.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.034+0000 7f8df6edb700 1 -- 172.21.15.57:0/1300470240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8df00743f0 con 0x7f8de00037f0 2023-11-28T12:01:50.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.033+0000 7fc2aed72700 1 -- 172.21.15.57:0/429670850 wait complete. 2023-11-28T12:01:50.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.035+0000 7fc2aed72700 1 Processor -- start 2023-11-28T12:01:50.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.035+0000 7f8ded7fa700 1 -- 172.21.15.57:0/1300470240 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ddc031020 con 0x7f8de00037f0 2023-11-28T12:01:50.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.036+0000 7f8ded7fa700 1 --2- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8dd40682f0 0x7f8dd406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.036+0000 7f8ded7fa700 1 -- 172.21.15.57:0/1300470240 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8ddc08f370 con 0x7f8de00037f0 2023-11-28T12:01:50.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.036+0000 7f8ded7fa700 1 --2- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x7f8dd4070b20 0x7f8dd4072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.036+0000 7f8ded7fa700 1 -- 172.21.15.57:0/1300470240 --> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8dd40735e0 con 0x7f8dd4070b20 2023-11-28T12:01:50.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.036+0000 7f8ded7fa700 1 -- 172.21.15.57:0/1300470240 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ddc091980 con 0x7f8de00037f0 2023-11-28T12:01:50.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.034+0000 7fc06b594700 1 -- 172.21.15.57:0/2202839217 shutdown_connections 2023-11-28T12:01:50.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.035+0000 7fc06b594700 1 -- 172.21.15.57:0/2202839217 wait complete. 2023-11-28T12:01:50.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.035+0000 7fc2aed72700 1 -- start start 2023-11-28T12:01:50.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.037+0000 7fc06b594700 1 Processor -- start 2023-11-28T12:01:50.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.036+0000 7fc2aed72700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2a806d230 0x7fc2a80b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.036+0000 7fc2aed72700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a80b0540 0x7fc2a80c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.036+0000 7fc2aed72700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a8158980 0x7fc2a8157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.036+0000 7fc2aed72700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80b34d0 con 0x7fc2a806d230 2023-11-28T12:01:50.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.036+0000 7fc2aed72700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80b37b0 con 0x7fc2a80b0540 2023-11-28T12:01:50.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.036+0000 7fc2aed72700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80b3a90 con 0x7fc2a8158980 2023-11-28T12:01:50.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.037+0000 7fc2acb0e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2a806d230 0x7fc2a80b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.037+0000 7fc2acb0e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2a806d230 0x7fc2a80b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47906/0 (socket says 172.21.15.57:47906) 2023-11-28T12:01:50.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.037+0000 7fc2acb0e700 1 -- 172.21.15.57:0/3260807598 learned_addr learned my addr 172.21.15.57:0/3260807598 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.033+0000 7f92c5346700 1 -- 172.21.15.57:0/258515980 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc004230 con 0x7f92c0065ef0 2023-11-28T12:01:50.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.033+0000 7f92c5346700 1 -- 172.21.15.57:0/258515980 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92bc004390 con 0x7f92c0065ef0 2023-11-28T12:01:50.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.033+0000 7f92c5346700 1 -- 172.21.15.57:0/258515980 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0036a0 con 0x7f92c0065ef0 2023-11-28T12:01:50.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.034+0000 7f92c85ac700 1 -- 172.21.15.57:0/258515980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92c0065ef0 msgr2=0x7f92c006c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.034+0000 7f92c85ac700 1 --2- 172.21.15.57:0/258515980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92c0065ef0 0x7f92c006c440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0097c0 tx=0x7f92bc009ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.037+0000 7f92c85ac700 1 -- 172.21.15.57:0/258515980 shutdown_connections 2023-11-28T12:01:50.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.037+0000 7f92c85ac700 1 --2- 172.21.15.57:0/258515980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92c0065ef0 0x7f92c006c440 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.037+0000 7f92c85ac700 1 -- 172.21.15.57:0/258515980 >> 172.21.15.57:0/258515980 conn(0x7f92c00cf400 msgr2=0x7f92c00cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.037+0000 7fc2acb0e700 1 -- 172.21.15.57:0/3260807598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a8158980 msgr2=0x7fc2a8157ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T12:01:50.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.037+0000 7fc2acb0e700 1 --2- 172.21.15.57:0/3260807598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a8158980 0x7fc2a8157ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.037+0000 7fc2acb0e700 1 -- 172.21.15.57:0/3260807598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a80b0540 msgr2=0x7fc2a80c60f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T12:01:50.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.037+0000 7fc2acb0e700 1 --2- 172.21.15.57:0/3260807598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a80b0540 0x7fc2a80c60f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.037+0000 7fc2acb0e700 1 -- 172.21.15.57:0/3260807598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2980079d0 con 0x7fc2a806d230 2023-11-28T12:01:50.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.038+0000 7fc2acb0e700 1 --2- 172.21.15.57:0/3260807598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2a806d230 0x7fc2a80b2dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400b450 tx=0x7fc2a400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.037+0000 7fc06b594700 1 -- start start 2023-11-28T12:01:50.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.038+0000 7fc06b594700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0640adda0 0x7fc064141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.038+0000 7fc06b594700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0640b0d20 0x7fc0640b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.038+0000 7fc06b594700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc064158820 0x7fc0640b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.038+0000 7fc06b594700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc064162ba0 con 0x7fc064158820 2023-11-28T12:01:50.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.038+0000 7fc06b594700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0640b4880 con 0x7fc0640b0d20 2023-11-28T12:01:50.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.038+0000 7fc06b594700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc064162220 con 0x7fc0640adda0 2023-11-28T12:01:50.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.039+0000 7fc069b31700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc064158820 0x7fc0640b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.039+0000 7fc069b31700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc064158820 0x7fc0640b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47922/0 (socket says 172.21.15.57:47922) 2023-11-28T12:01:50.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.039+0000 7fc069b31700 1 -- 172.21.15.57:0/1560501654 learned_addr learned my addr 172.21.15.57:0/1560501654 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.040+0000 7fc069b31700 1 -- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0640adda0 msgr2=0x7fc064141100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T12:01:50.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.040+0000 7fc069b31700 1 --2- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0640adda0 0x7fc064141100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.040+0000 7fc069b31700 1 -- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0640b0d20 msgr2=0x7fc0640b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.040+0000 7fc069b31700 1 --2- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0640b0d20 0x7fc0640b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.040+0000 7fc069b31700 1 -- 172.21.15.57:0/1560501654 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060009550 con 0x7fc064158820 2023-11-28T12:01:50.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.040+0000 7fc069b31700 1 --2- 172.21.15.57:0/1560501654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc064158820 0x7fc0640b4340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00b990 tx=0x7fc05c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.040+0000 7fc059ffb700 1 -- 172.21.15.57:0/1560501654 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c00c9d0 con 0x7fc064158820 2023-11-28T12:01:50.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.038+0000 7fc29d7fa700 1 -- 172.21.15.57:0/3260807598 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4011830 con 0x7fc2a806d230 2023-11-28T12:01:50.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.038+0000 7fc29d7fa700 1 -- 172.21.15.57:0/3260807598 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a40119b0 con 0x7fc2a806d230 2023-11-28T12:01:50.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.038+0000 7fc29d7fa700 1 -- 172.21.15.57:0/3260807598 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4011d60 con 0x7fc2a806d230 2023-11-28T12:01:50.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.040+0000 7fc2aed72700 1 -- 172.21.15.57:0/3260807598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a80cb090 con 0x7fc2a806d230 2023-11-28T12:01:50.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.040+0000 7fc2aed72700 1 -- 172.21.15.57:0/3260807598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a80cadb0 con 0x7fc2a806d230 2023-11-28T12:01:50.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.040+0000 7fc059ffb700 1 -- 172.21.15.57:0/1560501654 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c00cb30 con 0x7fc064158820 2023-11-28T12:01:50.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.040+0000 7fc059ffb700 1 -- 172.21.15.57:0/1560501654 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c014440 con 0x7fc064158820 2023-11-28T12:01:50.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.040+0000 7fc06b594700 1 -- 172.21.15.57:0/1560501654 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc064164a90 con 0x7fc064158820 2023-11-28T12:01:50.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.041+0000 7fc06b594700 1 -- 172.21.15.57:0/1560501654 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0641647b0 con 0x7fc064158820 2023-11-28T12:01:50.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.042+0000 7fc2aed72700 1 -- 172.21.15.57:0/3260807598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc2a8158d50 con 0x7fc2a806d230 2023-11-28T12:01:50.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.042+0000 7f92c85ac700 1 -- 172.21.15.57:0/258515980 shutdown_connections 2023-11-28T12:01:50.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.043+0000 7f92c85ac700 1 -- 172.21.15.57:0/258515980 wait complete. 2023-11-28T12:01:50.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.043+0000 7f8deffff700 1 --2- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8dd40682f0 0x7f8dd406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.043+0000 7f8df5478700 1 --2- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x7f8dd4070b20 0x7f8dd4072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.044+0000 7f8deffff700 1 --2- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8dd40682f0 0x7f8dd406a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8de4000fd0 tx=0x7f8de40016f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.043+0000 7fc29d7fa700 1 -- 172.21.15.57:0/3260807598 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc2a4021070 con 0x7fc2a806d230 2023-11-28T12:01:50.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.044+0000 7fc29d7fa700 1 --2- 172.21.15.57:0/3260807598 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc2940683c0 0x7fc29406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.044+0000 7fc29d7fa700 1 -- 172.21.15.57:0/3260807598 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc2a4085c00 con 0x7fc2a806d230 2023-11-28T12:01:50.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.044+0000 7fc29d7fa700 1 --2- 172.21.15.57:0/3260807598 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x7fc294070bf0 0x7fc294073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.044+0000 7fc29d7fa700 1 -- 172.21.15.57:0/3260807598 --> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc2940736b0 con 0x7fc294070bf0 2023-11-28T12:01:50.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.044+0000 7fc29d7fa700 1 -- 172.21.15.57:0/3260807598 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2a4052410 con 0x7fc2a806d230 2023-11-28T12:01:50.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.044+0000 7fc2ad30f700 1 --2- 172.21.15.57:0/3260807598 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x7fc294070bf0 0x7fc294073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.042+0000 7fc06b594700 1 -- 172.21.15.57:0/1560501654 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc064164d70 con 0x7fc064158820 2023-11-28T12:01:50.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.045+0000 7fc2ad30f700 1 --2- 172.21.15.57:0/3260807598 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x7fc294070bf0 0x7fc294073000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.045+0000 7f92c85ac700 1 Processor -- start 2023-11-28T12:01:50.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.045+0000 7f92c85ac700 1 -- start start 2023-11-28T12:01:50.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.046+0000 7fc29ffff700 1 --2- 172.21.15.57:0/3260807598 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc2940683c0 0x7fc29406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.046+0000 7f92c85ac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92c0065ef0 0x7f92c015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.046+0000 7f92c85ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92c0066360 0x7f92c00b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.046+0000 7f92c85ac700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92c00cef30 0x7f92c01fa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.046+0000 7f92c85ac700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00b5150 con 0x7f92c00cef30 2023-11-28T12:01:50.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.046+0000 7f92c85ac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00b4e70 con 0x7f92c0066360 2023-11-28T12:01:50.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.046+0000 7f92c85ac700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00b2d90 con 0x7f92c0065ef0 2023-11-28T12:01:50.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.046+0000 7fc29d7fa700 1 -- 172.21.15.57:0/3260807598 <== osd.1 v2:172.21.15.57:6808/120755 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc2940736b0 con 0x7fc294070bf0 2023-11-28T12:01:50.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.046+0000 7f92c6348700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92c0065ef0 0x7f92c015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.046+0000 7f92c6348700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92c0065ef0 0x7f92c015c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:60792/0 (socket says 172.21.15.57:60792) 2023-11-28T12:01:50.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.046+0000 7f92c6348700 1 -- 172.21.15.57:0/4206345698 learned_addr learned my addr 172.21.15.57:0/4206345698 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7f92c6b49700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92c00cef30 0x7f92c01fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7f92c6348700 1 -- 172.21.15.57:0/4206345698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92c0066360 msgr2=0x7f92c00b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7f92c6348700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92c0066360 0x7f92c00b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7f92c6348700 1 -- 172.21.15.57:0/4206345698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92c00cef30 msgr2=0x7f92c01fa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7f92c6348700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92c00cef30 0x7f92c01fa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7f92c6348700 1 -- 172.21.15.57:0/4206345698 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc0094a0 con 0x7f92c0065ef0 2023-11-28T12:01:50.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7f92c6b49700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92c00cef30 0x7f92c01fa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:50.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7fc29ffff700 1 --2- 172.21.15.57:0/3260807598 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc2940683c0 0x7fc29406a870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc29800d1e0 tx=0x7fc298012000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7f92c6348700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92c0065ef0 0x7f92c015c040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f92b000b470 tx=0x7f92b000b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7fcc1b9c0700 1 Processor -- start 2023-11-28T12:01:50.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7f92b6ffd700 1 -- 172.21.15.57:0/4206345698 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0011830 con 0x7f92c0065ef0 2023-11-28T12:01:50.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7fcc1b9c0700 1 -- start start 2023-11-28T12:01:50.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7f92b6ffd700 1 -- 172.21.15.57:0/4206345698 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b00119b0 con 0x7f92c0065ef0 2023-11-28T12:01:50.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7f92b6ffd700 1 -- 172.21.15.57:0/4206345698 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0011d60 con 0x7f92c0065ef0 2023-11-28T12:01:50.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.047+0000 7f92c85ac700 1 -- 172.21.15.57:0/4206345698 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c006a830 con 0x7f92c0065ef0 2023-11-28T12:01:50.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.048+0000 7f92c85ac700 1 -- 172.21.15.57:0/4206345698 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c006a550 con 0x7f92c0065ef0 2023-11-28T12:01:50.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.048+0000 7f8df5478700 1 --2- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x7f8dd4070b20 0x7f8dd4072f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.048+0000 7f8ded7fa700 1 -- 172.21.15.57:0/1300470240 <== osd.5 v2:172.21.15.181:6824/120977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8dd40735e0 con 0x7f8dd4070b20 2023-11-28T12:01:50.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7fc059ffb700 1 -- 172.21.15.57:0/1560501654 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc05c01b030 con 0x7fc064158820 2023-11-28T12:01:50.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.049+0000 7f92c85ac700 1 -- 172.21.15.57:0/4206345698 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f92a8000fc0 con 0x7f92c0065ef0 2023-11-28T12:01:50.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7f92b6ffd700 1 -- 172.21.15.57:0/4206345698 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f92b0021070 con 0x7f92c0065ef0 2023-11-28T12:01:50.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.048+0000 7fcc1b9c0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc140cf710 0x7fcc140ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.048+0000 7fcc1b9c0700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1415cc10 con 0x7fcc1406a980 2023-11-28T12:01:50.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.048+0000 7fcc1b9c0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1406c9b0 con 0x7fcc14071890 2023-11-28T12:01:50.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.048+0000 7fcc1b9c0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1406cc90 con 0x7fcc140cf710 2023-11-28T12:01:50.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.049+0000 7fcc1975c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc140cf710 0x7fcc140ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.049+0000 7fcc1975c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc140cf710 0x7fcc140ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:60806/0 (socket says 172.21.15.57:60806) 2023-11-28T12:01:50.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.049+0000 7fcc1975c700 1 -- 172.21.15.57:0/1227903680 learned_addr learned my addr 172.21.15.57:0/1227903680 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.049+0000 7fcc1975c700 1 -- 172.21.15.57:0/1227903680 >> v1:172.21.15.181:6789/0 conn(0x7fcc14071890 legacy=0x7fcc1406a270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T12:01:50.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.049+0000 7fcc1975c700 1 -- 172.21.15.57:0/1227903680 >> v1:172.21.15.57:6789/0 conn(0x7fcc1406a980 legacy=0x7fcc14160480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T12:01:50.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.049+0000 7fcc1975c700 1 -- 172.21.15.57:0/1227903680 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1406cf70 con 0x7fcc140cf710 2023-11-28T12:01:50.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.049+0000 7fcc1975c700 1 --2- 172.21.15.57:0/1227903680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc140cf710 0x7fcc140ce740 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009920 tx=0x7fcc10009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43c7ecc765f592a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.049+0000 7fcc0bfff700 1 -- 172.21.15.57:0/1227903680 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10004230 con 0x7fcc140cf710 2023-11-28T12:01:50.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.049+0000 7fcc0bfff700 1 -- 172.21.15.57:0/1227903680 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc10004390 con 0x7fcc140cf710 2023-11-28T12:01:50.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7fcc0bfff700 1 -- 172.21.15.57:0/1227903680 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10009e70 con 0x7fcc140cf710 2023-11-28T12:01:50.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7fc059ffb700 1 --2- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0480682f0 0x7fc04806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7f92b6ffd700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f92a40682f0 0x7f92a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7f92b6ffd700 1 -- 172.21.15.57:0/4206345698 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f92b0085be0 con 0x7f92c0065ef0 2023-11-28T12:01:50.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7fc059ffb700 1 -- 172.21.15.57:0/1560501654 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc05c0859d0 con 0x7fc064158820 2023-11-28T12:01:50.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7f92b6ffd700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] conn(0x7f92a4070b20 0x7f92a4072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7f92b6ffd700 1 -- 172.21.15.57:0/4206345698 --> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92a40735e0 con 0x7f92a4070b20 2023-11-28T12:01:50.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7f92b6ffd700 1 -- 172.21.15.57:0/4206345698 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92b008a2a0 con 0x7f92c0065ef0 2023-11-28T12:01:50.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7fc059ffb700 1 --2- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x7fc048070ba0 0x7fc048072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7fc059ffb700 1 -- 172.21.15.57:0/1560501654 --> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc048073660 con 0x7fc048070ba0 2023-11-28T12:01:50.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7fc059ffb700 1 -- 172.21.15.57:0/1560501654 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc05c08a2a0 con 0x7fc064158820 2023-11-28T12:01:50.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7fc069330700 1 --2- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0480682f0 0x7fc04806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7f92c5b47700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f92a40682f0 0x7f92a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.050+0000 7f92c6b49700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] conn(0x7f92a4070b20 0x7f92a4072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.051+0000 7fc068b2f700 1 --2- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x7fc048070ba0 0x7fc048072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.048+0000 7f557eec4700 1 -- 172.21.15.57:0/916540593 --> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5558005ca0 con 0x7f55580015e0 2023-11-28T12:01:50.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.048+0000 7f55757fa700 1 -- 172.21.15.57:0/916540593 <== osd.2 v2:172.21.15.57:6816/120771 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5558005ca0 con 0x7f55580015e0 2023-11-28T12:01:50.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.051+0000 7f92c5b47700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f92a40682f0 0x7f92a406a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f92c00cb690 tx=0x7f92bc010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.051+0000 7fc069330700 1 --2- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0480682f0 0x7fc04806a7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc06406e690 tx=0x7fc054008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.051+0000 7f92c6b49700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] conn(0x7f92a4070b20 0x7f92a4072f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.051+0000 7fc068b2f700 1 --2- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x7fc048070ba0 0x7fc048072fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.051+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/1227903680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc140cf710 msgr2=0x7fcc140ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.051+0000 7fcc1b9c0700 1 --2- 172.21.15.57:0/1227903680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc140cf710 0x7fcc140ce740 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009920 tx=0x7fcc10009c30 comp rx=0 tx=0).stop 2023-11-28T12:01:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.051+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/1227903680 shutdown_connections 2023-11-28T12:01:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.051+0000 7fcc1b9c0700 1 --2- 172.21.15.57:0/1227903680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc140cf710 0x7fcc140ce740 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.051+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/1227903680 >> 172.21.15.57:0/1227903680 conn(0x7fcc1406c010 msgr2=0x7fcc140cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.052+0000 7fc059ffb700 1 -- 172.21.15.57:0/1560501654 <== osd.6 v2:172.21.15.181:6816/120984 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc048073660 con 0x7fc048070ba0 2023-11-28T12:01:50.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.052+0000 7f92b6ffd700 1 -- 172.21.15.57:0/4206345698 <== osd.3 v2:172.21.15.57:6817/120770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f92a40735e0 con 0x7f92a4070b20 2023-11-28T12:01:50.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.053+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/1227903680 shutdown_connections 2023-11-28T12:01:50.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.053+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/1227903680 wait complete. 2023-11-28T12:01:50.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.055+0000 7fcc1b9c0700 1 Processor -- start 2023-11-28T12:01:50.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.055+0000 7fcc1b9c0700 1 -- start start 2023-11-28T12:01:50.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.056+0000 7fcc1b9c0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc1406a980 0x7fcc1415b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.056+0000 7fcc1b9c0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc14071890 0x7fcc141e58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.056+0000 7fcc1b9c0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc140cf710 0x7fcc140ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.056+0000 7fcc1b9c0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc141e6390 con 0x7fcc1406a980 2023-11-28T12:01:50.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.056+0000 7fcc1b9c0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc141e6670 con 0x7fcc14071890 2023-11-28T12:01:50.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.056+0000 7fcc1b9c0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc141e6950 con 0x7fcc140cf710 2023-11-28T12:01:50.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.056+0000 7fcc1975c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc1406a980 0x7fcc1415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.056+0000 7fcc1975c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc1406a980 0x7fcc1415b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47946/0 (socket says 172.21.15.57:47946) 2023-11-28T12:01:50.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.056+0000 7fcc1975c700 1 -- 172.21.15.57:0/2517693665 learned_addr learned my addr 172.21.15.57:0/2517693665 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.056+0000 7fcc18f5b700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc14071890 0x7fcc141e58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.056+0000 7fcc19f5d700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc140cf710 0x7fcc140ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.057+0000 7fcc1975c700 1 -- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc140cf710 msgr2=0x7fcc140ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.057+0000 7fcc1975c700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc140cf710 0x7fcc140ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.057+0000 7fcc1975c700 1 -- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc14071890 msgr2=0x7fcc141e58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.057+0000 7fcc1975c700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc14071890 0x7fcc141e58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.057+0000 7fcc1975c700 1 -- 172.21.15.57:0/2517693665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc10009550 con 0x7fcc1406a980 2023-11-28T12:01:50.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.057+0000 7fcc18f5b700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc14071890 0x7fcc141e58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:50.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.057+0000 7fcc19f5d700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc140cf710 0x7fcc140ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:50.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.057+0000 7fcc1975c700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc1406a980 0x7fcc1415b120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009870 tx=0x7fcc10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.057+0000 7fcc09ffb700 1 -- 172.21.15.57:0/2517693665 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc100036a0 con 0x7fcc1406a980 2023-11-28T12:01:50.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.057+0000 7fcc09ffb700 1 -- 172.21.15.57:0/2517693665 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc10003820 con 0x7fcc1406a980 2023-11-28T12:01:50.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.057+0000 7fcc09ffb700 1 -- 172.21.15.57:0/2517693665 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10003bd0 con 0x7fcc1406a980 2023-11-28T12:01:50.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.057+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/2517693665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1406d530 con 0x7fcc1406a980 2023-11-28T12:01:50.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.058+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/2517693665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1406d250 con 0x7fcc1406a980 2023-11-28T12:01:50.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.058+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/2517693665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc14071220 con 0x7fcc1406a980 2023-11-28T12:01:50.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.059+0000 7fcc09ffb700 1 -- 172.21.15.57:0/2517693665 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcc10017020 con 0x7fcc1406a980 2023-11-28T12:01:50.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.060+0000 7fcc09ffb700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcbf80682f0 0x7fcbf806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.060+0000 7fcc09ffb700 1 -- 172.21.15.57:0/2517693665 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcc10087670 con 0x7fcc1406a980 2023-11-28T12:01:50.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.060+0000 7fa6551a0700 1 -- 172.21.15.57:0/1047955357 --> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa630002d40 con 0x7fa638070c80 2023-11-28T12:01:50.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.060+0000 7fcc09ffb700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x7fcbf8070aa0 0x7fcbf8072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.060+0000 7fcc09ffb700 1 -- 172.21.15.57:0/2517693665 --> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcbf8073560 con 0x7fcbf8070aa0 2023-11-28T12:01:50.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.060+0000 7fcc18f5b700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcbf80682f0 0x7fcbf806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.061+0000 7fa6377fe700 1 -- 172.21.15.57:0/1047955357 <== osd.0 v2:172.21.15.57:6800/120754 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa630002d40 con 0x7fa638070c80 2023-11-28T12:01:50.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.061+0000 7fcc18f5b700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcbf80682f0 0x7fcbf806a7a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcc141e72a0 tx=0x7fcc04009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.061+0000 7fa6551a0700 1 -- 172.21.15.57:0/1047955357 >> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] conn(0x7fa638070c80 msgr2=0x7fa638073090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.061+0000 7fa6551a0700 1 --2- 172.21.15.57:0/1047955357 >> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] conn(0x7fa638070c80 0x7fa638073090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.061+0000 7fa6551a0700 1 -- 172.21.15.57:0/1047955357 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa638068450 msgr2=0x7fa63806a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.061+0000 7fa6551a0700 1 --2- 172.21.15.57:0/1047955357 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa638068450 0x7fa63806a900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6500ada30 tx=0x7fa648009040 comp rx=0 tx=0).stop 2023-11-28T12:01:50.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.061+0000 7fa6551a0700 1 -- 172.21.15.57:0/1047955357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6500b0a80 msgr2=0x7fa6500c7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.061+0000 7fa6551a0700 1 --2- 172.21.15.57:0/1047955357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6500b0a80 0x7fa6500c7e50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa64400b4d0 tx=0x7fa64400b7e0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.061+0000 7fa64f59e700 1 -- 172.21.15.57:0/1047955357 reap_dead start 2023-11-28T12:01:50.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.061+0000 7fa6551a0700 1 -- 172.21.15.57:0/1047955357 shutdown_connections 2023-11-28T12:01:50.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.061+0000 7fa6551a0700 1 -- 172.21.15.57:0/1047955357 >> 172.21.15.57:0/1047955357 conn(0x7fa65006aed0 msgr2=0x7fa6500cd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.061+0000 7f557eec4700 1 -- 172.21.15.57:0/916540593 >> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] conn(0x7f55580015e0 msgr2=0x7f5558003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.061+0000 7f557eec4700 1 --2- 172.21.15.57:0/916540593 >> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] conn(0x7f55580015e0 0x7f5558003a90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.062+0000 7f557eec4700 1 -- 172.21.15.57:0/916540593 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f555c0683c0 msgr2=0x7f555c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.062+0000 7f557eec4700 1 --2- 172.21.15.57:0/916540593 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f555c0683c0 0x7f555c06a870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55640096d0 tx=0x7f5564006c30 comp rx=0 tx=0).stop 2023-11-28T12:01:50.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.062+0000 7f557eec4700 1 -- 172.21.15.57:0/916540593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5578158e10 msgr2=0x7f55780c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.062+0000 7f557eec4700 1 --2- 172.21.15.57:0/916540593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5578158e10 0x7f55780c6fe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5570014000 tx=0x7f5570004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.062+0000 7f557d461700 1 -- 172.21.15.57:0/916540593 reap_dead start 2023-11-28T12:01:50.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.062+0000 7f557eec4700 1 -- 172.21.15.57:0/916540593 shutdown_connections 2023-11-28T12:01:50.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.062+0000 7f557eec4700 1 -- 172.21.15.57:0/916540593 >> 172.21.15.57:0/916540593 conn(0x7f55780cf400 msgr2=0x7f557815c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.062+0000 7f557eec4700 1 -- 172.21.15.57:0/916540593 shutdown_connections 2023-11-28T12:01:50.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.062+0000 7f557eec4700 1 -- 172.21.15.57:0/916540593 wait complete. 2023-11-28T12:01:50.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.062+0000 7f928bbab700 1 -- 172.21.15.57:0/915514456 --> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f92840b13d0 con 0x7f9268070a80 2023-11-28T12:01:50.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.063+0000 7f927a7fc700 1 -- 172.21.15.57:0/915514456 <== osd.4 v2:172.21.15.181:6800/120958 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f92840b13d0 con 0x7f9268070a80 2023-11-28T12:01:50.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.067+0000 7fa6551a0700 1 -- 172.21.15.57:0/1047955357 shutdown_connections 2023-11-28T12:01:50.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.067+0000 7fa6551a0700 1 -- 172.21.15.57:0/1047955357 wait complete. 2023-11-28T12:01:50.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.068+0000 7fcc19f5d700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x7fcbf8070aa0 0x7fcbf8072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.068+0000 7fcc09ffb700 1 -- 172.21.15.57:0/2517693665 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc10053d80 con 0x7fcc1406a980 2023-11-28T12:01:50.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.069+0000 7fcc19f5d700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x7fcbf8070aa0 0x7fcbf8072eb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.070+0000 7fcc09ffb700 1 -- 172.21.15.57:0/2517693665 <== osd.7 v2:172.21.15.181:6808/120959 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcbf8073560 con 0x7fcbf8070aa0 2023-11-28T12:01:50.115 INFO:teuthology.orchestra.run.smithi057.stdout:55834574852 2023-11-28T12:01:50.116 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T12:01:50.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.086+0000 7f8df6edb700 1 -- 172.21.15.57:0/1300470240 --> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8df00cc9f0 con 0x7f8dd4070b20 2023-11-28T12:01:50.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.087+0000 7f928bbab700 1 -- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] conn(0x7f9268070a80 msgr2=0x7f9268072e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.087+0000 7f928bbab700 1 --2- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] conn(0x7f9268070a80 0x7f9268072e90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.087+0000 7f8ded7fa700 1 -- 172.21.15.57:0/1300470240 <== osd.5 v2:172.21.15.181:6824/120977 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8df00cc9f0 con 0x7f8dd4070b20 2023-11-28T12:01:50.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.087+0000 7f928bbab700 1 -- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9268068340 msgr2=0x7f926806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.087+0000 7f928bbab700 1 --2- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9268068340 0x7f926806a7f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92740097f0 tx=0x7f9274008040 comp rx=0 tx=0).stop 2023-11-28T12:01:50.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.087+0000 7f928bbab700 1 -- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92840cbc50 msgr2=0x7f92840b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.087+0000 7f928bbab700 1 --2- 172.21.15.57:0/915514456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92840cbc50 0x7f92840b2060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f928406d180 tx=0x7f927c004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.087+0000 7f8df6edb700 1 -- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x7f8dd4070b20 msgr2=0x7f8dd4072f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.088+0000 7f8df6edb700 1 --2- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x7f8dd4070b20 0x7f8dd4072f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.088+0000 7f928a148700 1 -- 172.21.15.57:0/915514456 reap_dead start 2023-11-28T12:01:50.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.088+0000 7f8df6edb700 1 -- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8dd40682f0 msgr2=0x7f8dd406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.088+0000 7f8df6edb700 1 --2- 172.21.15.57:0/1300470240 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8dd40682f0 0x7f8dd406a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8de4000fd0 tx=0x7f8de40016f0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.088+0000 7f8df6edb700 1 -- 172.21.15.57:0/1300470240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8de00037f0 msgr2=0x7f8df00c3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.088+0000 7f8df6edb700 1 --2- 172.21.15.57:0/1300470240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8de00037f0 0x7f8df00c3000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8de000e620 tx=0x7f8ddc004860 comp rx=0 tx=0).stop 2023-11-28T12:01:50.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.088+0000 7f8df5478700 1 -- 172.21.15.57:0/1300470240 reap_dead start 2023-11-28T12:01:50.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.088+0000 7f928bbab700 1 -- 172.21.15.57:0/915514456 shutdown_connections 2023-11-28T12:01:50.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.088+0000 7f928bbab700 1 -- 172.21.15.57:0/915514456 >> 172.21.15.57:0/915514456 conn(0x7f928406aed0 msgr2=0x7f928406fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.088+0000 7f8df6edb700 1 -- 172.21.15.57:0/1300470240 shutdown_connections 2023-11-28T12:01:50.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.088+0000 7f8df6edb700 1 -- 172.21.15.57:0/1300470240 >> 172.21.15.57:0/1300470240 conn(0x7f8df006aed0 msgr2=0x7f8df00710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.091+0000 7f8df6edb700 1 -- 172.21.15.57:0/1300470240 shutdown_connections 2023-11-28T12:01:50.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.091+0000 7fc06b594700 1 -- 172.21.15.57:0/1560501654 --> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc0640743f0 con 0x7fc048070ba0 2023-11-28T12:01:50.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.091+0000 7f8df6edb700 1 -- 172.21.15.57:0/1300470240 wait complete. 2023-11-28T12:01:50.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.093+0000 7f92c85ac700 1 -- 172.21.15.57:0/4206345698 --> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f92a8002d40 con 0x7f92a4070b20 2023-11-28T12:01:50.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.093+0000 7f92b6ffd700 1 -- 172.21.15.57:0/4206345698 <== osd.3 v2:172.21.15.57:6817/120770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f92a8002d40 con 0x7f92a4070b20 2023-11-28T12:01:50.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.094+0000 7f92c85ac700 1 -- 172.21.15.57:0/4206345698 >> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] conn(0x7f92a4070b20 msgr2=0x7f92a4072f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.094+0000 7f92c85ac700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] conn(0x7f92a4070b20 0x7f92a4072f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.094+0000 7f92c85ac700 1 -- 172.21.15.57:0/4206345698 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f92a40682f0 msgr2=0x7f92a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.094+0000 7f92c85ac700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f92a40682f0 0x7f92a406a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f92c00cb690 tx=0x7f92bc010000 comp rx=0 tx=0).stop 2023-11-28T12:01:50.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.094+0000 7f92c85ac700 1 -- 172.21.15.57:0/4206345698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92c0065ef0 msgr2=0x7f92c015c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.094+0000 7f92c85ac700 1 --2- 172.21.15.57:0/4206345698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92c0065ef0 0x7f92c015c040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f92b000b470 tx=0x7f92b000b830 comp rx=0 tx=0).stop 2023-11-28T12:01:50.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.094+0000 7f92c6b49700 1 -- 172.21.15.57:0/4206345698 reap_dead start 2023-11-28T12:01:50.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.094+0000 7f928bbab700 1 -- 172.21.15.57:0/915514456 shutdown_connections 2023-11-28T12:01:50.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.094+0000 7f928bbab700 1 -- 172.21.15.57:0/915514456 wait complete. 2023-11-28T12:01:50.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.094+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/2517693665 --> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc140a9580 con 0x7fcbf8070aa0 2023-11-28T12:01:50.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7f92c85ac700 1 -- 172.21.15.57:0/4206345698 shutdown_connections 2023-11-28T12:01:50.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7f92c85ac700 1 -- 172.21.15.57:0/4206345698 >> 172.21.15.57:0/4206345698 conn(0x7f92c00cf400 msgr2=0x7f92c00b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7fc059ffb700 1 -- 172.21.15.57:0/1560501654 <== osd.6 v2:172.21.15.181:6816/120984 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc0640743f0 con 0x7fc048070ba0 2023-11-28T12:01:50.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7fcc09ffb700 1 -- 172.21.15.57:0/2517693665 <== osd.7 v2:172.21.15.181:6808/120959 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcc140a9580 con 0x7fcbf8070aa0 2023-11-28T12:01:50.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7fc2aed72700 1 -- 172.21.15.57:0/3260807598 --> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc2a80743f0 con 0x7fc294070bf0 2023-11-28T12:01:50.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7fc06b594700 1 -- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x7fc048070ba0 msgr2=0x7fc048072fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7fc06b594700 1 --2- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x7fc048070ba0 0x7fc048072fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x7fcbf8070aa0 msgr2=0x7fcbf8072eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7fcc1b9c0700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x7fcbf8070aa0 0x7fcbf8072eb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7fc06b594700 1 -- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0480682f0 msgr2=0x7fc04806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7fc06b594700 1 --2- 172.21.15.57:0/1560501654 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0480682f0 0x7fc04806a7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc06406e690 tx=0x7fc054008040 comp rx=0 tx=0).stop 2023-11-28T12:01:50.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7fc06b594700 1 -- 172.21.15.57:0/1560501654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc064158820 msgr2=0x7fc0640b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7fc06b594700 1 --2- 172.21.15.57:0/1560501654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc064158820 0x7fc0640b4340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00b990 tx=0x7fc05c00bd50 comp rx=0 tx=0).stop 2023-11-28T12:01:50.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.095+0000 7fc069b31700 1 -- 172.21.15.57:0/1560501654 reap_dead start 2023-11-28T12:01:50.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.096+0000 7fc06b594700 1 -- 172.21.15.57:0/1560501654 shutdown_connections 2023-11-28T12:01:50.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.096+0000 7fc06b594700 1 -- 172.21.15.57:0/1560501654 >> 172.21.15.57:0/1560501654 conn(0x7fc0640cb540 msgr2=0x7fc0640cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.096+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcbf80682f0 msgr2=0x7fcbf806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.096+0000 7fcc1b9c0700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcbf80682f0 0x7fcbf806a7a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcc141e72a0 tx=0x7fcc04009190 comp rx=0 tx=0).stop 2023-11-28T12:01:50.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.096+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/2517693665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc1406a980 msgr2=0x7fcc1415b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.096+0000 7fcc1b9c0700 1 --2- 172.21.15.57:0/2517693665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc1406a980 0x7fcc1415b120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009870 tx=0x7fcc10004750 comp rx=0 tx=0).stop 2023-11-28T12:01:50.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.096+0000 7fcc19f5d700 1 -- 172.21.15.57:0/2517693665 reap_dead start 2023-11-28T12:01:50.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.096+0000 7fc06b594700 1 -- 172.21.15.57:0/1560501654 shutdown_connections 2023-11-28T12:01:50.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.096+0000 7fc06b594700 1 -- 172.21.15.57:0/1560501654 wait complete. 2023-11-28T12:01:50.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.096+0000 7fc29d7fa700 1 -- 172.21.15.57:0/3260807598 <== osd.1 v2:172.21.15.57:6808/120755 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc2a80743f0 con 0x7fc294070bf0 2023-11-28T12:01:50.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.097+0000 7f92c85ac700 1 -- 172.21.15.57:0/4206345698 shutdown_connections 2023-11-28T12:01:50.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.097+0000 7f92c85ac700 1 -- 172.21.15.57:0/4206345698 wait complete. 2023-11-28T12:01:50.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.097+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/2517693665 shutdown_connections 2023-11-28T12:01:50.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.097+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/2517693665 >> 172.21.15.57:0/2517693665 conn(0x7fcc1406c010 msgr2=0x7fcc140b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.097+0000 7fc2aed72700 1 -- 172.21.15.57:0/3260807598 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x7fc294070bf0 msgr2=0x7fc294073000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.097+0000 7fc2aed72700 1 --2- 172.21.15.57:0/3260807598 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x7fc294070bf0 0x7fc294073000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.097+0000 7fc2aed72700 1 -- 172.21.15.57:0/3260807598 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc2940683c0 msgr2=0x7fc29406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.097+0000 7fc2aed72700 1 --2- 172.21.15.57:0/3260807598 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc2940683c0 0x7fc29406a870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc29800d1e0 tx=0x7fc298012000 comp rx=0 tx=0).stop 2023-11-28T12:01:50.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.097+0000 7fc2aed72700 1 -- 172.21.15.57:0/3260807598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2a806d230 msgr2=0x7fc2a80b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.097+0000 7fc2aed72700 1 --2- 172.21.15.57:0/3260807598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2a806d230 0x7fc2a80b2dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400b450 tx=0x7fc2a400b810 comp rx=0 tx=0).stop 2023-11-28T12:01:50.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.097+0000 7fc2ad30f700 1 -- 172.21.15.57:0/3260807598 reap_dead start 2023-11-28T12:01:50.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.097+0000 7fc2aed72700 1 -- 172.21.15.57:0/3260807598 shutdown_connections 2023-11-28T12:01:50.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.097+0000 7fc2aed72700 1 -- 172.21.15.57:0/3260807598 >> 172.21.15.57:0/3260807598 conn(0x7fc2a806aed0 msgr2=0x7fc2a80ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.100+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/2517693665 shutdown_connections 2023-11-28T12:01:50.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.100+0000 7fcc1b9c0700 1 -- 172.21.15.57:0/2517693665 wait complete. 2023-11-28T12:01:50.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.102+0000 7fc2aed72700 1 -- 172.21.15.57:0/3260807598 shutdown_connections 2023-11-28T12:01:50.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.102+0000 7fc2aed72700 1 -- 172.21.15.57:0/3260807598 wait complete. 2023-11-28T12:01:50.132 INFO:teuthology.orchestra.run.smithi057.stdout:55834574855 2023-11-28T12:01:50.132 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T12:01:50.133 INFO:teuthology.orchestra.run.smithi057.stdout:55834574855 2023-11-28T12:01:50.134 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T12:01:50.134 INFO:teuthology.orchestra.run.smithi057.stdout:55834574854 2023-11-28T12:01:50.135 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T12:01:50.135 INFO:teuthology.orchestra.run.smithi057.stdout:55834574853 2023-11-28T12:01:50.136 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T12:01:50.137 INFO:teuthology.orchestra.run.smithi057.stdout:55834574853 2023-11-28T12:01:50.137 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T12:01:50.138 INFO:teuthology.orchestra.run.smithi057.stdout:55834574853 2023-11-28T12:01:50.138 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T12:01:50.139 INFO:teuthology.orchestra.run.smithi057.stdout:55834574852 2023-11-28T12:01:50.139 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T12:01:50.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.407+0000 7fcc0368c700 1 Processor -- start 2023-11-28T12:01:50.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.408+0000 7fcc0368c700 1 -- start start 2023-11-28T12:01:50.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.409+0000 7fcc0368c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbfc06d230 0x7fcbfc0adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.409+0000 7fcc0368c700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbfc06d6c0 con 0x7fcbfc158980 2023-11-28T12:01:50.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.409+0000 7fcc0368c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbfc0ca7f0 con 0x7fcbfc0b0c50 2023-11-28T12:01:50.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.409+0000 7fcc0368c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc06c700 con 0x7fcbfc06d230 2023-11-28T12:01:50.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.410+0000 7fcc01428700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcbfc158980 0x7fcbfc0b0540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36978/0 (socket says 172.21.15.57:36978) 2023-11-28T12:01:50.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.410+0000 7fcc01428700 1 -- 172.21.15.57:0/3746242829 learned_addr learned my addr 172.21.15.57:0/3746242829 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:50.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.411+0000 7f13ec4fc700 1 Processor -- start 2023-11-28T12:01:50.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.411+0000 7fcc00c27700 1 --2- 172.21.15.57:0/3746242829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbfc06d230 0x7fcbfc0adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.411+0000 7f13ec4fc700 1 -- start start 2023-11-28T12:01:50.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.411+0000 7fcc00c27700 1 -- 172.21.15.57:0/3746242829 >> v1:172.21.15.181:6789/0 conn(0x7fcbfc0b0c50 legacy=0x7fcbfc0cdc40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T12:01:50.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.411+0000 7fcc00c27700 1 -- 172.21.15.57:0/3746242829 >> v1:172.21.15.57:6789/0 conn(0x7fcbfc158980 legacy=0x7fcbfc0b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.411+0000 7fcbf3fff700 1 -- 172.21.15.57:0/3746242829 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350301566 0 0) 0x7fcbfc06d6c0 con 0x7fcbfc158980 2023-11-28T12:01:50.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.411+0000 7fcc00c27700 1 -- 172.21.15.57:0/3746242829 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc0caad0 con 0x7fcbfc06d230 2023-11-28T12:01:50.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.412+0000 7fcc00c27700 1 --2- 172.21.15.57:0/3746242829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbfc06d230 0x7fcbfc0adf60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400aed0 tx=0x7fcbf400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9fd929ca555d66b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.412+0000 7fcbf3fff700 1 -- 172.21.15.57:0/3746242829 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf400f070 con 0x7fcbfc06d230 2023-11-28T12:01:50.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.412+0000 7fcbf3fff700 1 -- 172.21.15.57:0/3746242829 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf400ca40 con 0x7fcbfc06d230 2023-11-28T12:01:50.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.412+0000 7fcbf3fff700 1 -- 172.21.15.57:0/3746242829 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf400a320 con 0x7fcbfc06d230 2023-11-28T12:01:50.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.412+0000 7f529f798700 1 Processor -- start 2023-11-28T12:01:50.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.412+0000 7f6148797700 1 Processor -- start 2023-11-28T12:01:50.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.412+0000 7f13ec4fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f13e40cf710 0x7f13e40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.412+0000 7f13ec4fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13e406a980 0x7f13e4071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.412+0000 7f13ec4fc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e415cc10 con 0x7f13e4071890 2023-11-28T12:01:50.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.412+0000 7f13ec4fc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e406c9b0 con 0x7f13e40cf710 2023-11-28T12:01:50.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.412+0000 7f13ec4fc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e406cc90 con 0x7f13e406a980 2023-11-28T12:01:50.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.412+0000 7f6148797700 1 -- start start 2023-11-28T12:01:50.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.413+0000 7f13ea298700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f13e40cf710 0x7f13e40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.413+0000 7f13e9a97700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f13e4071890 0x7f13e406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:43910/0 (socket says 172.21.15.57:43910) 2023-11-28T12:01:50.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.413+0000 7f13e9a97700 1 -- 172.21.15.57:0/4022388774 learned_addr learned my addr 172.21.15.57:0/4022388774 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:50.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.413+0000 7f13eaa99700 1 --2- 172.21.15.57:0/4022388774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13e406a980 0x7f13e4071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.413+0000 7fcc0368c700 1 -- 172.21.15.57:0/3746242829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbfc06d230 msgr2=0x7fcbfc0adf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.413+0000 7fcc0368c700 1 --2- 172.21.15.57:0/3746242829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbfc06d230 0x7fcbfc0adf60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400aed0 tx=0x7fcbf400c330 comp rx=0 tx=0).stop 2023-11-28T12:01:50.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.413+0000 7fcc0368c700 1 -- 172.21.15.57:0/3746242829 shutdown_connections 2023-11-28T12:01:50.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7fcc0368c700 1 --2- 172.21.15.57:0/3746242829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbfc06d230 0x7fcbfc0adf60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7fcc0368c700 1 -- 172.21.15.57:0/3746242829 >> 172.21.15.57:0/3746242829 conn(0x7fcbfc06aed0 msgr2=0x7fcbfc0cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f13e9296700 1 -- 172.21.15.57:0/4022388774 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145529625 0 0) 0x7f13e415cc10 con 0x7f13e4071890 2023-11-28T12:01:50.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f13e9296700 1 -- 172.21.15.57:0/4022388774 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d00035b0 con 0x7f13e4071890 2023-11-28T12:01:50.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7fcc0368c700 1 -- 172.21.15.57:0/3746242829 shutdown_connections 2023-11-28T12:01:50.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f529f798700 1 -- start start 2023-11-28T12:01:50.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f6215044700 1 Processor -- start 2023-11-28T12:01:50.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f13ea298700 1 -- 172.21.15.57:0/4022388774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13e406a980 msgr2=0x7f13e4071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f13ea298700 1 --2- 172.21.15.57:0/4022388774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13e406a980 0x7f13e4071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f13ea298700 1 -- 172.21.15.57:0/4022388774 >> v1:172.21.15.181:6789/0 conn(0x7f13e4071890 legacy=0x7f13e406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f13ea298700 1 -- 172.21.15.57:0/4022388774 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e406cf70 con 0x7f13e40cf710 2023-11-28T12:01:50.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f6215044700 1 -- start start 2023-11-28T12:01:50.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f6148797700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6140158980 0x7f61400adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7fcc0368c700 1 -- 172.21.15.57:0/3746242829 wait complete. 2023-11-28T12:01:50.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f13ea298700 1 --2- 172.21.15.57:0/4022388774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e40cf710 0x7f13e40ce740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009870 tx=0x7f13e0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fee0b1faa8210a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f6148797700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f61400b0c50 0x7f614006b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f6148797700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614006d6c0 con 0x7f614006d230 2023-11-28T12:01:50.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f6148797700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614006c700 con 0x7f61400b0c50 2023-11-28T12:01:50.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.414+0000 7f6148797700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61400ca7f0 con 0x7f6140158980 2023-11-28T12:01:50.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6146533700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6140158980 0x7f61400adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6146533700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6140158980 0x7f61400adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:49402/0 (socket says 172.21.15.57:49402) 2023-11-28T12:01:50.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6146533700 1 -- 172.21.15.57:0/3571365071 learned_addr learned my addr 172.21.15.57:0/3571365071 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6146d34700 1 --2- 172.21.15.57:0/3571365071 >> v2:172.21.15.57:3300/0 conn(0x7f61400b0c50 0x7f614006b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6145531700 1 -- 172.21.15.57:0/3571365071 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634138757 0 0) 0x7f614006d6c0 con 0x7f614006d230 2023-11-28T12:01:50.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6145531700 1 -- 172.21.15.57:0/3571365071 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f612c0035b0 con 0x7f614006d230 2023-11-28T12:01:50.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6146d34700 1 -- 172.21.15.57:0/3571365071 >> v2:172.21.15.181:3300/0 conn(0x7f6140158980 msgr2=0x7f61400adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6146d34700 1 --2- 172.21.15.57:0/3571365071 >> v2:172.21.15.181:3300/0 conn(0x7f6140158980 0x7f61400adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6146d34700 1 -- 172.21.15.57:0/3571365071 >> v1:172.21.15.57:6789/0 conn(0x7f614006d230 legacy=0x7f61400b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6146d34700 1 -- 172.21.15.57:0/3571365071 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61400caad0 con 0x7f61400b0c50 2023-11-28T12:01:50.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f6146d34700 1 --2- 172.21.15.57:0/3571365071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61400b0c50 0x7f614006b770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6140071f90 tx=0x7f613c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa7f62a798746af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f6145531700 1 -- 172.21.15.57:0/3571365071 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c011830 con 0x7f61400b0c50 2023-11-28T12:01:50.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f6145531700 1 -- 172.21.15.57:0/3571365071 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613c011990 con 0x7f61400b0c50 2023-11-28T12:01:50.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f6145531700 1 -- 172.21.15.57:0/3571365071 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c011d00 con 0x7f61400b0c50 2023-11-28T12:01:50.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7fcc0368c700 1 Processor -- start 2023-11-28T12:01:50.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6215044700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f62080546a0 0x7f6208036610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6215044700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6208005f60 con 0x7f620803a6b0 2023-11-28T12:01:50.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6215044700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6208036fa0 con 0x7f6208036b50 2023-11-28T12:01:50.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f6215044700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6208006f50 con 0x7f62080546a0 2023-11-28T12:01:50.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f620ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f62080546a0 0x7f6208036610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f620ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f62080546a0 0x7f6208036610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47962/0 (socket says 172.21.15.57:47962) 2023-11-28T12:01:50.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f620ffff700 1 -- 172.21.15.57:0/95716236 learned_addr learned my addr 172.21.15.57:0/95716236 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f620ffff700 1 -- 172.21.15.57:0/95716236 >> v1:172.21.15.181:6789/0 conn(0x7f6208036b50 legacy=0x7f6208039fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T12:01:50.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f620ffff700 1 -- 172.21.15.57:0/95716236 >> v1:172.21.15.57:6789/0 conn(0x7f620803a6b0 legacy=0x7f6208052760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f620ffff700 1 -- 172.21.15.57:0/95716236 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62080deba0 con 0x7f62080546a0 2023-11-28T12:01:50.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f620ffff700 1 --2- 172.21.15.57:0/95716236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62080546a0 0x7f6208036610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f62000088f0 tx=0x7f620000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccf1150a9ba9557 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f13e9296700 1 -- 172.21.15.57:0/4022388774 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0011830 con 0x7f13e40cf710 2023-11-28T12:01:50.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f13e9296700 1 -- 172.21.15.57:0/4022388774 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e0011990 con 0x7f13e40cf710 2023-11-28T12:01:50.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f13e9296700 1 -- 172.21.15.57:0/4022388774 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0011d20 con 0x7f13e40cf710 2023-11-28T12:01:50.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f13ec4fc700 1 -- 172.21.15.57:0/4022388774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e40cf710 msgr2=0x7f13e40ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f13ec4fc700 1 --2- 172.21.15.57:0/4022388774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e40cf710 0x7f13e40ce740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009870 tx=0x7f13e0009b80 comp rx=0 tx=0).stop 2023-11-28T12:01:50.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f529f798700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f529806d230 0x7f52980adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f529f798700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52980cbc50 0x7f529806b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f529f798700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52980ca7f0 con 0x7f5298158710 2023-11-28T12:01:50.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f529f798700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529806c700 con 0x7f529806d230 2023-11-28T12:01:50.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.415+0000 7f529f798700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52980caad0 con 0x7f52980cbc50 2023-11-28T12:01:50.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f620effd700 1 -- 172.21.15.57:0/95716236 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200010070 con 0x7f62080546a0 2023-11-28T12:01:50.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f620effd700 1 -- 172.21.15.57:0/95716236 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620000da00 con 0x7f62080546a0 2023-11-28T12:01:50.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f620effd700 1 -- 172.21.15.57:0/95716236 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620000dd90 con 0x7f62080546a0 2023-11-28T12:01:50.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f13ec4fc700 1 -- 172.21.15.57:0/4022388774 shutdown_connections 2023-11-28T12:01:50.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f13ec4fc700 1 --2- 172.21.15.57:0/4022388774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e40cf710 0x7f13e40ce740 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f13ec4fc700 1 --2- 172.21.15.57:0/4022388774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13e406a980 0x7f13e4071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f13ec4fc700 1 -- 172.21.15.57:0/4022388774 >> 172.21.15.57:0/4022388774 conn(0x7f13e406c010 msgr2=0x7f13e40cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7fcc0368c700 1 -- start start 2023-11-28T12:01:50.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7fcc0368c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbfc06d230 0x7fcbfc0b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7fcc0368c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbfc0b0c50 0x7fcbfc14fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7fcc0368c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbfc158980 0x7fcbfc0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7fcc0368c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0b43a0 con 0x7fcbfc06d230 2023-11-28T12:01:50.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7fcc0368c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0b3720 con 0x7fcbfc158980 2023-11-28T12:01:50.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7fcc0368c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0b3a00 con 0x7fcbfc0b0c50 2023-11-28T12:01:50.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7fcc01428700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbfc06d230 0x7fcbfc0b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7fcc01428700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbfc06d230 0x7fcbfc0b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47974/0 (socket says 172.21.15.57:47974) 2023-11-28T12:01:50.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7fcc01428700 1 -- 172.21.15.57:0/534515674 learned_addr learned my addr 172.21.15.57:0/534515674 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7fcc01c29700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbfc158980 0x7fcbfc0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7fcc00c27700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbfc0b0c50 0x7fcbfc14fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7fcc01428700 1 -- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbfc0b0c50 msgr2=0x7fcbfc14fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7fcc01428700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbfc0b0c50 0x7fcbfc14fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7fcc01428700 1 -- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbfc158980 msgr2=0x7fcbfc0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7fcc01428700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbfc158980 0x7fcbfc0cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7fcc01428700 1 -- 172.21.15.57:0/534515674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf400ab80 con 0x7fcbfc06d230 2023-11-28T12:01:50.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f50c9e7b700 1 Processor -- start 2023-11-28T12:01:50.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f529dd35700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5298158710 0x7f52980b0540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:43922/0 (socket says 172.21.15.57:43922) 2023-11-28T12:01:50.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f529dd35700 1 -- 172.21.15.57:0/1195266383 learned_addr learned my addr 172.21.15.57:0/1195266383 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:50.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.416+0000 7f529cd33700 1 --2- 172.21.15.57:0/1195266383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52980cbc50 0x7f529806b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f528ffff700 1 -- 172.21.15.57:0/1195266383 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703605821 0 0) 0x7f52980ca7f0 con 0x7f5298158710 2023-11-28T12:01:50.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f528ffff700 1 -- 172.21.15.57:0/1195266383 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52840035b0 con 0x7f5298158710 2023-11-28T12:01:50.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f529cd33700 1 -- 172.21.15.57:0/1195266383 >> v2:172.21.15.181:3300/0 conn(0x7f529806d230 msgr2=0x7f52980adf60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T12:01:50.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f529cd33700 1 --2- 172.21.15.57:0/1195266383 >> v2:172.21.15.181:3300/0 conn(0x7f529806d230 0x7f52980adf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f529cd33700 1 -- 172.21.15.57:0/1195266383 >> v1:172.21.15.181:6789/0 conn(0x7f5298158710 legacy=0x7f52980b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f529cd33700 1 -- 172.21.15.57:0/1195266383 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52980cadb0 con 0x7f52980cbc50 2023-11-28T12:01:50.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.417+0000 7f529cd33700 1 --2- 172.21.15.57:0/1195266383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52980cbc50 0x7f529806b770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5294009840 tx=0x7f5294009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=349ce4dcfe77f826 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7fcc01428700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbfc06d230 0x7fcbfc0b2ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800c570 tx=0x7fcbf800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f528ffff700 1 -- 172.21.15.57:0/1195266383 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294011830 con 0x7f52980cbc50 2023-11-28T12:01:50.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f528ffff700 1 -- 172.21.15.57:0/1195266383 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5294011990 con 0x7f52980cbc50 2023-11-28T12:01:50.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f528ffff700 1 -- 172.21.15.57:0/1195266383 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294011d20 con 0x7f52980cbc50 2023-11-28T12:01:50.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f6215044700 1 -- 172.21.15.57:0/95716236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62080546a0 msgr2=0x7f6208036610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f6215044700 1 --2- 172.21.15.57:0/95716236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62080546a0 0x7f6208036610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f62000088f0 tx=0x7f620000d330 comp rx=0 tx=0).stop 2023-11-28T12:01:50.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f13ec4fc700 1 -- 172.21.15.57:0/4022388774 shutdown_connections 2023-11-28T12:01:50.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f6148797700 1 -- 172.21.15.57:0/3571365071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61400b0c50 msgr2=0x7f614006b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f6148797700 1 --2- 172.21.15.57:0/3571365071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61400b0c50 0x7f614006b770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6140071f90 tx=0x7f613c009b20 comp rx=0 tx=0).stop 2023-11-28T12:01:50.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7fcbf1ffb700 1 -- 172.21.15.57:0/534515674 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf80070d0 con 0x7fcbfc06d230 2023-11-28T12:01:50.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7fcbf1ffb700 1 -- 172.21.15.57:0/534515674 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf8007230 con 0x7fcbfc06d230 2023-11-28T12:01:50.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7fcbf1ffb700 1 -- 172.21.15.57:0/534515674 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf80075e0 con 0x7fcbfc06d230 2023-11-28T12:01:50.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f6215044700 1 -- 172.21.15.57:0/95716236 shutdown_connections 2023-11-28T12:01:50.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f6215044700 1 --2- 172.21.15.57:0/95716236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62080546a0 0x7f6208036610 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f6215044700 1 -- 172.21.15.57:0/95716236 >> 172.21.15.57:0/95716236 conn(0x7f6208051610 msgr2=0x7f6208051a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7fcc0368c700 1 -- 172.21.15.57:0/534515674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbfc0cb090 con 0x7fcbfc06d230 2023-11-28T12:01:50.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.419+0000 7fcc0368c700 1 -- 172.21.15.57:0/534515674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbfc0cadb0 con 0x7fcbfc06d230 2023-11-28T12:01:50.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f50c9e7b700 1 -- start start 2023-11-28T12:01:50.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.419+0000 7f50c9e7b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f50c40af0a0 0x7f50c415a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.419+0000 7f50c9e7b700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c4159ec0 con 0x7f50c40b1650 2023-11-28T12:01:50.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.419+0000 7f50c9e7b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c415d8a0 con 0x7f50c41599b0 2023-11-28T12:01:50.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.419+0000 7f50c9e7b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c415db80 con 0x7f50c40af0a0 2023-11-28T12:01:50.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.419+0000 7f6148797700 1 -- 172.21.15.57:0/3571365071 shutdown_connections 2023-11-28T12:01:50.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.419+0000 7f6148797700 1 --2- 172.21.15.57:0/3571365071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61400b0c50 0x7f614006b770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.419+0000 7f6148797700 1 --2- 172.21.15.57:0/3571365071 >> v2:172.21.15.181:3300/0 conn(0x7f6140158980 0x7f61400adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.419+0000 7f6148797700 1 -- 172.21.15.57:0/3571365071 >> 172.21.15.57:0/3571365071 conn(0x7f614006aed0 msgr2=0x7f61400cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.420+0000 7f6148797700 1 -- 172.21.15.57:0/3571365071 shutdown_connections 2023-11-28T12:01:50.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.420+0000 7f6148797700 1 -- 172.21.15.57:0/3571365071 wait complete. 2023-11-28T12:01:50.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.419+0000 7f6215044700 1 -- 172.21.15.57:0/95716236 shutdown_connections 2023-11-28T12:01:50.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.419+0000 7f6215044700 1 -- 172.21.15.57:0/95716236 wait complete. 2023-11-28T12:01:50.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.421+0000 7f6215044700 1 Processor -- start 2023-11-28T12:01:50.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.418+0000 7f13ec4fc700 1 -- 172.21.15.57:0/4022388774 wait complete. 2023-11-28T12:01:50.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.420+0000 7f13ec4fc700 1 Processor -- start 2023-11-28T12:01:50.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.419+0000 7f50baffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f50c41599b0 0x7f50c415d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:43954/0 (socket says 172.21.15.57:43954) 2023-11-28T12:01:50.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.419+0000 7f50baffd700 1 -- 172.21.15.57:0/1777955037 learned_addr learned my addr 172.21.15.57:0/1777955037 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:50.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.420+0000 7f50c3fff700 1 --2- 172.21.15.57:0/1777955037 >> v2:172.21.15.57:3300/0 conn(0x7f50c40af0a0 0x7f50c415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.420+0000 7f50c3fff700 1 -- 172.21.15.57:0/1777955037 >> v1:172.21.15.181:6789/0 conn(0x7f50c41599b0 legacy=0x7f50c415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.420+0000 7f50c3fff700 1 -- 172.21.15.57:0/1777955037 >> v1:172.21.15.57:6789/0 conn(0x7f50c40b1650 legacy=0x7f50c40b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.420+0000 7f50c3fff700 1 -- 172.21.15.57:0/1777955037 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c415de60 con 0x7f50c40af0a0 2023-11-28T12:01:50.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.421+0000 7f50c3fff700 1 --2- 172.21.15.57:0/1777955037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50c40af0a0 0x7f50c415a350 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0097c0 tx=0x7f50bc009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f82e52ef71ea4be server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7f6148797700 1 Processor -- start 2023-11-28T12:01:50.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.421+0000 7fcbf1ffb700 1 -- 172.21.15.57:0/534515674 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcbf801a020 con 0x7fcbfc06d230 2023-11-28T12:01:50.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7fcbf1ffb700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcbe00683c0 0x7fcbe006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7fcc0368c700 1 -- 172.21.15.57:0/534515674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbfc158d50 con 0x7fcbfc06d230 2023-11-28T12:01:50.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7fcbf1ffb700 1 -- 172.21.15.57:0/534515674 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbf80858c0 con 0x7fcbfc06d230 2023-11-28T12:01:50.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.420+0000 7f13ec4fc700 1 -- start start 2023-11-28T12:01:50.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.421+0000 7f13ec4fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e406a980 0x7f13e415b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.421+0000 7f13ec4fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e4071890 0x7f13e40af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.421+0000 7f13ec4fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13e40cf710 0x7f13e40af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.421+0000 7f13ec4fc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e415c720 con 0x7f13e4071890 2023-11-28T12:01:50.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.421+0000 7f13ec4fc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e40ae5a0 con 0x7f13e406a980 2023-11-28T12:01:50.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.421+0000 7f13ec4fc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e41ff130 con 0x7f13e40cf710 2023-11-28T12:01:50.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7f13eaa99700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13e40cf710 0x7f13e40af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7f13ea298700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e406a980 0x7f13e415b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7f13ea298700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e406a980 0x7f13e415b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:49430/0 (socket says 172.21.15.57:49430) 2023-11-28T12:01:50.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7f13ea298700 1 -- 172.21.15.57:0/3231298948 learned_addr learned my addr 172.21.15.57:0/3231298948 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7f529f798700 1 -- 172.21.15.57:0/1195266383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52980cbc50 msgr2=0x7f529806b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7f529f798700 1 --2- 172.21.15.57:0/1195266383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52980cbc50 0x7f529806b770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5294009840 tx=0x7f5294009b50 comp rx=0 tx=0).stop 2023-11-28T12:01:50.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7fcc00c27700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcbe00683c0 0x7fcbe006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.421+0000 7f6215044700 1 -- start start 2023-11-28T12:01:50.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7f6215044700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6208036b50 0x7f62080e0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7f6215044700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f620803a6b0 0x7f62080479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7f6215044700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62080546a0 0x7f6208047f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6215044700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6208048440 con 0x7f620803a6b0 2023-11-28T12:01:50.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6215044700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620817d3a0 con 0x7f6208036b50 2023-11-28T12:01:50.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6215044700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6208180d80 con 0x7f62080546a0 2023-11-28T12:01:50.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7f13e9a97700 1 --2- 172.21.15.57:0/3231298948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e4071890 0x7f13e40af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f13eaa99700 1 -- 172.21.15.57:0/3231298948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e406a980 msgr2=0x7f13e415b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f13eaa99700 1 --2- 172.21.15.57:0/3231298948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e406a980 0x7f13e415b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f13eaa99700 1 -- 172.21.15.57:0/3231298948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e4071890 msgr2=0x7f13e40af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f13eaa99700 1 --2- 172.21.15.57:0/3231298948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e4071890 0x7f13e40af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f13eaa99700 1 -- 172.21.15.57:0/3231298948 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e00094a0 con 0x7f13e40cf710 2023-11-28T12:01:50.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.422+0000 7f6148797700 1 -- start start 2023-11-28T12:01:50.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6148797700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f614006d230 0x7f6140070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6148797700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61400b0c50 0x7f61401f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6148797700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6140158980 0x7f61400c3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6148797700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61400c3000 con 0x7f6140158980 2023-11-28T12:01:50.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6148797700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61400ae500 con 0x7f614006d230 2023-11-28T12:01:50.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6148797700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6140071030 con 0x7f61400b0c50 2023-11-28T12:01:50.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6214843700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62080546a0 0x7f6208047f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6214843700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62080546a0 0x7f6208047f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:60858/0 (socket says 172.21.15.57:60858) 2023-11-28T12:01:50.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6214843700 1 -- 172.21.15.57:0/1791658669 learned_addr learned my addr 172.21.15.57:0/1791658669 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6146533700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f614006d230 0x7f6140070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6146533700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f614006d230 0x7f6140070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:49456/0 (socket says 172.21.15.57:49456) 2023-11-28T12:01:50.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6146533700 1 -- 172.21.15.57:0/2437829495 learned_addr learned my addr 172.21.15.57:0/2437829495 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7fcc00c27700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcbe00683c0 0x7fcbe006a870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4000f80 tx=0x7fcbf4009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f13eaa99700 1 --2- 172.21.15.57:0/3231298948 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13e40cf710 0x7f13e40af880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f13e406b2d0 tx=0x7f13dc00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6214843700 1 -- 172.21.15.57:0/1791658669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6208036b50 msgr2=0x7f62080e0240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T12:01:50.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6214843700 1 --2- 172.21.15.57:0/1791658669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6208036b50 0x7f62080e0240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6214843700 1 -- 172.21.15.57:0/1791658669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f620803a6b0 msgr2=0x7f62080479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6214843700 1 --2- 172.21.15.57:0/1791658669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f620803a6b0 0x7f62080479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.423+0000 7f6214843700 1 -- 172.21.15.57:0/1791658669 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62000085d0 con 0x7f62080546a0 2023-11-28T12:01:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.424+0000 7f6145d32700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61400b0c50 0x7f61401f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.424+0000 7f6146d34700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6140158980 0x7f61400c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.424+0000 7f6146533700 1 -- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61400b0c50 msgr2=0x7f61401f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.424+0000 7f6146533700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61400b0c50 0x7f61401f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.424+0000 7f6146533700 1 -- 172.21.15.57:0/2437829495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6140158980 msgr2=0x7f61400c3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.424+0000 7f6146533700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6140158980 0x7f61400c3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.424+0000 7f6146533700 1 -- 172.21.15.57:0/2437829495 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c0094a0 con 0x7f614006d230 2023-11-28T12:01:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.424+0000 7f6146533700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f614006d230 0x7f6140070af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6138008ac0 tx=0x7f613800b3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.424+0000 7f6136ffd700 1 -- 172.21.15.57:0/2437829495 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138010070 con 0x7f614006d230 2023-11-28T12:01:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.424+0000 7f6136ffd700 1 -- 172.21.15.57:0/2437829495 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613800a0d0 con 0x7f614006d230 2023-11-28T12:01:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.424+0000 7f6136ffd700 1 -- 172.21.15.57:0/2437829495 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613800a440 con 0x7f614006d230 2023-11-28T12:01:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.424+0000 7f6148797700 1 -- 172.21.15.57:0/2437829495 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61400cb090 con 0x7f614006d230 2023-11-28T12:01:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.424+0000 7f6148797700 1 -- 172.21.15.57:0/2437829495 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61400cadb0 con 0x7f614006d230 2023-11-28T12:01:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.425+0000 7f6148797700 1 -- 172.21.15.57:0/2437829495 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6140158d50 con 0x7f614006d230 2023-11-28T12:01:50.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.426+0000 7f6214843700 1 --2- 172.21.15.57:0/1791658669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62080546a0 0x7f6208047f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f621002d030 tx=0x7f6210068730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.426+0000 7f620cff9700 1 -- 172.21.15.57:0/1791658669 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210090070 con 0x7f62080546a0 2023-11-28T12:01:50.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.426+0000 7f620cff9700 1 -- 172.21.15.57:0/1791658669 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62100723d0 con 0x7f62080546a0 2023-11-28T12:01:50.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.426+0000 7f620cff9700 1 -- 172.21.15.57:0/1791658669 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210072760 con 0x7f62080546a0 2023-11-28T12:01:50.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.426+0000 7f6215044700 1 -- 172.21.15.57:0/1791658669 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6208006470 con 0x7f62080546a0 2023-11-28T12:01:50.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.426+0000 7f6215044700 1 -- 172.21.15.57:0/1791658669 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6208006190 con 0x7f62080546a0 2023-11-28T12:01:50.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.427+0000 7f13daffd700 1 -- 172.21.15.57:0/3231298948 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc00cc20 con 0x7f13e40cf710 2023-11-28T12:01:50.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.427+0000 7f13daffd700 1 -- 172.21.15.57:0/3231298948 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc00f040 con 0x7f13e40cf710 2023-11-28T12:01:50.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.427+0000 7f13daffd700 1 -- 172.21.15.57:0/3231298948 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc009e60 con 0x7f13e40cf710 2023-11-28T12:01:50.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.427+0000 7f13ec4fc700 1 -- 172.21.15.57:0/3231298948 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e406d530 con 0x7f13e40cf710 2023-11-28T12:01:50.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.427+0000 7f13ec4fc700 1 -- 172.21.15.57:0/3231298948 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e406d250 con 0x7f13e40cf710 2023-11-28T12:01:50.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.427+0000 7f6136ffd700 1 -- 172.21.15.57:0/2437829495 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f613801b030 con 0x7f614006d230 2023-11-28T12:01:50.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.428+0000 7f6215044700 1 -- 172.21.15.57:0/1791658669 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6208006750 con 0x7f62080546a0 2023-11-28T12:01:50.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.428+0000 7f6136ffd700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f612c068770 0x7f612c06ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.428+0000 7f6136ffd700 1 -- 172.21.15.57:0/2437829495 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61380862a0 con 0x7f614006d230 2023-11-28T12:01:50.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.428+0000 7f6145d32700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f612c068770 0x7f612c06ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.428+0000 7f6145d32700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f612c068770 0x7f612c06ac20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6130007fd0 tx=0x7f6130009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.429+0000 7f13ec4fc700 1 -- 172.21.15.57:0/3231298948 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13cc0052f0 con 0x7f13e40cf710 2023-11-28T12:01:50.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.429+0000 7f620cff9700 1 -- 172.21.15.57:0/1791658669 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6210095410 con 0x7f62080546a0 2023-11-28T12:01:50.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.429+0000 7f620cff9700 1 --2- 172.21.15.57:0/1791658669 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f61fc0682f0 0x7f61fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.429+0000 7f620cff9700 1 -- 172.21.15.57:0/1791658669 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6210100150 con 0x7f62080546a0 2023-11-28T12:01:50.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.429+0000 7f620ffff700 1 --2- 172.21.15.57:0/1791658669 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f61fc0682f0 0x7f61fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.429+0000 7f529f798700 1 -- 172.21.15.57:0/1195266383 shutdown_connections 2023-11-28T12:01:50.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.429+0000 7f529f798700 1 --2- 172.21.15.57:0/1195266383 >> v2:172.21.15.181:3300/0 conn(0x7f529806d230 0x7f52980adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.429+0000 7f529f798700 1 --2- 172.21.15.57:0/1195266383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52980cbc50 0x7f529806b770 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.430+0000 7f529f798700 1 -- 172.21.15.57:0/1195266383 >> 172.21.15.57:0/1195266383 conn(0x7f529806aed0 msgr2=0x7f52980cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.430+0000 7f13daffd700 1 -- 172.21.15.57:0/3231298948 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f13dc014480 con 0x7f13e40cf710 2023-11-28T12:01:50.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.430+0000 7f620ffff700 1 --2- 172.21.15.57:0/1791658669 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f61fc0682f0 0x7f61fc06a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f620000d370 tx=0x7f6200012000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.430+0000 7f50c2ffd700 1 -- 172.21.15.57:0/1777955037 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc004230 con 0x7f50c40af0a0 2023-11-28T12:01:50.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.430+0000 7f50c2ffd700 1 -- 172.21.15.57:0/1777955037 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50bc004390 con 0x7f50c40af0a0 2023-11-28T12:01:50.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.430+0000 7f50c2ffd700 1 -- 172.21.15.57:0/1777955037 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc0036a0 con 0x7f50c40af0a0 2023-11-28T12:01:50.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.430+0000 7fab4e489700 1 Processor -- start 2023-11-28T12:01:50.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.430+0000 7f529f798700 1 -- 172.21.15.57:0/1195266383 shutdown_connections 2023-11-28T12:01:50.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.430+0000 7f13daffd700 1 --2- 172.21.15.57:0/3231298948 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f13c80683c0 0x7f13c806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.430+0000 7f13daffd700 1 -- 172.21.15.57:0/3231298948 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f13dc0868b0 con 0x7f13e40cf710 2023-11-28T12:01:50.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.430+0000 7f529f798700 1 -- 172.21.15.57:0/1195266383 wait complete. 2023-11-28T12:01:50.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.430+0000 7fab4e489700 1 -- start start 2023-11-28T12:01:50.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.431+0000 7fcbf1ffb700 1 -- 172.21.15.57:0/534515674 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcbf8052050 con 0x7fcbfc06d230 2023-11-28T12:01:50.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.431+0000 7fab4e489700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fab4806b220 0x7fab480cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.431+0000 7f6136ffd700 1 -- 172.21.15.57:0/2437829495 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6138056570 con 0x7f614006d230 2023-11-28T12:01:50.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.431+0000 7fab4e489700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab480d22c0 con 0x7fab48160ac0 2023-11-28T12:01:50.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.431+0000 7fab4e489700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab480cf550 con 0x7fab481652c0 2023-11-28T12:01:50.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.431+0000 7fab4e489700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab480d0840 con 0x7fab4806b220 2023-11-28T12:01:50.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7fab4ca26700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fab481652c0 0x7fab480d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:43966/0 (socket says 172.21.15.57:43966) 2023-11-28T12:01:50.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7fab4ca26700 1 -- 172.21.15.57:0/1049840363 learned_addr learned my addr 172.21.15.57:0/1049840363 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:50.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7f13ea298700 1 --2- 172.21.15.57:0/3231298948 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f13c80683c0 0x7f13c806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7fab477fe700 1 --2- 172.21.15.57:0/1049840363 >> v2:172.21.15.57:3300/0 conn(0x7fab4806b220 0x7fab480cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7f50c9e7b700 1 -- 172.21.15.57:0/1777955037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50c40af0a0 msgr2=0x7f50c415a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7f50c9e7b700 1 --2- 172.21.15.57:0/1777955037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50c40af0a0 0x7f50c415a350 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0097c0 tx=0x7f50bc009ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7fab477fe700 1 -- 172.21.15.57:0/1049840363 >> v1:172.21.15.181:6789/0 conn(0x7fab481652c0 legacy=0x7fab480d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7fab477fe700 1 -- 172.21.15.57:0/1049840363 >> v1:172.21.15.57:6789/0 conn(0x7fab48160ac0 legacy=0x7fab4806ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:50.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7fab477fe700 1 -- 172.21.15.57:0/1049840363 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab481647e0 con 0x7fab4806b220 2023-11-28T12:01:50.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7fab477fe700 1 --2- 172.21.15.57:0/1049840363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab4806b220 0x7fab480cef20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fab38009d90 tx=0x7fab3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cc1885ff8d5652 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7f13ea298700 1 --2- 172.21.15.57:0/3231298948 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f13c80683c0 0x7f13c806a870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009bc0 tx=0x7f13e001c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7fab46ffd700 1 -- 172.21.15.57:0/1049840363 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800e070 con 0x7fab4806b220 2023-11-28T12:01:50.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7fab46ffd700 1 -- 172.21.15.57:0/1049840363 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3800bb90 con 0x7fab4806b220 2023-11-28T12:01:50.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7fab46ffd700 1 -- 172.21.15.57:0/1049840363 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38003a20 con 0x7fab4806b220 2023-11-28T12:01:50.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7f50c9e7b700 1 -- 172.21.15.57:0/1777955037 shutdown_connections 2023-11-28T12:01:50.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7f50c9e7b700 1 --2- 172.21.15.57:0/1777955037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50c40af0a0 0x7f50c415a350 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.432+0000 7f50c9e7b700 1 -- 172.21.15.57:0/1777955037 >> 172.21.15.57:0/1777955037 conn(0x7f50c406c010 msgr2=0x7f50c40cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.433+0000 7f50c9e7b700 1 -- 172.21.15.57:0/1777955037 shutdown_connections 2023-11-28T12:01:50.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.433+0000 7f529f798700 1 Processor -- start 2023-11-28T12:01:50.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.433+0000 7fab4e489700 1 -- 172.21.15.57:0/1049840363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab4806b220 msgr2=0x7fab480cef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.433+0000 7fab4e489700 1 --2- 172.21.15.57:0/1049840363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab4806b220 0x7fab480cef20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fab38009d90 tx=0x7fab3800b330 comp rx=0 tx=0).stop 2023-11-28T12:01:50.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.434+0000 7f620cff9700 1 -- 172.21.15.57:0/1791658669 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f62100d02f0 con 0x7f62080546a0 2023-11-28T12:01:50.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.433+0000 7f529f798700 1 -- start start 2023-11-28T12:01:50.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.434+0000 7f529f798700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f529806d230 0x7f529806fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.434+0000 7f529f798700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52980cbc50 0x7f52980b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.434+0000 7f529f798700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5298158710 0x7f52980afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.434+0000 7f529f798700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52980b2770 con 0x7f5298158710 2023-11-28T12:01:50.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.434+0000 7f529f798700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52980b1ba0 con 0x7f52980cbc50 2023-11-28T12:01:50.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.434+0000 7f529f798700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52980b1e40 con 0x7f529806d230 2023-11-28T12:01:50.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.434+0000 7f529dd35700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5298158710 0x7f52980afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.433+0000 7f50c9e7b700 1 -- 172.21.15.57:0/1777955037 wait complete. 2023-11-28T12:01:50.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.434+0000 7f529cd33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52980cbc50 0x7f52980b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.434+0000 7f529cd33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52980cbc50 0x7f52980b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:49458/0 (socket says 172.21.15.57:49458) 2023-11-28T12:01:50.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.434+0000 7f529cd33700 1 -- 172.21.15.57:0/383775948 learned_addr learned my addr 172.21.15.57:0/383775948 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.434+0000 7f529d534700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f529806d230 0x7f529806fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.434+0000 7f50c9e7b700 1 Processor -- start 2023-11-28T12:01:50.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7fab4e489700 1 -- 172.21.15.57:0/1049840363 shutdown_connections 2023-11-28T12:01:50.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7fab4e489700 1 --2- 172.21.15.57:0/1049840363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab4806b220 0x7fab480cef20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7fab4e489700 1 -- 172.21.15.57:0/1049840363 >> 172.21.15.57:0/1049840363 conn(0x7fab4806c520 msgr2=0x7fab4806c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7f529d534700 1 -- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52980cbc50 msgr2=0x7f52980b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7f529d534700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52980cbc50 0x7f52980b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7f529d534700 1 -- 172.21.15.57:0/383775948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5298158710 msgr2=0x7f52980afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7f529d534700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5298158710 0x7f52980afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7f529d534700 1 -- 172.21.15.57:0/383775948 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52940094a0 con 0x7f529806d230 2023-11-28T12:01:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7f529d534700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f529806d230 0x7f529806fd30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5288006fd0 tx=0x7f528800b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7f529dd35700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5298158710 0x7f52980afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:01:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7f529cd33700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52980cbc50 0x7f52980b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7fab4e489700 1 -- 172.21.15.57:0/1049840363 shutdown_connections 2023-11-28T12:01:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7fab4e489700 1 -- 172.21.15.57:0/1049840363 wait complete. 2023-11-28T12:01:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.435+0000 7f50c9e7b700 1 -- start start 2023-11-28T12:01:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.436+0000 7f50c9e7b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50c40af0a0 0x7f50c4158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.436+0000 7f50c9e7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c40b1650 0x7f50c40b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.436+0000 7f50c9e7b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50c40b5110 0x7f50c40c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.436+0000 7f50c9e7b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40c5150 con 0x7f50c40b5110 2023-11-28T12:01:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.436+0000 7f50c9e7b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40b5570 con 0x7f50c40b1650 2023-11-28T12:01:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.436+0000 7f50c9e7b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40c44d0 con 0x7f50c40af0a0 2023-11-28T12:01:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.436+0000 7f50c3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50c40b5110 0x7f50c40c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.436+0000 7f50c3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50c40b5110 0x7f50c40c6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48030/0 (socket says 172.21.15.57:48030) 2023-11-28T12:01:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.436+0000 7f50c3fff700 1 -- 172.21.15.57:0/2296640515 learned_addr learned my addr 172.21.15.57:0/2296640515 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.436+0000 7f50c37fe700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50c40af0a0 0x7f50c4158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.436+0000 7f50baffd700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c40b1650 0x7f50c40b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.437+0000 7f50baffd700 1 -- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50c40af0a0 msgr2=0x7f50c4158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.437+0000 7f50baffd700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50c40af0a0 0x7f50c4158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.437+0000 7f50baffd700 1 -- 172.21.15.57:0/2296640515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50c40b5110 msgr2=0x7f50c40c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.437+0000 7f50baffd700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50c40b5110 0x7f50c40c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.437+0000 7f50baffd700 1 -- 172.21.15.57:0/2296640515 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc0094a0 con 0x7f50c40b1650 2023-11-28T12:01:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.437+0000 7fab4e489700 1 Processor -- start 2023-11-28T12:01:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.437+0000 7f50baffd700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c40b1650 0x7f50c40b4bd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f50c406eb00 tx=0x7f50ac00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.437+0000 7f50c0ff9700 1 -- 172.21.15.57:0/2296640515 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac004c80 con 0x7f50c40b1650 2023-11-28T12:01:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.437+0000 7f50c0ff9700 1 -- 172.21.15.57:0/2296640515 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ac004de0 con 0x7f50c40b1650 2023-11-28T12:01:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.437+0000 7f50c9e7b700 1 -- 172.21.15.57:0/2296640515 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c415aa20 con 0x7f50c40b1650 2023-11-28T12:01:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.438+0000 7f50c9e7b700 1 -- 172.21.15.57:0/2296640515 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c415a890 con 0x7f50c40b1650 2023-11-28T12:01:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.439+0000 7f50c9e7b700 1 -- 172.21.15.57:0/2296640515 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c40743f0 con 0x7f50c40b1650 2023-11-28T12:01:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.439+0000 7f50c0ff9700 1 -- 172.21.15.57:0/2296640515 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac007310 con 0x7f50c40b1650 2023-11-28T12:01:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.440+0000 7f20b57dc700 1 Processor -- start 2023-11-28T12:01:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.441+0000 7f20b57dc700 1 -- start start 2023-11-28T12:01:50.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.441+0000 7f50c0ff9700 1 -- 172.21.15.57:0/2296640515 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f50ac01a020 con 0x7f50c40b1650 2023-11-28T12:01:50.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.441+0000 7f50c0ff9700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50a40682f0 0x7f50a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.442+0000 7f50c0ff9700 1 -- 172.21.15.57:0/2296640515 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f50ac085db0 con 0x7f50c40b1650 2023-11-28T12:01:50.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.442+0000 7f20b57dc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f20b00cec00 0x7f20b0065fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.442+0000 7f50c37fe700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50a40682f0 0x7f50a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.442+0000 7f50c37fe700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50a40682f0 0x7f50a406a7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50b4001f90 tx=0x7f50b4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.442+0000 7f20b57dc700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b015ceb0 con 0x7f20b0160300 2023-11-28T12:01:50.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.442+0000 7f20b57dc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b01608c0 con 0x7f20b00cac90 2023-11-28T12:01:50.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.442+0000 7f20b57dc700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b0160ba0 con 0x7f20b00cec00 2023-11-28T12:01:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.442+0000 7f20aeffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f20b00cec00 0x7f20b0065fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.442+0000 7f20aeffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f20b00cec00 0x7f20b0065fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48034/0 (socket says 172.21.15.57:48034) 2023-11-28T12:01:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.442+0000 7f20aeffd700 1 -- 172.21.15.57:0/1882869674 learned_addr learned my addr 172.21.15.57:0/1882869674 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.442+0000 7f20aeffd700 1 -- 172.21.15.57:0/1882869674 >> v1:172.21.15.181:6789/0 conn(0x7f20b00cac90 legacy=0x7f20b006c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.442+0000 7f20aeffd700 1 -- 172.21.15.57:0/1882869674 >> v1:172.21.15.57:6789/0 conn(0x7f20b0160300 legacy=0x7f20b015d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.442+0000 7f20aeffd700 1 -- 172.21.15.57:0/1882869674 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b0160e80 con 0x7f20b00cec00 2023-11-28T12:01:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.443+0000 7f20aeffd700 1 --2- 172.21.15.57:0/1882869674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20b00cec00 0x7f20b0065fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f20a00098f0 tx=0x7f20a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9106939449a56ad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.443+0000 7f20adffb700 1 -- 172.21.15.57:0/1882869674 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a000b8a0 con 0x7f20b00cec00 2023-11-28T12:01:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.443+0000 7f20adffb700 1 -- 172.21.15.57:0/1882869674 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a000ba00 con 0x7f20b00cec00 2023-11-28T12:01:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.443+0000 7f20adffb700 1 -- 172.21.15.57:0/1882869674 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a000bd90 con 0x7f20b00cec00 2023-11-28T12:01:50.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.445+0000 7f50c0ff9700 1 -- 172.21.15.57:0/2296640515 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f50ac052610 con 0x7f50c40b1650 2023-11-28T12:01:50.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.445+0000 7f20b57dc700 1 -- 172.21.15.57:0/1882869674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20b00cec00 msgr2=0x7f20b0065fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.445+0000 7f20b57dc700 1 --2- 172.21.15.57:0/1882869674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20b00cec00 0x7f20b0065fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f20a00098f0 tx=0x7f20a0009c00 comp rx=0 tx=0).stop 2023-11-28T12:01:50.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.438+0000 7f528dffb700 1 -- 172.21.15.57:0/383775948 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288011830 con 0x7f529806d230 2023-11-28T12:01:50.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.438+0000 7f528dffb700 1 -- 172.21.15.57:0/383775948 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5288011990 con 0x7f529806d230 2023-11-28T12:01:50.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.438+0000 7f528dffb700 1 -- 172.21.15.57:0/383775948 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288011d00 con 0x7f529806d230 2023-11-28T12:01:50.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.443+0000 7f529f798700 1 -- 172.21.15.57:0/383775948 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52980adba0 con 0x7f529806d230 2023-11-28T12:01:50.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.443+0000 7f529f798700 1 -- 172.21.15.57:0/383775948 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52980cb090 con 0x7f529806d230 2023-11-28T12:01:50.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.444+0000 7f529f798700 1 -- 172.21.15.57:0/383775948 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52980add30 con 0x7f529806d230 2023-11-28T12:01:50.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.445+0000 7f528dffb700 1 -- 172.21.15.57:0/383775948 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5288021070 con 0x7f529806d230 2023-11-28T12:01:50.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.445+0000 7f528dffb700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f527c0682f0 0x7f527c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.445+0000 7f528dffb700 1 -- 172.21.15.57:0/383775948 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5288085b70 con 0x7f529806d230 2023-11-28T12:01:50.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.446+0000 7f20b57dc700 1 -- 172.21.15.57:0/1882869674 shutdown_connections 2023-11-28T12:01:50.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.446+0000 7f20b57dc700 1 --2- 172.21.15.57:0/1882869674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20b00cec00 0x7f20b0065fd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.446+0000 7f20b57dc700 1 -- 172.21.15.57:0/1882869674 >> 172.21.15.57:0/1882869674 conn(0x7f20b00cf400 msgr2=0x7f20b00cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.446+0000 7f20b57dc700 1 -- 172.21.15.57:0/1882869674 shutdown_connections 2023-11-28T12:01:50.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.447+0000 7f13daffd700 1 -- 172.21.15.57:0/3231298948 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f13dc053040 con 0x7f13e40cf710 2023-11-28T12:01:50.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.447+0000 7f529cd33700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f527c0682f0 0x7f527c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.446+0000 7fab4e489700 1 -- start start 2023-11-28T12:01:50.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.447+0000 7fab4e489700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab4806b220 0x7fab481566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.447+0000 7fab4e489700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab48160ac0 0x7fab480b8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.447+0000 7fab4e489700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab481652c0 0x7fab480b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.447+0000 7fab4e489700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480b8d70 con 0x7fab481652c0 2023-11-28T12:01:50.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.447+0000 7fab4e489700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480b6cc0 con 0x7fab4806b220 2023-11-28T12:01:50.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.447+0000 7fab4e489700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab480b6f20 con 0x7fab48160ac0 2023-11-28T12:01:50.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.447+0000 7fab477fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab48160ac0 0x7fab480b8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.447+0000 7fab477fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab48160ac0 0x7fab480b8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:60910/0 (socket says 172.21.15.57:60910) 2023-11-28T12:01:50.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.447+0000 7fab477fe700 1 -- 172.21.15.57:0/915487523 learned_addr learned my addr 172.21.15.57:0/915487523 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.447+0000 7fab47fff700 1 --2- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab4806b220 0x7fab481566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.448+0000 7fab477fe700 1 -- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab4806b220 msgr2=0x7fab481566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.448+0000 7fab477fe700 1 --2- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab4806b220 0x7fab481566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.448+0000 7fab477fe700 1 -- 172.21.15.57:0/915487523 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab481652c0 msgr2=0x7fab480b6780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T12:01:50.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.448+0000 7fab477fe700 1 --2- 172.21.15.57:0/915487523 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab481652c0 0x7fab480b6780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.448+0000 7fab477fe700 1 -- 172.21.15.57:0/915487523 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab38009a70 con 0x7fab48160ac0 2023-11-28T12:01:50.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.448+0000 7fab477fe700 1 --2- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab48160ac0 0x7fab480b8830 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fab3800b730 tx=0x7fab38004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.446+0000 7f20b57dc700 1 -- 172.21.15.57:0/1882869674 wait complete. 2023-11-28T12:01:50.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.448+0000 7f20b57dc700 1 Processor -- start 2023-11-28T12:01:50.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.450+0000 7fab44ff9700 1 -- 172.21.15.57:0/915487523 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38012630 con 0x7fab48160ac0 2023-11-28T12:01:50.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.450+0000 7fab44ff9700 1 -- 172.21.15.57:0/915487523 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab38012790 con 0x7fab48160ac0 2023-11-28T12:01:50.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.450+0000 7fab44ff9700 1 -- 172.21.15.57:0/915487523 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38012b40 con 0x7fab48160ac0 2023-11-28T12:01:50.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.450+0000 7fab4e489700 1 -- 172.21.15.57:0/915487523 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab48164da0 con 0x7fab48160ac0 2023-11-28T12:01:50.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.450+0000 7fab4e489700 1 -- 172.21.15.57:0/915487523 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab48164ac0 con 0x7fab48160ac0 2023-11-28T12:01:50.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.450+0000 7f528dffb700 1 -- 172.21.15.57:0/383775948 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f52880523d0 con 0x7f529806d230 2023-11-28T12:01:50.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.451+0000 7fab4e489700 1 -- 172.21.15.57:0/915487523 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab48160e90 con 0x7fab48160ac0 2023-11-28T12:01:50.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.449+0000 7f20b57dc700 1 -- start start 2023-11-28T12:01:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.450+0000 7f20b57dc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b00cac90 0x7f20b00c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.450+0000 7f20b57dc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b00cec00 0x7f20b01ff220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.450+0000 7f20b57dc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20b0160300 0x7f20b00c3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.450+0000 7f20b57dc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b0200250 con 0x7f20b0160300 2023-11-28T12:01:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.450+0000 7f20b57dc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b01555e0 con 0x7f20b00cec00 2023-11-28T12:01:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.450+0000 7f20b57dc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b01ff760 con 0x7f20b00cac90 2023-11-28T12:01:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.451+0000 7f20ae7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b00cec00 0x7f20b01ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.451+0000 7f20aeffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b00cac90 0x7f20b00c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.451+0000 7f20af7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20b0160300 0x7f20b00c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.451+0000 7f20af7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20b0160300 0x7f20b00c3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48050/0 (socket says 172.21.15.57:48050) 2023-11-28T12:01:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.451+0000 7f20af7fe700 1 -- 172.21.15.57:0/196203622 learned_addr learned my addr 172.21.15.57:0/196203622 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.453+0000 7f20aeffd700 1 -- 172.21.15.57:0/196203622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b00cec00 msgr2=0x7f20b01ff220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.453+0000 7f20aeffd700 1 --2- 172.21.15.57:0/196203622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b00cec00 0x7f20b01ff220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.453+0000 7f20aeffd700 1 -- 172.21.15.57:0/196203622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20b0160300 msgr2=0x7f20b00c3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.453+0000 7f20aeffd700 1 --2- 172.21.15.57:0/196203622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20b0160300 0x7f20b00c3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.453+0000 7f20aeffd700 1 -- 172.21.15.57:0/196203622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a0009550 con 0x7f20b00cac90 2023-11-28T12:01:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.454+0000 7f20aeffd700 1 --2- 172.21.15.57:0/196203622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b00cac90 0x7f20b00c8e30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f20a0016000 tx=0x7f20a0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.454+0000 7f20977fe700 1 -- 172.21.15.57:0/196203622 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00044d0 con 0x7f20b00cac90 2023-11-28T12:01:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.454+0000 7f20977fe700 1 -- 172.21.15.57:0/196203622 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a001f070 con 0x7f20b00cac90 2023-11-28T12:01:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.454+0000 7f20977fe700 1 -- 172.21.15.57:0/196203622 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00038e0 con 0x7f20b00cac90 2023-11-28T12:01:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.458+0000 7fab44ff9700 1 -- 172.21.15.57:0/915487523 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fab38012ca0 con 0x7fab48160ac0 2023-11-28T12:01:50.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.458+0000 7fab44ff9700 1 --2- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fab34068340 0x7fab3406a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.458+0000 7fab44ff9700 1 -- 172.21.15.57:0/915487523 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fab38088850 con 0x7fab48160ac0 2023-11-28T12:01:50.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.459+0000 7fab47fff700 1 --2- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fab34068340 0x7fab3406a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.455+0000 7f20b57dc700 1 -- 172.21.15.57:0/196203622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b0066510 con 0x7f20b00cac90 2023-11-28T12:01:50.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.456+0000 7f20b57dc700 1 -- 172.21.15.57:0/196203622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b0161160 con 0x7f20b00cac90 2023-11-28T12:01:50.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.458+0000 7f20977fe700 1 -- 172.21.15.57:0/196203622 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f20a000f050 con 0x7f20b00cac90 2023-11-28T12:01:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.459+0000 7f20b57dc700 1 -- 172.21.15.57:0/196203622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b00cefd0 con 0x7f20b00cac90 2023-11-28T12:01:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.459+0000 7f20977fe700 1 --2- 172.21.15.57:0/196203622 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f209c0683c0 0x7f209c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.459+0000 7f20977fe700 1 -- 172.21.15.57:0/196203622 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f20a0021070 con 0x7f20b00cac90 2023-11-28T12:01:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.459+0000 7fab47fff700 1 --2- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fab34068340 0x7fab3406a7f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fab40007fd0 tx=0x7fab40009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.459+0000 7f20ae7fc700 1 --2- 172.21.15.57:0/196203622 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f209c0683c0 0x7f209c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:50.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.464+0000 7fab44ff9700 1 -- 172.21.15.57:0/915487523 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fab38055040 con 0x7fab48160ac0 2023-11-28T12:01:50.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.463+0000 7f20ae7fc700 1 --2- 172.21.15.57:0/196203622 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f209c0683c0 0x7f209c06a870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f20b00c2aa0 tx=0x7f20a4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.467+0000 7f529cd33700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f527c0682f0 0x7f527c06a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52980ce240 tx=0x7f5294008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:50.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.474+0000 7f20977fe700 1 -- 172.21.15.57:0/196203622 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f20a0053d90 con 0x7f20b00cac90 2023-11-28T12:01:50.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.834+0000 7f6148797700 1 -- 172.21.15.57:0/2437829495 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f61400743f0 con 0x7f614006d230 2023-11-28T12:01:50.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.837+0000 7f6136ffd700 1 -- 172.21.15.57:0/2437829495 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6138023090 con 0x7f614006d230 2023-11-28T12:01:50.841 INFO:teuthology.orchestra.run.smithi057.stdout:55834574854 2023-11-28T12:01:50.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.847+0000 7f6148797700 1 -- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f612c068770 msgr2=0x7f612c06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.847+0000 7f6148797700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f612c068770 0x7f612c06ac20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6130007fd0 tx=0x7f6130009040 comp rx=0 tx=0).stop 2023-11-28T12:01:50.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.847+0000 7f6148797700 1 -- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f614006d230 msgr2=0x7f6140070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.847+0000 7f6148797700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f614006d230 0x7f6140070af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6138008ac0 tx=0x7f613800b3d0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.847+0000 7f6148797700 1 -- 172.21.15.57:0/2437829495 shutdown_connections 2023-11-28T12:01:50.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.847+0000 7f6148797700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f612c068770 0x7f612c06ac20 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6130007fd0 tx=0x7f6130009040 comp rx=0 tx=0).stop 2023-11-28T12:01:50.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.847+0000 7f6148797700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6140158980 0x7f61400c3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.847+0000 7f6148797700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f614006d230 0x7f6140070af0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.847+0000 7f6148797700 1 --2- 172.21.15.57:0/2437829495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61400b0c50 0x7f61401f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.847+0000 7f6148797700 1 -- 172.21.15.57:0/2437829495 >> 172.21.15.57:0/2437829495 conn(0x7f614006aed0 msgr2=0x7f61400ac8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.849+0000 7f6148797700 1 -- 172.21.15.57:0/2437829495 shutdown_connections 2023-11-28T12:01:50.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.849+0000 7f6148797700 1 -- 172.21.15.57:0/2437829495 wait complete. 2023-11-28T12:01:50.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.857+0000 7f13ec4fc700 1 -- 172.21.15.57:0/3231298948 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f13cc0055a0 con 0x7f13e40cf710 2023-11-28T12:01:50.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.858+0000 7f13daffd700 1 -- 172.21.15.57:0/3231298948 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f13dc056660 con 0x7f13e40cf710 2023-11-28T12:01:50.863 INFO:teuthology.orchestra.run.smithi057.stdout:55834574853 2023-11-28T12:01:50.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.862+0000 7f13ec4fc700 1 -- 172.21.15.57:0/3231298948 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f13c80683c0 msgr2=0x7f13c806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.862+0000 7f13ec4fc700 1 --2- 172.21.15.57:0/3231298948 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f13c80683c0 0x7f13c806a870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009bc0 tx=0x7f13e001c040 comp rx=0 tx=0).stop 2023-11-28T12:01:50.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.862+0000 7f13ec4fc700 1 -- 172.21.15.57:0/3231298948 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13e40cf710 msgr2=0x7f13e40af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.862+0000 7f13ec4fc700 1 --2- 172.21.15.57:0/3231298948 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13e40cf710 0x7f13e40af880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f13e406b2d0 tx=0x7f13dc00ef90 comp rx=0 tx=0).stop 2023-11-28T12:01:50.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.862+0000 7f13ec4fc700 1 -- 172.21.15.57:0/3231298948 shutdown_connections 2023-11-28T12:01:50.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.862+0000 7f13ec4fc700 1 --2- 172.21.15.57:0/3231298948 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f13c80683c0 0x7f13c806a870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.862+0000 7f13ec4fc700 1 --2- 172.21.15.57:0/3231298948 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13e40cf710 0x7f13e40af880 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.863+0000 7f13ec4fc700 1 --2- 172.21.15.57:0/3231298948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13e406a980 0x7f13e415b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.863+0000 7f13ec4fc700 1 --2- 172.21.15.57:0/3231298948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e4071890 0x7f13e40af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.863+0000 7f13ec4fc700 1 -- 172.21.15.57:0/3231298948 >> 172.21.15.57:0/3231298948 conn(0x7f13e406c010 msgr2=0x7f13e40b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.863+0000 7f50c9e7b700 1 -- 172.21.15.57:0/2296640515 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f50c406cd10 con 0x7f50c40b1650 2023-11-28T12:01:50.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.864+0000 7f13ec4fc700 1 -- 172.21.15.57:0/3231298948 shutdown_connections 2023-11-28T12:01:50.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.864+0000 7f13ec4fc700 1 -- 172.21.15.57:0/3231298948 wait complete. 2023-11-28T12:01:50.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.864+0000 7fcc0368c700 1 -- 172.21.15.57:0/534515674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcbfc0c9d20 con 0x7fcbfc06d230 2023-11-28T12:01:50.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.865+0000 7f6215044700 1 -- 172.21.15.57:0/1791658669 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f62080053d0 con 0x7f62080546a0 2023-11-28T12:01:50.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.865+0000 7fcbf1ffb700 1 -- 172.21.15.57:0/534515674 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcbf8055670 con 0x7fcbfc06d230 2023-11-28T12:01:50.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.866+0000 7f620cff9700 1 -- 172.21.15.57:0/1791658669 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f62100cfe80 con 0x7f62080546a0 2023-11-28T12:01:50.867 INFO:teuthology.orchestra.run.smithi057.stdout:55834574854 2023-11-28T12:01:50.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.867+0000 7f50c0ff9700 1 -- 172.21.15.57:0/2296640515 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f50ac055c30 con 0x7f50c40b1650 2023-11-28T12:01:50.869 INFO:teuthology.orchestra.run.smithi057.stdout:55834574851 2023-11-28T12:01:50.871 INFO:teuthology.orchestra.run.smithi057.stdout:55834574852 2023-11-28T12:01:50.872 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2023-11-28T12:01:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7fcc0368c700 1 -- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcbe00683c0 msgr2=0x7fcbe006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7fcc0368c700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcbe00683c0 0x7fcbe006a870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4000f80 tx=0x7fcbf4009000 comp rx=0 tx=0).stop 2023-11-28T12:01:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7fcc0368c700 1 -- 172.21.15.57:0/534515674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbfc06d230 msgr2=0x7fcbfc0b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7fcc0368c700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbfc06d230 0x7fcbfc0b2ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800c570 tx=0x7fcbf800c880 comp rx=0 tx=0).stop 2023-11-28T12:01:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7fcc0368c700 1 -- 172.21.15.57:0/534515674 shutdown_connections 2023-11-28T12:01:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7fcc0368c700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcbe00683c0 0x7fcbe006a870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7fcc0368c700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbfc158980 0x7fcbfc0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7fcc0368c700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbfc06d230 0x7fcbfc0b2ed0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7fcc0368c700 1 --2- 172.21.15.57:0/534515674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbfc0b0c50 0x7fcbfc14fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7fcc0368c700 1 -- 172.21.15.57:0/534515674 >> 172.21.15.57:0/534515674 conn(0x7fcbfc06aed0 msgr2=0x7fcbfc0cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7fcc0368c700 1 -- 172.21.15.57:0/534515674 shutdown_connections 2023-11-28T12:01:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7f6215044700 1 -- 172.21.15.57:0/1791658669 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f61fc0682f0 msgr2=0x7f61fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7f6215044700 1 --2- 172.21.15.57:0/1791658669 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f61fc0682f0 0x7f61fc06a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f620000d370 tx=0x7f6200012000 comp rx=0 tx=0).stop 2023-11-28T12:01:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7f6215044700 1 -- 172.21.15.57:0/1791658669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62080546a0 msgr2=0x7f6208047f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7fcc0368c700 1 -- 172.21.15.57:0/534515674 wait complete. 2023-11-28T12:01:50.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.871+0000 7f6215044700 1 --2- 172.21.15.57:0/1791658669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62080546a0 0x7f6208047f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f621002d030 tx=0x7f6210068730 comp rx=0 tx=0).stop 2023-11-28T12:01:50.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.874+0000 7f50c9e7b700 1 -- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50a40682f0 msgr2=0x7f50a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.874+0000 7f50c9e7b700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50a40682f0 0x7f50a406a7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50b4001f90 tx=0x7f50b4009040 comp rx=0 tx=0).stop 2023-11-28T12:01:50.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.874+0000 7f50c9e7b700 1 -- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c40b1650 msgr2=0x7f50c40b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.874+0000 7f50c9e7b700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c40b1650 0x7f50c40b4bd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f50c406eb00 tx=0x7f50ac00c800 comp rx=0 tx=0).stop 2023-11-28T12:01:50.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.874+0000 7f50c9e7b700 1 -- 172.21.15.57:0/2296640515 shutdown_connections 2023-11-28T12:01:50.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.874+0000 7f50c9e7b700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50a40682f0 0x7f50a406a7a0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50b4001f90 tx=0x7f50b4009040 comp rx=0 tx=0).stop 2023-11-28T12:01:50.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.874+0000 7f50c9e7b700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50c40b5110 0x7f50c40c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.874+0000 7f50c9e7b700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50c40af0a0 0x7f50c4158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.874+0000 7f50c9e7b700 1 --2- 172.21.15.57:0/2296640515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c40b1650 0x7f50c40b4bd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.874+0000 7f50c9e7b700 1 -- 172.21.15.57:0/2296640515 >> 172.21.15.57:0/2296640515 conn(0x7f50c406c010 msgr2=0x7f50c406d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.875+0000 7f529f798700 1 -- 172.21.15.57:0/383775948 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f52980c4620 con 0x7f529806d230 2023-11-28T12:01:50.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.875+0000 7f6215044700 1 -- 172.21.15.57:0/1791658669 shutdown_connections 2023-11-28T12:01:50.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.875+0000 7f6215044700 1 --2- 172.21.15.57:0/1791658669 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f61fc0682f0 0x7f61fc06a7a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.875+0000 7f6215044700 1 --2- 172.21.15.57:0/1791658669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62080546a0 0x7f6208047f00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.875+0000 7f6215044700 1 --2- 172.21.15.57:0/1791658669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6208036b50 0x7f62080e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.875+0000 7f6215044700 1 --2- 172.21.15.57:0/1791658669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f620803a6b0 0x7f62080479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.875+0000 7f6215044700 1 -- 172.21.15.57:0/1791658669 >> 172.21.15.57:0/1791658669 conn(0x7f6208051610 msgr2=0x7f62080dee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.876+0000 7f50c9e7b700 1 -- 172.21.15.57:0/2296640515 shutdown_connections 2023-11-28T12:01:50.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.876+0000 7f6215044700 1 -- 172.21.15.57:0/1791658669 shutdown_connections 2023-11-28T12:01:50.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.876+0000 7fab4e489700 1 -- 172.21.15.57:0/915487523 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fab48067d70 con 0x7fab48160ac0 2023-11-28T12:01:50.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.876+0000 7f50c9e7b700 1 -- 172.21.15.57:0/2296640515 wait complete. 2023-11-28T12:01:50.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.877+0000 7f528dffb700 1 -- 172.21.15.57:0/383775948 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5288015020 con 0x7f529806d230 2023-11-28T12:01:50.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.877+0000 7fab44ff9700 1 -- 172.21.15.57:0/915487523 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fab38058660 con 0x7fab48160ac0 2023-11-28T12:01:50.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.877+0000 7f6215044700 1 -- 172.21.15.57:0/1791658669 wait complete. 2023-11-28T12:01:50.881 INFO:teuthology.orchestra.run.smithi057.stdout:55834574852 2023-11-28T12:01:50.881 INFO:teuthology.orchestra.run.smithi057.stdout:55834574852 2023-11-28T12:01:50.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.879+0000 7fab4e489700 1 -- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fab34068340 msgr2=0x7fab3406a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.879+0000 7fab4e489700 1 --2- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fab34068340 0x7fab3406a7f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fab40007fd0 tx=0x7fab40009040 comp rx=0 tx=0).stop 2023-11-28T12:01:50.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.879+0000 7fab4e489700 1 -- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab48160ac0 msgr2=0x7fab480b8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.879+0000 7fab4e489700 1 --2- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab48160ac0 0x7fab480b8830 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fab3800b730 tx=0x7fab38004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.882+0000 7f529f798700 1 -- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f527c0682f0 msgr2=0x7f527c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.882+0000 7f529f798700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f527c0682f0 0x7f527c06a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52980ce240 tx=0x7f5294008030 comp rx=0 tx=0).stop 2023-11-28T12:01:50.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.882+0000 7f529f798700 1 -- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f529806d230 msgr2=0x7f529806fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.882+0000 7f529f798700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f529806d230 0x7f529806fd30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5288006fd0 tx=0x7f528800b7e0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.882+0000 7fab4e489700 1 -- 172.21.15.57:0/915487523 shutdown_connections 2023-11-28T12:01:50.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.882+0000 7fab4e489700 1 --2- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fab34068340 0x7fab3406a7f0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fab40007fd0 tx=0x7fab40009040 comp rx=0 tx=0).stop 2023-11-28T12:01:50.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.882+0000 7fab4e489700 1 --2- 172.21.15.57:0/915487523 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab481652c0 0x7fab480b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.882+0000 7fab4e489700 1 --2- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab4806b220 0x7fab481566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.882+0000 7fab4e489700 1 --2- 172.21.15.57:0/915487523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab48160ac0 0x7fab480b8830 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.882+0000 7fab4e489700 1 -- 172.21.15.57:0/915487523 >> 172.21.15.57:0/915487523 conn(0x7fab4806c520 msgr2=0x7fab480b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.883+0000 7fab4e489700 1 -- 172.21.15.57:0/915487523 shutdown_connections 2023-11-28T12:01:50.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.883+0000 7f529f798700 1 -- 172.21.15.57:0/383775948 shutdown_connections 2023-11-28T12:01:50.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.883+0000 7f529f798700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f527c0682f0 0x7f527c06a7a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.883+0000 7f529f798700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5298158710 0x7f52980afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.883+0000 7f529f798700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f529806d230 0x7f529806fd30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.883+0000 7f529f798700 1 --2- 172.21.15.57:0/383775948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52980cbc50 0x7f52980b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.883+0000 7f529f798700 1 -- 172.21.15.57:0/383775948 >> 172.21.15.57:0/383775948 conn(0x7f529806aed0 msgr2=0x7f52980c4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.883+0000 7fab4e489700 1 -- 172.21.15.57:0/915487523 wait complete. 2023-11-28T12:01:50.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.887+0000 7f529f798700 1 -- 172.21.15.57:0/383775948 shutdown_connections 2023-11-28T12:01:50.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.887+0000 7f529f798700 1 -- 172.21.15.57:0/383775948 wait complete. 2023-11-28T12:01:50.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2023-11-28T12:01:50.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.893+0000 7f20b57dc700 1 -- 172.21.15.57:0/196203622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f20b00c98a0 con 0x7f20b00cac90 2023-11-28T12:01:50.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.893+0000 7f20977fe700 1 -- 172.21.15.57:0/196203622 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f20a00573b0 con 0x7f20b00cac90 2023-11-28T12:01:50.895 INFO:teuthology.orchestra.run.smithi057.stdout:55834574851 2023-11-28T12:01:50.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2023-11-28T12:01:50.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.896+0000 7f20b57dc700 1 -- 172.21.15.57:0/196203622 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f209c0683c0 msgr2=0x7f209c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.896+0000 7f20b57dc700 1 --2- 172.21.15.57:0/196203622 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f209c0683c0 0x7f209c06a870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f20b00c2aa0 tx=0x7f20a4009040 comp rx=0 tx=0).stop 2023-11-28T12:01:50.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.896+0000 7f20b57dc700 1 -- 172.21.15.57:0/196203622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b00cac90 msgr2=0x7f20b00c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:50.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.896+0000 7f20b57dc700 1 --2- 172.21.15.57:0/196203622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b00cac90 0x7f20b00c8e30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f20a0016000 tx=0x7f20a0004750 comp rx=0 tx=0).stop 2023-11-28T12:01:50.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.896+0000 7f20b57dc700 1 -- 172.21.15.57:0/196203622 shutdown_connections 2023-11-28T12:01:50.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.896+0000 7f20b57dc700 1 --2- 172.21.15.57:0/196203622 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f209c0683c0 0x7f209c06a870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.896+0000 7f20b57dc700 1 --2- 172.21.15.57:0/196203622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20b0160300 0x7f20b00c3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.896+0000 7f20b57dc700 1 --2- 172.21.15.57:0/196203622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b00cac90 0x7f20b00c8e30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.896+0000 7f20b57dc700 1 --2- 172.21.15.57:0/196203622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b00cec00 0x7f20b01ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:50.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.896+0000 7f20b57dc700 1 -- 172.21.15.57:0/196203622 >> 172.21.15.57:0/196203622 conn(0x7f20b00cf400 msgr2=0x7f20b006afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:50.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.896+0000 7f20b57dc700 1 -- 172.21.15.57:0/196203622 shutdown_connections 2023-11-28T12:01:50.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:50.896+0000 7f20b57dc700 1 -- 172.21.15.57:0/196203622 wait complete. 2023-11-28T12:01:50.902 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2023-11-28T12:01:50.910 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-11-28T12:01:50.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2023-11-28T12:01:50.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2023-11-28T12:01:50.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2023-11-28T12:01:51.873 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T12:01:51.893 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T12:01:51.898 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T12:01:51.903 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T12:01:51.911 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T12:01:51.912 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T12:01:51.914 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T12:01:51.915 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T12:01:52.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.094+0000 7f6f76cad700 1 Processor -- start 2023-11-28T12:01:52.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.094+0000 7f6f76cad700 1 -- start start 2023-11-28T12:01:52.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.095+0000 7f6f76cad700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f6f70158980 0x7f6f700adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.095+0000 7f6f76cad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f7006d230 0x7f6f7006b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.095+0000 7f6f76cad700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6f700b0540 0x7f6f700b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.095+0000 7f6f76cad700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7006c700 con 0x7f6f70158980 2023-11-28T12:01:52.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.095+0000 7f6f76cad700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f700b0ee0 con 0x7f6f700b0540 2023-11-28T12:01:52.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.095+0000 7f6f76cad700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7006d6c0 con 0x7f6f7006d230 2023-11-28T12:01:52.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.096+0000 7f6f6ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f7006d230 0x7f6f7006b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.097+0000 7f6f6ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f7006d230 0x7f6f7006b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:60938/0 (socket says 172.21.15.57:60938) 2023-11-28T12:01:52.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.097+0000 7f6f6ffff700 1 -- 172.21.15.57:0/3051286840 learned_addr learned my addr 172.21.15.57:0/3051286840 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:52.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.097+0000 7f6f7524a700 1 --2- 172.21.15.57:0/3051286840 >> v2:172.21.15.181:3300/0 conn(0x7f6f700b0540 0x7f6f700b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.097+0000 7f6f7524a700 1 -- 172.21.15.57:0/3051286840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f7006d230 msgr2=0x7f6f7006b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.097+0000 7f6f7524a700 1 --2- 172.21.15.57:0/3051286840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f7006d230 0x7f6f7006b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.097+0000 7f6f7524a700 1 -- 172.21.15.57:0/3051286840 >> v2:172.21.15.57:3300/0 conn(0x7f6f70158980 msgr2=0x7f6f700adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.097+0000 7f6f7524a700 1 --2- 172.21.15.57:0/3051286840 >> v2:172.21.15.57:3300/0 conn(0x7f6f70158980 0x7f6f700adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.097+0000 7f6f7524a700 1 -- 172.21.15.57:0/3051286840 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f700ca7f0 con 0x7f6f700b0540 2023-11-28T12:01:52.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.098+0000 7f6f7524a700 1 --2- 172.21.15.57:0/3051286840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f700b0540 0x7f6f700b09a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f700723e0 tx=0x7f6f6400ac30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b6acbae331aa0b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.098+0000 7f6f6f7fe700 1 -- 172.21.15.57:0/3051286840 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64005c20 con 0x7f6f700b0540 2023-11-28T12:01:52.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.098+0000 7f6f6f7fe700 1 -- 172.21.15.57:0/3051286840 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f64005d80 con 0x7f6f700b0540 2023-11-28T12:01:52.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.098+0000 7f6f6f7fe700 1 -- 172.21.15.57:0/3051286840 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64013990 con 0x7f6f700b0540 2023-11-28T12:01:52.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.100+0000 7f6f76cad700 1 -- 172.21.15.57:0/3051286840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f700b0540 msgr2=0x7f6f700b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.100+0000 7f6f76cad700 1 --2- 172.21.15.57:0/3051286840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f700b0540 0x7f6f700b09a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f700723e0 tx=0x7f6f6400ac30 comp rx=0 tx=0).stop 2023-11-28T12:01:52.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.100+0000 7f6f76cad700 1 -- 172.21.15.57:0/3051286840 shutdown_connections 2023-11-28T12:01:52.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.100+0000 7f6f76cad700 1 --2- 172.21.15.57:0/3051286840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f700b0540 0x7f6f700b09a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.100+0000 7f6f76cad700 1 --2- 172.21.15.57:0/3051286840 >> v2:172.21.15.57:3300/0 conn(0x7f6f70158980 0x7f6f700adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.100+0000 7f6f76cad700 1 --2- 172.21.15.57:0/3051286840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f7006d230 0x7f6f7006b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.100+0000 7f6f76cad700 1 -- 172.21.15.57:0/3051286840 >> 172.21.15.57:0/3051286840 conn(0x7f6f7006aed0 msgr2=0x7f6f700cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.100+0000 7f6f76cad700 1 -- 172.21.15.57:0/3051286840 shutdown_connections 2023-11-28T12:01:52.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.100+0000 7f6f76cad700 1 -- 172.21.15.57:0/3051286840 wait complete. 2023-11-28T12:01:52.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.102+0000 7f6f76cad700 1 Processor -- start 2023-11-28T12:01:52.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.103+0000 7f6f76cad700 1 -- start start 2023-11-28T12:01:52.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.104+0000 7f6f76cad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f7006d230 0x7f6f700c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.104+0000 7f6f76cad700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f700b0540 0x7f6f700c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.104+0000 7f6f76cad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f70158980 0x7f6f700c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.104+0000 7f6f76cad700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f70150dc0 con 0x7f6f700b0540 2023-11-28T12:01:52.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.104+0000 7f6f76cad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f70150520 con 0x7f6f70158980 2023-11-28T12:01:52.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.104+0000 7f6f76cad700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f70150800 con 0x7f6f7006d230 2023-11-28T12:01:52.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.105+0000 7f6f6ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f700b0540 0x7f6f700c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.105+0000 7f6f6ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f700b0540 0x7f6f700c5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48064/0 (socket says 172.21.15.57:48064) 2023-11-28T12:01:52.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.105+0000 7f6f6ffff700 1 -- 172.21.15.57:0/501164978 learned_addr learned my addr 172.21.15.57:0/501164978 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:52.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.105+0000 7f6f74a49700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f7006d230 0x7f6f700c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.105+0000 7f6f6ffff700 1 -- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f7006d230 msgr2=0x7f6f700c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.105+0000 7f6f6ffff700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f7006d230 0x7f6f700c4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.105+0000 7f6f6ffff700 1 -- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f70158980 msgr2=0x7f6f700c60d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T12:01:52.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.105+0000 7f6f6ffff700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f70158980 0x7f6f700c60d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.105+0000 7f6f6ffff700 1 -- 172.21.15.57:0/501164978 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6400a5b0 con 0x7f6f700b0540 2023-11-28T12:01:52.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.106+0000 7f6f6ffff700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f700b0540 0x7f6f700c5b90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6f68007a10 tx=0x7f6f68007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.112+0000 7f6f74a49700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f7006d230 0x7f6f700c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:01:52.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.116+0000 7f6f6d7fa700 1 -- 172.21.15.57:0/501164978 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6800e960 con 0x7f6f700b0540 2023-11-28T12:01:52.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.116+0000 7f6f6d7fa700 1 -- 172.21.15.57:0/501164978 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6800eac0 con 0x7f6f700b0540 2023-11-28T12:01:52.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.116+0000 7f6f6d7fa700 1 -- 172.21.15.57:0/501164978 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6800ee30 con 0x7f6f700b0540 2023-11-28T12:01:52.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.116+0000 7f6f76cad700 1 -- 172.21.15.57:0/501164978 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f700cadb0 con 0x7f6f700b0540 2023-11-28T12:01:52.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.117+0000 7f6f76cad700 1 -- 172.21.15.57:0/501164978 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f700caad0 con 0x7f6f700b0540 2023-11-28T12:01:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.119+0000 7f6f6d7fa700 1 -- 172.21.15.57:0/501164978 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6f68020020 con 0x7f6f700b0540 2023-11-28T12:01:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.120+0000 7f6f6d7fa700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6f5c0683c0 0x7f6f5c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.120+0000 7f6f6d7fa700 1 -- 172.21.15.57:0/501164978 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6f68086220 con 0x7f6f700b0540 2023-11-28T12:01:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.121+0000 7f6f74a49700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6f5c0683c0 0x7f6f5c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.124+0000 7f6f76cad700 1 -- 172.21.15.57:0/501164978 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f500052f0 con 0x7f6f700b0540 2023-11-28T12:01:52.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.133+0000 7f6f6d7fa700 1 -- 172.21.15.57:0/501164978 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6f68052a10 con 0x7f6f700b0540 2023-11-28T12:01:52.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.135+0000 7f6f74a49700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6f5c0683c0 0x7f6f5c06a870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f60007900 tx=0x7f6f60008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.152+0000 7f560644a700 1 Processor -- start 2023-11-28T12:01:52.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.152+0000 7f560644a700 1 -- start start 2023-11-28T12:01:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.154+0000 7f560644a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f56001609c0 0x7f56000ca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.154+0000 7f560644a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f560006cdd0 0x7f56000b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.154+0000 7f560644a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56000b0a80 0x7f5600164630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.154+0000 7f560644a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56000cc690 con 0x7f56001609c0 2023-11-28T12:01:52.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.154+0000 7f560644a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56000b0ee0 con 0x7f560006cdd0 2023-11-28T12:01:52.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.154+0000 7f560644a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56000cc8c0 con 0x7f56000b0a80 2023-11-28T12:01:52.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.154+0000 7f55fffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f56001609c0 0x7f56000ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.154+0000 7f55fffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f56001609c0 0x7f56000ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48074/0 (socket says 172.21.15.57:48074) 2023-11-28T12:01:52.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.154+0000 7f55fffff700 1 -- 172.21.15.57:0/2750057514 learned_addr learned my addr 172.21.15.57:0/2750057514 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:52.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.155+0000 7f56049e7700 1 --2- 172.21.15.57:0/2750057514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56000b0a80 0x7f5600164630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.155+0000 7f55fffff700 1 -- 172.21.15.57:0/2750057514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56000b0a80 msgr2=0x7f5600164630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.155+0000 7f55fffff700 1 --2- 172.21.15.57:0/2750057514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56000b0a80 0x7f5600164630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.155+0000 7f55fffff700 1 -- 172.21.15.57:0/2750057514 >> v2:172.21.15.181:3300/0 conn(0x7f560006cdd0 msgr2=0x7f56000b0540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T12:01:52.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.155+0000 7f55fffff700 1 --2- 172.21.15.57:0/2750057514 >> v2:172.21.15.181:3300/0 conn(0x7f560006cdd0 0x7f56000b0540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.155+0000 7f55fffff700 1 -- 172.21.15.57:0/2750057514 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56000caa30 con 0x7f56001609c0 2023-11-28T12:01:52.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.155+0000 7f24426fc700 1 Processor -- start 2023-11-28T12:01:52.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.155+0000 7f24426fc700 1 -- start start 2023-11-28T12:01:52.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.156+0000 7f55fffff700 1 --2- 172.21.15.57:0/2750057514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56001609c0 0x7f56000ca150 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56000672e0 tx=0x7f55f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c995932bd1c62a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.156+0000 7f24426fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f243c1609c0 0x7f243c0ca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.156+0000 7f55feffd700 1 -- 172.21.15.57:0/2750057514 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f800e070 con 0x7f56001609c0 2023-11-28T12:01:52.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.156+0000 7f55feffd700 1 -- 172.21.15.57:0/2750057514 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f800b9c0 con 0x7f56001609c0 2023-11-28T12:01:52.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.156+0000 7f55feffd700 1 -- 172.21.15.57:0/2750057514 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f800bd70 con 0x7f56001609c0 2023-11-28T12:01:52.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.157+0000 7f560644a700 1 -- 172.21.15.57:0/2750057514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56001609c0 msgr2=0x7f56000ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.157+0000 7f560644a700 1 --2- 172.21.15.57:0/2750057514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56001609c0 0x7f56000ca150 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56000672e0 tx=0x7f55f800b330 comp rx=0 tx=0).stop 2023-11-28T12:01:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.156+0000 7f24426fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c0b0c50 0x7f243c06bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.157+0000 7f24426fc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243c0cc690 con 0x7f243c06cdd0 2023-11-28T12:01:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.157+0000 7f24426fc700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243c0cc8c0 con 0x7f243c1609c0 2023-11-28T12:01:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.157+0000 7f24426fc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0caa30 con 0x7f243c0b0c50 2023-11-28T12:01:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.158+0000 7f2440c99700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c0b0c50 0x7f243c06bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.158+0000 7f560644a700 1 -- 172.21.15.57:0/2750057514 shutdown_connections 2023-11-28T12:01:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.158+0000 7f560644a700 1 --2- 172.21.15.57:0/2750057514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56000b0a80 0x7f5600164630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.158+0000 7f560644a700 1 --2- 172.21.15.57:0/2750057514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56001609c0 0x7f56000ca150 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.158+0000 7f560644a700 1 --2- 172.21.15.57:0/2750057514 >> v2:172.21.15.181:3300/0 conn(0x7f560006cdd0 0x7f56000b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.158+0000 7f560644a700 1 -- 172.21.15.57:0/2750057514 >> 172.21.15.57:0/2750057514 conn(0x7f560006aed0 msgr2=0x7f56000cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.158+0000 7f243b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f243c06cdd0 0x7f243c0b0540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:43994/0 (socket says 172.21.15.57:43994) 2023-11-28T12:01:52.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.158+0000 7f243b7fe700 1 -- 172.21.15.57:0/2546582218 learned_addr learned my addr 172.21.15.57:0/2546582218 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:52.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.158+0000 7f560644a700 1 -- 172.21.15.57:0/2750057514 shutdown_connections 2023-11-28T12:01:52.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.158+0000 7f560644a700 1 -- 172.21.15.57:0/2750057514 wait complete. 2023-11-28T12:01:52.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.158+0000 7f2440c99700 1 -- 172.21.15.57:0/2546582218 >> v2:172.21.15.57:3300/0 conn(0x7f243c1609c0 msgr2=0x7f243c0ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.158+0000 7f243bfff700 1 --2- 172.21.15.57:0/2546582218 >> v2:172.21.15.57:3300/0 conn(0x7f243c1609c0 0x7f243c0ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.158+0000 7f2440c99700 1 --2- 172.21.15.57:0/2546582218 >> v2:172.21.15.57:3300/0 conn(0x7f243c1609c0 0x7f243c0ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.159+0000 7f243affd700 1 -- 172.21.15.57:0/2546582218 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284013744 0 0) 0x7f243c0cc690 con 0x7f243c06cdd0 2023-11-28T12:01:52.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.160+0000 7f560644a700 1 Processor -- start 2023-11-28T12:01:52.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.161+0000 7f560644a700 1 -- start start 2023-11-28T12:01:52.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.160+0000 7f2440c99700 1 -- 172.21.15.57:0/2546582218 >> v1:172.21.15.181:6789/0 conn(0x7f243c06cdd0 legacy=0x7f243c0b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.160+0000 7f2440c99700 1 -- 172.21.15.57:0/2546582218 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c0cad10 con 0x7f243c0b0c50 2023-11-28T12:01:52.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.161+0000 7f2440c99700 1 --2- 172.21.15.57:0/2546582218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c0b0c50 0x7f243c06bd30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2434009d90 tx=0x7f243400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de8ed175ea7b62e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.162+0000 7f560644a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f560006cdd0 0x7f56000b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.162+0000 7f560644a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56000b0a80 0x7f56000b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.162+0000 7f560644a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56001609c0 0x7f56000c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.162+0000 7f560644a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56000c25d0 con 0x7f56001609c0 2023-11-28T12:01:52.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.162+0000 7f560644a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56000c2820 con 0x7f560006cdd0 2023-11-28T12:01:52.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.162+0000 7f560644a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56000c2b00 con 0x7f56000b0a80 2023-11-28T12:01:52.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.162+0000 7f243affd700 1 -- 172.21.15.57:0/2546582218 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243400e070 con 0x7f243c0b0c50 2023-11-28T12:01:52.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.162+0000 7f243affd700 1 -- 172.21.15.57:0/2546582218 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243400bbd0 con 0x7f243c0b0c50 2023-11-28T12:01:52.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.162+0000 7f243affd700 1 -- 172.21.15.57:0/2546582218 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434012430 con 0x7f243c0b0c50 2023-11-28T12:01:52.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.162+0000 7f56049e7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56001609c0 0x7f56000c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.162+0000 7f56049e7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56001609c0 0x7f56000c38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48096/0 (socket says 172.21.15.57:48096) 2023-11-28T12:01:52.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.162+0000 7f56049e7700 1 -- 172.21.15.57:0/1002551293 learned_addr learned my addr 172.21.15.57:0/1002551293 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:52.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f56049e7700 1 -- 172.21.15.57:0/1002551293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56000b0a80 msgr2=0x7f56000b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f56049e7700 1 --2- 172.21.15.57:0/1002551293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56000b0a80 0x7f56000b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f56049e7700 1 -- 172.21.15.57:0/1002551293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f560006cdd0 msgr2=0x7f56000b2300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T12:01:52.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f56049e7700 1 --2- 172.21.15.57:0/1002551293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f560006cdd0 0x7f56000b2300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f56049e7700 1 -- 172.21.15.57:0/1002551293 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f8009b30 con 0x7f56001609c0 2023-11-28T12:01:52.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f56049e7700 1 --2- 172.21.15.57:0/1002551293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56001609c0 0x7f56000c38c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f55f000e810 tx=0x7f55f000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f55fcff9700 1 -- 172.21.15.57:0/1002551293 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0013610 con 0x7f56001609c0 2023-11-28T12:01:52.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f24426fc700 1 -- 172.21.15.57:0/2546582218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c0b0c50 msgr2=0x7f243c06bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f24426fc700 1 --2- 172.21.15.57:0/2546582218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c0b0c50 0x7f243c06bd30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2434009d90 tx=0x7f243400b330 comp rx=0 tx=0).stop 2023-11-28T12:01:52.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f24426fc700 1 -- 172.21.15.57:0/2546582218 shutdown_connections 2023-11-28T12:01:52.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f24426fc700 1 --2- 172.21.15.57:0/2546582218 >> v2:172.21.15.57:3300/0 conn(0x7f243c1609c0 0x7f243c0ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f24426fc700 1 --2- 172.21.15.57:0/2546582218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c0b0c50 0x7f243c06bd30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f24426fc700 1 -- 172.21.15.57:0/2546582218 >> 172.21.15.57:0/2546582218 conn(0x7f243c06aed0 msgr2=0x7f243c0cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f24426fc700 1 -- 172.21.15.57:0/2546582218 shutdown_connections 2023-11-28T12:01:52.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f24426fc700 1 -- 172.21.15.57:0/2546582218 wait complete. 2023-11-28T12:01:52.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f55fcff9700 1 -- 172.21.15.57:0/1002551293 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f0013790 con 0x7f56001609c0 2023-11-28T12:01:52.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.163+0000 7f55fcff9700 1 -- 172.21.15.57:0/1002551293 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0013b40 con 0x7f56001609c0 2023-11-28T12:01:52.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.164+0000 7f560644a700 1 -- 172.21.15.57:0/1002551293 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56000caff0 con 0x7f56001609c0 2023-11-28T12:01:52.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.164+0000 7f560644a700 1 -- 172.21.15.57:0/1002551293 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56000cad10 con 0x7f56001609c0 2023-11-28T12:01:52.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.165+0000 7f24426fc700 1 Processor -- start 2023-11-28T12:01:52.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.166+0000 7efeddd65700 1 Processor -- start 2023-11-28T12:01:52.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.166+0000 7efeddd65700 1 -- start start 2023-11-28T12:01:52.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.167+0000 7efeddd65700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed8158980 0x7efed80adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.167+0000 7efeddd65700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efed806d230 0x7efed806b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.167+0000 7efeddd65700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed806d6c0 con 0x7efed80b0540 2023-11-28T12:01:52.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.167+0000 7efeddd65700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed806c700 con 0x7efed806d230 2023-11-28T12:01:52.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.167+0000 7efeddd65700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed80ca7f0 con 0x7efed8158980 2023-11-28T12:01:52.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.166+0000 7f24426fc700 1 -- start start 2023-11-28T12:01:52.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.167+0000 7f24426fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c06cdd0 0x7f243c0c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.167+0000 7f24426fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c0b0c50 0x7f243c202100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.167+0000 7f24426fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f243c1609c0 0x7f243c0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.167+0000 7f24426fc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0c3fc0 con 0x7f243c1609c0 2023-11-28T12:01:52.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.167+0000 7f24426fc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0b02c0 con 0x7f243c06cdd0 2023-11-28T12:01:52.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.167+0000 7f24426fc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0c3620 con 0x7f243c0b0c50 2023-11-28T12:01:52.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7f243b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c0b0c50 0x7f243c202100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7f243bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c06cdd0 0x7f243c0c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7f243bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c06cdd0 0x7f243c0c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:49500/0 (socket says 172.21.15.57:49500) 2023-11-28T12:01:52.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7f243bfff700 1 -- 172.21.15.57:0/2134496774 learned_addr learned my addr 172.21.15.57:0/2134496774 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:52.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7f2440c99700 1 --2- 172.21.15.57:0/2134496774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f243c1609c0 0x7f243c0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7f243b7fe700 1 -- 172.21.15.57:0/2134496774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c06cdd0 msgr2=0x7f243c0c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7f243b7fe700 1 --2- 172.21.15.57:0/2134496774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c06cdd0 0x7f243c0c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7f243b7fe700 1 -- 172.21.15.57:0/2134496774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f243c1609c0 msgr2=0x7f243c0704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7f243b7fe700 1 --2- 172.21.15.57:0/2134496774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f243c1609c0 0x7f243c0704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7f243b7fe700 1 -- 172.21.15.57:0/2134496774 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2434009a70 con 0x7f243c0b0c50 2023-11-28T12:01:52.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7f55fcff9700 1 -- 172.21.15.57:0/1002551293 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f55f0023070 con 0x7f56001609c0 2023-11-28T12:01:52.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7f55fcff9700 1 --2- 172.21.15.57:0/1002551293 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f55ec0682f0 0x7f55ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7f55fcff9700 1 -- 172.21.15.57:0/1002551293 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f55f0085a00 con 0x7f56001609c0 2023-11-28T12:01:52.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7f55fffff700 1 --2- 172.21.15.57:0/1002551293 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f55ec0682f0 0x7f55ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.167+0000 7efed7fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efed80b0540 0x7efed80cdc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:43996/0 (socket says 172.21.15.57:43996) 2023-11-28T12:01:52.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.167+0000 7efed7fff700 1 -- 172.21.15.57:0/3581635196 learned_addr learned my addr 172.21.15.57:0/3581635196 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:52.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791242742 0 0) 0x7efed806d6c0 con 0x7efed80b0540 2023-11-28T12:01:52.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efec0003590 con 0x7efed80b0540 2023-11-28T12:01:52.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 639201657 0 0) 0x7efec0003590 con 0x7efed80b0540 2023-11-28T12:01:52.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed8158980 msgr2=0x7efed80adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7efed67fc700 1 --2- 172.21.15.57:0/3581635196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed8158980 0x7efed80adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 >> v2:172.21.15.181:3300/0 conn(0x7efed806d230 msgr2=0x7efed806b770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T12:01:52.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7efed67fc700 1 --2- 172.21.15.57:0/3581635196 >> v2:172.21.15.181:3300/0 conn(0x7efed806d230 0x7efed806b770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed80caad0 con 0x7efed80b0540 2023-11-28T12:01:52.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7efec8004d60 con 0x7efed80b0540 2023-11-28T12:01:52.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 >> v1:172.21.15.181:6789/0 conn(0x7efed80b0540 legacy=0x7efed80cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.168+0000 7efed67fc700 1 --2- 172.21.15.57:0/3581635196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec0003bb0 0x7efec0003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7efed67fc700 1 --2- 172.21.15.57:0/3581635196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efec00067f0 0x7efec0008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7efed67fc700 1 --2- 172.21.15.57:0/3581635196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec0009470 0x7efec00098e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed806c700 con 0x7efec00067f0 2023-11-28T12:01:52.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed80ca7f0 con 0x7efec0003bb0 2023-11-28T12:01:52.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec000a1f0 con 0x7efec0009470 2023-11-28T12:01:52.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7efed7fff700 1 --2- 172.21.15.57:0/3581635196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec0009470 0x7efec00098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7efed7fff700 1 -- 172.21.15.57:0/3581635196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec0003bb0 msgr2=0x7efec0003fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T12:01:52.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7efed7fff700 1 --2- 172.21.15.57:0/3581635196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec0003bb0 0x7efec0003fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7efed7fff700 1 -- 172.21.15.57:0/3581635196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efec00067f0 msgr2=0x7efec0008c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T12:01:52.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7efed7fff700 1 --2- 172.21.15.57:0/3581635196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efec00067f0 0x7efec0008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7efed7fff700 1 -- 172.21.15.57:0/3581635196 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec0003590 con 0x7efec0009470 2023-11-28T12:01:52.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.169+0000 7efed7fff700 1 --2- 172.21.15.57:0/3581635196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec0009470 0x7efec00098e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efec8001e50 tx=0x7efec0003840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5de4972e1d71e5da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.170+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8001940 con 0x7efec0009470 2023-11-28T12:01:52.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.170+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec800de30 con 0x7efec0009470 2023-11-28T12:01:52.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.170+0000 7efed67fc700 1 -- 172.21.15.57:0/3581635196 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec800f600 con 0x7efec0009470 2023-11-28T12:01:52.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.170+0000 7f243b7fe700 1 --2- 172.21.15.57:0/2134496774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c0b0c50 0x7f243c202100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f243000c410 tx=0x7f243000c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.170+0000 7f2438ff9700 1 -- 172.21.15.57:0/2134496774 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430004c80 con 0x7f243c0b0c50 2023-11-28T12:01:52.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.170+0000 7f2438ff9700 1 -- 172.21.15.57:0/2134496774 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430004de0 con 0x7f243c0b0c50 2023-11-28T12:01:52.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.170+0000 7f2438ff9700 1 -- 172.21.15.57:0/2134496774 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430007310 con 0x7f243c0b0c50 2023-11-28T12:01:52.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.170+0000 7f24426fc700 1 -- 172.21.15.57:0/2134496774 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243c0cb2d0 con 0x7f243c0b0c50 2023-11-28T12:01:52.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.170+0000 7f24426fc700 1 -- 172.21.15.57:0/2134496774 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243c0caff0 con 0x7f243c0b0c50 2023-11-28T12:01:52.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.171+0000 7efeddd65700 1 -- 172.21.15.57:0/3581635196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec0009470 msgr2=0x7efec00098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.171+0000 7efeddd65700 1 --2- 172.21.15.57:0/3581635196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efec0009470 0x7efec00098e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efec8001e50 tx=0x7efec0003840 comp rx=0 tx=0).stop 2023-11-28T12:01:52.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.171+0000 7efed7fff700 1 -- 172.21.15.57:0/3581635196 reap_dead start 2023-11-28T12:01:52.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.171+0000 7efeddd65700 1 -- 172.21.15.57:0/3581635196 shutdown_connections 2023-11-28T12:01:52.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.171+0000 7efeddd65700 1 -- 172.21.15.57:0/3581635196 >> 172.21.15.57:0/3581635196 conn(0x7efed806aed0 msgr2=0x7efed80cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.171+0000 7ff5b7bd6700 1 Processor -- start 2023-11-28T12:01:52.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.171+0000 7ff5b7bd6700 1 -- start start 2023-11-28T12:01:52.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.173+0000 7ff5b7bd6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b0160c30 0x7ff5b00ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.173+0000 7ff5b7bd6700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b00c2460 con 0x7ff5b00b0ef0 2023-11-28T12:01:52.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.173+0000 7ff5b7bd6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b006b770 con 0x7ff5b00adda0 2023-11-28T12:01:52.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.173+0000 7ff5b7bd6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b00ae210 con 0x7ff5b0160c30 2023-11-28T12:01:52.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.173+0000 7f2438ff9700 1 -- 172.21.15.57:0/2134496774 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f243001a020 con 0x7f243c0b0c50 2023-11-28T12:01:52.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.173+0000 7f24426fc700 1 -- 172.21.15.57:0/2134496774 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243c06b810 con 0x7f243c0b0c50 2023-11-28T12:01:52.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.173+0000 7ff5b6173700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff5b00b0ef0 0x7ff5b00ca220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37052/0 (socket says 172.21.15.57:37052) 2023-11-28T12:01:52.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.173+0000 7ff5b6173700 1 -- 172.21.15.57:0/478377383 learned_addr learned my addr 172.21.15.57:0/478377383 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:52.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.173+0000 7f2438ff9700 1 --2- 172.21.15.57:0/2134496774 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f24280682f0 0x7f242806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.173+0000 7f2438ff9700 1 -- 172.21.15.57:0/2134496774 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2430085770 con 0x7f243c0b0c50 2023-11-28T12:01:52.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7f243bfff700 1 --2- 172.21.15.57:0/2134496774 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f24280682f0 0x7f242806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625059773 0 0) 0x7ff5b00c2460 con 0x7ff5b00b0ef0 2023-11-28T12:01:52.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b5972700 1 --2- 172.21.15.57:0/478377383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b0160c30 0x7ff5b00ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff59c0035b0 con 0x7ff5b00b0ef0 2023-11-28T12:01:52.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3995609454 0 0) 0x7ff59c0035b0 con 0x7ff5b00b0ef0 2023-11-28T12:01:52.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b0160c30 msgr2=0x7ff5b00ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 --2- 172.21.15.57:0/478377383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b0160c30 0x7ff5b00ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 >> v1:172.21.15.181:6789/0 conn(0x7ff5b00adda0 legacy=0x7ff5b00b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b0161000 con 0x7ff5b00b0ef0 2023-11-28T12:01:52.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7ff5ac001670 con 0x7ff5b00b0ef0 2023-11-28T12:01:52.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b5972700 1 --2- 172.21.15.57:0/478377383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b0160c30 0x7ff5b00ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:52.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 >> v1:172.21.15.57:6789/0 conn(0x7ff5b00b0ef0 legacy=0x7ff5b00ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 --2- 172.21.15.57:0/478377383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59c003a30 0x7ff59c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 --2- 172.21.15.57:0/478377383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b0160c30 0x7ff59c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 --2- 172.21.15.57:0/478377383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff59c004d90 0x7ff59c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b00ae210 con 0x7ff59c004d90 2023-11-28T12:01:52.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c00bac0 con 0x7ff59c003a30 2023-11-28T12:01:52.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c00bc30 con 0x7ff5b0160c30 2023-11-28T12:01:52.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b5171700 1 --2- 172.21.15.57:0/478377383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b0160c30 0x7ff59c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.174+0000 7ff5b6173700 1 --2- 172.21.15.57:0/478377383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff59c004d90 0x7ff59c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.175+0000 7ff5b5972700 1 --2- 172.21.15.57:0/478377383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59c003a30 0x7ff59c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.175+0000 7ff5b6173700 1 -- 172.21.15.57:0/478377383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b0160c30 msgr2=0x7ff59c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.175+0000 7ff5b6173700 1 --2- 172.21.15.57:0/478377383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b0160c30 0x7ff59c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.175+0000 7ff5b6173700 1 -- 172.21.15.57:0/478377383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59c003a30 msgr2=0x7ff59c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.175+0000 7ff5b6173700 1 --2- 172.21.15.57:0/478377383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59c003a30 0x7ff59c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.175+0000 7ff5b6173700 1 -- 172.21.15.57:0/478377383 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a00094a0 con 0x7ff59c004d90 2023-11-28T12:01:52.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.175+0000 7ff5b6173700 1 --2- 172.21.15.57:0/478377383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff59c004d90 0x7ff59c005200 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00c210 tx=0x7ff59c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7ad34ca1a3f5ac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.175+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac007b70 con 0x7ff59c004d90 2023-11-28T12:01:52.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.175+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ac00cd50 con 0x7ff59c004d90 2023-11-28T12:01:52.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.175+0000 7ff5b4970700 1 -- 172.21.15.57:0/478377383 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac005b80 con 0x7ff59c004d90 2023-11-28T12:01:52.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.175+0000 7f560644a700 1 -- 172.21.15.57:0/1002551293 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56000cb2b0 con 0x7f56001609c0 2023-11-28T12:01:52.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.176+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/478377383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff59c004d90 msgr2=0x7ff59c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.176+0000 7ff5b7bd6700 1 --2- 172.21.15.57:0/478377383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff59c004d90 0x7ff59c005200 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00c210 tx=0x7ff59c005880 comp rx=0 tx=0).stop 2023-11-28T12:01:52.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.176+0000 7f243bfff700 1 --2- 172.21.15.57:0/2134496774 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f24280682f0 0x7f242806a7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f242c00ac10 tx=0x7f242c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.178+0000 7fc4993d0700 1 Processor -- start 2023-11-28T12:01:52.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.178+0000 7fc4993d0700 1 -- start start 2023-11-28T12:01:52.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.179+0000 7f55fffff700 1 --2- 172.21.15.57:0/1002551293 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f55ec0682f0 0x7f55ec06a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55f8000f80 tx=0x7f55f8009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7fc4993d0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc4941609c0 0x7fc4940ca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7fc4993d0700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4940cc690 con 0x7fc49406cdd0 2023-11-28T12:01:52.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7fc4993d0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4940cc8c0 con 0x7fc4940b0c50 2023-11-28T12:01:52.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7fc4993d0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4940caa30 con 0x7fc4941609c0 2023-11-28T12:01:52.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7fc492ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc4941609c0 0x7fc4940ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7fc492ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc4941609c0 0x7fc4940ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:49524/0 (socket says 172.21.15.57:49524) 2023-11-28T12:01:52.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7fc492ffd700 1 -- 172.21.15.57:0/1043382023 learned_addr learned my addr 172.21.15.57:0/1043382023 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:52.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7fc492ffd700 1 -- 172.21.15.57:0/1043382023 >> v1:172.21.15.181:6789/0 conn(0x7fc4940b0c50 legacy=0x7fc49406bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7fc492ffd700 1 -- 172.21.15.57:0/1043382023 >> v1:172.21.15.57:6789/0 conn(0x7fc49406cdd0 legacy=0x7fc4940b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7fc492ffd700 1 -- 172.21.15.57:0/1043382023 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4940cad10 con 0x7fc4941609c0 2023-11-28T12:01:52.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7fc492ffd700 1 --2- 172.21.15.57:0/1043382023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4941609c0 0x7fc4940ca150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0098f0 tx=0x7fc47c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=737c0e11dd12d18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.177+0000 7efeddd65700 1 -- 172.21.15.57:0/3581635196 shutdown_connections 2023-11-28T12:01:52.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.177+0000 7efeddd65700 1 -- 172.21.15.57:0/3581635196 wait complete. 2023-11-28T12:01:52.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.178+0000 7efeddd65700 1 Processor -- start 2023-11-28T12:01:52.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.178+0000 7efeddd65700 1 -- start start 2023-11-28T12:01:52.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.179+0000 7efeddd65700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efec0009470 0x7efed80c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.179+0000 7efeddd65700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed80aeaa0 0x7efed81e2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.179+0000 7efeddd65700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed80c7060 0x7efed80c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.179+0000 7efeddd65700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed80c7a40 con 0x7efec0009470 2023-11-28T12:01:52.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.179+0000 7efeddd65700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed80c5430 con 0x7efed80c7060 2023-11-28T12:01:52.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.179+0000 7efeddd65700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed80aefa0 con 0x7efed80aeaa0 2023-11-28T12:01:52.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.179+0000 7efed6ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed80aeaa0 0x7efed81e2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.179+0000 7efed6ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed80aeaa0 0x7efed81e2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:32774/0 (socket says 172.21.15.57:32774) 2023-11-28T12:01:52.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.179+0000 7efed6ffd700 1 -- 172.21.15.57:0/4181134426 learned_addr learned my addr 172.21.15.57:0/4181134426 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:52.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7efed6ffd700 1 -- 172.21.15.57:0/4181134426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed80c7060 msgr2=0x7efed80c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7efed6ffd700 1 --2- 172.21.15.57:0/4181134426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed80c7060 0x7efed80c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7efed6ffd700 1 -- 172.21.15.57:0/4181134426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efec0009470 msgr2=0x7efed80c4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7efed6ffd700 1 --2- 172.21.15.57:0/4181134426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efec0009470 0x7efed80c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.180+0000 7efed6ffd700 1 -- 172.21.15.57:0/4181134426 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed80cb090 con 0x7efed80aeaa0 2023-11-28T12:01:52.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.181+0000 7fc491ffb700 1 -- 172.21.15.57:0/1043382023 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c004230 con 0x7fc4941609c0 2023-11-28T12:01:52.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.181+0000 7ff5b6173700 1 -- 172.21.15.57:0/478377383 reap_dead start 2023-11-28T12:01:52.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.181+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/478377383 shutdown_connections 2023-11-28T12:01:52.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.181+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/478377383 >> 172.21.15.57:0/478377383 conn(0x7ff5b00cb540 msgr2=0x7ff5b00cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.181+0000 7fc491ffb700 1 -- 172.21.15.57:0/1043382023 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47c004390 con 0x7fc4941609c0 2023-11-28T12:01:52.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.181+0000 7fc491ffb700 1 -- 172.21.15.57:0/1043382023 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0036a0 con 0x7fc4941609c0 2023-11-28T12:01:52.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.182+0000 7efed6ffd700 1 --2- 172.21.15.57:0/4181134426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed80aeaa0 0x7efed81e2880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efed000b9a0 tx=0x7efed000bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.182+0000 7fc4993d0700 1 -- 172.21.15.57:0/1043382023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4941609c0 msgr2=0x7fc4940ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.182+0000 7fc4993d0700 1 --2- 172.21.15.57:0/1043382023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4941609c0 0x7fc4940ca150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0098f0 tx=0x7fc47c009c00 comp rx=0 tx=0).stop 2023-11-28T12:01:52.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.182+0000 7efebffff700 1 -- 172.21.15.57:0/4181134426 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed000c9b0 con 0x7efed80aeaa0 2023-11-28T12:01:52.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.182+0000 7efebffff700 1 -- 172.21.15.57:0/4181134426 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed000cb10 con 0x7efed80aeaa0 2023-11-28T12:01:52.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.182+0000 7efebffff700 1 -- 172.21.15.57:0/4181134426 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0014440 con 0x7efed80aeaa0 2023-11-28T12:01:52.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.182+0000 7efeddd65700 1 -- 172.21.15.57:0/4181134426 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed80cadb0 con 0x7efed80aeaa0 2023-11-28T12:01:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.182+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/478377383 shutdown_connections 2023-11-28T12:01:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.182+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/478377383 wait complete. 2023-11-28T12:01:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.182+0000 7efeddd65700 1 -- 172.21.15.57:0/4181134426 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed8158d50 con 0x7efed80aeaa0 2023-11-28T12:01:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.184+0000 7ff5b7bd6700 1 Processor -- start 2023-11-28T12:01:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.182+0000 7f55fcff9700 1 -- 172.21.15.57:0/1002551293 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f55f0055ba0 con 0x7f56001609c0 2023-11-28T12:01:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.184+0000 7ff5b7bd6700 1 -- start start 2023-11-28T12:01:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.183+0000 7fc4993d0700 1 -- 172.21.15.57:0/1043382023 shutdown_connections 2023-11-28T12:01:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.183+0000 7fc4993d0700 1 --2- 172.21.15.57:0/1043382023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4941609c0 0x7fc4940ca150 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.183+0000 7fc4993d0700 1 -- 172.21.15.57:0/1043382023 >> 172.21.15.57:0/1043382023 conn(0x7fc49406aed0 msgr2=0x7fc4940cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.184+0000 7f2438ff9700 1 -- 172.21.15.57:0/2134496774 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2430051ee0 con 0x7f243c0b0c50 2023-11-28T12:01:52.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.186+0000 7ff5b7bd6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5b01f5fd0 0x7ff5b00b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.187+0000 7ff5b7bd6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b01f9b90 0x7ff5b01fa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.187+0000 7ff5b7bd6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5b01f63a0 0x7ff5b01f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.187+0000 7ff5b7bd6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b01fa540 con 0x7ff5b01f5fd0 2023-11-28T12:01:52.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.187+0000 7ff5b7bd6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b00b33f0 con 0x7ff5b01f63a0 2023-11-28T12:01:52.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.187+0000 7ff5b7bd6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b00b4810 con 0x7ff5b01f9b90 2023-11-28T12:01:52.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.187+0000 7ff5b6173700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5b01f63a0 0x7ff5b01f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.187+0000 7ff5b6173700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5b01f63a0 0x7ff5b01f6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:49540/0 (socket says 172.21.15.57:49540) 2023-11-28T12:01:52.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.187+0000 7ff5b6173700 1 -- 172.21.15.57:0/217585561 learned_addr learned my addr 172.21.15.57:0/217585561 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:52.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.187+0000 7ff5b5171700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b01f9b90 0x7ff5b01fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.187+0000 7ff5b5972700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5b01f5fd0 0x7ff5b00b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.188+0000 7ff5b5171700 1 -- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5b01f63a0 msgr2=0x7ff5b01f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.188+0000 7ff5b5171700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5b01f63a0 0x7ff5b01f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.188+0000 7ff5b5171700 1 -- 172.21.15.57:0/217585561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5b01f5fd0 msgr2=0x7ff5b00b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.188+0000 7ff5b5171700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5b01f5fd0 0x7ff5b00b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.188+0000 7ff5b5171700 1 -- 172.21.15.57:0/217585561 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b0164a90 con 0x7ff5b01f9b90 2023-11-28T12:01:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.188+0000 7ff5b5171700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b01f9b90 0x7ff5b01fa000 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800fe40 tx=0x7ff5a800d430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.189+0000 7ff5a67fc700 1 -- 172.21.15.57:0/217585561 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8005240 con 0x7ff5b01f9b90 2023-11-28T12:01:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.189+0000 7ff5a67fc700 1 -- 172.21.15.57:0/217585561 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a800d990 con 0x7ff5b01f9b90 2023-11-28T12:01:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.189+0000 7ff5a67fc700 1 -- 172.21.15.57:0/217585561 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a800dd40 con 0x7ff5b01f9b90 2023-11-28T12:01:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.186+0000 7fc4993d0700 1 -- 172.21.15.57:0/1043382023 shutdown_connections 2023-11-28T12:01:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.187+0000 7fc4993d0700 1 -- 172.21.15.57:0/1043382023 wait complete. 2023-11-28T12:01:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.189+0000 7fc4993d0700 1 Processor -- start 2023-11-28T12:01:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.189+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/217585561 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b01647b0 con 0x7ff5b01f9b90 2023-11-28T12:01:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.189+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/217585561 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b0164d70 con 0x7ff5b01f9b90 2023-11-28T12:01:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.186+0000 7efebffff700 1 -- 172.21.15.57:0/4181134426 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efed001b030 con 0x7efed80aeaa0 2023-11-28T12:01:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.187+0000 7efebffff700 1 --2- 172.21.15.57:0/4181134426 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efec40682f0 0x7efec406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.187+0000 7efebffff700 1 -- 172.21.15.57:0/4181134426 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efed0085900 con 0x7efed80aeaa0 2023-11-28T12:01:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.188+0000 7efed77fe700 1 --2- 172.21.15.57:0/4181134426 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efec40682f0 0x7efec406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.188+0000 7efed77fe700 1 --2- 172.21.15.57:0/4181134426 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efec40682f0 0x7efec406a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efed806cbf0 tx=0x7efecc006d20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.189+0000 7fc4993d0700 1 -- start start 2023-11-28T12:01:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.190+0000 7fc4993d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc49406cdd0 0x7fc4941f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.190+0000 7fc4993d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4940b0c50 0x7fc494159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.190+0000 7fc4993d0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4941609c0 0x7fc4940b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.190+0000 7fc4993d0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940c4ca0 con 0x7fc49406cdd0 2023-11-28T12:01:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.190+0000 7fc4993d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940c40d0 con 0x7fc4940b0c50 2023-11-28T12:01:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.190+0000 7fc4993d0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940c4370 con 0x7fc4941609c0 2023-11-28T12:01:52.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.191+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/217585561 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff594005320 con 0x7ff5b01f9b90 2023-11-28T12:01:52.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.191+0000 7fc4937fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4941609c0 0x7fc4940b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.191+0000 7fc4937fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4941609c0 0x7fc4940b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:32796/0 (socket says 172.21.15.57:32796) 2023-11-28T12:01:52.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.191+0000 7fc4937fe700 1 -- 172.21.15.57:0/2946741241 learned_addr learned my addr 172.21.15.57:0/2946741241 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:52.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.191+0000 7fc4937fe700 1 -- 172.21.15.57:0/2946741241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4940b0c50 msgr2=0x7fc494159670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T12:01:52.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.191+0000 7fc4937fe700 1 --2- 172.21.15.57:0/2946741241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4940b0c50 0x7fc494159670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.191+0000 7fc4937fe700 1 -- 172.21.15.57:0/2946741241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc49406cdd0 msgr2=0x7fc4941f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.191+0000 7fc4937fe700 1 --2- 172.21.15.57:0/2946741241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc49406cdd0 0x7fc4941f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.191+0000 7fc4937fe700 1 -- 172.21.15.57:0/2946741241 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c009550 con 0x7fc4941609c0 2023-11-28T12:01:52.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.191+0000 7fc4937fe700 1 --2- 172.21.15.57:0/2946741241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4941609c0 0x7fc4940b1ae0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00b990 tx=0x7fc48c00bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.192+0000 7fc48b7fe700 1 -- 172.21.15.57:0/2946741241 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c00c9d0 con 0x7fc4941609c0 2023-11-28T12:01:52.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.192+0000 7fc48b7fe700 1 -- 172.21.15.57:0/2946741241 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c00cb30 con 0x7fc4941609c0 2023-11-28T12:01:52.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.192+0000 7fc48b7fe700 1 -- 172.21.15.57:0/2946741241 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c014440 con 0x7fc4941609c0 2023-11-28T12:01:52.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.192+0000 7fc4993d0700 1 -- 172.21.15.57:0/2946741241 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4940cb2d0 con 0x7fc4941609c0 2023-11-28T12:01:52.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.192+0000 7fc4993d0700 1 -- 172.21.15.57:0/2946741241 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4940caff0 con 0x7fc4941609c0 2023-11-28T12:01:52.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.194+0000 7fc385dcc700 1 Processor -- start 2023-11-28T12:01:52.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.194+0000 7fc385dcc700 1 -- start start 2023-11-28T12:01:52.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.194+0000 7fc4993d0700 1 -- 172.21.15.57:0/2946741241 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4780052f0 con 0x7fc4941609c0 2023-11-28T12:01:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.195+0000 7fc385dcc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800b0540 0x7fc3800b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.195+0000 7fc385dcc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc3800c9b00 0x7fc3801598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.195+0000 7fc385dcc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38015cf40 con 0x7fc3800adf30 2023-11-28T12:01:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.195+0000 7fc385dcc700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3800b0e50 con 0x7fc3800c9b00 2023-11-28T12:01:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.195+0000 7fc385dcc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38015ddc0 con 0x7fc3800b0540 2023-11-28T12:01:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.195+0000 7fc37f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800b0540 0x7fc3800b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.195+0000 7fc37f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800b0540 0x7fc3800b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:32806/0 (socket says 172.21.15.57:32806) 2023-11-28T12:01:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.195+0000 7fc37f7fe700 1 -- 172.21.15.57:0/4002662228 learned_addr learned my addr 172.21.15.57:0/4002662228 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.195+0000 7fc37effd700 1 --2- 172.21.15.57:0/4002662228 >> v2:172.21.15.57:3300/0 conn(0x7fc3800c9b00 0x7fc3801598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.196+0000 7fc37f7fe700 1 -- 172.21.15.57:0/4002662228 >> v2:172.21.15.57:3300/0 conn(0x7fc3800c9b00 msgr2=0x7fc3801598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.196+0000 7fc37f7fe700 1 --2- 172.21.15.57:0/4002662228 >> v2:172.21.15.57:3300/0 conn(0x7fc3800c9b00 0x7fc3801598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.196+0000 7fc37f7fe700 1 -- 172.21.15.57:0/4002662228 >> v1:172.21.15.181:6789/0 conn(0x7fc3800adf30 legacy=0x7fc38015d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.196+0000 7fc37f7fe700 1 -- 172.21.15.57:0/4002662228 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38015a1c0 con 0x7fc3800b0540 2023-11-28T12:01:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.196+0000 7fc37effd700 1 --2- 172.21.15.57:0/4002662228 >> v2:172.21.15.57:3300/0 conn(0x7fc3800c9b00 0x7fc3801598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.196+0000 7fc37f7fe700 1 --2- 172.21.15.57:0/4002662228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800b0540 0x7fc3800b0910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc37800d150 tx=0x7fc37800d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c0cfdc6c19f958b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.191+0000 7efeddd65700 1 -- 172.21.15.57:0/4181134426 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb80052f0 con 0x7efed80aeaa0 2023-11-28T12:01:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.197+0000 7ff5a67fc700 1 -- 172.21.15.57:0/217585561 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff5a8014470 con 0x7ff5b01f9b90 2023-11-28T12:01:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.197+0000 7ff5a67fc700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff59c069820 0x7ff59c06bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.197+0000 7fc37e7fc700 1 -- 172.21.15.57:0/4002662228 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378010070 con 0x7fc3800b0540 2023-11-28T12:01:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.197+0000 7ff5a67fc700 1 -- 172.21.15.57:0/217585561 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff5a80864f0 con 0x7ff5b01f9b90 2023-11-28T12:01:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.198+0000 7efebffff700 1 -- 172.21.15.57:0/4181134426 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efed00520e0 con 0x7efed80aeaa0 2023-11-28T12:01:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.198+0000 7ff5b5972700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff59c069820 0x7ff59c06bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.200+0000 7f1951352700 1 Processor -- start 2023-11-28T12:01:52.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.201+0000 7f1951352700 1 -- start start 2023-11-28T12:01:52.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.200+0000 7fc48b7fe700 1 -- 172.21.15.57:0/2946741241 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc48c01b030 con 0x7fc4941609c0 2023-11-28T12:01:52.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.200+0000 7fc48b7fe700 1 --2- 172.21.15.57:0/2946741241 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc474068340 0x7fc47406a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.201+0000 7fc492ffd700 1 --2- 172.21.15.57:0/2946741241 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc474068340 0x7fc47406a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.201+0000 7ff5b5972700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff59c069820 0x7ff59c06bcd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009470 tx=0x7ff5a0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.201+0000 7fc492ffd700 1 --2- 172.21.15.57:0/2946741241 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc474068340 0x7fc47406a7f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009c40 tx=0x7fc47c010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.202+0000 7f1951352700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f194c0af0a0 0x7f194c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.202+0000 7f1951352700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c159ec0 con 0x7f194c1599b0 2023-11-28T12:01:52.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.202+0000 7f1951352700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c15d8a0 con 0x7f194c0b1650 2023-11-28T12:01:52.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.202+0000 7f1951352700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c160370 con 0x7f194c0af0a0 2023-11-28T12:01:52.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.202+0000 7f194affd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f194c1599b0 0x7f194c15d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37082/0 (socket says 172.21.15.57:37082) 2023-11-28T12:01:52.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.202+0000 7f194affd700 1 -- 172.21.15.57:0/2118771704 learned_addr learned my addr 172.21.15.57:0/2118771704 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:52.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.202+0000 7ff5a67fc700 1 -- 172.21.15.57:0/217585561 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff5a8052d50 con 0x7ff5b01f9b90 2023-11-28T12:01:52.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.202+0000 7fc48b7fe700 1 -- 172.21.15.57:0/2946741241 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc48c05e350 con 0x7fc4941609c0 2023-11-28T12:01:52.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.207+0000 7fc48b7fe700 1 -- 172.21.15.57:0/2946741241 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc48c052570 con 0x7fc4941609c0 2023-11-28T12:01:52.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345137231 0 0) 0x7f194c159ec0 con 0x7f194c1599b0 2023-11-28T12:01:52.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19340035b0 con 0x7f194c1599b0 2023-11-28T12:01:52.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123742652 0 0) 0x7f194c15d8a0 con 0x7f194c0b1650 2023-11-28T12:01:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f194c159ec0 con 0x7f194c0b1650 2023-11-28T12:01:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1779603313 0 0) 0x7f19340035b0 con 0x7f194c1599b0 2023-11-28T12:01:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f194c0af0a0 msgr2=0x7f194c15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 --2- 172.21.15.57:0/2118771704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f194c0af0a0 0x7f194c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 >> v1:172.21.15.181:6789/0 conn(0x7f194c0b1650 legacy=0x7f194c06cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c15db80 con 0x7f194c1599b0 2023-11-28T12:01:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f193c0015f0 con 0x7f194c1599b0 2023-11-28T12:01:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 >> v1:172.21.15.57:6789/0 conn(0x7f194c1599b0 legacy=0x7f194c15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 --2- 172.21.15.57:0/2118771704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1934003bc0 0x7f1934003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 --2- 172.21.15.57:0/2118771704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1934004850 0x7f1934004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 --2- 172.21.15.57:0/2118771704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1934005250 0x7f19340079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c160370 con 0x7f1934005250 2023-11-28T12:01:52.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1934008140 con 0x7f1934003bc0 2023-11-28T12:01:52.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1934008300 con 0x7f1934004850 2023-11-28T12:01:52.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.204+0000 7f194a7fc700 1 --2- 172.21.15.57:0/2118771704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1934004850 0x7f1934004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.204+0000 7f194a7fc700 1 -- 172.21.15.57:0/2118771704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1934003bc0 msgr2=0x7f1934003fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T12:01:52.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.204+0000 7f194a7fc700 1 --2- 172.21.15.57:0/2118771704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1934003bc0 0x7f1934003fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.204+0000 7f194a7fc700 1 -- 172.21.15.57:0/2118771704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1934005250 msgr2=0x7f19340079f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:52.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.204+0000 7f194a7fc700 1 --2- 172.21.15.57:0/2118771704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1934005250 0x7f19340079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.204+0000 7f194a7fc700 1 -- 172.21.15.57:0/2118771704 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c15db80 con 0x7f1934004850 2023-11-28T12:01:52.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.204+0000 7f194a7fc700 1 --2- 172.21.15.57:0/2118771704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1934004850 0x7f1934004cc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f194c06c730 tx=0x7f1934005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46f0f0794cb4bd57 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.207+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940011620 con 0x7f1934004850 2023-11-28T12:01:52.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.207+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1940011780 con 0x7f1934004850 2023-11-28T12:01:52.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.207+0000 7f1949ffb700 1 -- 172.21.15.57:0/2118771704 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940011b10 con 0x7f1934004850 2023-11-28T12:01:52.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.202+0000 7fc37e7fc700 1 -- 172.21.15.57:0/4002662228 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37800dd80 con 0x7fc3800b0540 2023-11-28T12:01:52.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.202+0000 7fc37e7fc700 1 -- 172.21.15.57:0/4002662228 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37800b310 con 0x7fc3800b0540 2023-11-28T12:01:52.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7fc385dcc700 1 -- 172.21.15.57:0/4002662228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800b0540 msgr2=0x7fc3800b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.203+0000 7fc385dcc700 1 --2- 172.21.15.57:0/4002662228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800b0540 0x7fc3800b0910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc37800d150 tx=0x7fc37800d460 comp rx=0 tx=0).stop 2023-11-28T12:01:52.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.211+0000 7f1951352700 1 -- 172.21.15.57:0/2118771704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1934004850 msgr2=0x7f1934004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.211+0000 7f1951352700 1 --2- 172.21.15.57:0/2118771704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1934004850 0x7f1934004cc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f194c06c730 tx=0x7f1934005880 comp rx=0 tx=0).stop 2023-11-28T12:01:52.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.211+0000 7f194b7fe700 1 -- 172.21.15.57:0/2118771704 reap_dead start 2023-11-28T12:01:52.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.211+0000 7f1951352700 1 -- 172.21.15.57:0/2118771704 shutdown_connections 2023-11-28T12:01:52.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.211+0000 7f1951352700 1 -- 172.21.15.57:0/2118771704 >> 172.21.15.57:0/2118771704 conn(0x7f194c06c010 msgr2=0x7f194c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.211+0000 7f1951352700 1 -- 172.21.15.57:0/2118771704 shutdown_connections 2023-11-28T12:01:52.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.211+0000 7f1951352700 1 -- 172.21.15.57:0/2118771704 wait complete. 2023-11-28T12:01:52.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.213+0000 7f1951352700 1 Processor -- start 2023-11-28T12:01:52.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.214+0000 7fc385dcc700 1 -- 172.21.15.57:0/4002662228 shutdown_connections 2023-11-28T12:01:52.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.214+0000 7fc385dcc700 1 --2- 172.21.15.57:0/4002662228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800b0540 0x7fc3800b0910 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.214+0000 7fc385dcc700 1 --2- 172.21.15.57:0/4002662228 >> v2:172.21.15.57:3300/0 conn(0x7fc3800c9b00 0x7fc3801598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.214+0000 7fc385dcc700 1 -- 172.21.15.57:0/4002662228 >> 172.21.15.57:0/4002662228 conn(0x7fc3800cb540 msgr2=0x7fc3800cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.214+0000 7fc385dcc700 1 -- 172.21.15.57:0/4002662228 shutdown_connections 2023-11-28T12:01:52.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.214+0000 7fc385dcc700 1 -- 172.21.15.57:0/4002662228 wait complete. 2023-11-28T12:01:52.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.216+0000 7fc385dcc700 1 Processor -- start 2023-11-28T12:01:52.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.214+0000 7f1951352700 1 -- start start 2023-11-28T12:01:52.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.215+0000 7f1951352700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1934004850 0x7f194c13fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.215+0000 7f1951352700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f194c0b5660 0x7f194c0c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.215+0000 7f1951352700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f194c0b2a00 0x7f194c0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.215+0000 7f1951352700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0b4670 con 0x7f194c0b2a00 2023-11-28T12:01:52.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.215+0000 7f1951352700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0b3170 con 0x7f1934004850 2023-11-28T12:01:52.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.215+0000 7f1951352700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0b2ea0 con 0x7f194c0b5660 2023-11-28T12:01:52.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.215+0000 7f194b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f194c0b2a00 0x7f194c0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.215+0000 7f194b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f194c0b2a00 0x7f194c0b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48198/0 (socket says 172.21.15.57:48198) 2023-11-28T12:01:52.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.215+0000 7f194b7fe700 1 -- 172.21.15.57:0/3226779621 learned_addr learned my addr 172.21.15.57:0/3226779621 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:52.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.216+0000 7fc385dcc700 1 -- start start 2023-11-28T12:01:52.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.218+0000 7fc385dcc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800adf30 0x7fc38015c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.218+0000 7fc385dcc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3800b0540 0x7fc3800ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.218+0000 7fc385dcc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3800c9b00 0x7fc380154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.218+0000 7fc385dcc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38015b910 con 0x7fc3800b0540 2023-11-28T12:01:52.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.218+0000 7fc385dcc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38015ac90 con 0x7fc3800c9b00 2023-11-28T12:01:52.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.218+0000 7fc385dcc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38015af70 con 0x7fc3800adf30 2023-11-28T12:01:52.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.216+0000 7f194b7fe700 1 -- 172.21.15.57:0/3226779621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f194c0b5660 msgr2=0x7f194c0c61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T12:01:52.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.216+0000 7f194b7fe700 1 --2- 172.21.15.57:0/3226779621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f194c0b5660 0x7f194c0c61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.216+0000 7f194b7fe700 1 -- 172.21.15.57:0/3226779621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1934004850 msgr2=0x7f194c13fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T12:01:52.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.216+0000 7f194b7fe700 1 --2- 172.21.15.57:0/3226779621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1934004850 0x7f194c13fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.216+0000 7f194b7fe700 1 -- 172.21.15.57:0/3226779621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c06d4a0 con 0x7f194c0b2a00 2023-11-28T12:01:52.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.221+0000 7f194b7fe700 1 --2- 172.21.15.57:0/3226779621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f194c0b2a00 0x7f194c0b06d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f194c072090 tx=0x7f194400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.218+0000 7fc37f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800adf30 0x7fc38015c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.218+0000 7fc37f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800adf30 0x7fc38015c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:32848/0 (socket says 172.21.15.57:32848) 2023-11-28T12:01:52.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.218+0000 7fc37f7fe700 1 -- 172.21.15.57:0/245180373 learned_addr learned my addr 172.21.15.57:0/245180373 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:52.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.219+0000 7fc37f7fe700 1 -- 172.21.15.57:0/245180373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3800c9b00 msgr2=0x7fc380154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.219+0000 7fc37f7fe700 1 --2- 172.21.15.57:0/245180373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3800c9b00 0x7fc380154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.219+0000 7fc37f7fe700 1 -- 172.21.15.57:0/245180373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3800b0540 msgr2=0x7fc3800ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.219+0000 7fc37f7fe700 1 --2- 172.21.15.57:0/245180373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3800b0540 0x7fc3800ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.219+0000 7fc37f7fe700 1 -- 172.21.15.57:0/245180373 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37800bd40 con 0x7fc3800adf30 2023-11-28T12:01:52.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.219+0000 7fc37f7fe700 1 --2- 172.21.15.57:0/245180373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800adf30 0x7fc38015c480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc378017040 tx=0x7fc378004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.219+0000 7fc367fff700 1 -- 172.21.15.57:0/245180373 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37800b760 con 0x7fc3800adf30 2023-11-28T12:01:52.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.219+0000 7fc367fff700 1 -- 172.21.15.57:0/245180373 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc378009a10 con 0x7fc3800adf30 2023-11-28T12:01:52.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.221+0000 7fc367fff700 1 -- 172.21.15.57:0/245180373 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378009dc0 con 0x7fc3800adf30 2023-11-28T12:01:52.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.221+0000 7fc385dcc700 1 -- 172.21.15.57:0/245180373 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38015a780 con 0x7fc3800adf30 2023-11-28T12:01:52.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.221+0000 7fc385dcc700 1 -- 172.21.15.57:0/245180373 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38015a4a0 con 0x7fc3800adf30 2023-11-28T12:01:52.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.223+0000 7fc367fff700 1 -- 172.21.15.57:0/245180373 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc378004030 con 0x7fc3800adf30 2023-11-28T12:01:52.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.224+0000 7fc367fff700 1 --2- 172.21.15.57:0/245180373 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc36c068340 0x7fc36c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.224+0000 7fc367fff700 1 -- 172.21.15.57:0/245180373 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc378087680 con 0x7fc3800adf30 2023-11-28T12:01:52.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.224+0000 7f19337fe700 1 -- 172.21.15.57:0/3226779621 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194400a760 con 0x7f194c0b2a00 2023-11-28T12:01:52.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.224+0000 7f19337fe700 1 -- 172.21.15.57:0/3226779621 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194400a8c0 con 0x7f194c0b2a00 2023-11-28T12:01:52.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.224+0000 7f19337fe700 1 -- 172.21.15.57:0/3226779621 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194400ac70 con 0x7f194c0b2a00 2023-11-28T12:01:52.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.225+0000 7fc37effd700 1 --2- 172.21.15.57:0/245180373 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc36c068340 0x7fc36c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.225+0000 7f1951352700 1 -- 172.21.15.57:0/3226779621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c15de60 con 0x7f194c0b2a00 2023-11-28T12:01:52.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.225+0000 7f1951352700 1 -- 172.21.15.57:0/3226779621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c06d630 con 0x7f194c0b2a00 2023-11-28T12:01:52.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.228+0000 7fc37effd700 1 --2- 172.21.15.57:0/245180373 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc36c068340 0x7fc36c06a7f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc374006fd0 tx=0x7fc374008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.228+0000 7f1951352700 1 -- 172.21.15.57:0/3226779621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c0743f0 con 0x7f194c0b2a00 2023-11-28T12:01:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.228+0000 7f19337fe700 1 -- 172.21.15.57:0/3226779621 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1944019030 con 0x7f194c0b2a00 2023-11-28T12:01:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.228+0000 7f19337fe700 1 --2- 172.21.15.57:0/3226779621 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f19380683c0 0x7f193806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.229+0000 7f19337fe700 1 -- 172.21.15.57:0/3226779621 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1944085480 con 0x7f194c0b2a00 2023-11-28T12:01:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.229+0000 7fc385dcc700 1 -- 172.21.15.57:0/245180373 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38015d190 con 0x7fc3800adf30 2023-11-28T12:01:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.229+0000 7f194affd700 1 --2- 172.21.15.57:0/3226779621 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f19380683c0 0x7f193806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.230+0000 7f194affd700 1 --2- 172.21.15.57:0/3226779621 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f19380683c0 0x7f193806a870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f194c06e5b0 tx=0x7f193c001460 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:52.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.234+0000 7fc367fff700 1 -- 172.21.15.57:0/245180373 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc378053df0 con 0x7fc3800adf30 2023-11-28T12:01:52.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.238+0000 7f19337fe700 1 -- 172.21.15.57:0/3226779621 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f19440556a0 con 0x7f194c0b2a00 2023-11-28T12:01:52.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.546+0000 7f6f76cad700 1 -- 172.21.15.57:0/501164978 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6f500055a0 con 0x7f6f700b0540 2023-11-28T12:01:52.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.547+0000 7f6f6d7fa700 1 -- 172.21.15.57:0/501164978 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6f68056030 con 0x7f6f700b0540 2023-11-28T12:01:52.551 INFO:teuthology.orchestra.run.smithi057.stdout:55834574854 2023-11-28T12:01:52.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.551+0000 7f6f76cad700 1 -- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6f5c0683c0 msgr2=0x7f6f5c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.551+0000 7f6f76cad700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6f5c0683c0 0x7f6f5c06a870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f60007900 tx=0x7f6f60008040 comp rx=0 tx=0).stop 2023-11-28T12:01:52.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.551+0000 7f6f76cad700 1 -- 172.21.15.57:0/501164978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f700b0540 msgr2=0x7f6f700c5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.551+0000 7f6f76cad700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f700b0540 0x7f6f700c5b90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6f68007a10 tx=0x7f6f68007af0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.552+0000 7f6f76cad700 1 -- 172.21.15.57:0/501164978 shutdown_connections 2023-11-28T12:01:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.552+0000 7f6f76cad700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6f5c0683c0 0x7f6f5c06a870 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.552+0000 7f6f76cad700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f70158980 0x7f6f700c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.552+0000 7f6f76cad700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f7006d230 0x7f6f700c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.552+0000 7f6f76cad700 1 --2- 172.21.15.57:0/501164978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f700b0540 0x7f6f700c5b90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.552+0000 7f6f76cad700 1 -- 172.21.15.57:0/501164978 >> 172.21.15.57:0/501164978 conn(0x7f6f7006aed0 msgr2=0x7f6f700ce810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.552+0000 7f6f76cad700 1 -- 172.21.15.57:0/501164978 shutdown_connections 2023-11-28T12:01:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.552+0000 7f6f76cad700 1 -- 172.21.15.57:0/501164978 wait complete. 2023-11-28T12:01:52.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2023-11-28T12:01:52.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.591+0000 7f560644a700 1 -- 172.21.15.57:0/1002551293 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f56000cc2d0 con 0x7f56001609c0 2023-11-28T12:01:52.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.591+0000 7f55fcff9700 1 -- 172.21.15.57:0/1002551293 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f55f0021020 con 0x7f56001609c0 2023-11-28T12:01:52.593 INFO:teuthology.orchestra.run.smithi057.stdout:55834574854 2023-11-28T12:01:52.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.595+0000 7f560644a700 1 -- 172.21.15.57:0/1002551293 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f55ec0682f0 msgr2=0x7f55ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.595+0000 7f560644a700 1 --2- 172.21.15.57:0/1002551293 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f55ec0682f0 0x7f55ec06a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55f8000f80 tx=0x7f55f8009000 comp rx=0 tx=0).stop 2023-11-28T12:01:52.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.595+0000 7f560644a700 1 -- 172.21.15.57:0/1002551293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56001609c0 msgr2=0x7f56000c38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.595+0000 7f560644a700 1 --2- 172.21.15.57:0/1002551293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56001609c0 0x7f56000c38c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f55f000e810 tx=0x7f55f000ebd0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.596+0000 7f560644a700 1 -- 172.21.15.57:0/1002551293 shutdown_connections 2023-11-28T12:01:52.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.596+0000 7f560644a700 1 --2- 172.21.15.57:0/1002551293 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f55ec0682f0 0x7f55ec06a7a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.596+0000 7f560644a700 1 --2- 172.21.15.57:0/1002551293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56001609c0 0x7f56000c38c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.596+0000 7f560644a700 1 --2- 172.21.15.57:0/1002551293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f560006cdd0 0x7f56000b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.596+0000 7f560644a700 1 --2- 172.21.15.57:0/1002551293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56000b0a80 0x7f56000b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.596+0000 7f560644a700 1 -- 172.21.15.57:0/1002551293 >> 172.21.15.57:0/1002551293 conn(0x7f560006aed0 msgr2=0x7f56000ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.597+0000 7f24426fc700 1 -- 172.21.15.57:0/2134496774 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f243c0743f0 con 0x7f243c0b0c50 2023-11-28T12:01:52.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.598+0000 7f560644a700 1 -- 172.21.15.57:0/1002551293 shutdown_connections 2023-11-28T12:01:52.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.598+0000 7f560644a700 1 -- 172.21.15.57:0/1002551293 wait complete. 2023-11-28T12:01:52.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.598+0000 7efeddd65700 1 -- 172.21.15.57:0/4181134426 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7efeb80055a0 con 0x7efed80aeaa0 2023-11-28T12:01:52.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.599+0000 7efebffff700 1 -- 172.21.15.57:0/4181134426 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efed0019020 con 0x7efed80aeaa0 2023-11-28T12:01:52.601 INFO:teuthology.orchestra.run.smithi057.stdout:55834574855 2023-11-28T12:01:52.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.604+0000 7efeddd65700 1 -- 172.21.15.57:0/4181134426 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efec40682f0 msgr2=0x7efec406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.604+0000 7efeddd65700 1 --2- 172.21.15.57:0/4181134426 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efec40682f0 0x7efec406a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efed806cbf0 tx=0x7efecc006d20 comp rx=0 tx=0).stop 2023-11-28T12:01:52.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.604+0000 7efeddd65700 1 -- 172.21.15.57:0/4181134426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed80aeaa0 msgr2=0x7efed81e2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.604+0000 7efeddd65700 1 --2- 172.21.15.57:0/4181134426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed80aeaa0 0x7efed81e2880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efed000b9a0 tx=0x7efed000bd60 comp rx=0 tx=0).stop 2023-11-28T12:01:52.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.604+0000 7efeddd65700 1 -- 172.21.15.57:0/4181134426 shutdown_connections 2023-11-28T12:01:52.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.604+0000 7efeddd65700 1 --2- 172.21.15.57:0/4181134426 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efec40682f0 0x7efec406a7a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.604+0000 7efeddd65700 1 --2- 172.21.15.57:0/4181134426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed80c7060 0x7efed80c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.604+0000 7efeddd65700 1 --2- 172.21.15.57:0/4181134426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efec0009470 0x7efed80c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.604+0000 7efeddd65700 1 --2- 172.21.15.57:0/4181134426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed80aeaa0 0x7efed81e2880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.604+0000 7efeddd65700 1 -- 172.21.15.57:0/4181134426 >> 172.21.15.57:0/4181134426 conn(0x7efed806aed0 msgr2=0x7efed80b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.606+0000 7f2438ff9700 1 -- 172.21.15.57:0/2134496774 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2430055500 con 0x7f243c0b0c50 2023-11-28T12:01:52.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.606+0000 7efeddd65700 1 -- 172.21.15.57:0/4181134426 shutdown_connections 2023-11-28T12:01:52.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.606+0000 7efeddd65700 1 -- 172.21.15.57:0/4181134426 wait complete. 2023-11-28T12:01:52.610 INFO:teuthology.orchestra.run.smithi057.stdout:55834574852 2023-11-28T12:01:52.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.607+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/217585561 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff5940055d0 con 0x7ff5b01f9b90 2023-11-28T12:01:52.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.609+0000 7ff5a67fc700 1 -- 172.21.15.57:0/217585561 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff5a800dea0 con 0x7ff5b01f9b90 2023-11-28T12:01:52.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.610+0000 7f24426fc700 1 -- 172.21.15.57:0/2134496774 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f24280682f0 msgr2=0x7f242806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.610+0000 7f24426fc700 1 --2- 172.21.15.57:0/2134496774 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f24280682f0 0x7f242806a7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f242c00ac10 tx=0x7f242c008040 comp rx=0 tx=0).stop 2023-11-28T12:01:52.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.610+0000 7f24426fc700 1 -- 172.21.15.57:0/2134496774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c0b0c50 msgr2=0x7f243c202100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.610+0000 7f24426fc700 1 --2- 172.21.15.57:0/2134496774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c0b0c50 0x7f243c202100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f243000c410 tx=0x7f243000c720 comp rx=0 tx=0).stop 2023-11-28T12:01:52.612 INFO:teuthology.orchestra.run.smithi057.stdout:55834574853 2023-11-28T12:01:52.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.611+0000 7f24426fc700 1 -- 172.21.15.57:0/2134496774 shutdown_connections 2023-11-28T12:01:52.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.611+0000 7f24426fc700 1 --2- 172.21.15.57:0/2134496774 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f24280682f0 0x7f242806a7a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.611+0000 7f24426fc700 1 --2- 172.21.15.57:0/2134496774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f243c1609c0 0x7f243c0704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.611+0000 7f24426fc700 1 --2- 172.21.15.57:0/2134496774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f243c06cdd0 0x7f243c0c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.611+0000 7f24426fc700 1 --2- 172.21.15.57:0/2134496774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f243c0b0c50 0x7f243c202100 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.611+0000 7f24426fc700 1 -- 172.21.15.57:0/2134496774 >> 172.21.15.57:0/2134496774 conn(0x7f243c06aed0 msgr2=0x7f243c0715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.614+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff59c069820 msgr2=0x7ff59c06bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.614+0000 7ff5b7bd6700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff59c069820 0x7ff59c06bcd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009470 tx=0x7ff5a0008040 comp rx=0 tx=0).stop 2023-11-28T12:01:52.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.614+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b01f9b90 msgr2=0x7ff5b01fa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.614+0000 7ff5b7bd6700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b01f9b90 0x7ff5b01fa000 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800fe40 tx=0x7ff5a800d430 comp rx=0 tx=0).stop 2023-11-28T12:01:52.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.615+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/217585561 shutdown_connections 2023-11-28T12:01:52.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.615+0000 7ff5b7bd6700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff59c069820 0x7ff59c06bcd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.615+0000 7ff5b7bd6700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5b01f63a0 0x7ff5b01f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.615+0000 7ff5b7bd6700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5b01f5fd0 0x7ff5b00b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.615+0000 7ff5b7bd6700 1 --2- 172.21.15.57:0/217585561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5b01f9b90 0x7ff5b01fa000 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.615+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/217585561 >> 172.21.15.57:0/217585561 conn(0x7ff5b00cb540 msgr2=0x7ff5b00c5160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.615+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/217585561 shutdown_connections 2023-11-28T12:01:52.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.615+0000 7ff5b7bd6700 1 -- 172.21.15.57:0/217585561 wait complete. 2023-11-28T12:01:52.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.619+0000 7f24426fc700 1 -- 172.21.15.57:0/2134496774 shutdown_connections 2023-11-28T12:01:52.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.619+0000 7f24426fc700 1 -- 172.21.15.57:0/2134496774 wait complete. 2023-11-28T12:01:52.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-11-28T12:01:52.623 DEBUG:teuthology.parallel:result is None 2023-11-28T12:01:52.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-11-28T12:01:52.628 DEBUG:teuthology.parallel:result is None 2023-11-28T12:01:52.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.629+0000 7f1951352700 1 -- 172.21.15.57:0/3226779621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f194c06e620 con 0x7f194c0b2a00 2023-11-28T12:01:52.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.630+0000 7fc4993d0700 1 -- 172.21.15.57:0/2946741241 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc478005870 con 0x7fc4941609c0 2023-11-28T12:01:52.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.630+0000 7f19337fe700 1 -- 172.21.15.57:0/3226779621 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1944001cc0 con 0x7f194c0b2a00 2023-11-28T12:01:52.631 INFO:teuthology.orchestra.run.smithi057.stdout:55834574853 2023-11-28T12:01:52.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.630+0000 7fc48b7fe700 1 -- 172.21.15.57:0/2946741241 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc48c019070 con 0x7fc4941609c0 2023-11-28T12:01:52.632 INFO:teuthology.orchestra.run.smithi057.stdout:55834574852 2023-11-28T12:01:52.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.633+0000 7fc4993d0700 1 -- 172.21.15.57:0/2946741241 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc474068340 msgr2=0x7fc47406a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.633+0000 7fc4993d0700 1 --2- 172.21.15.57:0/2946741241 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc474068340 0x7fc47406a7f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009c40 tx=0x7fc47c010000 comp rx=0 tx=0).stop 2023-11-28T12:01:52.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.633+0000 7fc4993d0700 1 -- 172.21.15.57:0/2946741241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4941609c0 msgr2=0x7fc4940b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.633+0000 7fc4993d0700 1 --2- 172.21.15.57:0/2946741241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4941609c0 0x7fc4940b1ae0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00b990 tx=0x7fc48c00bd50 comp rx=0 tx=0).stop 2023-11-28T12:01:52.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.633+0000 7fc4993d0700 1 -- 172.21.15.57:0/2946741241 shutdown_connections 2023-11-28T12:01:52.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.633+0000 7fc4993d0700 1 --2- 172.21.15.57:0/2946741241 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc474068340 0x7fc47406a7f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.633+0000 7fc4993d0700 1 --2- 172.21.15.57:0/2946741241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4941609c0 0x7fc4940b1ae0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.633+0000 7fc4993d0700 1 --2- 172.21.15.57:0/2946741241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc49406cdd0 0x7fc4941f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.633+0000 7fc4993d0700 1 --2- 172.21.15.57:0/2946741241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4940b0c50 0x7fc494159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.633+0000 7fc4993d0700 1 -- 172.21.15.57:0/2946741241 >> 172.21.15.57:0/2946741241 conn(0x7fc49406aed0 msgr2=0x7fc4940ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.633+0000 7fc4993d0700 1 -- 172.21.15.57:0/2946741241 shutdown_connections 2023-11-28T12:01:52.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.633+0000 7fc4993d0700 1 -- 172.21.15.57:0/2946741241 wait complete. 2023-11-28T12:01:52.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.634+0000 7f1951352700 1 -- 172.21.15.57:0/3226779621 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f19380683c0 msgr2=0x7f193806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.634+0000 7f1951352700 1 --2- 172.21.15.57:0/3226779621 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f19380683c0 0x7f193806a870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f194c06e5b0 tx=0x7f193c001460 comp rx=0 tx=0).stop 2023-11-28T12:01:52.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.634+0000 7f1951352700 1 -- 172.21.15.57:0/3226779621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f194c0b2a00 msgr2=0x7f194c0b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.634+0000 7f1951352700 1 --2- 172.21.15.57:0/3226779621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f194c0b2a00 0x7f194c0b06d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f194c072090 tx=0x7f194400a060 comp rx=0 tx=0).stop 2023-11-28T12:01:52.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.634+0000 7f1951352700 1 -- 172.21.15.57:0/3226779621 shutdown_connections 2023-11-28T12:01:52.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.634+0000 7f1951352700 1 --2- 172.21.15.57:0/3226779621 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f19380683c0 0x7f193806a870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.634+0000 7f1951352700 1 --2- 172.21.15.57:0/3226779621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f194c0b2a00 0x7f194c0b06d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.634+0000 7f1951352700 1 --2- 172.21.15.57:0/3226779621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1934004850 0x7f194c13fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.634+0000 7f1951352700 1 --2- 172.21.15.57:0/3226779621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f194c0b5660 0x7f194c0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.634+0000 7f1951352700 1 -- 172.21.15.57:0/3226779621 >> 172.21.15.57:0/3226779621 conn(0x7f194c06c010 msgr2=0x7f194c158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.634+0000 7f1951352700 1 -- 172.21.15.57:0/3226779621 shutdown_connections 2023-11-28T12:01:52.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.634+0000 7f1951352700 1 -- 172.21.15.57:0/3226779621 wait complete. 2023-11-28T12:01:52.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2023-11-28T12:01:52.641 DEBUG:teuthology.parallel:result is None 2023-11-28T12:01:52.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.642+0000 7fc385dcc700 1 -- 172.21.15.57:0/245180373 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc3800743f0 con 0x7fc3800adf30 2023-11-28T12:01:52.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.643+0000 7fc367fff700 1 -- 172.21.15.57:0/245180373 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc378021090 con 0x7fc3800adf30 2023-11-28T12:01:52.644 INFO:teuthology.orchestra.run.smithi057.stdout:55834574853 2023-11-28T12:01:52.647 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2023-11-28T12:01:52.649 DEBUG:teuthology.parallel:result is None 2023-11-28T12:01:52.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.645+0000 7fc385dcc700 1 -- 172.21.15.57:0/245180373 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc36c068340 msgr2=0x7fc36c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.645+0000 7fc385dcc700 1 --2- 172.21.15.57:0/245180373 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc36c068340 0x7fc36c06a7f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc374006fd0 tx=0x7fc374008040 comp rx=0 tx=0).stop 2023-11-28T12:01:52.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.645+0000 7fc385dcc700 1 -- 172.21.15.57:0/245180373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800adf30 msgr2=0x7fc38015c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:52.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.645+0000 7fc385dcc700 1 --2- 172.21.15.57:0/245180373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800adf30 0x7fc38015c480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc378017040 tx=0x7fc378004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.646+0000 7fc385dcc700 1 -- 172.21.15.57:0/245180373 shutdown_connections 2023-11-28T12:01:52.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.646+0000 7fc385dcc700 1 --2- 172.21.15.57:0/245180373 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc36c068340 0x7fc36c06a7f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.646+0000 7fc385dcc700 1 --2- 172.21.15.57:0/245180373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3800c9b00 0x7fc380154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.646+0000 7fc385dcc700 1 --2- 172.21.15.57:0/245180373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3800adf30 0x7fc38015c480 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.646+0000 7fc385dcc700 1 --2- 172.21.15.57:0/245180373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3800b0540 0x7fc3800ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:52.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.646+0000 7fc385dcc700 1 -- 172.21.15.57:0/245180373 >> 172.21.15.57:0/245180373 conn(0x7fc3800cb540 msgr2=0x7fc3800cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:52.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.646+0000 7fc385dcc700 1 -- 172.21.15.57:0/245180373 shutdown_connections 2023-11-28T12:01:52.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:52.646+0000 7fc385dcc700 1 -- 172.21.15.57:0/245180373 wait complete. 2023-11-28T12:01:52.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2023-11-28T12:01:52.656 DEBUG:teuthology.parallel:result is None 2023-11-28T12:01:52.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2023-11-28T12:01:52.656 DEBUG:teuthology.parallel:result is None 2023-11-28T12:01:52.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2023-11-28T12:01:52.661 DEBUG:teuthology.parallel:result is None 2023-11-28T12:01:53.579 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T12:01:53.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.723+0000 7fd0d5336700 1 Processor -- start 2023-11-28T12:01:53.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.723+0000 7fd0d5336700 1 -- start start 2023-11-28T12:01:53.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.724+0000 7fd0d5336700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0d015c8c0 0x7fd0d0160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:53.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.724+0000 7fd0d5336700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd0d00b5dc0 0x7fd0d006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:53.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.724+0000 7fd0d5336700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d00d2d30 con 0x7fd0d006b050 2023-11-28T12:01:53.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.724+0000 7fd0d5336700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d006b4d0 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.724+0000 7fd0d5336700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00d0b40 con 0x7fd0d015c8c0 2023-11-28T12:01:53.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.724+0000 7fd0ce7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd0d00b5dc0 0x7fd0d006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.724+0000 7fd0ceffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0d015c8c0 0x7fd0d0160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.724+0000 7fd0cf7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd0d006b050 0x7fd0d00d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:44048/0 (socket says 172.21.15.57:44048) 2023-11-28T12:01:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.724+0000 7fd0cf7fe700 1 -- 172.21.15.57:0/3810445254 learned_addr learned my addr 172.21.15.57:0/3810445254 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.725+0000 7fd0cdffb700 1 -- 172.21.15.57:0/3810445254 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1070464800 0 0) 0x7fd0c8001490 con 0x7fd0d006b050 2023-11-28T12:01:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.725+0000 7fd0cdffb700 1 -- 172.21.15.57:0/3810445254 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0bc0035b0 con 0x7fd0d006b050 2023-11-28T12:01:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.725+0000 7fd0ceffd700 1 -- 172.21.15.57:0/3810445254 >> v2:172.21.15.181:3300/0 conn(0x7fd0d00b5dc0 msgr2=0x7fd0d006ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.725+0000 7fd0ceffd700 1 --2- 172.21.15.57:0/3810445254 >> v2:172.21.15.181:3300/0 conn(0x7fd0d00b5dc0 0x7fd0d006ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.725+0000 7fd0ceffd700 1 -- 172.21.15.57:0/3810445254 >> v1:172.21.15.181:6789/0 conn(0x7fd0d006b050 legacy=0x7fd0d00d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.725+0000 7fd0ceffd700 1 -- 172.21.15.57:0/3810445254 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d01611f0 con 0x7fd0d015c8c0 2023-11-28T12:01:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.725+0000 7fd0ce7fc700 1 --2- 172.21.15.57:0/3810445254 >> v2:172.21.15.181:3300/0 conn(0x7fd0d00b5dc0 0x7fd0d006ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.725+0000 7fd0ceffd700 1 --2- 172.21.15.57:0/3810445254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0d015c8c0 0x7fd0d0160cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8009840 tx=0x7fd0b8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e8ba070c7779b3e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.725+0000 7fd0cdffb700 1 -- 172.21.15.57:0/3810445254 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8011830 con 0x7fd0d015c8c0 2023-11-28T12:01:53.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.725+0000 7fd0cdffb700 1 -- 172.21.15.57:0/3810445254 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b8011990 con 0x7fd0d015c8c0 2023-11-28T12:01:53.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.725+0000 7fd0cdffb700 1 -- 172.21.15.57:0/3810445254 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8011d20 con 0x7fd0d015c8c0 2023-11-28T12:01:53.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.726+0000 7fd0d5336700 1 -- 172.21.15.57:0/3810445254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0d015c8c0 msgr2=0x7fd0d0160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:53.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.726+0000 7fd0d5336700 1 --2- 172.21.15.57:0/3810445254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0d015c8c0 0x7fd0d0160cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8009840 tx=0x7fd0b8009b50 comp rx=0 tx=0).stop 2023-11-28T12:01:53.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.726+0000 7fd0d5336700 1 -- 172.21.15.57:0/3810445254 shutdown_connections 2023-11-28T12:01:53.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.726+0000 7fd0d5336700 1 --2- 172.21.15.57:0/3810445254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0d015c8c0 0x7fd0d0160cb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:53.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.726+0000 7fd0d5336700 1 --2- 172.21.15.57:0/3810445254 >> v2:172.21.15.181:3300/0 conn(0x7fd0d00b5dc0 0x7fd0d006ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:53.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.726+0000 7fd0d5336700 1 -- 172.21.15.57:0/3810445254 >> 172.21.15.57:0/3810445254 conn(0x7fd0d006c520 msgr2=0x7fd0d006c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:53.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.726+0000 7fd0d5336700 1 -- 172.21.15.57:0/3810445254 shutdown_connections 2023-11-28T12:01:53.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.726+0000 7fd0d5336700 1 -- 172.21.15.57:0/3810445254 wait complete. 2023-11-28T12:01:53.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.727+0000 7fd0d5336700 1 Processor -- start 2023-11-28T12:01:53.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.727+0000 7fd0d5336700 1 -- start start 2023-11-28T12:01:53.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0d5336700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0d006b050 0x7fd0d014d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:53.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0d5336700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0d00b5dc0 0x7fd0d014df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:53.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0d5336700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0d015c8c0 0x7fd0d015cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:53.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0d5336700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d014f3f0 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0d5336700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d014d9f0 con 0x7fd0d015c8c0 2023-11-28T12:01:53.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0d5336700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d014e7e0 con 0x7fd0d006b050 2023-11-28T12:01:53.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0ce7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0d00b5dc0 0x7fd0d014df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:53.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0ce7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0d00b5dc0 0x7fd0d014df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48212/0 (socket says 172.21.15.57:48212) 2023-11-28T12:01:53.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0ce7fc700 1 -- 172.21.15.57:0/1788389886 learned_addr learned my addr 172.21.15.57:0/1788389886 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:53.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0cf7fe700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0d015c8c0 0x7fd0d015cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:53.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0ceffd700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0d006b050 0x7fd0d014d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:53.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0ce7fc700 1 -- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0d006b050 msgr2=0x7fd0d014d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:53.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0ce7fc700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0d006b050 0x7fd0d014d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:53.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0ce7fc700 1 -- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0d015c8c0 msgr2=0x7fd0d015cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:53.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0ce7fc700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0d015c8c0 0x7fd0d015cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:53.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.728+0000 7fd0ce7fc700 1 -- 172.21.15.57:0/1788389886 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c00094a0 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.729+0000 7fd0ce7fc700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0d00b5dc0 0x7fd0d014df90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000ec10 tx=0x7fd0c000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:53.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.729+0000 7fd0c77fe700 1 -- 172.21.15.57:0/1788389886 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000c850 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.729+0000 7fd0c77fe700 1 -- 172.21.15.57:0/1788389886 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c000c9b0 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.729+0000 7fd0d5336700 1 -- 172.21.15.57:0/1788389886 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b80094a0 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.729+0000 7fd0c77fe700 1 -- 172.21.15.57:0/1788389886 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000cd20 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.729+0000 7fd0d5336700 1 -- 172.21.15.57:0/1788389886 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d00cd760 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.730+0000 7fd0d5336700 1 -- 172.21.15.57:0/1788389886 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d00d2a50 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.730+0000 7fd0c77fe700 1 -- 172.21.15.57:0/1788389886 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd0c0019030 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.731+0000 7fd0c77fe700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0b00682f0 0x7fd0b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:53.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.731+0000 7fd0c77fe700 1 -- 172.21.15.57:0/1788389886 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd0c0085ba0 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.731+0000 7fd0ceffd700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0b00682f0 0x7fd0b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:53.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.731+0000 7fd0ceffd700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0b00682f0 0x7fd0b006a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8000c00 tx=0x7fd0b8008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:53.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.735+0000 7fd0c77fe700 1 -- 172.21.15.57:0/1788389886 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd0c0052400 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.938+0000 7fd0d5336700 1 -- 172.21.15.57:0/1788389886 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd0d0067d70 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.939+0000 7fd0c77fe700 1 -- 172.21.15.57:0/1788389886 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd0c0017090 con 0x7fd0d00b5dc0 2023-11-28T12:01:53.940 INFO:teuthology.orchestra.run.smithi057.stdout:55834574855 2023-11-28T12:01:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.941+0000 7fd0d5336700 1 -- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0b00682f0 msgr2=0x7fd0b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.941+0000 7fd0d5336700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0b00682f0 0x7fd0b006a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8000c00 tx=0x7fd0b8008030 comp rx=0 tx=0).stop 2023-11-28T12:01:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.941+0000 7fd0d5336700 1 -- 172.21.15.57:0/1788389886 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0d00b5dc0 msgr2=0x7fd0d014df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.941+0000 7fd0d5336700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0d00b5dc0 0x7fd0d014df90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000ec10 tx=0x7fd0c000ef20 comp rx=0 tx=0).stop 2023-11-28T12:01:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.941+0000 7fd0d5336700 1 -- 172.21.15.57:0/1788389886 shutdown_connections 2023-11-28T12:01:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.941+0000 7fd0d5336700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0b00682f0 0x7fd0b006a7a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.941+0000 7fd0d5336700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0d015c8c0 0x7fd0d015cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.941+0000 7fd0d5336700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0d006b050 0x7fd0d014d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.941+0000 7fd0d5336700 1 --2- 172.21.15.57:0/1788389886 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0d00b5dc0 0x7fd0d014df90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.941+0000 7fd0d5336700 1 -- 172.21.15.57:0/1788389886 >> 172.21.15.57:0/1788389886 conn(0x7fd0d006c520 msgr2=0x7fd0d00cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.941+0000 7fd0d5336700 1 -- 172.21.15.57:0/1788389886 shutdown_connections 2023-11-28T12:01:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:53.941+0000 7fd0d5336700 1 -- 172.21.15.57:0/1788389886 wait complete. 2023-11-28T12:01:53.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-11-28T12:01:53.956 DEBUG:teuthology.parallel:result is None 2023-11-28T12:01:53.956 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T12:01:53.956 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:01:54.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.089+0000 7fa22bfd0700 1 Processor -- start 2023-11-28T12:01:54.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.089+0000 7fa22bfd0700 1 -- start start 2023-11-28T12:01:54.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.090+0000 7fa22bfd0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa22406e400 0x7fa22406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.090+0000 7fa22bfd0700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22415af30 con 0x7fa22415e010 2023-11-28T12:01:54.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.090+0000 7fa22bfd0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22415ef40 con 0x7fa2240d15f0 2023-11-28T12:01:54.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.090+0000 7fa22bfd0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22415a9d0 con 0x7fa22406e400 2023-11-28T12:01:54.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.091+0000 7fa22a56d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa22415e010 0x7fa2240ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37088/0 (socket says 172.21.15.57:37088) 2023-11-28T12:01:54.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.091+0000 7fa22a56d700 1 -- 172.21.15.57:0/2949769445 learned_addr learned my addr 172.21.15.57:0/2949769445 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:54.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.091+0000 7fa229d6c700 1 --2- 172.21.15.57:0/2949769445 >> v2:172.21.15.181:3300/0 conn(0x7fa22406e400 0x7fa22406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.091+0000 7fa228d6a700 1 -- 172.21.15.57:0/2949769445 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596146483 0 0) 0x7fa22415af30 con 0x7fa22415e010 2023-11-28T12:01:54.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.091+0000 7fa228d6a700 1 -- 172.21.15.57:0/2949769445 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2100035b0 con 0x7fa22415e010 2023-11-28T12:01:54.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.091+0000 7fa229d6c700 1 -- 172.21.15.57:0/2949769445 >> v1:172.21.15.181:6789/0 conn(0x7fa2240d15f0 legacy=0x7fa22415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.091+0000 7fa229d6c700 1 -- 172.21.15.57:0/2949769445 >> v1:172.21.15.57:6789/0 conn(0x7fa22415e010 legacy=0x7fa2240ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.091+0000 7fa228d6a700 1 -- 172.21.15.57:0/2949769445 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572434917 0 0) 0x7fa22415ef40 con 0x7fa2240d15f0 2023-11-28T12:01:54.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.091+0000 7fa229d6c700 1 -- 172.21.15.57:0/2949769445 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22406ed10 con 0x7fa22406e400 2023-11-28T12:01:54.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.091+0000 7fa229d6c700 1 --2- 172.21.15.57:0/2949769445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa22406e400 0x7fa22406e7d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa214009840 tx=0x7fa214009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81fb9b56533a39bb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:54.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.091+0000 7fa228d6a700 1 -- 172.21.15.57:0/2949769445 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214011830 con 0x7fa22406e400 2023-11-28T12:01:54.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.091+0000 7fa228d6a700 1 -- 172.21.15.57:0/2949769445 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa214011990 con 0x7fa22406e400 2023-11-28T12:01:54.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.091+0000 7fa228d6a700 1 -- 172.21.15.57:0/2949769445 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214011d20 con 0x7fa22406e400 2023-11-28T12:01:54.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.092+0000 7fa22bfd0700 1 -- 172.21.15.57:0/2949769445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa22406e400 msgr2=0x7fa22406e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.092+0000 7fa22bfd0700 1 --2- 172.21.15.57:0/2949769445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa22406e400 0x7fa22406e7d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa214009840 tx=0x7fa214009b50 comp rx=0 tx=0).stop 2023-11-28T12:01:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.092+0000 7fa22bfd0700 1 -- 172.21.15.57:0/2949769445 shutdown_connections 2023-11-28T12:01:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.092+0000 7fa22bfd0700 1 --2- 172.21.15.57:0/2949769445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa22406e400 0x7fa22406e7d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.092+0000 7fa22bfd0700 1 -- 172.21.15.57:0/2949769445 >> 172.21.15.57:0/2949769445 conn(0x7fa22406a270 msgr2=0x7fa22406a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.092+0000 7fa22bfd0700 1 -- 172.21.15.57:0/2949769445 shutdown_connections 2023-11-28T12:01:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.092+0000 7fa22bfd0700 1 -- 172.21.15.57:0/2949769445 wait complete. 2023-11-28T12:01:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.093+0000 7fa22bfd0700 1 Processor -- start 2023-11-28T12:01:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.094+0000 7fa22bfd0700 1 -- start start 2023-11-28T12:01:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.094+0000 7fa22bfd0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa22406e400 0x7fa22406d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.094+0000 7fa22bfd0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2240d15f0 0x7fa22406b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.094+0000 7fa22bfd0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa22415e010 0x7fa2240cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.094+0000 7fa22bfd0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22406c3e0 con 0x7fa22406e400 2023-11-28T12:01:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.094+0000 7fa22bfd0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2240ca750 con 0x7fa2240d15f0 2023-11-28T12:01:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.094+0000 7fa22bfd0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22406b8c0 con 0x7fa22415e010 2023-11-28T12:01:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.094+0000 7fa229d6c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa22406e400 0x7fa22406d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.094+0000 7fa229d6c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa22406e400 0x7fa22406d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48214/0 (socket says 172.21.15.57:48214) 2023-11-28T12:01:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.094+0000 7fa229d6c700 1 -- 172.21.15.57:0/322000500 learned_addr learned my addr 172.21.15.57:0/322000500 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa22a56d700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa22415e010 0x7fa2240cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa229d6c700 1 -- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa22415e010 msgr2=0x7fa2240cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa22956b700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2240d15f0 0x7fa22406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa229d6c700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa22415e010 0x7fa2240cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa229d6c700 1 -- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2240d15f0 msgr2=0x7fa22406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa229d6c700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2240d15f0 0x7fa22406b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa229d6c700 1 -- 172.21.15.57:0/322000500 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2140094a0 con 0x7fa22406e400 2023-11-28T12:01:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa22a56d700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa22415e010 0x7fa2240cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:01:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa22956b700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2240d15f0 0x7fa22406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:01:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa229d6c700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa22406e400 0x7fa22406d000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa2140090b0 tx=0x7fa214004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa21a7fc700 1 -- 172.21.15.57:0/322000500 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21401a910 con 0x7fa22406e400 2023-11-28T12:01:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa21a7fc700 1 -- 172.21.15.57:0/322000500 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21401aa70 con 0x7fa22406e400 2023-11-28T12:01:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa22bfd0700 1 -- 172.21.15.57:0/322000500 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22415ec60 con 0x7fa22406e400 2023-11-28T12:01:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa21a7fc700 1 -- 172.21.15.57:0/322000500 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21401ade0 con 0x7fa22406e400 2023-11-28T12:01:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.095+0000 7fa22bfd0700 1 -- 172.21.15.57:0/322000500 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22415e980 con 0x7fa22406e400 2023-11-28T12:01:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.096+0000 7fa22bfd0700 1 -- 172.21.15.57:0/322000500 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22415abc0 con 0x7fa22406e400 2023-11-28T12:01:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.097+0000 7fa21a7fc700 1 -- 172.21.15.57:0/322000500 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa21402b030 con 0x7fa22406e400 2023-11-28T12:01:54.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.097+0000 7fa21a7fc700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2080682f0 0x7fa20806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.097+0000 7fa21a7fc700 1 -- 172.21.15.57:0/322000500 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa214086fd0 con 0x7fa22406e400 2023-11-28T12:01:54.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.097+0000 7fa22956b700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2080682f0 0x7fa20806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.098+0000 7fa22956b700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2080682f0 0x7fa20806a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa22415c020 tx=0x7fa220009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:54.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.102+0000 7fa21a7fc700 1 -- 172.21.15.57:0/322000500 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa214053830 con 0x7fa22406e400 2023-11-28T12:01:54.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.297+0000 7fa22bfd0700 1 -- 172.21.15.57:0/322000500 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2240b5e30 con 0x7fa2080682f0 2023-11-28T12:01:54.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.305+0000 7fa21a7fc700 1 -- 172.21.15.57:0/322000500 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286827 (secure 0 0 0) 0x7fa2240b5e30 con 0x7fa2080682f0 2023-11-28T12:01:54.310 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:54.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.308+0000 7fa22bfd0700 1 -- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2080682f0 msgr2=0x7fa20806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.309+0000 7fa22bfd0700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2080682f0 0x7fa20806a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa22415c020 tx=0x7fa220009040 comp rx=0 tx=0).stop 2023-11-28T12:01:54.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.309+0000 7fa22bfd0700 1 -- 172.21.15.57:0/322000500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa22406e400 msgr2=0x7fa22406d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.309+0000 7fa22bfd0700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa22406e400 0x7fa22406d000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa2140090b0 tx=0x7fa214004750 comp rx=0 tx=0).stop 2023-11-28T12:01:54.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.309+0000 7fa22bfd0700 1 -- 172.21.15.57:0/322000500 shutdown_connections 2023-11-28T12:01:54.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.309+0000 7fa22bfd0700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2080682f0 0x7fa20806a7a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.309+0000 7fa22bfd0700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa22415e010 0x7fa2240cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.309+0000 7fa22bfd0700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa22406e400 0x7fa22406d000 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.309+0000 7fa22bfd0700 1 --2- 172.21.15.57:0/322000500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2240d15f0 0x7fa22406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.309+0000 7fa22bfd0700 1 -- 172.21.15.57:0/322000500 >> 172.21.15.57:0/322000500 conn(0x7fa22406a270 msgr2=0x7fa2240b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:54.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.309+0000 7fa22bfd0700 1 -- 172.21.15.57:0/322000500 shutdown_connections 2023-11-28T12:01:54.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.309+0000 7fa22bfd0700 1 -- 172.21.15.57:0/322000500 wait complete. 2023-11-28T12:01:54.313 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:01:54.323 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-28T12:01:53.660658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":65960,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":754908760,"statfs":{"total":773094113280,"available":773026570240,"internally_reserved":0,"allocated":2990080,"data_stored":1821076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.735559"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542856+0000","last_change":"2023-11-28T12:01:43.953143+0000","last_active":"2023-11-28T12:01:46.542856+0000","last_peered":"2023-11-28T12:01:46.542856+0000","last_clean":"2023-11-28T12:01:46.542856+0000","last_became_active":"2023-11-28T12:01:43.952673+0000","last_became_peered":"2023-11-28T12:01:43.952673+0000","last_unstale":"2023-11-28T12:01:46.542856+0000","last_undegraded":"2023-11-28T12:01:46.542856+0000","last_fullsized":"2023-11-28T12:01:46.542856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:18.949383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545399+0000","last_change":"2023-11-28T12:01:43.947886+0000","last_active":"2023-11-28T12:01:46.545399+0000","last_peered":"2023-11-28T12:01:46.545399+0000","last_clean":"2023-11-28T12:01:46.545399+0000","last_became_active":"2023-11-28T12:01:43.947468+0000","last_became_peered":"2023-11-28T12:01:43.947468+0000","last_unstale":"2023-11-28T12:01:46.545399+0000","last_undegraded":"2023-11-28T12:01:46.545399+0000","last_fullsized":"2023-11-28T12:01:46.545399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:41.736417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938403+0000","last_change":"2023-11-28T12:01:43.941069+0000","last_active":"2023-11-28T12:01:44.938403+0000","last_peered":"2023-11-28T12:01:44.938403+0000","last_clean":"2023-11-28T12:01:44.938403+0000","last_became_active":"2023-11-28T12:01:43.940366+0000","last_became_peered":"2023-11-28T12:01:43.940366+0000","last_unstale":"2023-11-28T12:01:44.938403+0000","last_undegraded":"2023-11-28T12:01:44.938403+0000","last_fullsized":"2023-11-28T12:01:44.938403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:40.988886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937690+0000","last_change":"2023-11-28T12:01:43.947292+0000","last_active":"2023-11-28T12:01:44.937690+0000","last_peered":"2023-11-28T12:01:44.937690+0000","last_clean":"2023-11-28T12:01:44.937690+0000","last_became_active":"2023-11-28T12:01:43.943027+0000","last_became_peered":"2023-11-28T12:01:43.943027+0000","last_unstale":"2023-11-28T12:01:44.937690+0000","last_undegraded":"2023-11-28T12:01:44.937690+0000","last_fullsized":"2023-11-28T12:01:44.937690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:19.706888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497843+0000","last_change":"2023-11-28T12:01:43.954299+0000","last_active":"2023-11-28T12:01:46.497843+0000","last_peered":"2023-11-28T12:01:46.497843+0000","last_clean":"2023-11-28T12:01:46.497843+0000","last_became_active":"2023-11-28T12:01:43.953967+0000","last_became_peered":"2023-11-28T12:01:43.953967+0000","last_unstale":"2023-11-28T12:01:46.497843+0000","last_undegraded":"2023-11-28T12:01:46.497843+0000","last_fullsized":"2023-11-28T12:01:46.497843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:35.755196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937402+0000","last_change":"2023-11-28T12:01:43.938748+0000","last_active":"2023-11-28T12:01:44.937402+0000","last_peered":"2023-11-28T12:01:44.937402+0000","last_clean":"2023-11-28T12:01:44.937402+0000","last_became_active":"2023-11-28T12:01:43.938430+0000","last_became_peered":"2023-11-28T12:01:43.938430+0000","last_unstale":"2023-11-28T12:01:44.937402+0000","last_undegraded":"2023-11-28T12:01:44.937402+0000","last_fullsized":"2023-11-28T12:01:44.937402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:49.922116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853429+0000","last_change":"2023-11-28T12:01:43.950997+0000","last_active":"2023-11-28T12:01:45.853429+0000","last_peered":"2023-11-28T12:01:45.853429+0000","last_clean":"2023-11-28T12:01:45.853429+0000","last_became_active":"2023-11-28T12:01:43.950532+0000","last_became_peered":"2023-11-28T12:01:43.950532+0000","last_unstale":"2023-11-28T12:01:45.853429+0000","last_undegraded":"2023-11-28T12:01:45.853429+0000","last_fullsized":"2023-11-28T12:01:45.853429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545106+0000","last_change":"2023-11-28T12:01:43.945419+0000","last_active":"2023-11-28T12:01:46.545106+0000","last_peered":"2023-11-28T12:01:46.545106+0000","last_clean":"2023-11-28T12:01:46.545106+0000","last_became_active":"2023-11-28T12:01:43.942913+0000","last_became_peered":"2023-11-28T12:01:43.942913+0000","last_unstale":"2023-11-28T12:01:46.545106+0000","last_undegraded":"2023-11-28T12:01:46.545106+0000","last_fullsized":"2023-11-28T12:01:46.545106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497879+0000","last_change":"2023-11-28T12:01:43.954900+0000","last_active":"2023-11-28T12:01:46.497879+0000","last_peered":"2023-11-28T12:01:46.497879+0000","last_clean":"2023-11-28T12:01:46.497879+0000","last_became_active":"2023-11-28T12:01:43.954524+0000","last_became_peered":"2023-11-28T12:01:43.954524+0000","last_unstale":"2023-11-28T12:01:46.497879+0000","last_undegraded":"2023-11-28T12:01:46.497879+0000","last_fullsized":"2023-11-28T12:01:46.497879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:10.298310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545527+0000","last_change":"2023-11-28T12:01:43.944809+0000","last_active":"2023-11-28T12:01:46.545527+0000","last_peered":"2023-11-28T12:01:46.545527+0000","last_clean":"2023-11-28T12:01:46.545527+0000","last_became_active":"2023-11-28T12:01:43.943120+0000","last_became_peered":"2023-11-28T12:01:43.943120+0000","last_unstale":"2023-11-28T12:01:46.545527+0000","last_undegraded":"2023-11-28T12:01:46.545527+0000","last_fullsized":"2023-11-28T12:01:46.545527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938327+0000","last_change":"2023-11-28T12:01:43.941446+0000","last_active":"2023-11-28T12:01:44.938327+0000","last_peered":"2023-11-28T12:01:44.938327+0000","last_clean":"2023-11-28T12:01:44.938327+0000","last_became_active":"2023-11-28T12:01:43.940640+0000","last_became_peered":"2023-11-28T12:01:43.940640+0000","last_unstale":"2023-11-28T12:01:44.938327+0000","last_undegraded":"2023-11-28T12:01:44.938327+0000","last_fullsized":"2023-11-28T12:01:44.938327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:41.605113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542711+0000","last_change":"2023-11-28T12:01:43.946959+0000","last_active":"2023-11-28T12:01:46.542711+0000","last_peered":"2023-11-28T12:01:46.542711+0000","last_clean":"2023-11-28T12:01:46.542711+0000","last_became_active":"2023-11-28T12:01:43.946674+0000","last_became_peered":"2023-11-28T12:01:43.946674+0000","last_unstale":"2023-11-28T12:01:46.542711+0000","last_undegraded":"2023-11-28T12:01:46.542711+0000","last_fullsized":"2023-11-28T12:01:46.542711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:18.910451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542620+0000","last_change":"2023-11-28T12:01:43.952292+0000","last_active":"2023-11-28T12:01:46.542620+0000","last_peered":"2023-11-28T12:01:46.542620+0000","last_clean":"2023-11-28T12:01:46.542620+0000","last_became_active":"2023-11-28T12:01:43.951948+0000","last_became_peered":"2023-11-28T12:01:43.951948+0000","last_unstale":"2023-11-28T12:01:46.542620+0000","last_undegraded":"2023-11-28T12:01:46.542620+0000","last_fullsized":"2023-11-28T12:01:46.542620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:30.672727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545157+0000","last_change":"2023-11-28T12:01:43.945700+0000","last_active":"2023-11-28T12:01:46.545157+0000","last_peered":"2023-11-28T12:01:46.545157+0000","last_clean":"2023-11-28T12:01:46.545157+0000","last_became_active":"2023-11-28T12:01:43.943097+0000","last_became_peered":"2023-11-28T12:01:43.943097+0000","last_unstale":"2023-11-28T12:01:46.545157+0000","last_undegraded":"2023-11-28T12:01:46.545157+0000","last_fullsized":"2023-11-28T12:01:46.545157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:36.019691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853575+0000","last_change":"2023-11-28T12:01:43.951123+0000","last_active":"2023-11-28T12:01:45.853575+0000","last_peered":"2023-11-28T12:01:45.853575+0000","last_clean":"2023-11-28T12:01:45.853575+0000","last_became_active":"2023-11-28T12:01:43.950867+0000","last_became_peered":"2023-11-28T12:01:43.950867+0000","last_unstale":"2023-11-28T12:01:45.853575+0000","last_undegraded":"2023-11-28T12:01:45.853575+0000","last_fullsized":"2023-11-28T12:01:45.853575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:32.123066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937717+0000","last_change":"2023-11-28T12:01:43.942191+0000","last_active":"2023-11-28T12:01:44.937717+0000","last_peered":"2023-11-28T12:01:44.937717+0000","last_clean":"2023-11-28T12:01:44.937717+0000","last_became_active":"2023-11-28T12:01:43.941576+0000","last_became_peered":"2023-11-28T12:01:43.941576+0000","last_unstale":"2023-11-28T12:01:44.937717+0000","last_undegraded":"2023-11-28T12:01:44.937717+0000","last_fullsized":"2023-11-28T12:01:44.937717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:43.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497887+0000","last_change":"2023-11-28T12:01:43.954787+0000","last_active":"2023-11-28T12:01:46.497887+0000","last_peered":"2023-11-28T12:01:46.497887+0000","last_clean":"2023-11-28T12:01:46.497887+0000","last_became_active":"2023-11-28T12:01:43.954463+0000","last_became_peered":"2023-11-28T12:01:43.954463+0000","last_unstale":"2023-11-28T12:01:46.497887+0000","last_undegraded":"2023-11-28T12:01:46.497887+0000","last_fullsized":"2023-11-28T12:01:46.497887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:23.540287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938451+0000","last_change":"2023-11-28T12:01:43.939593+0000","last_active":"2023-11-28T12:01:44.938451+0000","last_peered":"2023-11-28T12:01:44.938451+0000","last_clean":"2023-11-28T12:01:44.938451+0000","last_became_active":"2023-11-28T12:01:43.937190+0000","last_became_peered":"2023-11-28T12:01:43.937190+0000","last_unstale":"2023-11-28T12:01:44.938451+0000","last_undegraded":"2023-11-28T12:01:44.938451+0000","last_fullsized":"2023-11-28T12:01:44.938451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:16.230349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542842+0000","last_change":"2023-11-28T12:01:43.942910+0000","last_active":"2023-11-28T12:01:46.542842+0000","last_peered":"2023-11-28T12:01:46.542842+0000","last_clean":"2023-11-28T12:01:46.542842+0000","last_became_active":"2023-11-28T12:01:43.942609+0000","last_became_peered":"2023-11-28T12:01:43.942609+0000","last_unstale":"2023-11-28T12:01:46.542842+0000","last_undegraded":"2023-11-28T12:01:46.542842+0000","last_fullsized":"2023-11-28T12:01:46.542842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:37.541651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938215+0000","last_change":"2023-11-28T12:01:43.945798+0000","last_active":"2023-11-28T12:01:44.938215+0000","last_peered":"2023-11-28T12:01:44.938215+0000","last_clean":"2023-11-28T12:01:44.938215+0000","last_became_active":"2023-11-28T12:01:43.945473+0000","last_became_peered":"2023-11-28T12:01:43.945473+0000","last_unstale":"2023-11-28T12:01:44.938215+0000","last_undegraded":"2023-11-28T12:01:44.938215+0000","last_fullsized":"2023-11-28T12:01:44.938215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:29.033626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853662+0000","last_change":"2023-11-28T12:01:43.954303+0000","last_active":"2023-11-28T12:01:45.853662+0000","last_peered":"2023-11-28T12:01:45.853662+0000","last_clean":"2023-11-28T12:01:45.853662+0000","last_became_active":"2023-11-28T12:01:43.954076+0000","last_became_peered":"2023-11-28T12:01:43.954076+0000","last_unstale":"2023-11-28T12:01:45.853662+0000","last_undegraded":"2023-11-28T12:01:45.853662+0000","last_fullsized":"2023-11-28T12:01:45.853662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:41.736417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497578+0000","last_change":"2023-11-28T12:01:43.946172+0000","last_active":"2023-11-28T12:01:46.497578+0000","last_peered":"2023-11-28T12:01:46.497578+0000","last_clean":"2023-11-28T12:01:46.497578+0000","last_became_active":"2023-11-28T12:01:43.945881+0000","last_became_peered":"2023-11-28T12:01:43.945881+0000","last_unstale":"2023-11-28T12:01:46.497578+0000","last_undegraded":"2023-11-28T12:01:46.497578+0000","last_fullsized":"2023-11-28T12:01:46.497578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:43.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938826+0000","last_change":"2023-11-28T12:01:43.951156+0000","last_active":"2023-11-28T12:01:44.938826+0000","last_peered":"2023-11-28T12:01:44.938826+0000","last_clean":"2023-11-28T12:01:44.938826+0000","last_became_active":"2023-11-28T12:01:43.950873+0000","last_became_peered":"2023-11-28T12:01:43.950873+0000","last_unstale":"2023-11-28T12:01:44.938826+0000","last_undegraded":"2023-11-28T12:01:44.938826+0000","last_fullsized":"2023-11-28T12:01:44.938826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:32.718654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853513+0000","last_change":"2023-11-28T12:01:43.953938+0000","last_active":"2023-11-28T12:01:45.853513+0000","last_peered":"2023-11-28T12:01:45.853513+0000","last_clean":"2023-11-28T12:01:45.853513+0000","last_became_active":"2023-11-28T12:01:43.953649+0000","last_became_peered":"2023-11-28T12:01:43.953649+0000","last_unstale":"2023-11-28T12:01:45.853513+0000","last_undegraded":"2023-11-28T12:01:45.853513+0000","last_fullsized":"2023-11-28T12:01:45.853513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923145+0000","last_change":"2023-11-28T12:01:43.941856+0000","last_active":"2023-11-28T12:01:45.923145+0000","last_peered":"2023-11-28T12:01:45.923145+0000","last_clean":"2023-11-28T12:01:45.923145+0000","last_became_active":"2023-11-28T12:01:43.941555+0000","last_became_peered":"2023-11-28T12:01:43.941555+0000","last_unstale":"2023-11-28T12:01:45.923145+0000","last_undegraded":"2023-11-28T12:01:45.923145+0000","last_fullsized":"2023-11-28T12:01:45.923145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:01.126989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923247+0000","last_change":"2023-11-28T12:01:43.940084+0000","last_active":"2023-11-28T12:01:45.923247+0000","last_peered":"2023-11-28T12:01:45.923247+0000","last_clean":"2023-11-28T12:01:45.923247+0000","last_became_active":"2023-11-28T12:01:43.939856+0000","last_became_peered":"2023-11-28T12:01:43.939856+0000","last_unstale":"2023-11-28T12:01:45.923247+0000","last_undegraded":"2023-11-28T12:01:45.923247+0000","last_fullsized":"2023-11-28T12:01:45.923247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:50.516316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545353+0000","last_change":"2023-11-28T12:01:43.943624+0000","last_active":"2023-11-28T12:01:46.545353+0000","last_peered":"2023-11-28T12:01:46.545353+0000","last_clean":"2023-11-28T12:01:46.545353+0000","last_became_active":"2023-11-28T12:01:43.943313+0000","last_became_peered":"2023-11-28T12:01:43.943313+0000","last_unstale":"2023-11-28T12:01:46.545353+0000","last_undegraded":"2023-11-28T12:01:46.545353+0000","last_fullsized":"2023-11-28T12:01:46.545353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:10.298310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938040+0000","last_change":"2023-11-28T12:01:43.500890+0000","last_active":"2023-11-28T12:01:44.938040+0000","last_peered":"2023-11-28T12:01:44.938040+0000","last_clean":"2023-11-28T12:01:44.938040+0000","last_became_active":"2023-11-28T12:01:43.500621+0000","last_became_peered":"2023-11-28T12:01:43.500621+0000","last_unstale":"2023-11-28T12:01:44.938040+0000","last_undegraded":"2023-11-28T12:01:44.938040+0000","last_fullsized":"2023-11-28T12:01:44.938040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939156+0000","last_change":"2023-11-28T12:01:43.951003+0000","last_active":"2023-11-28T12:01:44.939156+0000","last_peered":"2023-11-28T12:01:44.939156+0000","last_clean":"2023-11-28T12:01:44.939156+0000","last_became_active":"2023-11-28T12:01:43.950717+0000","last_became_peered":"2023-11-28T12:01:43.950717+0000","last_unstale":"2023-11-28T12:01:44.939156+0000","last_undegraded":"2023-11-28T12:01:44.939156+0000","last_fullsized":"2023-11-28T12:01:44.939156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:32.123066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922884+0000","last_change":"2023-11-28T12:01:43.500086+0000","last_active":"2023-11-28T12:01:45.922884+0000","last_peered":"2023-11-28T12:01:45.922884+0000","last_clean":"2023-11-28T12:01:45.922884+0000","last_became_active":"2023-11-28T12:01:43.499317+0000","last_became_peered":"2023-11-28T12:01:43.499317+0000","last_unstale":"2023-11-28T12:01:45.922884+0000","last_undegraded":"2023-11-28T12:01:45.922884+0000","last_fullsized":"2023-11-28T12:01:45.922884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.498026+0000","last_change":"2023-11-28T12:01:43.952329+0000","last_active":"2023-11-28T12:01:46.498026+0000","last_peered":"2023-11-28T12:01:46.498026+0000","last_clean":"2023-11-28T12:01:46.498026+0000","last_became_active":"2023-11-28T12:01:43.952003+0000","last_became_peered":"2023-11-28T12:01:43.952003+0000","last_unstale":"2023-11-28T12:01:46.498026+0000","last_undegraded":"2023-11-28T12:01:46.498026+0000","last_fullsized":"2023-11-28T12:01:46.498026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938921+0000","last_change":"2023-11-28T12:01:43.499958+0000","last_active":"2023-11-28T12:01:44.938921+0000","last_peered":"2023-11-28T12:01:44.938921+0000","last_clean":"2023-11-28T12:01:44.938921+0000","last_became_active":"2023-11-28T12:01:43.499623+0000","last_became_peered":"2023-11-28T12:01:43.499623+0000","last_unstale":"2023-11-28T12:01:44.938921+0000","last_undegraded":"2023-11-28T12:01:44.938921+0000","last_fullsized":"2023-11-28T12:01:44.938921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:46.276522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937707+0000","last_change":"2023-11-28T12:01:43.945390+0000","last_active":"2023-11-28T12:01:44.937707+0000","last_peered":"2023-11-28T12:01:44.937707+0000","last_clean":"2023-11-28T12:01:44.937707+0000","last_became_active":"2023-11-28T12:01:43.945125+0000","last_became_peered":"2023-11-28T12:01:43.945125+0000","last_unstale":"2023-11-28T12:01:44.937707+0000","last_undegraded":"2023-11-28T12:01:44.937707+0000","last_fullsized":"2023-11-28T12:01:44.937707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:01.126989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497989+0000","last_change":"2023-11-28T12:01:43.497913+0000","last_active":"2023-11-28T12:01:46.497989+0000","last_peered":"2023-11-28T12:01:46.497989+0000","last_clean":"2023-11-28T12:01:46.497989+0000","last_became_active":"2023-11-28T12:01:43.497597+0000","last_became_peered":"2023-11-28T12:01:43.497597+0000","last_unstale":"2023-11-28T12:01:46.497989+0000","last_undegraded":"2023-11-28T12:01:46.497989+0000","last_fullsized":"2023-11-28T12:01:46.497989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938856+0000","last_change":"2023-11-28T12:01:43.938776+0000","last_active":"2023-11-28T12:01:44.938856+0000","last_peered":"2023-11-28T12:01:44.938856+0000","last_clean":"2023-11-28T12:01:44.938856+0000","last_became_active":"2023-11-28T12:01:43.938258+0000","last_became_peered":"2023-11-28T12:01:43.938258+0000","last_unstale":"2023-11-28T12:01:44.938856+0000","last_undegraded":"2023-11-28T12:01:44.938856+0000","last_fullsized":"2023-11-28T12:01:44.938856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:10.443506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545384+0000","last_change":"2023-11-28T12:01:43.494977+0000","last_active":"2023-11-28T12:01:46.545384+0000","last_peered":"2023-11-28T12:01:46.545384+0000","last_clean":"2023-11-28T12:01:46.545384+0000","last_became_active":"2023-11-28T12:01:43.494659+0000","last_became_peered":"2023-11-28T12:01:43.494659+0000","last_unstale":"2023-11-28T12:01:46.545384+0000","last_undegraded":"2023-11-28T12:01:46.545384+0000","last_fullsized":"2023-11-28T12:01:46.545384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:29.667067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938938+0000","last_change":"2023-11-28T12:01:43.949219+0000","last_active":"2023-11-28T12:01:44.938938+0000","last_peered":"2023-11-28T12:01:44.938938+0000","last_clean":"2023-11-28T12:01:44.938938+0000","last_became_active":"2023-11-28T12:01:43.948599+0000","last_became_peered":"2023-11-28T12:01:43.948599+0000","last_unstale":"2023-11-28T12:01:44.938938+0000","last_undegraded":"2023-11-28T12:01:44.938938+0000","last_fullsized":"2023-11-28T12:01:44.938938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:43.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853768+0000","last_change":"2023-11-28T12:01:43.504068+0000","last_active":"2023-11-28T12:01:45.853768+0000","last_peered":"2023-11-28T12:01:45.853768+0000","last_clean":"2023-11-28T12:01:45.853768+0000","last_became_active":"2023-11-28T12:01:43.503819+0000","last_became_peered":"2023-11-28T12:01:43.503819+0000","last_unstale":"2023-11-28T12:01:45.853768+0000","last_undegraded":"2023-11-28T12:01:45.853768+0000","last_fullsized":"2023-11-28T12:01:45.853768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:49.510656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922795+0000","last_change":"2023-11-28T12:01:43.941737+0000","last_active":"2023-11-28T12:01:45.922795+0000","last_peered":"2023-11-28T12:01:45.922795+0000","last_clean":"2023-11-28T12:01:45.922795+0000","last_became_active":"2023-11-28T12:01:43.941456+0000","last_became_peered":"2023-11-28T12:01:43.941456+0000","last_unstale":"2023-11-28T12:01:45.922795+0000","last_undegraded":"2023-11-28T12:01:45.922795+0000","last_fullsized":"2023-11-28T12:01:45.922795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:36.019691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938329+0000","last_change":"2023-11-28T12:01:43.492482+0000","last_active":"2023-11-28T12:01:44.938329+0000","last_peered":"2023-11-28T12:01:44.938329+0000","last_clean":"2023-11-28T12:01:44.938329+0000","last_became_active":"2023-11-28T12:01:43.492180+0000","last_became_peered":"2023-11-28T12:01:43.492180+0000","last_unstale":"2023-11-28T12:01:44.938329+0000","last_undegraded":"2023-11-28T12:01:44.938329+0000","last_fullsized":"2023-11-28T12:01:44.938329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938187+0000","last_change":"2023-11-28T12:01:43.944780+0000","last_active":"2023-11-28T12:01:44.938187+0000","last_peered":"2023-11-28T12:01:44.938187+0000","last_clean":"2023-11-28T12:01:44.938187+0000","last_became_active":"2023-11-28T12:01:43.944467+0000","last_became_peered":"2023-11-28T12:01:43.944467+0000","last_unstale":"2023-11-28T12:01:44.938187+0000","last_undegraded":"2023-11-28T12:01:44.938187+0000","last_fullsized":"2023-11-28T12:01:44.938187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:57.245235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853599+0000","last_change":"2023-11-28T12:01:43.499400+0000","last_active":"2023-11-28T12:01:45.853599+0000","last_peered":"2023-11-28T12:01:45.853599+0000","last_clean":"2023-11-28T12:01:45.853599+0000","last_became_active":"2023-11-28T12:01:43.499116+0000","last_became_peered":"2023-11-28T12:01:43.499116+0000","last_unstale":"2023-11-28T12:01:45.853599+0000","last_undegraded":"2023-11-28T12:01:45.853599+0000","last_fullsized":"2023-11-28T12:01:45.853599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:00.731296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939194+0000","last_change":"2023-11-28T12:01:43.954953+0000","last_active":"2023-11-28T12:01:44.939194+0000","last_peered":"2023-11-28T12:01:44.939194+0000","last_clean":"2023-11-28T12:01:44.939194+0000","last_became_active":"2023-11-28T12:01:43.952565+0000","last_became_peered":"2023-11-28T12:01:43.952565+0000","last_unstale":"2023-11-28T12:01:44.939194+0000","last_undegraded":"2023-11-28T12:01:44.939194+0000","last_fullsized":"2023-11-28T12:01:44.939194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:29.033626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497689+0000","last_change":"2023-11-28T12:01:43.496294+0000","last_active":"2023-11-28T12:01:46.497689+0000","last_peered":"2023-11-28T12:01:46.497689+0000","last_clean":"2023-11-28T12:01:46.497689+0000","last_became_active":"2023-11-28T12:01:43.493574+0000","last_became_peered":"2023-11-28T12:01:43.493574+0000","last_unstale":"2023-11-28T12:01:46.497689+0000","last_undegraded":"2023-11-28T12:01:46.497689+0000","last_fullsized":"2023-11-28T12:01:46.497689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545401+0000","last_change":"2023-11-28T12:01:43.942727+0000","last_active":"2023-11-28T12:01:46.545401+0000","last_peered":"2023-11-28T12:01:46.545401+0000","last_clean":"2023-11-28T12:01:46.545401+0000","last_became_active":"2023-11-28T12:01:43.942041+0000","last_became_peered":"2023-11-28T12:01:43.942041+0000","last_unstale":"2023-11-28T12:01:46.545401+0000","last_undegraded":"2023-11-28T12:01:46.545401+0000","last_fullsized":"2023-11-28T12:01:46.545401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:23.540287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853448+0000","last_change":"2023-11-28T12:01:43.499858+0000","last_active":"2023-11-28T12:01:45.853448+0000","last_peered":"2023-11-28T12:01:45.853448+0000","last_clean":"2023-11-28T12:01:45.853448+0000","last_became_active":"2023-11-28T12:01:43.499006+0000","last_became_peered":"2023-11-28T12:01:43.499006+0000","last_unstale":"2023-11-28T12:01:45.853448+0000","last_undegraded":"2023-11-28T12:01:45.853448+0000","last_fullsized":"2023-11-28T12:01:45.853448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:41:02.952529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497880+0000","last_change":"2023-11-28T12:01:43.950190+0000","last_active":"2023-11-28T12:01:46.497880+0000","last_peered":"2023-11-28T12:01:46.497880+0000","last_clean":"2023-11-28T12:01:46.497880+0000","last_became_active":"2023-11-28T12:01:43.949923+0000","last_became_peered":"2023-11-28T12:01:43.949923+0000","last_unstale":"2023-11-28T12:01:46.497880+0000","last_undegraded":"2023-11-28T12:01:46.497880+0000","last_fullsized":"2023-11-28T12:01:46.497880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:32.123066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937355+0000","last_change":"2023-11-28T12:01:43.493841+0000","last_active":"2023-11-28T12:01:44.937355+0000","last_peered":"2023-11-28T12:01:44.937355+0000","last_clean":"2023-11-28T12:01:44.937355+0000","last_became_active":"2023-11-28T12:01:43.493559+0000","last_became_peered":"2023-11-28T12:01:43.493559+0000","last_unstale":"2023-11-28T12:01:44.937355+0000","last_undegraded":"2023-11-28T12:01:44.937355+0000","last_fullsized":"2023-11-28T12:01:44.937355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939401+0000","last_change":"2023-11-28T12:01:43.954120+0000","last_active":"2023-11-28T12:01:44.939401+0000","last_peered":"2023-11-28T12:01:44.939401+0000","last_clean":"2023-11-28T12:01:44.939401+0000","last_became_active":"2023-11-28T12:01:43.953634+0000","last_became_peered":"2023-11-28T12:01:43.953634+0000","last_unstale":"2023-11-28T12:01:44.939401+0000","last_undegraded":"2023-11-28T12:01:44.939401+0000","last_fullsized":"2023-11-28T12:01:44.939401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:46.302273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853794+0000","last_change":"2023-11-28T12:01:43.499686+0000","last_active":"2023-11-28T12:01:45.853794+0000","last_peered":"2023-11-28T12:01:45.853794+0000","last_clean":"2023-11-28T12:01:45.853794+0000","last_became_active":"2023-11-28T12:01:43.499379+0000","last_became_peered":"2023-11-28T12:01:43.499379+0000","last_unstale":"2023-11-28T12:01:45.853794+0000","last_undegraded":"2023-11-28T12:01:45.853794+0000","last_fullsized":"2023-11-28T12:01:45.853794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:07.361707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545042+0000","last_change":"2023-11-28T12:01:43.943016+0000","last_active":"2023-11-28T12:01:46.545042+0000","last_peered":"2023-11-28T12:01:46.545042+0000","last_clean":"2023-11-28T12:01:46.545042+0000","last_became_active":"2023-11-28T12:01:43.941928+0000","last_became_peered":"2023-11-28T12:01:43.941928+0000","last_unstale":"2023-11-28T12:01:46.545042+0000","last_undegraded":"2023-11-28T12:01:46.545042+0000","last_fullsized":"2023-11-28T12:01:46.545042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:01.126989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853782+0000","last_change":"2023-11-28T12:01:43.499257+0000","last_active":"2023-11-28T12:01:45.853782+0000","last_peered":"2023-11-28T12:01:45.853782+0000","last_clean":"2023-11-28T12:01:45.853782+0000","last_became_active":"2023-11-28T12:01:43.498973+0000","last_became_peered":"2023-11-28T12:01:43.498973+0000","last_unstale":"2023-11-28T12:01:45.853782+0000","last_undegraded":"2023-11-28T12:01:45.853782+0000","last_fullsized":"2023-11-28T12:01:45.853782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:46:41.932790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937789+0000","last_change":"2023-11-28T12:01:43.937615+0000","last_active":"2023-11-28T12:01:44.937789+0000","last_peered":"2023-11-28T12:01:44.937789+0000","last_clean":"2023-11-28T12:01:44.937789+0000","last_became_active":"2023-11-28T12:01:43.937115+0000","last_became_peered":"2023-11-28T12:01:43.937115+0000","last_unstale":"2023-11-28T12:01:44.937789+0000","last_undegraded":"2023-11-28T12:01:44.937789+0000","last_fullsized":"2023-11-28T12:01:44.937789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:51.664836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853717+0000","last_change":"2023-11-28T12:01:43.496645+0000","last_active":"2023-11-28T12:01:45.853717+0000","last_peered":"2023-11-28T12:01:45.853717+0000","last_clean":"2023-11-28T12:01:45.853717+0000","last_became_active":"2023-11-28T12:01:43.496393+0000","last_became_peered":"2023-11-28T12:01:43.496393+0000","last_unstale":"2023-11-28T12:01:45.853717+0000","last_undegraded":"2023-11-28T12:01:45.853717+0000","last_fullsized":"2023-11-28T12:01:45.853717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:18.701227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937935+0000","last_change":"2023-11-28T12:01:43.946845+0000","last_active":"2023-11-28T12:01:44.937935+0000","last_peered":"2023-11-28T12:01:44.937935+0000","last_clean":"2023-11-28T12:01:44.937935+0000","last_became_active":"2023-11-28T12:01:43.946572+0000","last_became_peered":"2023-11-28T12:01:43.946572+0000","last_unstale":"2023-11-28T12:01:44.937935+0000","last_undegraded":"2023-11-28T12:01:44.937935+0000","last_fullsized":"2023-11-28T12:01:44.937935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545284+0000","last_change":"2023-11-28T12:01:43.500124+0000","last_active":"2023-11-28T12:01:46.545284+0000","last_peered":"2023-11-28T12:01:46.545284+0000","last_clean":"2023-11-28T12:01:46.545284+0000","last_became_active":"2023-11-28T12:01:43.499857+0000","last_became_peered":"2023-11-28T12:01:43.499857+0000","last_unstale":"2023-11-28T12:01:46.545284+0000","last_undegraded":"2023-11-28T12:01:46.545284+0000","last_fullsized":"2023-11-28T12:01:46.545284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497627+0000","last_change":"2023-11-28T12:01:43.945003+0000","last_active":"2023-11-28T12:01:46.497627+0000","last_peered":"2023-11-28T12:01:46.497627+0000","last_clean":"2023-11-28T12:01:46.497627+0000","last_became_active":"2023-11-28T12:01:43.944725+0000","last_became_peered":"2023-11-28T12:01:43.944725+0000","last_unstale":"2023-11-28T12:01:46.497627+0000","last_undegraded":"2023-11-28T12:01:46.497627+0000","last_fullsized":"2023-11-28T12:01:46.497627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:46.302273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939198+0000","last_change":"2023-11-28T12:01:43.500073+0000","last_active":"2023-11-28T12:01:44.939198+0000","last_peered":"2023-11-28T12:01:44.939198+0000","last_clean":"2023-11-28T12:01:44.939198+0000","last_became_active":"2023-11-28T12:01:43.499770+0000","last_became_peered":"2023-11-28T12:01:43.499770+0000","last_unstale":"2023-11-28T12:01:44.939198+0000","last_undegraded":"2023-11-28T12:01:44.939198+0000","last_fullsized":"2023-11-28T12:01:44.939198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938256+0000","last_change":"2023-11-28T12:01:43.954024+0000","last_active":"2023-11-28T12:01:44.938256+0000","last_peered":"2023-11-28T12:01:44.938256+0000","last_clean":"2023-11-28T12:01:44.938256+0000","last_became_active":"2023-11-28T12:01:43.953743+0000","last_became_peered":"2023-11-28T12:01:43.953743+0000","last_unstale":"2023-11-28T12:01:44.938256+0000","last_undegraded":"2023-11-28T12:01:44.938256+0000","last_fullsized":"2023-11-28T12:01:44.938256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:35.755196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937423+0000","last_change":"2023-11-28T12:01:43.491888+0000","last_active":"2023-11-28T12:01:44.937423+0000","last_peered":"2023-11-28T12:01:44.937423+0000","last_clean":"2023-11-28T12:01:44.937423+0000","last_became_active":"2023-11-28T12:01:43.491585+0000","last_became_peered":"2023-11-28T12:01:43.491585+0000","last_unstale":"2023-11-28T12:01:44.937423+0000","last_undegraded":"2023-11-28T12:01:44.937423+0000","last_fullsized":"2023-11-28T12:01:44.937423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938535+0000","last_change":"2023-11-28T12:01:43.940969+0000","last_active":"2023-11-28T12:01:44.938535+0000","last_peered":"2023-11-28T12:01:44.938535+0000","last_clean":"2023-11-28T12:01:44.938535+0000","last_became_active":"2023-11-28T12:01:43.940012+0000","last_became_peered":"2023-11-28T12:01:43.940012+0000","last_unstale":"2023-11-28T12:01:44.938535+0000","last_undegraded":"2023-11-28T12:01:44.938535+0000","last_fullsized":"2023-11-28T12:01:44.938535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:20:45.411292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937934+0000","last_change":"2023-11-28T12:01:43.493475+0000","last_active":"2023-11-28T12:01:44.937934+0000","last_peered":"2023-11-28T12:01:44.937934+0000","last_clean":"2023-11-28T12:01:44.937934+0000","last_became_active":"2023-11-28T12:01:43.493206+0000","last_became_peered":"2023-11-28T12:01:43.493206+0000","last_unstale":"2023-11-28T12:01:44.937934+0000","last_undegraded":"2023-11-28T12:01:44.937934+0000","last_fullsized":"2023-11-28T12:01:44.937934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938568+0000","last_change":"2023-11-28T12:01:43.944744+0000","last_active":"2023-11-28T12:01:44.938568+0000","last_peered":"2023-11-28T12:01:44.938568+0000","last_clean":"2023-11-28T12:01:44.938568+0000","last_became_active":"2023-11-28T12:01:43.943939+0000","last_became_peered":"2023-11-28T12:01:43.943939+0000","last_unstale":"2023-11-28T12:01:44.938568+0000","last_undegraded":"2023-11-28T12:01:44.938568+0000","last_fullsized":"2023-11-28T12:01:44.938568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:10.443506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938813+0000","last_change":"2023-11-28T12:01:43.497932+0000","last_active":"2023-11-28T12:01:44.938813+0000","last_peered":"2023-11-28T12:01:44.938813+0000","last_clean":"2023-11-28T12:01:44.938813+0000","last_became_active":"2023-11-28T12:01:43.497623+0000","last_became_peered":"2023-11-28T12:01:43.497623+0000","last_unstale":"2023-11-28T12:01:44.938813+0000","last_undegraded":"2023-11-28T12:01:44.938813+0000","last_fullsized":"2023-11-28T12:01:44.938813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:46:41.932790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923287+0000","last_change":"2023-11-28T12:01:43.943517+0000","last_active":"2023-11-28T12:01:45.923287+0000","last_peered":"2023-11-28T12:01:45.923287+0000","last_clean":"2023-11-28T12:01:45.923287+0000","last_became_active":"2023-11-28T12:01:43.943220+0000","last_became_peered":"2023-11-28T12:01:43.943220+0000","last_unstale":"2023-11-28T12:01:45.923287+0000","last_undegraded":"2023-11-28T12:01:45.923287+0000","last_fullsized":"2023-11-28T12:01:45.923287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939358+0000","last_change":"2023-11-28T12:01:43.495923+0000","last_active":"2023-11-28T12:01:44.939358+0000","last_peered":"2023-11-28T12:01:44.939358+0000","last_clean":"2023-11-28T12:01:44.939358+0000","last_became_active":"2023-11-28T12:01:43.494593+0000","last_became_peered":"2023-11-28T12:01:43.494593+0000","last_unstale":"2023-11-28T12:01:44.939358+0000","last_undegraded":"2023-11-28T12:01:44.939358+0000","last_fullsized":"2023-11-28T12:01:44.939358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.928332+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:01:43.928332+0000","last_peered":"2023-11-28T12:01:43.928332+0000","last_clean":"2023-11-28T12:01:43.928332+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:01:43.928332+0000","last_undegraded":"2023-11-28T12:01:43.928332+0000","last_fullsized":"2023-11-28T12:01:43.928332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938743+0000","last_change":"2023-11-28T12:01:43.938577+0000","last_active":"2023-11-28T12:01:44.938743+0000","last_peered":"2023-11-28T12:01:44.938743+0000","last_clean":"2023-11-28T12:01:44.938743+0000","last_became_active":"2023-11-28T12:01:43.937923+0000","last_became_peered":"2023-11-28T12:01:43.937923+0000","last_unstale":"2023-11-28T12:01:44.938743+0000","last_undegraded":"2023-11-28T12:01:44.938743+0000","last_fullsized":"2023-11-28T12:01:44.938743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:40.988886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.928705+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:01:43.928705+0000","last_peered":"2023-11-28T12:01:43.928705+0000","last_clean":"2023-11-28T12:01:43.928705+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:01:43.928705+0000","last_undegraded":"2023-11-28T12:01:43.928705+0000","last_fullsized":"2023-11-28T12:01:43.928705+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.734302+0000","last_change":"2023-11-28T12:01:43.500640+0000","last_active":"2023-11-28T12:01:46.734302+0000","last_peered":"2023-11-28T12:01:46.734302+0000","last_clean":"2023-11-28T12:01:46.734302+0000","last_became_active":"2023-11-28T12:01:43.500348+0000","last_became_peered":"2023-11-28T12:01:43.500348+0000","last_unstale":"2023-11-28T12:01:46.734302+0000","last_undegraded":"2023-11-28T12:01:46.734302+0000","last_fullsized":"2023-11-28T12:01:46.734302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:40.727625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937843+0000","last_change":"2023-11-28T12:01:43.944208+0000","last_active":"2023-11-28T12:01:44.937843+0000","last_peered":"2023-11-28T12:01:44.937843+0000","last_clean":"2023-11-28T12:01:44.937843+0000","last_became_active":"2023-11-28T12:01:43.943957+0000","last_became_peered":"2023-11-28T12:01:43.943957+0000","last_unstale":"2023-11-28T12:01:44.937843+0000","last_undegraded":"2023-11-28T12:01:44.937843+0000","last_fullsized":"2023-11-28T12:01:44.937843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:54:35.260564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.929563+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:01:43.929563+0000","last_peered":"2023-11-28T12:01:43.929563+0000","last_clean":"2023-11-28T12:01:43.929563+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:01:43.929563+0000","last_undegraded":"2023-11-28T12:01:43.929563+0000","last_fullsized":"2023-11-28T12:01:43.929563+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.934971+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:01:43.934971+0000","last_peered":"2023-11-28T12:01:43.934971+0000","last_clean":"2023-11-28T12:01:43.934971+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:01:43.934971+0000","last_undegraded":"2023-11-28T12:01:43.934971+0000","last_fullsized":"2023-11-28T12:01:43.934971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497590+0000","last_change":"2023-11-28T12:01:43.498540+0000","last_active":"2023-11-28T12:01:46.497590+0000","last_peered":"2023-11-28T12:01:46.497590+0000","last_clean":"2023-11-28T12:01:46.497590+0000","last_became_active":"2023-11-28T12:01:43.498040+0000","last_became_peered":"2023-11-28T12:01:43.498040+0000","last_unstale":"2023-11-28T12:01:46.497590+0000","last_undegraded":"2023-11-28T12:01:46.497590+0000","last_fullsized":"2023-11-28T12:01:46.497590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:29.667067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497902+0000","last_change":"2023-11-28T12:01:43.946239+0000","last_active":"2023-11-28T12:01:46.497902+0000","last_peered":"2023-11-28T12:01:46.497902+0000","last_clean":"2023-11-28T12:01:46.497902+0000","last_became_active":"2023-11-28T12:01:43.945937+0000","last_became_peered":"2023-11-28T12:01:43.945937+0000","last_unstale":"2023-11-28T12:01:46.497902+0000","last_undegraded":"2023-11-28T12:01:46.497902+0000","last_fullsized":"2023-11-28T12:01:46.497902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:40.988886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.935299+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:01:43.935299+0000","last_peered":"2023-11-28T12:01:43.935299+0000","last_clean":"2023-11-28T12:01:43.935299+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:01:43.935299+0000","last_undegraded":"2023-11-28T12:01:43.935299+0000","last_fullsized":"2023-11-28T12:01:43.935299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922707+0000","last_change":"2023-11-28T12:01:43.495719+0000","last_active":"2023-11-28T12:01:45.922707+0000","last_peered":"2023-11-28T12:01:45.922707+0000","last_clean":"2023-11-28T12:01:45.922707+0000","last_became_active":"2023-11-28T12:01:43.494865+0000","last_became_peered":"2023-11-28T12:01:43.494865+0000","last_unstale":"2023-11-28T12:01:45.922707+0000","last_undegraded":"2023-11-28T12:01:45.922707+0000","last_fullsized":"2023-11-28T12:01:45.922707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:29.667067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923227+0000","last_change":"2023-11-28T12:01:43.946289+0000","last_active":"2023-11-28T12:01:45.923227+0000","last_peered":"2023-11-28T12:01:45.923227+0000","last_clean":"2023-11-28T12:01:45.923227+0000","last_became_active":"2023-11-28T12:01:43.945759+0000","last_became_peered":"2023-11-28T12:01:43.945759+0000","last_unstale":"2023-11-28T12:01:45.923227+0000","last_undegraded":"2023-11-28T12:01:45.923227+0000","last_fullsized":"2023-11-28T12:01:45.923227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:41.736417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.934867+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:01:43.934867+0000","last_peered":"2023-11-28T12:01:43.934867+0000","last_clean":"2023-11-28T12:01:43.934867+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:01:43.934867+0000","last_undegraded":"2023-11-28T12:01:43.934867+0000","last_fullsized":"2023-11-28T12:01:43.934867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938960+0000","last_change":"2023-11-28T12:01:43.496920+0000","last_active":"2023-11-28T12:01:44.938960+0000","last_peered":"2023-11-28T12:01:44.938960+0000","last_clean":"2023-11-28T12:01:44.938960+0000","last_became_active":"2023-11-28T12:01:43.495644+0000","last_became_peered":"2023-11-28T12:01:43.495644+0000","last_unstale":"2023-11-28T12:01:44.938960+0000","last_undegraded":"2023-11-28T12:01:44.938960+0000","last_fullsized":"2023-11-28T12:01:44.938960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937511+0000","last_change":"2023-11-28T12:01:43.950061+0000","last_active":"2023-11-28T12:01:44.937511+0000","last_peered":"2023-11-28T12:01:44.937511+0000","last_clean":"2023-11-28T12:01:44.937511+0000","last_became_active":"2023-11-28T12:01:43.949778+0000","last_became_peered":"2023-11-28T12:01:43.949778+0000","last_unstale":"2023-11-28T12:01:44.937511+0000","last_undegraded":"2023-11-28T12:01:44.937511+0000","last_fullsized":"2023-11-28T12:01:44.937511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:36.019691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.932921+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:01:43.932921+0000","last_peered":"2023-11-28T12:01:43.932921+0000","last_clean":"2023-11-28T12:01:43.932921+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:01:43.932921+0000","last_undegraded":"2023-11-28T12:01:43.932921+0000","last_fullsized":"2023-11-28T12:01:43.932921+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938196+0000","last_change":"2023-11-28T12:01:43.499396+0000","last_active":"2023-11-28T12:01:44.938196+0000","last_peered":"2023-11-28T12:01:44.938196+0000","last_clean":"2023-11-28T12:01:44.938196+0000","last_became_active":"2023-11-28T12:01:43.499103+0000","last_became_peered":"2023-11-28T12:01:43.499103+0000","last_unstale":"2023-11-28T12:01:44.938196+0000","last_undegraded":"2023-11-28T12:01:44.938196+0000","last_fullsized":"2023-11-28T12:01:44.938196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923174+0000","last_change":"2023-11-28T12:01:43.941086+0000","last_active":"2023-11-28T12:01:45.923174+0000","last_peered":"2023-11-28T12:01:45.923174+0000","last_clean":"2023-11-28T12:01:45.923174+0000","last_became_active":"2023-11-28T12:01:43.940781+0000","last_became_peered":"2023-11-28T12:01:43.940781+0000","last_unstale":"2023-11-28T12:01:45.923174+0000","last_undegraded":"2023-11-28T12:01:45.923174+0000","last_fullsized":"2023-11-28T12:01:45.923174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:19.706888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.935713+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:01:43.935713+0000","last_peered":"2023-11-28T12:01:43.935713+0000","last_clean":"2023-11-28T12:01:43.935713+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:01:43.935713+0000","last_undegraded":"2023-11-28T12:01:43.935713+0000","last_fullsized":"2023-11-28T12:01:43.935713+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923245+0000","last_change":"2023-11-28T12:01:43.501876+0000","last_active":"2023-11-28T12:01:45.923245+0000","last_peered":"2023-11-28T12:01:45.923245+0000","last_clean":"2023-11-28T12:01:45.923245+0000","last_became_active":"2023-11-28T12:01:43.501509+0000","last_became_peered":"2023-11-28T12:01:43.501509+0000","last_unstale":"2023-11-28T12:01:45.923245+0000","last_undegraded":"2023-11-28T12:01:45.923245+0000","last_fullsized":"2023-11-28T12:01:45.923245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497880+0000","last_change":"2023-11-28T12:01:43.947364+0000","last_active":"2023-11-28T12:01:46.497880+0000","last_peered":"2023-11-28T12:01:46.497880+0000","last_clean":"2023-11-28T12:01:46.497880+0000","last_became_active":"2023-11-28T12:01:43.947043+0000","last_became_peered":"2023-11-28T12:01:43.947043+0000","last_unstale":"2023-11-28T12:01:46.497880+0000","last_undegraded":"2023-11-28T12:01:46.497880+0000","last_fullsized":"2023-11-28T12:01:46.497880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.936792+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:01:43.936792+0000","last_peered":"2023-11-28T12:01:43.936792+0000","last_clean":"2023-11-28T12:01:43.936792+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:01:43.936792+0000","last_undegraded":"2023-11-28T12:01:43.936792+0000","last_fullsized":"2023-11-28T12:01:43.936792+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545482+0000","last_change":"2023-11-28T12:01:43.497088+0000","last_active":"2023-11-28T12:01:46.545482+0000","last_peered":"2023-11-28T12:01:46.545482+0000","last_clean":"2023-11-28T12:01:46.545482+0000","last_became_active":"2023-11-28T12:01:43.496456+0000","last_became_peered":"2023-11-28T12:01:43.496456+0000","last_unstale":"2023-11-28T12:01:46.545482+0000","last_undegraded":"2023-11-28T12:01:46.545482+0000","last_fullsized":"2023-11-28T12:01:46.545482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923037+0000","last_change":"2023-11-28T12:01:43.946582+0000","last_active":"2023-11-28T12:01:45.923037+0000","last_peered":"2023-11-28T12:01:45.923037+0000","last_clean":"2023-11-28T12:01:45.923037+0000","last_became_active":"2023-11-28T12:01:43.946281+0000","last_became_peered":"2023-11-28T12:01:43.946281+0000","last_unstale":"2023-11-28T12:01:45.923037+0000","last_undegraded":"2023-11-28T12:01:45.923037+0000","last_fullsized":"2023-11-28T12:01:45.923037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:10.298310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497709+0000","last_change":"2023-11-28T12:01:43.499312+0000","last_active":"2023-11-28T12:01:46.497709+0000","last_peered":"2023-11-28T12:01:46.497709+0000","last_clean":"2023-11-28T12:01:46.497709+0000","last_became_active":"2023-11-28T12:01:43.499020+0000","last_became_peered":"2023-11-28T12:01:43.499020+0000","last_unstale":"2023-11-28T12:01:46.497709+0000","last_undegraded":"2023-11-28T12:01:46.497709+0000","last_fullsized":"2023-11-28T12:01:46.497709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542759+0000","last_change":"2023-11-28T12:01:43.945612+0000","last_active":"2023-11-28T12:01:46.542759+0000","last_peered":"2023-11-28T12:01:46.542759+0000","last_clean":"2023-11-28T12:01:46.542759+0000","last_became_active":"2023-11-28T12:01:43.945304+0000","last_became_peered":"2023-11-28T12:01:43.945304+0000","last_unstale":"2023-11-28T12:01:46.542759+0000","last_undegraded":"2023-11-28T12:01:46.542759+0000","last_fullsized":"2023-11-28T12:01:46.542759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:20.571945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545110+0000","last_change":"2023-11-28T12:01:43.496842+0000","last_active":"2023-11-28T12:01:46.545110+0000","last_peered":"2023-11-28T12:01:46.545110+0000","last_clean":"2023-11-28T12:01:46.545110+0000","last_became_active":"2023-11-28T12:01:43.496567+0000","last_became_peered":"2023-11-28T12:01:43.496567+0000","last_unstale":"2023-11-28T12:01:46.545110+0000","last_undegraded":"2023-11-28T12:01:46.545110+0000","last_fullsized":"2023-11-28T12:01:46.545110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938343+0000","last_change":"2023-11-28T12:01:43.941160+0000","last_active":"2023-11-28T12:01:44.938343+0000","last_peered":"2023-11-28T12:01:44.938343+0000","last_clean":"2023-11-28T12:01:44.938343+0000","last_became_active":"2023-11-28T12:01:43.940593+0000","last_became_peered":"2023-11-28T12:01:43.940593+0000","last_unstale":"2023-11-28T12:01:44.938343+0000","last_undegraded":"2023-11-28T12:01:44.938343+0000","last_fullsized":"2023-11-28T12:01:44.938343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:46.302273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545507+0000","last_change":"2023-11-28T12:01:43.497657+0000","last_active":"2023-11-28T12:01:46.545507+0000","last_peered":"2023-11-28T12:01:46.545507+0000","last_clean":"2023-11-28T12:01:46.545507+0000","last_became_active":"2023-11-28T12:01:43.496814+0000","last_became_peered":"2023-11-28T12:01:43.496814+0000","last_unstale":"2023-11-28T12:01:46.545507+0000","last_undegraded":"2023-11-28T12:01:46.545507+0000","last_fullsized":"2023-11-28T12:01:46.545507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938078+0000","last_change":"2023-11-28T12:01:43.938233+0000","last_active":"2023-11-28T12:01:44.938078+0000","last_peered":"2023-11-28T12:01:44.938078+0000","last_clean":"2023-11-28T12:01:44.938078+0000","last_became_active":"2023-11-28T12:01:43.937825+0000","last_became_peered":"2023-11-28T12:01:43.937825+0000","last_unstale":"2023-11-28T12:01:44.938078+0000","last_undegraded":"2023-11-28T12:01:44.938078+0000","last_fullsized":"2023-11-28T12:01:44.938078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:04.509152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939417+0000","last_change":"2023-11-28T12:01:43.485835+0000","last_active":"2023-11-28T12:01:44.939417+0000","last_peered":"2023-11-28T12:01:44.939417+0000","last_clean":"2023-11-28T12:01:44.939417+0000","last_became_active":"2023-11-28T12:01:43.484610+0000","last_became_peered":"2023-11-28T12:01:43.484610+0000","last_unstale":"2023-11-28T12:01:44.939417+0000","last_undegraded":"2023-11-28T12:01:44.939417+0000","last_fullsized":"2023-11-28T12:01:44.939417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:40.727625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939028+0000","last_change":"2023-11-28T12:01:43.949382+0000","last_active":"2023-11-28T12:01:44.939028+0000","last_peered":"2023-11-28T12:01:44.939028+0000","last_clean":"2023-11-28T12:01:44.939028+0000","last_became_active":"2023-11-28T12:01:43.948949+0000","last_became_peered":"2023-11-28T12:01:43.948949+0000","last_unstale":"2023-11-28T12:01:44.939028+0000","last_undegraded":"2023-11-28T12:01:44.939028+0000","last_fullsized":"2023-11-28T12:01:44.939028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:35.755196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497980+0000","last_change":"2023-11-28T12:01:43.498156+0000","last_active":"2023-11-28T12:01:46.497980+0000","last_peered":"2023-11-28T12:01:46.497980+0000","last_clean":"2023-11-28T12:01:46.497980+0000","last_became_active":"2023-11-28T12:01:43.496510+0000","last_became_peered":"2023-11-28T12:01:43.496510+0000","last_unstale":"2023-11-28T12:01:46.497980+0000","last_undegraded":"2023-11-28T12:01:46.497980+0000","last_fullsized":"2023-11-28T12:01:46.497980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:46.276522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937629+0000","last_change":"2023-11-28T12:01:43.944985+0000","last_active":"2023-11-28T12:01:44.937629+0000","last_peered":"2023-11-28T12:01:44.937629+0000","last_clean":"2023-11-28T12:01:44.937629+0000","last_became_active":"2023-11-28T12:01:43.943580+0000","last_became_peered":"2023-11-28T12:01:43.943580+0000","last_unstale":"2023-11-28T12:01:44.937629+0000","last_undegraded":"2023-11-28T12:01:44.937629+0000","last_fullsized":"2023-11-28T12:01:44.937629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937476+0000","last_change":"2023-11-28T12:01:43.941917+0000","last_active":"2023-11-28T12:01:44.937476+0000","last_peered":"2023-11-28T12:01:44.937476+0000","last_clean":"2023-11-28T12:01:44.937476+0000","last_became_active":"2023-11-28T12:01:43.941625+0000","last_became_peered":"2023-11-28T12:01:43.941625+0000","last_unstale":"2023-11-28T12:01:44.937476+0000","last_undegraded":"2023-11-28T12:01:44.937476+0000","last_fullsized":"2023-11-28T12:01:44.937476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:23.540287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938902+0000","last_change":"2023-11-28T12:01:43.943568+0000","last_active":"2023-11-28T12:01:44.938902+0000","last_peered":"2023-11-28T12:01:44.938902+0000","last_clean":"2023-11-28T12:01:44.938902+0000","last_became_active":"2023-11-28T12:01:43.943204+0000","last_became_peered":"2023-11-28T12:01:43.943204+0000","last_unstale":"2023-11-28T12:01:44.938902+0000","last_undegraded":"2023-11-28T12:01:44.938902+0000","last_fullsized":"2023-11-28T12:01:44.938902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937986+0000","last_change":"2023-11-28T12:01:43.943816+0000","last_active":"2023-11-28T12:01:44.937986+0000","last_peered":"2023-11-28T12:01:44.937986+0000","last_clean":"2023-11-28T12:01:44.937986+0000","last_became_active":"2023-11-28T12:01:43.943506+0000","last_became_peered":"2023-11-28T12:01:43.943506+0000","last_unstale":"2023-11-28T12:01:44.937986+0000","last_undegraded":"2023-11-28T12:01:44.937986+0000","last_fullsized":"2023-11-28T12:01:44.937986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:28.088904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939262+0000","last_change":"2023-11-28T12:01:43.950413+0000","last_active":"2023-11-28T12:01:44.939262+0000","last_peered":"2023-11-28T12:01:44.939262+0000","last_clean":"2023-11-28T12:01:44.939262+0000","last_became_active":"2023-11-28T12:01:43.950110+0000","last_became_peered":"2023-11-28T12:01:43.950110+0000","last_unstale":"2023-11-28T12:01:44.939262+0000","last_undegraded":"2023-11-28T12:01:44.939262+0000","last_fullsized":"2023-11-28T12:01:44.939262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:57.245235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922987+0000","last_change":"2023-11-28T12:01:43.938293+0000","last_active":"2023-11-28T12:01:45.922987+0000","last_peered":"2023-11-28T12:01:45.922987+0000","last_clean":"2023-11-28T12:01:45.922987+0000","last_became_active":"2023-11-28T12:01:43.937951+0000","last_became_peered":"2023-11-28T12:01:43.937951+0000","last_unstale":"2023-11-28T12:01:45.922987+0000","last_undegraded":"2023-11-28T12:01:45.922987+0000","last_fullsized":"2023-11-28T12:01:45.922987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9084,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362756,"statfs":{"total":96636764160,"available":96627462144,"internally_reserved":0,"allocated":839680,"data_stored":671592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7900,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363940,"statfs":{"total":96636764160,"available":96628674560,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9056,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362784,"statfs":{"total":96636764160,"available":96627490816,"internally_reserved":0,"allocated":811008,"data_stored":669828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7968,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363872,"statfs":{"total":96636764160,"available":96628604928,"internally_reserved":0,"allocated":221184,"data_stored":80026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7900,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363940,"statfs":{"total":96636764160,"available":96628674560,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8032,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363808,"statfs":{"total":96636764160,"available":96628539392,"internally_reserved":0,"allocated":221184,"data_stored":79550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8048,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363792,"statfs":{"total":96636764160,"available":96628523008,"internally_reserved":0,"allocated":237568,"data_stored":81064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7972,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363868,"statfs":{"total":96636764160,"available":96628600832,"internally_reserved":0,"allocated":225280,"data_stored":80096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:01:54.328 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:01:54.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.501+0000 7f48b5617700 1 Processor -- start 2023-11-28T12:01:54.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.501+0000 7f48b5617700 1 -- start start 2023-11-28T12:01:54.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48b5617700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f48b006ab10 0x7f48b006aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48b5617700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48b00d0450 0x7f48b0160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48b5617700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b00d1ee0 con 0x7f48b00cd5a0 2023-11-28T12:01:54.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48b5617700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b006b420 con 0x7f48b006ab10 2023-11-28T12:01:54.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48b5617700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00cdb10 con 0x7f48b00d0450 2023-11-28T12:01:54.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48ae7fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f48b00cd5a0 0x7f48b0160300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41136/0 (socket says 172.21.15.57:41136) 2023-11-28T12:01:54.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48ae7fc700 1 -- 172.21.15.57:0/2777467340 learned_addr learned my addr 172.21.15.57:0/2777467340 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:54.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48af7fe700 1 --2- 172.21.15.57:0/2777467340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48b00d0450 0x7f48b0160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48aeffd700 1 --2- 172.21.15.57:0/2777467340 >> v2:172.21.15.181:3300/0 conn(0x7f48b006ab10 0x7f48b006aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104037028 0 0) 0x7f48b00d1ee0 con 0x7f48b00cd5a0 2023-11-28T12:01:54.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f489c0035b0 con 0x7f48b00cd5a0 2023-11-28T12:01:54.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2305584802 0 0) 0x7f489c0035b0 con 0x7f48b00cd5a0 2023-11-28T12:01:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48b00d0450 msgr2=0x7f48b0160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.502+0000 7f48adffb700 1 --2- 172.21.15.57:0/2777467340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48b00d0450 0x7f48b0160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 >> v2:172.21.15.181:3300/0 conn(0x7f48b006ab10 msgr2=0x7f48b006aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 --2- 172.21.15.57:0/2777467340 >> v2:172.21.15.181:3300/0 conn(0x7f48b006ab10 0x7f48b006aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b00d1890 con 0x7f48b00cd5a0 2023-11-28T12:01:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f4898001670 con 0x7f48b00cd5a0 2023-11-28T12:01:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 >> v1:172.21.15.57:6789/0 conn(0x7f48b00cd5a0 legacy=0x7f48b0160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 --2- 172.21.15.57:0/2777467340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f489c003a00 0x7f489c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 --2- 172.21.15.57:0/2777467340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48b00d0450 0x7f489c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 --2- 172.21.15.57:0/2777467340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f489c004d90 0x7f489c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b006b420 con 0x7f489c003a00 2023-11-28T12:01:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00cdb10 con 0x7f489c004d90 2023-11-28T12:01:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489c00ba10 con 0x7f48b00d0450 2023-11-28T12:01:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48aeffd700 1 --2- 172.21.15.57:0/2777467340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f489c003a00 0x7f489c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48ae7fc700 1 --2- 172.21.15.57:0/2777467340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48b00d0450 0x7f489c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48af7fe700 1 --2- 172.21.15.57:0/2777467340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f489c004d90 0x7f489c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48aeffd700 1 -- 172.21.15.57:0/2777467340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48b00d0450 msgr2=0x7f489c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48aeffd700 1 --2- 172.21.15.57:0/2777467340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48b00d0450 0x7f489c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48aeffd700 1 -- 172.21.15.57:0/2777467340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f489c004d90 msgr2=0x7f489c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:54.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48aeffd700 1 --2- 172.21.15.57:0/2777467340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f489c004d90 0x7f489c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48aeffd700 1 -- 172.21.15.57:0/2777467340 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c0035b0 con 0x7f489c003a00 2023-11-28T12:01:54.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48aeffd700 1 --2- 172.21.15.57:0/2777467340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f489c003a00 0x7f489c003e10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009b60 tx=0x7f489c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa8193af9844f4d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:54.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a000e660 con 0x7f489c003a00 2023-11-28T12:01:54.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a000e7c0 con 0x7f489c003a00 2023-11-28T12:01:54.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.503+0000 7f48adffb700 1 -- 172.21.15.57:0/2777467340 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a000eb30 con 0x7f489c003a00 2023-11-28T12:01:54.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.504+0000 7f48b5617700 1 -- 172.21.15.57:0/2777467340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f489c003a00 msgr2=0x7f489c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.504+0000 7f48b5617700 1 --2- 172.21.15.57:0/2777467340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f489c003a00 0x7f489c003e10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009b60 tx=0x7f489c005880 comp rx=0 tx=0).stop 2023-11-28T12:01:54.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.504+0000 7f48af7fe700 1 -- 172.21.15.57:0/2777467340 reap_dead start 2023-11-28T12:01:54.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.504+0000 7f48b5617700 1 -- 172.21.15.57:0/2777467340 shutdown_connections 2023-11-28T12:01:54.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.504+0000 7f48b5617700 1 -- 172.21.15.57:0/2777467340 >> 172.21.15.57:0/2777467340 conn(0x7f48b006c520 msgr2=0x7f48b006c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:54.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.504+0000 7f48b5617700 1 -- 172.21.15.57:0/2777467340 shutdown_connections 2023-11-28T12:01:54.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.504+0000 7f48b5617700 1 -- 172.21.15.57:0/2777467340 wait complete. 2023-11-28T12:01:54.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.506+0000 7f48b5617700 1 Processor -- start 2023-11-28T12:01:54.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.506+0000 7f48b5617700 1 -- start start 2023-11-28T12:01:54.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.506+0000 7f48b5617700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b01f2180 0x7f48b0156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.506+0000 7f48b5617700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48b0156750 0x7f48b0155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.506+0000 7f48b5617700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48b0155ba0 0x7f48b0159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.506+0000 7f48b5617700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b0156bc0 con 0x7f48b0156750 2023-11-28T12:01:54.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.506+0000 7f48b5617700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b0156040 con 0x7f48b01f2180 2023-11-28T12:01:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.506+0000 7f48b5617700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b0159e70 con 0x7f48b0155ba0 2023-11-28T12:01:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.506+0000 7f48ae7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48b0156750 0x7f48b0155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.506+0000 7f48ae7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48b0156750 0x7f48b0155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38794/0 (socket says 172.21.15.57:38794) 2023-11-28T12:01:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48ae7fc700 1 -- 172.21.15.57:0/2158125442 learned_addr learned my addr 172.21.15.57:0/2158125442 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48aeffd700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b01f2180 0x7f48b0156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48af7fe700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48b0155ba0 0x7f48b0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48ae7fc700 1 -- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48b0155ba0 msgr2=0x7f48b0159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48ae7fc700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48b0155ba0 0x7f48b0159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48ae7fc700 1 -- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b01f2180 msgr2=0x7f48b0156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48ae7fc700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b01f2180 0x7f48b0156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48ae7fc700 1 -- 172.21.15.57:0/2158125442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b015cad0 con 0x7f48b0156750 2023-11-28T12:01:54.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48ae7fc700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48b0156750 0x7f48b0155660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f48980070e0 tx=0x7f48980074a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:54.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48a77fe700 1 -- 172.21.15.57:0/2158125442 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800f600 con 0x7f48b0156750 2023-11-28T12:01:54.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48a77fe700 1 -- 172.21.15.57:0/2158125442 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489800f7a0 con 0x7f48b0156750 2023-11-28T12:01:54.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48a77fe700 1 -- 172.21.15.57:0/2158125442 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800fb50 con 0x7f48b0156750 2023-11-28T12:01:54.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48b5617700 1 -- 172.21.15.57:0/2158125442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b00d1b70 con 0x7f48b0156750 2023-11-28T12:01:54.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.507+0000 7f48b5617700 1 -- 172.21.15.57:0/2158125442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b00b1ec0 con 0x7f48b0156750 2023-11-28T12:01:54.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.508+0000 7f48b5617700 1 -- 172.21.15.57:0/2158125442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b00743f0 con 0x7f48b0156750 2023-11-28T12:01:54.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.509+0000 7f48a77fe700 1 -- 172.21.15.57:0/2158125442 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f489800fcb0 con 0x7f48b0156750 2023-11-28T12:01:54.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.509+0000 7f48a77fe700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4890068320 0x7f489006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.509+0000 7f48a77fe700 1 -- 172.21.15.57:0/2158125442 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4898085d20 con 0x7f48b0156750 2023-11-28T12:01:54.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.509+0000 7f48aeffd700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4890068320 0x7f489006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.510+0000 7f48aeffd700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4890068320 0x7f489006a7d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009a80 tx=0x7f48a001e040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:54.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.513+0000 7f48a77fe700 1 -- 172.21.15.57:0/2158125442 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4898052580 con 0x7f48b0156750 2023-11-28T12:01:54.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.710+0000 7f48b5617700 1 -- 172.21.15.57:0/2158125442 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48b00d2c80 con 0x7f4890068320 2023-11-28T12:01:54.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.718+0000 7f48a77fe700 1 -- 172.21.15.57:0/2158125442 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286827 (secure 0 0 0) 0x7f48b00d2c80 con 0x7f4890068320 2023-11-28T12:01:54.723 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:54.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.722+0000 7f48b5617700 1 -- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4890068320 msgr2=0x7f489006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.722+0000 7f48b5617700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4890068320 0x7f489006a7d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009a80 tx=0x7f48a001e040 comp rx=0 tx=0).stop 2023-11-28T12:01:54.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.722+0000 7f48b5617700 1 -- 172.21.15.57:0/2158125442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48b0156750 msgr2=0x7f48b0155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.722+0000 7f48b5617700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48b0156750 0x7f48b0155660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f48980070e0 tx=0x7f48980074a0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.722+0000 7f48b5617700 1 -- 172.21.15.57:0/2158125442 shutdown_connections 2023-11-28T12:01:54.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.722+0000 7f48b5617700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4890068320 0x7f489006a7d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.722+0000 7f48b5617700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48b0155ba0 0x7f48b0159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.722+0000 7f48b5617700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b01f2180 0x7f48b0156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.722+0000 7f48b5617700 1 --2- 172.21.15.57:0/2158125442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48b0156750 0x7f48b0155660 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.722+0000 7f48b5617700 1 -- 172.21.15.57:0/2158125442 >> 172.21.15.57:0/2158125442 conn(0x7f48b006c520 msgr2=0x7f48b00b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:54.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.722+0000 7f48b5617700 1 -- 172.21.15.57:0/2158125442 shutdown_connections 2023-11-28T12:01:54.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.722+0000 7f48b5617700 1 -- 172.21.15.57:0/2158125442 wait complete. 2023-11-28T12:01:54.727 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:01:54.736 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-28T12:01:53.660658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":65960,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":754908760,"statfs":{"total":773094113280,"available":773026570240,"internally_reserved":0,"allocated":2990080,"data_stored":1821076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.735559"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542856+0000","last_change":"2023-11-28T12:01:43.953143+0000","last_active":"2023-11-28T12:01:46.542856+0000","last_peered":"2023-11-28T12:01:46.542856+0000","last_clean":"2023-11-28T12:01:46.542856+0000","last_became_active":"2023-11-28T12:01:43.952673+0000","last_became_peered":"2023-11-28T12:01:43.952673+0000","last_unstale":"2023-11-28T12:01:46.542856+0000","last_undegraded":"2023-11-28T12:01:46.542856+0000","last_fullsized":"2023-11-28T12:01:46.542856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:18.949383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545399+0000","last_change":"2023-11-28T12:01:43.947886+0000","last_active":"2023-11-28T12:01:46.545399+0000","last_peered":"2023-11-28T12:01:46.545399+0000","last_clean":"2023-11-28T12:01:46.545399+0000","last_became_active":"2023-11-28T12:01:43.947468+0000","last_became_peered":"2023-11-28T12:01:43.947468+0000","last_unstale":"2023-11-28T12:01:46.545399+0000","last_undegraded":"2023-11-28T12:01:46.545399+0000","last_fullsized":"2023-11-28T12:01:46.545399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:41.736417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938403+0000","last_change":"2023-11-28T12:01:43.941069+0000","last_active":"2023-11-28T12:01:44.938403+0000","last_peered":"2023-11-28T12:01:44.938403+0000","last_clean":"2023-11-28T12:01:44.938403+0000","last_became_active":"2023-11-28T12:01:43.940366+0000","last_became_peered":"2023-11-28T12:01:43.940366+0000","last_unstale":"2023-11-28T12:01:44.938403+0000","last_undegraded":"2023-11-28T12:01:44.938403+0000","last_fullsized":"2023-11-28T12:01:44.938403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:40.988886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937690+0000","last_change":"2023-11-28T12:01:43.947292+0000","last_active":"2023-11-28T12:01:44.937690+0000","last_peered":"2023-11-28T12:01:44.937690+0000","last_clean":"2023-11-28T12:01:44.937690+0000","last_became_active":"2023-11-28T12:01:43.943027+0000","last_became_peered":"2023-11-28T12:01:43.943027+0000","last_unstale":"2023-11-28T12:01:44.937690+0000","last_undegraded":"2023-11-28T12:01:44.937690+0000","last_fullsized":"2023-11-28T12:01:44.937690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:19.706888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497843+0000","last_change":"2023-11-28T12:01:43.954299+0000","last_active":"2023-11-28T12:01:46.497843+0000","last_peered":"2023-11-28T12:01:46.497843+0000","last_clean":"2023-11-28T12:01:46.497843+0000","last_became_active":"2023-11-28T12:01:43.953967+0000","last_became_peered":"2023-11-28T12:01:43.953967+0000","last_unstale":"2023-11-28T12:01:46.497843+0000","last_undegraded":"2023-11-28T12:01:46.497843+0000","last_fullsized":"2023-11-28T12:01:46.497843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:35.755196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937402+0000","last_change":"2023-11-28T12:01:43.938748+0000","last_active":"2023-11-28T12:01:44.937402+0000","last_peered":"2023-11-28T12:01:44.937402+0000","last_clean":"2023-11-28T12:01:44.937402+0000","last_became_active":"2023-11-28T12:01:43.938430+0000","last_became_peered":"2023-11-28T12:01:43.938430+0000","last_unstale":"2023-11-28T12:01:44.937402+0000","last_undegraded":"2023-11-28T12:01:44.937402+0000","last_fullsized":"2023-11-28T12:01:44.937402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:49.922116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853429+0000","last_change":"2023-11-28T12:01:43.950997+0000","last_active":"2023-11-28T12:01:45.853429+0000","last_peered":"2023-11-28T12:01:45.853429+0000","last_clean":"2023-11-28T12:01:45.853429+0000","last_became_active":"2023-11-28T12:01:43.950532+0000","last_became_peered":"2023-11-28T12:01:43.950532+0000","last_unstale":"2023-11-28T12:01:45.853429+0000","last_undegraded":"2023-11-28T12:01:45.853429+0000","last_fullsized":"2023-11-28T12:01:45.853429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545106+0000","last_change":"2023-11-28T12:01:43.945419+0000","last_active":"2023-11-28T12:01:46.545106+0000","last_peered":"2023-11-28T12:01:46.545106+0000","last_clean":"2023-11-28T12:01:46.545106+0000","last_became_active":"2023-11-28T12:01:43.942913+0000","last_became_peered":"2023-11-28T12:01:43.942913+0000","last_unstale":"2023-11-28T12:01:46.545106+0000","last_undegraded":"2023-11-28T12:01:46.545106+0000","last_fullsized":"2023-11-28T12:01:46.545106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497879+0000","last_change":"2023-11-28T12:01:43.954900+0000","last_active":"2023-11-28T12:01:46.497879+0000","last_peered":"2023-11-28T12:01:46.497879+0000","last_clean":"2023-11-28T12:01:46.497879+0000","last_became_active":"2023-11-28T12:01:43.954524+0000","last_became_peered":"2023-11-28T12:01:43.954524+0000","last_unstale":"2023-11-28T12:01:46.497879+0000","last_undegraded":"2023-11-28T12:01:46.497879+0000","last_fullsized":"2023-11-28T12:01:46.497879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:10.298310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545527+0000","last_change":"2023-11-28T12:01:43.944809+0000","last_active":"2023-11-28T12:01:46.545527+0000","last_peered":"2023-11-28T12:01:46.545527+0000","last_clean":"2023-11-28T12:01:46.545527+0000","last_became_active":"2023-11-28T12:01:43.943120+0000","last_became_peered":"2023-11-28T12:01:43.943120+0000","last_unstale":"2023-11-28T12:01:46.545527+0000","last_undegraded":"2023-11-28T12:01:46.545527+0000","last_fullsized":"2023-11-28T12:01:46.545527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938327+0000","last_change":"2023-11-28T12:01:43.941446+0000","last_active":"2023-11-28T12:01:44.938327+0000","last_peered":"2023-11-28T12:01:44.938327+0000","last_clean":"2023-11-28T12:01:44.938327+0000","last_became_active":"2023-11-28T12:01:43.940640+0000","last_became_peered":"2023-11-28T12:01:43.940640+0000","last_unstale":"2023-11-28T12:01:44.938327+0000","last_undegraded":"2023-11-28T12:01:44.938327+0000","last_fullsized":"2023-11-28T12:01:44.938327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:41.605113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542711+0000","last_change":"2023-11-28T12:01:43.946959+0000","last_active":"2023-11-28T12:01:46.542711+0000","last_peered":"2023-11-28T12:01:46.542711+0000","last_clean":"2023-11-28T12:01:46.542711+0000","last_became_active":"2023-11-28T12:01:43.946674+0000","last_became_peered":"2023-11-28T12:01:43.946674+0000","last_unstale":"2023-11-28T12:01:46.542711+0000","last_undegraded":"2023-11-28T12:01:46.542711+0000","last_fullsized":"2023-11-28T12:01:46.542711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:18.910451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542620+0000","last_change":"2023-11-28T12:01:43.952292+0000","last_active":"2023-11-28T12:01:46.542620+0000","last_peered":"2023-11-28T12:01:46.542620+0000","last_clean":"2023-11-28T12:01:46.542620+0000","last_became_active":"2023-11-28T12:01:43.951948+0000","last_became_peered":"2023-11-28T12:01:43.951948+0000","last_unstale":"2023-11-28T12:01:46.542620+0000","last_undegraded":"2023-11-28T12:01:46.542620+0000","last_fullsized":"2023-11-28T12:01:46.542620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:30.672727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545157+0000","last_change":"2023-11-28T12:01:43.945700+0000","last_active":"2023-11-28T12:01:46.545157+0000","last_peered":"2023-11-28T12:01:46.545157+0000","last_clean":"2023-11-28T12:01:46.545157+0000","last_became_active":"2023-11-28T12:01:43.943097+0000","last_became_peered":"2023-11-28T12:01:43.943097+0000","last_unstale":"2023-11-28T12:01:46.545157+0000","last_undegraded":"2023-11-28T12:01:46.545157+0000","last_fullsized":"2023-11-28T12:01:46.545157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:36.019691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853575+0000","last_change":"2023-11-28T12:01:43.951123+0000","last_active":"2023-11-28T12:01:45.853575+0000","last_peered":"2023-11-28T12:01:45.853575+0000","last_clean":"2023-11-28T12:01:45.853575+0000","last_became_active":"2023-11-28T12:01:43.950867+0000","last_became_peered":"2023-11-28T12:01:43.950867+0000","last_unstale":"2023-11-28T12:01:45.853575+0000","last_undegraded":"2023-11-28T12:01:45.853575+0000","last_fullsized":"2023-11-28T12:01:45.853575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:32.123066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937717+0000","last_change":"2023-11-28T12:01:43.942191+0000","last_active":"2023-11-28T12:01:44.937717+0000","last_peered":"2023-11-28T12:01:44.937717+0000","last_clean":"2023-11-28T12:01:44.937717+0000","last_became_active":"2023-11-28T12:01:43.941576+0000","last_became_peered":"2023-11-28T12:01:43.941576+0000","last_unstale":"2023-11-28T12:01:44.937717+0000","last_undegraded":"2023-11-28T12:01:44.937717+0000","last_fullsized":"2023-11-28T12:01:44.937717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:43.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497887+0000","last_change":"2023-11-28T12:01:43.954787+0000","last_active":"2023-11-28T12:01:46.497887+0000","last_peered":"2023-11-28T12:01:46.497887+0000","last_clean":"2023-11-28T12:01:46.497887+0000","last_became_active":"2023-11-28T12:01:43.954463+0000","last_became_peered":"2023-11-28T12:01:43.954463+0000","last_unstale":"2023-11-28T12:01:46.497887+0000","last_undegraded":"2023-11-28T12:01:46.497887+0000","last_fullsized":"2023-11-28T12:01:46.497887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:23.540287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938451+0000","last_change":"2023-11-28T12:01:43.939593+0000","last_active":"2023-11-28T12:01:44.938451+0000","last_peered":"2023-11-28T12:01:44.938451+0000","last_clean":"2023-11-28T12:01:44.938451+0000","last_became_active":"2023-11-28T12:01:43.937190+0000","last_became_peered":"2023-11-28T12:01:43.937190+0000","last_unstale":"2023-11-28T12:01:44.938451+0000","last_undegraded":"2023-11-28T12:01:44.938451+0000","last_fullsized":"2023-11-28T12:01:44.938451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:16.230349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542842+0000","last_change":"2023-11-28T12:01:43.942910+0000","last_active":"2023-11-28T12:01:46.542842+0000","last_peered":"2023-11-28T12:01:46.542842+0000","last_clean":"2023-11-28T12:01:46.542842+0000","last_became_active":"2023-11-28T12:01:43.942609+0000","last_became_peered":"2023-11-28T12:01:43.942609+0000","last_unstale":"2023-11-28T12:01:46.542842+0000","last_undegraded":"2023-11-28T12:01:46.542842+0000","last_fullsized":"2023-11-28T12:01:46.542842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:37.541651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938215+0000","last_change":"2023-11-28T12:01:43.945798+0000","last_active":"2023-11-28T12:01:44.938215+0000","last_peered":"2023-11-28T12:01:44.938215+0000","last_clean":"2023-11-28T12:01:44.938215+0000","last_became_active":"2023-11-28T12:01:43.945473+0000","last_became_peered":"2023-11-28T12:01:43.945473+0000","last_unstale":"2023-11-28T12:01:44.938215+0000","last_undegraded":"2023-11-28T12:01:44.938215+0000","last_fullsized":"2023-11-28T12:01:44.938215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:29.033626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853662+0000","last_change":"2023-11-28T12:01:43.954303+0000","last_active":"2023-11-28T12:01:45.853662+0000","last_peered":"2023-11-28T12:01:45.853662+0000","last_clean":"2023-11-28T12:01:45.853662+0000","last_became_active":"2023-11-28T12:01:43.954076+0000","last_became_peered":"2023-11-28T12:01:43.954076+0000","last_unstale":"2023-11-28T12:01:45.853662+0000","last_undegraded":"2023-11-28T12:01:45.853662+0000","last_fullsized":"2023-11-28T12:01:45.853662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:41.736417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497578+0000","last_change":"2023-11-28T12:01:43.946172+0000","last_active":"2023-11-28T12:01:46.497578+0000","last_peered":"2023-11-28T12:01:46.497578+0000","last_clean":"2023-11-28T12:01:46.497578+0000","last_became_active":"2023-11-28T12:01:43.945881+0000","last_became_peered":"2023-11-28T12:01:43.945881+0000","last_unstale":"2023-11-28T12:01:46.497578+0000","last_undegraded":"2023-11-28T12:01:46.497578+0000","last_fullsized":"2023-11-28T12:01:46.497578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:43.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938826+0000","last_change":"2023-11-28T12:01:43.951156+0000","last_active":"2023-11-28T12:01:44.938826+0000","last_peered":"2023-11-28T12:01:44.938826+0000","last_clean":"2023-11-28T12:01:44.938826+0000","last_became_active":"2023-11-28T12:01:43.950873+0000","last_became_peered":"2023-11-28T12:01:43.950873+0000","last_unstale":"2023-11-28T12:01:44.938826+0000","last_undegraded":"2023-11-28T12:01:44.938826+0000","last_fullsized":"2023-11-28T12:01:44.938826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:32.718654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853513+0000","last_change":"2023-11-28T12:01:43.953938+0000","last_active":"2023-11-28T12:01:45.853513+0000","last_peered":"2023-11-28T12:01:45.853513+0000","last_clean":"2023-11-28T12:01:45.853513+0000","last_became_active":"2023-11-28T12:01:43.953649+0000","last_became_peered":"2023-11-28T12:01:43.953649+0000","last_unstale":"2023-11-28T12:01:45.853513+0000","last_undegraded":"2023-11-28T12:01:45.853513+0000","last_fullsized":"2023-11-28T12:01:45.853513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923145+0000","last_change":"2023-11-28T12:01:43.941856+0000","last_active":"2023-11-28T12:01:45.923145+0000","last_peered":"2023-11-28T12:01:45.923145+0000","last_clean":"2023-11-28T12:01:45.923145+0000","last_became_active":"2023-11-28T12:01:43.941555+0000","last_became_peered":"2023-11-28T12:01:43.941555+0000","last_unstale":"2023-11-28T12:01:45.923145+0000","last_undegraded":"2023-11-28T12:01:45.923145+0000","last_fullsized":"2023-11-28T12:01:45.923145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:01.126989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923247+0000","last_change":"2023-11-28T12:01:43.940084+0000","last_active":"2023-11-28T12:01:45.923247+0000","last_peered":"2023-11-28T12:01:45.923247+0000","last_clean":"2023-11-28T12:01:45.923247+0000","last_became_active":"2023-11-28T12:01:43.939856+0000","last_became_peered":"2023-11-28T12:01:43.939856+0000","last_unstale":"2023-11-28T12:01:45.923247+0000","last_undegraded":"2023-11-28T12:01:45.923247+0000","last_fullsized":"2023-11-28T12:01:45.923247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:50.516316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545353+0000","last_change":"2023-11-28T12:01:43.943624+0000","last_active":"2023-11-28T12:01:46.545353+0000","last_peered":"2023-11-28T12:01:46.545353+0000","last_clean":"2023-11-28T12:01:46.545353+0000","last_became_active":"2023-11-28T12:01:43.943313+0000","last_became_peered":"2023-11-28T12:01:43.943313+0000","last_unstale":"2023-11-28T12:01:46.545353+0000","last_undegraded":"2023-11-28T12:01:46.545353+0000","last_fullsized":"2023-11-28T12:01:46.545353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:10.298310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938040+0000","last_change":"2023-11-28T12:01:43.500890+0000","last_active":"2023-11-28T12:01:44.938040+0000","last_peered":"2023-11-28T12:01:44.938040+0000","last_clean":"2023-11-28T12:01:44.938040+0000","last_became_active":"2023-11-28T12:01:43.500621+0000","last_became_peered":"2023-11-28T12:01:43.500621+0000","last_unstale":"2023-11-28T12:01:44.938040+0000","last_undegraded":"2023-11-28T12:01:44.938040+0000","last_fullsized":"2023-11-28T12:01:44.938040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939156+0000","last_change":"2023-11-28T12:01:43.951003+0000","last_active":"2023-11-28T12:01:44.939156+0000","last_peered":"2023-11-28T12:01:44.939156+0000","last_clean":"2023-11-28T12:01:44.939156+0000","last_became_active":"2023-11-28T12:01:43.950717+0000","last_became_peered":"2023-11-28T12:01:43.950717+0000","last_unstale":"2023-11-28T12:01:44.939156+0000","last_undegraded":"2023-11-28T12:01:44.939156+0000","last_fullsized":"2023-11-28T12:01:44.939156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:32.123066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922884+0000","last_change":"2023-11-28T12:01:43.500086+0000","last_active":"2023-11-28T12:01:45.922884+0000","last_peered":"2023-11-28T12:01:45.922884+0000","last_clean":"2023-11-28T12:01:45.922884+0000","last_became_active":"2023-11-28T12:01:43.499317+0000","last_became_peered":"2023-11-28T12:01:43.499317+0000","last_unstale":"2023-11-28T12:01:45.922884+0000","last_undegraded":"2023-11-28T12:01:45.922884+0000","last_fullsized":"2023-11-28T12:01:45.922884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.498026+0000","last_change":"2023-11-28T12:01:43.952329+0000","last_active":"2023-11-28T12:01:46.498026+0000","last_peered":"2023-11-28T12:01:46.498026+0000","last_clean":"2023-11-28T12:01:46.498026+0000","last_became_active":"2023-11-28T12:01:43.952003+0000","last_became_peered":"2023-11-28T12:01:43.952003+0000","last_unstale":"2023-11-28T12:01:46.498026+0000","last_undegraded":"2023-11-28T12:01:46.498026+0000","last_fullsized":"2023-11-28T12:01:46.498026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938921+0000","last_change":"2023-11-28T12:01:43.499958+0000","last_active":"2023-11-28T12:01:44.938921+0000","last_peered":"2023-11-28T12:01:44.938921+0000","last_clean":"2023-11-28T12:01:44.938921+0000","last_became_active":"2023-11-28T12:01:43.499623+0000","last_became_peered":"2023-11-28T12:01:43.499623+0000","last_unstale":"2023-11-28T12:01:44.938921+0000","last_undegraded":"2023-11-28T12:01:44.938921+0000","last_fullsized":"2023-11-28T12:01:44.938921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:46.276522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937707+0000","last_change":"2023-11-28T12:01:43.945390+0000","last_active":"2023-11-28T12:01:44.937707+0000","last_peered":"2023-11-28T12:01:44.937707+0000","last_clean":"2023-11-28T12:01:44.937707+0000","last_became_active":"2023-11-28T12:01:43.945125+0000","last_became_peered":"2023-11-28T12:01:43.945125+0000","last_unstale":"2023-11-28T12:01:44.937707+0000","last_undegraded":"2023-11-28T12:01:44.937707+0000","last_fullsized":"2023-11-28T12:01:44.937707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:01.126989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497989+0000","last_change":"2023-11-28T12:01:43.497913+0000","last_active":"2023-11-28T12:01:46.497989+0000","last_peered":"2023-11-28T12:01:46.497989+0000","last_clean":"2023-11-28T12:01:46.497989+0000","last_became_active":"2023-11-28T12:01:43.497597+0000","last_became_peered":"2023-11-28T12:01:43.497597+0000","last_unstale":"2023-11-28T12:01:46.497989+0000","last_undegraded":"2023-11-28T12:01:46.497989+0000","last_fullsized":"2023-11-28T12:01:46.497989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938856+0000","last_change":"2023-11-28T12:01:43.938776+0000","last_active":"2023-11-28T12:01:44.938856+0000","last_peered":"2023-11-28T12:01:44.938856+0000","last_clean":"2023-11-28T12:01:44.938856+0000","last_became_active":"2023-11-28T12:01:43.938258+0000","last_became_peered":"2023-11-28T12:01:43.938258+0000","last_unstale":"2023-11-28T12:01:44.938856+0000","last_undegraded":"2023-11-28T12:01:44.938856+0000","last_fullsized":"2023-11-28T12:01:44.938856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:10.443506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545384+0000","last_change":"2023-11-28T12:01:43.494977+0000","last_active":"2023-11-28T12:01:46.545384+0000","last_peered":"2023-11-28T12:01:46.545384+0000","last_clean":"2023-11-28T12:01:46.545384+0000","last_became_active":"2023-11-28T12:01:43.494659+0000","last_became_peered":"2023-11-28T12:01:43.494659+0000","last_unstale":"2023-11-28T12:01:46.545384+0000","last_undegraded":"2023-11-28T12:01:46.545384+0000","last_fullsized":"2023-11-28T12:01:46.545384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:29.667067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938938+0000","last_change":"2023-11-28T12:01:43.949219+0000","last_active":"2023-11-28T12:01:44.938938+0000","last_peered":"2023-11-28T12:01:44.938938+0000","last_clean":"2023-11-28T12:01:44.938938+0000","last_became_active":"2023-11-28T12:01:43.948599+0000","last_became_peered":"2023-11-28T12:01:43.948599+0000","last_unstale":"2023-11-28T12:01:44.938938+0000","last_undegraded":"2023-11-28T12:01:44.938938+0000","last_fullsized":"2023-11-28T12:01:44.938938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:43.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853768+0000","last_change":"2023-11-28T12:01:43.504068+0000","last_active":"2023-11-28T12:01:45.853768+0000","last_peered":"2023-11-28T12:01:45.853768+0000","last_clean":"2023-11-28T12:01:45.853768+0000","last_became_active":"2023-11-28T12:01:43.503819+0000","last_became_peered":"2023-11-28T12:01:43.503819+0000","last_unstale":"2023-11-28T12:01:45.853768+0000","last_undegraded":"2023-11-28T12:01:45.853768+0000","last_fullsized":"2023-11-28T12:01:45.853768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:49.510656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922795+0000","last_change":"2023-11-28T12:01:43.941737+0000","last_active":"2023-11-28T12:01:45.922795+0000","last_peered":"2023-11-28T12:01:45.922795+0000","last_clean":"2023-11-28T12:01:45.922795+0000","last_became_active":"2023-11-28T12:01:43.941456+0000","last_became_peered":"2023-11-28T12:01:43.941456+0000","last_unstale":"2023-11-28T12:01:45.922795+0000","last_undegraded":"2023-11-28T12:01:45.922795+0000","last_fullsized":"2023-11-28T12:01:45.922795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:36.019691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938329+0000","last_change":"2023-11-28T12:01:43.492482+0000","last_active":"2023-11-28T12:01:44.938329+0000","last_peered":"2023-11-28T12:01:44.938329+0000","last_clean":"2023-11-28T12:01:44.938329+0000","last_became_active":"2023-11-28T12:01:43.492180+0000","last_became_peered":"2023-11-28T12:01:43.492180+0000","last_unstale":"2023-11-28T12:01:44.938329+0000","last_undegraded":"2023-11-28T12:01:44.938329+0000","last_fullsized":"2023-11-28T12:01:44.938329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938187+0000","last_change":"2023-11-28T12:01:43.944780+0000","last_active":"2023-11-28T12:01:44.938187+0000","last_peered":"2023-11-28T12:01:44.938187+0000","last_clean":"2023-11-28T12:01:44.938187+0000","last_became_active":"2023-11-28T12:01:43.944467+0000","last_became_peered":"2023-11-28T12:01:43.944467+0000","last_unstale":"2023-11-28T12:01:44.938187+0000","last_undegraded":"2023-11-28T12:01:44.938187+0000","last_fullsized":"2023-11-28T12:01:44.938187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:57.245235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853599+0000","last_change":"2023-11-28T12:01:43.499400+0000","last_active":"2023-11-28T12:01:45.853599+0000","last_peered":"2023-11-28T12:01:45.853599+0000","last_clean":"2023-11-28T12:01:45.853599+0000","last_became_active":"2023-11-28T12:01:43.499116+0000","last_became_peered":"2023-11-28T12:01:43.499116+0000","last_unstale":"2023-11-28T12:01:45.853599+0000","last_undegraded":"2023-11-28T12:01:45.853599+0000","last_fullsized":"2023-11-28T12:01:45.853599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:00.731296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939194+0000","last_change":"2023-11-28T12:01:43.954953+0000","last_active":"2023-11-28T12:01:44.939194+0000","last_peered":"2023-11-28T12:01:44.939194+0000","last_clean":"2023-11-28T12:01:44.939194+0000","last_became_active":"2023-11-28T12:01:43.952565+0000","last_became_peered":"2023-11-28T12:01:43.952565+0000","last_unstale":"2023-11-28T12:01:44.939194+0000","last_undegraded":"2023-11-28T12:01:44.939194+0000","last_fullsized":"2023-11-28T12:01:44.939194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:29.033626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497689+0000","last_change":"2023-11-28T12:01:43.496294+0000","last_active":"2023-11-28T12:01:46.497689+0000","last_peered":"2023-11-28T12:01:46.497689+0000","last_clean":"2023-11-28T12:01:46.497689+0000","last_became_active":"2023-11-28T12:01:43.493574+0000","last_became_peered":"2023-11-28T12:01:43.493574+0000","last_unstale":"2023-11-28T12:01:46.497689+0000","last_undegraded":"2023-11-28T12:01:46.497689+0000","last_fullsized":"2023-11-28T12:01:46.497689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545401+0000","last_change":"2023-11-28T12:01:43.942727+0000","last_active":"2023-11-28T12:01:46.545401+0000","last_peered":"2023-11-28T12:01:46.545401+0000","last_clean":"2023-11-28T12:01:46.545401+0000","last_became_active":"2023-11-28T12:01:43.942041+0000","last_became_peered":"2023-11-28T12:01:43.942041+0000","last_unstale":"2023-11-28T12:01:46.545401+0000","last_undegraded":"2023-11-28T12:01:46.545401+0000","last_fullsized":"2023-11-28T12:01:46.545401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:23.540287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853448+0000","last_change":"2023-11-28T12:01:43.499858+0000","last_active":"2023-11-28T12:01:45.853448+0000","last_peered":"2023-11-28T12:01:45.853448+0000","last_clean":"2023-11-28T12:01:45.853448+0000","last_became_active":"2023-11-28T12:01:43.499006+0000","last_became_peered":"2023-11-28T12:01:43.499006+0000","last_unstale":"2023-11-28T12:01:45.853448+0000","last_undegraded":"2023-11-28T12:01:45.853448+0000","last_fullsized":"2023-11-28T12:01:45.853448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:41:02.952529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497880+0000","last_change":"2023-11-28T12:01:43.950190+0000","last_active":"2023-11-28T12:01:46.497880+0000","last_peered":"2023-11-28T12:01:46.497880+0000","last_clean":"2023-11-28T12:01:46.497880+0000","last_became_active":"2023-11-28T12:01:43.949923+0000","last_became_peered":"2023-11-28T12:01:43.949923+0000","last_unstale":"2023-11-28T12:01:46.497880+0000","last_undegraded":"2023-11-28T12:01:46.497880+0000","last_fullsized":"2023-11-28T12:01:46.497880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:32.123066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937355+0000","last_change":"2023-11-28T12:01:43.493841+0000","last_active":"2023-11-28T12:01:44.937355+0000","last_peered":"2023-11-28T12:01:44.937355+0000","last_clean":"2023-11-28T12:01:44.937355+0000","last_became_active":"2023-11-28T12:01:43.493559+0000","last_became_peered":"2023-11-28T12:01:43.493559+0000","last_unstale":"2023-11-28T12:01:44.937355+0000","last_undegraded":"2023-11-28T12:01:44.937355+0000","last_fullsized":"2023-11-28T12:01:44.937355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939401+0000","last_change":"2023-11-28T12:01:43.954120+0000","last_active":"2023-11-28T12:01:44.939401+0000","last_peered":"2023-11-28T12:01:44.939401+0000","last_clean":"2023-11-28T12:01:44.939401+0000","last_became_active":"2023-11-28T12:01:43.953634+0000","last_became_peered":"2023-11-28T12:01:43.953634+0000","last_unstale":"2023-11-28T12:01:44.939401+0000","last_undegraded":"2023-11-28T12:01:44.939401+0000","last_fullsized":"2023-11-28T12:01:44.939401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:46.302273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853794+0000","last_change":"2023-11-28T12:01:43.499686+0000","last_active":"2023-11-28T12:01:45.853794+0000","last_peered":"2023-11-28T12:01:45.853794+0000","last_clean":"2023-11-28T12:01:45.853794+0000","last_became_active":"2023-11-28T12:01:43.499379+0000","last_became_peered":"2023-11-28T12:01:43.499379+0000","last_unstale":"2023-11-28T12:01:45.853794+0000","last_undegraded":"2023-11-28T12:01:45.853794+0000","last_fullsized":"2023-11-28T12:01:45.853794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:07.361707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545042+0000","last_change":"2023-11-28T12:01:43.943016+0000","last_active":"2023-11-28T12:01:46.545042+0000","last_peered":"2023-11-28T12:01:46.545042+0000","last_clean":"2023-11-28T12:01:46.545042+0000","last_became_active":"2023-11-28T12:01:43.941928+0000","last_became_peered":"2023-11-28T12:01:43.941928+0000","last_unstale":"2023-11-28T12:01:46.545042+0000","last_undegraded":"2023-11-28T12:01:46.545042+0000","last_fullsized":"2023-11-28T12:01:46.545042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:01.126989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853782+0000","last_change":"2023-11-28T12:01:43.499257+0000","last_active":"2023-11-28T12:01:45.853782+0000","last_peered":"2023-11-28T12:01:45.853782+0000","last_clean":"2023-11-28T12:01:45.853782+0000","last_became_active":"2023-11-28T12:01:43.498973+0000","last_became_peered":"2023-11-28T12:01:43.498973+0000","last_unstale":"2023-11-28T12:01:45.853782+0000","last_undegraded":"2023-11-28T12:01:45.853782+0000","last_fullsized":"2023-11-28T12:01:45.853782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:46:41.932790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937789+0000","last_change":"2023-11-28T12:01:43.937615+0000","last_active":"2023-11-28T12:01:44.937789+0000","last_peered":"2023-11-28T12:01:44.937789+0000","last_clean":"2023-11-28T12:01:44.937789+0000","last_became_active":"2023-11-28T12:01:43.937115+0000","last_became_peered":"2023-11-28T12:01:43.937115+0000","last_unstale":"2023-11-28T12:01:44.937789+0000","last_undegraded":"2023-11-28T12:01:44.937789+0000","last_fullsized":"2023-11-28T12:01:44.937789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:51.664836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.853717+0000","last_change":"2023-11-28T12:01:43.496645+0000","last_active":"2023-11-28T12:01:45.853717+0000","last_peered":"2023-11-28T12:01:45.853717+0000","last_clean":"2023-11-28T12:01:45.853717+0000","last_became_active":"2023-11-28T12:01:43.496393+0000","last_became_peered":"2023-11-28T12:01:43.496393+0000","last_unstale":"2023-11-28T12:01:45.853717+0000","last_undegraded":"2023-11-28T12:01:45.853717+0000","last_fullsized":"2023-11-28T12:01:45.853717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:18.701227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937935+0000","last_change":"2023-11-28T12:01:43.946845+0000","last_active":"2023-11-28T12:01:44.937935+0000","last_peered":"2023-11-28T12:01:44.937935+0000","last_clean":"2023-11-28T12:01:44.937935+0000","last_became_active":"2023-11-28T12:01:43.946572+0000","last_became_peered":"2023-11-28T12:01:43.946572+0000","last_unstale":"2023-11-28T12:01:44.937935+0000","last_undegraded":"2023-11-28T12:01:44.937935+0000","last_fullsized":"2023-11-28T12:01:44.937935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545284+0000","last_change":"2023-11-28T12:01:43.500124+0000","last_active":"2023-11-28T12:01:46.545284+0000","last_peered":"2023-11-28T12:01:46.545284+0000","last_clean":"2023-11-28T12:01:46.545284+0000","last_became_active":"2023-11-28T12:01:43.499857+0000","last_became_peered":"2023-11-28T12:01:43.499857+0000","last_unstale":"2023-11-28T12:01:46.545284+0000","last_undegraded":"2023-11-28T12:01:46.545284+0000","last_fullsized":"2023-11-28T12:01:46.545284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497627+0000","last_change":"2023-11-28T12:01:43.945003+0000","last_active":"2023-11-28T12:01:46.497627+0000","last_peered":"2023-11-28T12:01:46.497627+0000","last_clean":"2023-11-28T12:01:46.497627+0000","last_became_active":"2023-11-28T12:01:43.944725+0000","last_became_peered":"2023-11-28T12:01:43.944725+0000","last_unstale":"2023-11-28T12:01:46.497627+0000","last_undegraded":"2023-11-28T12:01:46.497627+0000","last_fullsized":"2023-11-28T12:01:46.497627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:46.302273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939198+0000","last_change":"2023-11-28T12:01:43.500073+0000","last_active":"2023-11-28T12:01:44.939198+0000","last_peered":"2023-11-28T12:01:44.939198+0000","last_clean":"2023-11-28T12:01:44.939198+0000","last_became_active":"2023-11-28T12:01:43.499770+0000","last_became_peered":"2023-11-28T12:01:43.499770+0000","last_unstale":"2023-11-28T12:01:44.939198+0000","last_undegraded":"2023-11-28T12:01:44.939198+0000","last_fullsized":"2023-11-28T12:01:44.939198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938256+0000","last_change":"2023-11-28T12:01:43.954024+0000","last_active":"2023-11-28T12:01:44.938256+0000","last_peered":"2023-11-28T12:01:44.938256+0000","last_clean":"2023-11-28T12:01:44.938256+0000","last_became_active":"2023-11-28T12:01:43.953743+0000","last_became_peered":"2023-11-28T12:01:43.953743+0000","last_unstale":"2023-11-28T12:01:44.938256+0000","last_undegraded":"2023-11-28T12:01:44.938256+0000","last_fullsized":"2023-11-28T12:01:44.938256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:35.755196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937423+0000","last_change":"2023-11-28T12:01:43.491888+0000","last_active":"2023-11-28T12:01:44.937423+0000","last_peered":"2023-11-28T12:01:44.937423+0000","last_clean":"2023-11-28T12:01:44.937423+0000","last_became_active":"2023-11-28T12:01:43.491585+0000","last_became_peered":"2023-11-28T12:01:43.491585+0000","last_unstale":"2023-11-28T12:01:44.937423+0000","last_undegraded":"2023-11-28T12:01:44.937423+0000","last_fullsized":"2023-11-28T12:01:44.937423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938535+0000","last_change":"2023-11-28T12:01:43.940969+0000","last_active":"2023-11-28T12:01:44.938535+0000","last_peered":"2023-11-28T12:01:44.938535+0000","last_clean":"2023-11-28T12:01:44.938535+0000","last_became_active":"2023-11-28T12:01:43.940012+0000","last_became_peered":"2023-11-28T12:01:43.940012+0000","last_unstale":"2023-11-28T12:01:44.938535+0000","last_undegraded":"2023-11-28T12:01:44.938535+0000","last_fullsized":"2023-11-28T12:01:44.938535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:20:45.411292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937934+0000","last_change":"2023-11-28T12:01:43.493475+0000","last_active":"2023-11-28T12:01:44.937934+0000","last_peered":"2023-11-28T12:01:44.937934+0000","last_clean":"2023-11-28T12:01:44.937934+0000","last_became_active":"2023-11-28T12:01:43.493206+0000","last_became_peered":"2023-11-28T12:01:43.493206+0000","last_unstale":"2023-11-28T12:01:44.937934+0000","last_undegraded":"2023-11-28T12:01:44.937934+0000","last_fullsized":"2023-11-28T12:01:44.937934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938568+0000","last_change":"2023-11-28T12:01:43.944744+0000","last_active":"2023-11-28T12:01:44.938568+0000","last_peered":"2023-11-28T12:01:44.938568+0000","last_clean":"2023-11-28T12:01:44.938568+0000","last_became_active":"2023-11-28T12:01:43.943939+0000","last_became_peered":"2023-11-28T12:01:43.943939+0000","last_unstale":"2023-11-28T12:01:44.938568+0000","last_undegraded":"2023-11-28T12:01:44.938568+0000","last_fullsized":"2023-11-28T12:01:44.938568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:10.443506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938813+0000","last_change":"2023-11-28T12:01:43.497932+0000","last_active":"2023-11-28T12:01:44.938813+0000","last_peered":"2023-11-28T12:01:44.938813+0000","last_clean":"2023-11-28T12:01:44.938813+0000","last_became_active":"2023-11-28T12:01:43.497623+0000","last_became_peered":"2023-11-28T12:01:43.497623+0000","last_unstale":"2023-11-28T12:01:44.938813+0000","last_undegraded":"2023-11-28T12:01:44.938813+0000","last_fullsized":"2023-11-28T12:01:44.938813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:46:41.932790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923287+0000","last_change":"2023-11-28T12:01:43.943517+0000","last_active":"2023-11-28T12:01:45.923287+0000","last_peered":"2023-11-28T12:01:45.923287+0000","last_clean":"2023-11-28T12:01:45.923287+0000","last_became_active":"2023-11-28T12:01:43.943220+0000","last_became_peered":"2023-11-28T12:01:43.943220+0000","last_unstale":"2023-11-28T12:01:45.923287+0000","last_undegraded":"2023-11-28T12:01:45.923287+0000","last_fullsized":"2023-11-28T12:01:45.923287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:28.507708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939358+0000","last_change":"2023-11-28T12:01:43.495923+0000","last_active":"2023-11-28T12:01:44.939358+0000","last_peered":"2023-11-28T12:01:44.939358+0000","last_clean":"2023-11-28T12:01:44.939358+0000","last_became_active":"2023-11-28T12:01:43.494593+0000","last_became_peered":"2023-11-28T12:01:43.494593+0000","last_unstale":"2023-11-28T12:01:44.939358+0000","last_undegraded":"2023-11-28T12:01:44.939358+0000","last_fullsized":"2023-11-28T12:01:44.939358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:19.566285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.928332+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:01:43.928332+0000","last_peered":"2023-11-28T12:01:43.928332+0000","last_clean":"2023-11-28T12:01:43.928332+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:01:43.928332+0000","last_undegraded":"2023-11-28T12:01:43.928332+0000","last_fullsized":"2023-11-28T12:01:43.928332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938743+0000","last_change":"2023-11-28T12:01:43.938577+0000","last_active":"2023-11-28T12:01:44.938743+0000","last_peered":"2023-11-28T12:01:44.938743+0000","last_clean":"2023-11-28T12:01:44.938743+0000","last_became_active":"2023-11-28T12:01:43.937923+0000","last_became_peered":"2023-11-28T12:01:43.937923+0000","last_unstale":"2023-11-28T12:01:44.938743+0000","last_undegraded":"2023-11-28T12:01:44.938743+0000","last_fullsized":"2023-11-28T12:01:44.938743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:40.988886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.928705+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:01:43.928705+0000","last_peered":"2023-11-28T12:01:43.928705+0000","last_clean":"2023-11-28T12:01:43.928705+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:01:43.928705+0000","last_undegraded":"2023-11-28T12:01:43.928705+0000","last_fullsized":"2023-11-28T12:01:43.928705+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.734302+0000","last_change":"2023-11-28T12:01:43.500640+0000","last_active":"2023-11-28T12:01:46.734302+0000","last_peered":"2023-11-28T12:01:46.734302+0000","last_clean":"2023-11-28T12:01:46.734302+0000","last_became_active":"2023-11-28T12:01:43.500348+0000","last_became_peered":"2023-11-28T12:01:43.500348+0000","last_unstale":"2023-11-28T12:01:46.734302+0000","last_undegraded":"2023-11-28T12:01:46.734302+0000","last_fullsized":"2023-11-28T12:01:46.734302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:40.727625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937843+0000","last_change":"2023-11-28T12:01:43.944208+0000","last_active":"2023-11-28T12:01:44.937843+0000","last_peered":"2023-11-28T12:01:44.937843+0000","last_clean":"2023-11-28T12:01:44.937843+0000","last_became_active":"2023-11-28T12:01:43.943957+0000","last_became_peered":"2023-11-28T12:01:43.943957+0000","last_unstale":"2023-11-28T12:01:44.937843+0000","last_undegraded":"2023-11-28T12:01:44.937843+0000","last_fullsized":"2023-11-28T12:01:44.937843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:54:35.260564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.929563+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:01:43.929563+0000","last_peered":"2023-11-28T12:01:43.929563+0000","last_clean":"2023-11-28T12:01:43.929563+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:01:43.929563+0000","last_undegraded":"2023-11-28T12:01:43.929563+0000","last_fullsized":"2023-11-28T12:01:43.929563+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.934971+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:01:43.934971+0000","last_peered":"2023-11-28T12:01:43.934971+0000","last_clean":"2023-11-28T12:01:43.934971+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:01:43.934971+0000","last_undegraded":"2023-11-28T12:01:43.934971+0000","last_fullsized":"2023-11-28T12:01:43.934971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497590+0000","last_change":"2023-11-28T12:01:43.498540+0000","last_active":"2023-11-28T12:01:46.497590+0000","last_peered":"2023-11-28T12:01:46.497590+0000","last_clean":"2023-11-28T12:01:46.497590+0000","last_became_active":"2023-11-28T12:01:43.498040+0000","last_became_peered":"2023-11-28T12:01:43.498040+0000","last_unstale":"2023-11-28T12:01:46.497590+0000","last_undegraded":"2023-11-28T12:01:46.497590+0000","last_fullsized":"2023-11-28T12:01:46.497590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:29.667067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497902+0000","last_change":"2023-11-28T12:01:43.946239+0000","last_active":"2023-11-28T12:01:46.497902+0000","last_peered":"2023-11-28T12:01:46.497902+0000","last_clean":"2023-11-28T12:01:46.497902+0000","last_became_active":"2023-11-28T12:01:43.945937+0000","last_became_peered":"2023-11-28T12:01:43.945937+0000","last_unstale":"2023-11-28T12:01:46.497902+0000","last_undegraded":"2023-11-28T12:01:46.497902+0000","last_fullsized":"2023-11-28T12:01:46.497902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:40.988886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.935299+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:01:43.935299+0000","last_peered":"2023-11-28T12:01:43.935299+0000","last_clean":"2023-11-28T12:01:43.935299+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:01:43.935299+0000","last_undegraded":"2023-11-28T12:01:43.935299+0000","last_fullsized":"2023-11-28T12:01:43.935299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922707+0000","last_change":"2023-11-28T12:01:43.495719+0000","last_active":"2023-11-28T12:01:45.922707+0000","last_peered":"2023-11-28T12:01:45.922707+0000","last_clean":"2023-11-28T12:01:45.922707+0000","last_became_active":"2023-11-28T12:01:43.494865+0000","last_became_peered":"2023-11-28T12:01:43.494865+0000","last_unstale":"2023-11-28T12:01:45.922707+0000","last_undegraded":"2023-11-28T12:01:45.922707+0000","last_fullsized":"2023-11-28T12:01:45.922707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:29.667067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923227+0000","last_change":"2023-11-28T12:01:43.946289+0000","last_active":"2023-11-28T12:01:45.923227+0000","last_peered":"2023-11-28T12:01:45.923227+0000","last_clean":"2023-11-28T12:01:45.923227+0000","last_became_active":"2023-11-28T12:01:43.945759+0000","last_became_peered":"2023-11-28T12:01:43.945759+0000","last_unstale":"2023-11-28T12:01:45.923227+0000","last_undegraded":"2023-11-28T12:01:45.923227+0000","last_fullsized":"2023-11-28T12:01:45.923227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:41.736417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.934867+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:01:43.934867+0000","last_peered":"2023-11-28T12:01:43.934867+0000","last_clean":"2023-11-28T12:01:43.934867+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:01:43.934867+0000","last_undegraded":"2023-11-28T12:01:43.934867+0000","last_fullsized":"2023-11-28T12:01:43.934867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938960+0000","last_change":"2023-11-28T12:01:43.496920+0000","last_active":"2023-11-28T12:01:44.938960+0000","last_peered":"2023-11-28T12:01:44.938960+0000","last_clean":"2023-11-28T12:01:44.938960+0000","last_became_active":"2023-11-28T12:01:43.495644+0000","last_became_peered":"2023-11-28T12:01:43.495644+0000","last_unstale":"2023-11-28T12:01:44.938960+0000","last_undegraded":"2023-11-28T12:01:44.938960+0000","last_fullsized":"2023-11-28T12:01:44.938960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937511+0000","last_change":"2023-11-28T12:01:43.950061+0000","last_active":"2023-11-28T12:01:44.937511+0000","last_peered":"2023-11-28T12:01:44.937511+0000","last_clean":"2023-11-28T12:01:44.937511+0000","last_became_active":"2023-11-28T12:01:43.949778+0000","last_became_peered":"2023-11-28T12:01:43.949778+0000","last_unstale":"2023-11-28T12:01:44.937511+0000","last_undegraded":"2023-11-28T12:01:44.937511+0000","last_fullsized":"2023-11-28T12:01:44.937511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:36.019691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.932921+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:01:43.932921+0000","last_peered":"2023-11-28T12:01:43.932921+0000","last_clean":"2023-11-28T12:01:43.932921+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:01:43.932921+0000","last_undegraded":"2023-11-28T12:01:43.932921+0000","last_fullsized":"2023-11-28T12:01:43.932921+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938196+0000","last_change":"2023-11-28T12:01:43.499396+0000","last_active":"2023-11-28T12:01:44.938196+0000","last_peered":"2023-11-28T12:01:44.938196+0000","last_clean":"2023-11-28T12:01:44.938196+0000","last_became_active":"2023-11-28T12:01:43.499103+0000","last_became_peered":"2023-11-28T12:01:43.499103+0000","last_unstale":"2023-11-28T12:01:44.938196+0000","last_undegraded":"2023-11-28T12:01:44.938196+0000","last_fullsized":"2023-11-28T12:01:44.938196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923174+0000","last_change":"2023-11-28T12:01:43.941086+0000","last_active":"2023-11-28T12:01:45.923174+0000","last_peered":"2023-11-28T12:01:45.923174+0000","last_clean":"2023-11-28T12:01:45.923174+0000","last_became_active":"2023-11-28T12:01:43.940781+0000","last_became_peered":"2023-11-28T12:01:43.940781+0000","last_unstale":"2023-11-28T12:01:45.923174+0000","last_undegraded":"2023-11-28T12:01:45.923174+0000","last_fullsized":"2023-11-28T12:01:45.923174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:19.706888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.935713+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:01:43.935713+0000","last_peered":"2023-11-28T12:01:43.935713+0000","last_clean":"2023-11-28T12:01:43.935713+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:01:43.935713+0000","last_undegraded":"2023-11-28T12:01:43.935713+0000","last_fullsized":"2023-11-28T12:01:43.935713+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923245+0000","last_change":"2023-11-28T12:01:43.501876+0000","last_active":"2023-11-28T12:01:45.923245+0000","last_peered":"2023-11-28T12:01:45.923245+0000","last_clean":"2023-11-28T12:01:45.923245+0000","last_became_active":"2023-11-28T12:01:43.501509+0000","last_became_peered":"2023-11-28T12:01:43.501509+0000","last_unstale":"2023-11-28T12:01:45.923245+0000","last_undegraded":"2023-11-28T12:01:45.923245+0000","last_fullsized":"2023-11-28T12:01:45.923245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497880+0000","last_change":"2023-11-28T12:01:43.947364+0000","last_active":"2023-11-28T12:01:46.497880+0000","last_peered":"2023-11-28T12:01:46.497880+0000","last_clean":"2023-11-28T12:01:46.497880+0000","last_became_active":"2023-11-28T12:01:43.947043+0000","last_became_peered":"2023-11-28T12:01:43.947043+0000","last_unstale":"2023-11-28T12:01:46.497880+0000","last_undegraded":"2023-11-28T12:01:46.497880+0000","last_fullsized":"2023-11-28T12:01:46.497880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T12:01:43.936792+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:01:43.936792+0000","last_peered":"2023-11-28T12:01:43.936792+0000","last_clean":"2023-11-28T12:01:43.936792+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:01:43.936792+0000","last_undegraded":"2023-11-28T12:01:43.936792+0000","last_fullsized":"2023-11-28T12:01:43.936792+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545482+0000","last_change":"2023-11-28T12:01:43.497088+0000","last_active":"2023-11-28T12:01:46.545482+0000","last_peered":"2023-11-28T12:01:46.545482+0000","last_clean":"2023-11-28T12:01:46.545482+0000","last_became_active":"2023-11-28T12:01:43.496456+0000","last_became_peered":"2023-11-28T12:01:43.496456+0000","last_unstale":"2023-11-28T12:01:46.545482+0000","last_undegraded":"2023-11-28T12:01:46.545482+0000","last_fullsized":"2023-11-28T12:01:46.545482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:17.904790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.923037+0000","last_change":"2023-11-28T12:01:43.946582+0000","last_active":"2023-11-28T12:01:45.923037+0000","last_peered":"2023-11-28T12:01:45.923037+0000","last_clean":"2023-11-28T12:01:45.923037+0000","last_became_active":"2023-11-28T12:01:43.946281+0000","last_became_peered":"2023-11-28T12:01:43.946281+0000","last_unstale":"2023-11-28T12:01:45.923037+0000","last_undegraded":"2023-11-28T12:01:45.923037+0000","last_fullsized":"2023-11-28T12:01:45.923037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:10.298310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497709+0000","last_change":"2023-11-28T12:01:43.499312+0000","last_active":"2023-11-28T12:01:46.497709+0000","last_peered":"2023-11-28T12:01:46.497709+0000","last_clean":"2023-11-28T12:01:46.497709+0000","last_became_active":"2023-11-28T12:01:43.499020+0000","last_became_peered":"2023-11-28T12:01:43.499020+0000","last_unstale":"2023-11-28T12:01:46.497709+0000","last_undegraded":"2023-11-28T12:01:46.497709+0000","last_fullsized":"2023-11-28T12:01:46.497709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.542759+0000","last_change":"2023-11-28T12:01:43.945612+0000","last_active":"2023-11-28T12:01:46.542759+0000","last_peered":"2023-11-28T12:01:46.542759+0000","last_clean":"2023-11-28T12:01:46.542759+0000","last_became_active":"2023-11-28T12:01:43.945304+0000","last_became_peered":"2023-11-28T12:01:43.945304+0000","last_unstale":"2023-11-28T12:01:46.542759+0000","last_undegraded":"2023-11-28T12:01:46.542759+0000","last_fullsized":"2023-11-28T12:01:46.542759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:20.571945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545110+0000","last_change":"2023-11-28T12:01:43.496842+0000","last_active":"2023-11-28T12:01:46.545110+0000","last_peered":"2023-11-28T12:01:46.545110+0000","last_clean":"2023-11-28T12:01:46.545110+0000","last_became_active":"2023-11-28T12:01:43.496567+0000","last_became_peered":"2023-11-28T12:01:43.496567+0000","last_unstale":"2023-11-28T12:01:46.545110+0000","last_undegraded":"2023-11-28T12:01:46.545110+0000","last_fullsized":"2023-11-28T12:01:46.545110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:49.641983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938343+0000","last_change":"2023-11-28T12:01:43.941160+0000","last_active":"2023-11-28T12:01:44.938343+0000","last_peered":"2023-11-28T12:01:44.938343+0000","last_clean":"2023-11-28T12:01:44.938343+0000","last_became_active":"2023-11-28T12:01:43.940593+0000","last_became_peered":"2023-11-28T12:01:43.940593+0000","last_unstale":"2023-11-28T12:01:44.938343+0000","last_undegraded":"2023-11-28T12:01:44.938343+0000","last_fullsized":"2023-11-28T12:01:44.938343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:46.302273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.545507+0000","last_change":"2023-11-28T12:01:43.497657+0000","last_active":"2023-11-28T12:01:46.545507+0000","last_peered":"2023-11-28T12:01:46.545507+0000","last_clean":"2023-11-28T12:01:46.545507+0000","last_became_active":"2023-11-28T12:01:43.496814+0000","last_became_peered":"2023-11-28T12:01:43.496814+0000","last_unstale":"2023-11-28T12:01:46.545507+0000","last_undegraded":"2023-11-28T12:01:46.545507+0000","last_fullsized":"2023-11-28T12:01:46.545507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:37.840594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938078+0000","last_change":"2023-11-28T12:01:43.938233+0000","last_active":"2023-11-28T12:01:44.938078+0000","last_peered":"2023-11-28T12:01:44.938078+0000","last_clean":"2023-11-28T12:01:44.938078+0000","last_became_active":"2023-11-28T12:01:43.937825+0000","last_became_peered":"2023-11-28T12:01:43.937825+0000","last_unstale":"2023-11-28T12:01:44.938078+0000","last_undegraded":"2023-11-28T12:01:44.938078+0000","last_fullsized":"2023-11-28T12:01:44.938078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:04.509152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939417+0000","last_change":"2023-11-28T12:01:43.485835+0000","last_active":"2023-11-28T12:01:44.939417+0000","last_peered":"2023-11-28T12:01:44.939417+0000","last_clean":"2023-11-28T12:01:44.939417+0000","last_became_active":"2023-11-28T12:01:43.484610+0000","last_became_peered":"2023-11-28T12:01:43.484610+0000","last_unstale":"2023-11-28T12:01:44.939417+0000","last_undegraded":"2023-11-28T12:01:44.939417+0000","last_fullsized":"2023-11-28T12:01:44.939417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:40.727625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939028+0000","last_change":"2023-11-28T12:01:43.949382+0000","last_active":"2023-11-28T12:01:44.939028+0000","last_peered":"2023-11-28T12:01:44.939028+0000","last_clean":"2023-11-28T12:01:44.939028+0000","last_became_active":"2023-11-28T12:01:43.948949+0000","last_became_peered":"2023-11-28T12:01:43.948949+0000","last_unstale":"2023-11-28T12:01:44.939028+0000","last_undegraded":"2023-11-28T12:01:44.939028+0000","last_fullsized":"2023-11-28T12:01:44.939028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:35.755196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:46.497980+0000","last_change":"2023-11-28T12:01:43.498156+0000","last_active":"2023-11-28T12:01:46.497980+0000","last_peered":"2023-11-28T12:01:46.497980+0000","last_clean":"2023-11-28T12:01:46.497980+0000","last_became_active":"2023-11-28T12:01:43.496510+0000","last_became_peered":"2023-11-28T12:01:43.496510+0000","last_unstale":"2023-11-28T12:01:46.497980+0000","last_undegraded":"2023-11-28T12:01:46.497980+0000","last_fullsized":"2023-11-28T12:01:46.497980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:42.470324+0000","last_clean_scrub_stamp":"2023-11-28T12:01:42.470324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:46.276522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937629+0000","last_change":"2023-11-28T12:01:43.944985+0000","last_active":"2023-11-28T12:01:44.937629+0000","last_peered":"2023-11-28T12:01:44.937629+0000","last_clean":"2023-11-28T12:01:44.937629+0000","last_became_active":"2023-11-28T12:01:43.943580+0000","last_became_peered":"2023-11-28T12:01:43.943580+0000","last_unstale":"2023-11-28T12:01:44.937629+0000","last_undegraded":"2023-11-28T12:01:44.937629+0000","last_fullsized":"2023-11-28T12:01:44.937629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937476+0000","last_change":"2023-11-28T12:01:43.941917+0000","last_active":"2023-11-28T12:01:44.937476+0000","last_peered":"2023-11-28T12:01:44.937476+0000","last_clean":"2023-11-28T12:01:44.937476+0000","last_became_active":"2023-11-28T12:01:43.941625+0000","last_became_peered":"2023-11-28T12:01:43.941625+0000","last_unstale":"2023-11-28T12:01:44.937476+0000","last_undegraded":"2023-11-28T12:01:44.937476+0000","last_fullsized":"2023-11-28T12:01:44.937476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:23.540287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.938902+0000","last_change":"2023-11-28T12:01:43.943568+0000","last_active":"2023-11-28T12:01:44.938902+0000","last_peered":"2023-11-28T12:01:44.938902+0000","last_clean":"2023-11-28T12:01:44.938902+0000","last_became_active":"2023-11-28T12:01:43.943204+0000","last_became_peered":"2023-11-28T12:01:43.943204+0000","last_unstale":"2023-11-28T12:01:44.938902+0000","last_undegraded":"2023-11-28T12:01:44.938902+0000","last_fullsized":"2023-11-28T12:01:44.938902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:14.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.937986+0000","last_change":"2023-11-28T12:01:43.943816+0000","last_active":"2023-11-28T12:01:44.937986+0000","last_peered":"2023-11-28T12:01:44.937986+0000","last_clean":"2023-11-28T12:01:44.937986+0000","last_became_active":"2023-11-28T12:01:43.943506+0000","last_became_peered":"2023-11-28T12:01:43.943506+0000","last_unstale":"2023-11-28T12:01:44.937986+0000","last_undegraded":"2023-11-28T12:01:44.937986+0000","last_fullsized":"2023-11-28T12:01:44.937986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:28.088904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:44.939262+0000","last_change":"2023-11-28T12:01:43.950413+0000","last_active":"2023-11-28T12:01:44.939262+0000","last_peered":"2023-11-28T12:01:44.939262+0000","last_clean":"2023-11-28T12:01:44.939262+0000","last_became_active":"2023-11-28T12:01:43.950110+0000","last_became_peered":"2023-11-28T12:01:43.950110+0000","last_unstale":"2023-11-28T12:01:44.939262+0000","last_undegraded":"2023-11-28T12:01:44.939262+0000","last_fullsized":"2023-11-28T12:01:44.939262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:57.245235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T12:01:45.922987+0000","last_change":"2023-11-28T12:01:43.938293+0000","last_active":"2023-11-28T12:01:45.922987+0000","last_peered":"2023-11-28T12:01:45.922987+0000","last_clean":"2023-11-28T12:01:45.922987+0000","last_became_active":"2023-11-28T12:01:43.937951+0000","last_became_peered":"2023-11-28T12:01:43.937951+0000","last_unstale":"2023-11-28T12:01:45.922987+0000","last_undegraded":"2023-11-28T12:01:45.922987+0000","last_fullsized":"2023-11-28T12:01:45.922987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:43.475984+0000","last_clean_scrub_stamp":"2023-11-28T12:01:43.475984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9084,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362756,"statfs":{"total":96636764160,"available":96627462144,"internally_reserved":0,"allocated":839680,"data_stored":671592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7900,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363940,"statfs":{"total":96636764160,"available":96628674560,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9056,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362784,"statfs":{"total":96636764160,"available":96627490816,"internally_reserved":0,"allocated":811008,"data_stored":669828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7968,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363872,"statfs":{"total":96636764160,"available":96628604928,"internally_reserved":0,"allocated":221184,"data_stored":80026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7900,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363940,"statfs":{"total":96636764160,"available":96628674560,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8032,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363808,"statfs":{"total":96636764160,"available":96628539392,"internally_reserved":0,"allocated":221184,"data_stored":79550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8048,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363792,"statfs":{"total":96636764160,"available":96628523008,"internally_reserved":0,"allocated":237568,"data_stored":81064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7972,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363868,"statfs":{"total":96636764160,"available":96628600832,"internally_reserved":0,"allocated":225280,"data_stored":80096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:01:54.741 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T12:01:54.741 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-28T12:01:54.741 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-28T12:01:54.742 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:01:54.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.913+0000 7fa219a9b700 1 Processor -- start 2023-11-28T12:01:54.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.913+0000 7fa219a9b700 1 -- start start 2023-11-28T12:01:54.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa219a9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa21406e400 0x7fa21406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa219a9b700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21415af30 con 0x7fa2140d15f0 2023-11-28T12:01:54.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa219a9b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21415ef40 con 0x7fa21415e010 2023-11-28T12:01:54.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa219a9b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21415a9d0 con 0x7fa21406e400 2023-11-28T12:01:54.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa212ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa2140d15f0 0x7fa21415b3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41152/0 (socket says 172.21.15.57:41152) 2023-11-28T12:01:54.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa212ffd700 1 -- 172.21.15.57:0/4177568215 learned_addr learned my addr 172.21.15.57:0/4177568215 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:54.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2137fe700 1 --2- 172.21.15.57:0/4177568215 >> v2:172.21.15.181:3300/0 conn(0x7fa21406e400 0x7fa21406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995638788 0 0) 0x7fa21415af30 con 0x7fa2140d15f0 2023-11-28T12:01:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1fc0035b0 con 0x7fa2140d15f0 2023-11-28T12:01:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749668710 0 0) 0x7fa21415ef40 con 0x7fa21415e010 2023-11-28T12:01:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa21415af30 con 0x7fa21415e010 2023-11-28T12:01:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 424288025 0 0) 0x7fa1fc0035b0 con 0x7fa2140d15f0 2023-11-28T12:01:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 >> v2:172.21.15.181:3300/0 conn(0x7fa21406e400 msgr2=0x7fa21406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 --2- 172.21.15.57:0/4177568215 >> v2:172.21.15.181:3300/0 conn(0x7fa21406e400 0x7fa21406e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 >> v1:172.21.15.181:6789/0 conn(0x7fa21415e010 legacy=0x7fa2140ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21406ed10 con 0x7fa2140d15f0 2023-11-28T12:01:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fa208001670 con 0x7fa2140d15f0 2023-11-28T12:01:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2137fe700 1 --2- 172.21.15.57:0/4177568215 >> v2:172.21.15.181:3300/0 conn(0x7fa21406e400 0x7fa21406e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 >> v1:172.21.15.57:6789/0 conn(0x7fa2140d15f0 legacy=0x7fa21415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 --2- 172.21.15.57:0/4177568215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1fc003b90 0x7fa1fc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 --2- 172.21.15.57:0/4177568215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1fc004850 0x7fa1fc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.914+0000 7fa2127fc700 1 --2- 172.21.15.57:0/4177568215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1fc005200 0x7fa1fc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc00c030 con 0x7fa1fc005200 2023-11-28T12:01:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc00c1a0 con 0x7fa1fc003b90 2023-11-28T12:01:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc00c360 con 0x7fa1fc004850 2023-11-28T12:01:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa2137fe700 1 --2- 172.21.15.57:0/4177568215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1fc003b90 0x7fa1fc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa212ffd700 1 --2- 172.21.15.57:0/4177568215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1fc004850 0x7fa1fc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa213fff700 1 --2- 172.21.15.57:0/4177568215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1fc005200 0x7fa1fc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa2137fe700 1 -- 172.21.15.57:0/4177568215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1fc004850 msgr2=0x7fa1fc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa2137fe700 1 --2- 172.21.15.57:0/4177568215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1fc004850 0x7fa1fc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa2137fe700 1 -- 172.21.15.57:0/4177568215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1fc005200 msgr2=0x7fa1fc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa2137fe700 1 --2- 172.21.15.57:0/4177568215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1fc005200 0x7fa1fc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa2137fe700 1 -- 172.21.15.57:0/4177568215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc0035b0 con 0x7fa1fc003b90 2023-11-28T12:01:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa213fff700 1 --2- 172.21.15.57:0/4177568215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1fc005200 0x7fa1fc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa213fff700 1 -- 172.21.15.57:0/4177568215 reap_dead start 2023-11-28T12:01:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa2137fe700 1 --2- 172.21.15.57:0/4177568215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1fc003b90 0x7fa1fc003f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa204009d30 tx=0x7fa1fc005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47eb73d4c656894b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204009e80 con 0x7fa1fc003b90 2023-11-28T12:01:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20400e9a0 con 0x7fa1fc003b90 2023-11-28T12:01:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.915+0000 7fa2127fc700 1 -- 172.21.15.57:0/4177568215 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20400ed10 con 0x7fa1fc003b90 2023-11-28T12:01:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.916+0000 7fa219a9b700 1 -- 172.21.15.57:0/4177568215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1fc003b90 msgr2=0x7fa1fc003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.916+0000 7fa219a9b700 1 --2- 172.21.15.57:0/4177568215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1fc003b90 0x7fa1fc003f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa204009d30 tx=0x7fa1fc005880 comp rx=0 tx=0).stop 2023-11-28T12:01:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.916+0000 7fa219a9b700 1 -- 172.21.15.57:0/4177568215 shutdown_connections 2023-11-28T12:01:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.916+0000 7fa219a9b700 1 --2- 172.21.15.57:0/4177568215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1fc003b90 0x7fa1fc003f80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.916+0000 7fa219a9b700 1 -- 172.21.15.57:0/4177568215 >> 172.21.15.57:0/4177568215 conn(0x7fa21406a270 msgr2=0x7fa21406a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:54.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.916+0000 7fa219a9b700 1 -- 172.21.15.57:0/4177568215 shutdown_connections 2023-11-28T12:01:54.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.916+0000 7fa219a9b700 1 -- 172.21.15.57:0/4177568215 wait complete. 2023-11-28T12:01:54.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.917+0000 7fa219a9b700 1 Processor -- start 2023-11-28T12:01:54.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.918+0000 7fa219a9b700 1 -- start start 2023-11-28T12:01:54.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.918+0000 7fa219a9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1fc003b90 0x7fa2141491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.918+0000 7fa219a9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa21415b2c0 0x7fa21406c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.918+0000 7fa219a9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2140cc040 0x7fa2140cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.918+0000 7fa219a9b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2140cc9f0 con 0x7fa2140cc040 2023-11-28T12:01:54.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.918+0000 7fa219a9b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21406b380 con 0x7fa1fc003b90 2023-11-28T12:01:54.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.918+0000 7fa219a9b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21406b660 con 0x7fa21415b2c0 2023-11-28T12:01:54.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.918+0000 7fa213fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2140cc040 0x7fa2140cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.918+0000 7fa213fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2140cc040 0x7fa2140cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38808/0 (socket says 172.21.15.57:38808) 2023-11-28T12:01:54.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.918+0000 7fa213fff700 1 -- 172.21.15.57:0/3908396349 learned_addr learned my addr 172.21.15.57:0/3908396349 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:54.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.919+0000 7fa2137fe700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1fc003b90 0x7fa2141491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.919+0000 7fa212ffd700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa21415b2c0 0x7fa21406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.919+0000 7fa213fff700 1 -- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa21415b2c0 msgr2=0x7fa21406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.919+0000 7fa213fff700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa21415b2c0 0x7fa21406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.919+0000 7fa213fff700 1 -- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1fc003b90 msgr2=0x7fa2141491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:54.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.919+0000 7fa213fff700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1fc003b90 0x7fa2141491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:54.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.919+0000 7fa213fff700 1 -- 172.21.15.57:0/3908396349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21415ec60 con 0x7fa2140cc040 2023-11-28T12:01:54.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.919+0000 7fa213fff700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2140cc040 0x7fa2140cc4b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa21415e3e0 tx=0x7fa20c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:54.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.919+0000 7fa1fbfff700 1 -- 172.21.15.57:0/3908396349 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c00dc70 con 0x7fa2140cc040 2023-11-28T12:01:54.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.919+0000 7fa1fbfff700 1 -- 172.21.15.57:0/3908396349 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c00ddf0 con 0x7fa2140cc040 2023-11-28T12:01:54.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.919+0000 7fa219a9b700 1 -- 172.21.15.57:0/3908396349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21415e980 con 0x7fa2140cc040 2023-11-28T12:01:54.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.919+0000 7fa1fbfff700 1 -- 172.21.15.57:0/3908396349 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c003c40 con 0x7fa2140cc040 2023-11-28T12:01:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.919+0000 7fa219a9b700 1 -- 172.21.15.57:0/3908396349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21415abc0 con 0x7fa2140cc040 2023-11-28T12:01:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.920+0000 7fa219a9b700 1 -- 172.21.15.57:0/3908396349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2140743f0 con 0x7fa2140cc040 2023-11-28T12:01:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.921+0000 7fa1fbfff700 1 -- 172.21.15.57:0/3908396349 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa20c0050d0 con 0x7fa2140cc040 2023-11-28T12:01:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.921+0000 7fa1fbfff700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2000682f0 0x7fa20006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.921+0000 7fa1fbfff700 1 -- 172.21.15.57:0/3908396349 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa20c086150 con 0x7fa2140cc040 2023-11-28T12:01:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.921+0000 7fa2137fe700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2000682f0 0x7fa20006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.922+0000 7fa2137fe700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2000682f0 0x7fa20006a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa2040090b0 tx=0x7fa204008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:54.925+0000 7fa1fbfff700 1 -- 172.21.15.57:0/3908396349 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa20c0529b0 con 0x7fa2140cc040 2023-11-28T12:01:55.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.172+0000 7fa219a9b700 1 -- 172.21.15.57:0/3908396349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa21406e1e0 con 0x7fa2140cc040 2023-11-28T12:01:55.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.173+0000 7fa1fbfff700 1 -- 172.21.15.57:0/3908396349 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa20c055fd0 con 0x7fa2140cc040 2023-11-28T12:01:55.174 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:55.174 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-28T12:01:55.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.175+0000 7fa219a9b700 1 -- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2000682f0 msgr2=0x7fa20006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.175+0000 7fa219a9b700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2000682f0 0x7fa20006a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa2040090b0 tx=0x7fa204008030 comp rx=0 tx=0).stop 2023-11-28T12:01:55.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.175+0000 7fa219a9b700 1 -- 172.21.15.57:0/3908396349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2140cc040 msgr2=0x7fa2140cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.175+0000 7fa219a9b700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2140cc040 0x7fa2140cc4b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa21415e3e0 tx=0x7fa20c00d710 comp rx=0 tx=0).stop 2023-11-28T12:01:55.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.175+0000 7fa219a9b700 1 -- 172.21.15.57:0/3908396349 shutdown_connections 2023-11-28T12:01:55.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.175+0000 7fa219a9b700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2000682f0 0x7fa20006a7a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.175+0000 7fa219a9b700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2140cc040 0x7fa2140cc4b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.175+0000 7fa219a9b700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1fc003b90 0x7fa2141491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.175+0000 7fa219a9b700 1 --2- 172.21.15.57:0/3908396349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa21415b2c0 0x7fa21406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.175+0000 7fa219a9b700 1 -- 172.21.15.57:0/3908396349 >> 172.21.15.57:0/3908396349 conn(0x7fa21406a270 msgr2=0x7fa214154d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:55.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.175+0000 7fa219a9b700 1 -- 172.21.15.57:0/3908396349 shutdown_connections 2023-11-28T12:01:55.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.175+0000 7fa219a9b700 1 -- 172.21.15.57:0/3908396349 wait complete. 2023-11-28T12:01:55.190 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-28T12:01:55.190 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:01:55.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.324+0000 7fd1c50a1700 1 Processor -- start 2023-11-28T12:01:55.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.324+0000 7fd1c50a1700 1 -- start start 2023-11-28T12:01:55.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.325+0000 7fd1c50a1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd1c00d0c60 0x7fd1c00b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:55.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.325+0000 7fd1c50a1700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd1c006e400 0x7fd1c00d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:55.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.325+0000 7fd1c50a1700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c00d1570 con 0x7fd1c0160e70 2023-11-28T12:01:55.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.325+0000 7fd1c50a1700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c006e900 con 0x7fd1c006e400 2023-11-28T12:01:55.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.325+0000 7fd1c50a1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c0160390 con 0x7fd1c00d0c60 2023-11-28T12:01:55.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.325+0000 7fd1be59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd1c006e400 0x7fd1c00d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:55.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.325+0000 7fd1bf59e700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd1c0160e70 0x7fd1c006f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41164/0 (socket says 172.21.15.57:41164) 2023-11-28T12:01:55.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.325+0000 7fd1be59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd1c006e400 0x7fd1c00d1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38822/0 (socket says 172.21.15.57:38822) 2023-11-28T12:01:55.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.325+0000 7fd1bf59e700 1 -- 172.21.15.57:0/4029441992 learned_addr learned my addr 172.21.15.57:0/4029441992 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:55.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.326+0000 7fd1bed9d700 1 --2- 172.21.15.57:0/4029441992 >> v2:172.21.15.181:3300/0 conn(0x7fd1c00d0c60 0x7fd1c00b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:55.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.326+0000 7fd1be59c700 1 -- 172.21.15.57:0/4029441992 >> v2:172.21.15.181:3300/0 conn(0x7fd1c00d0c60 msgr2=0x7fd1c00b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:55.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.326+0000 7fd1be59c700 1 --2- 172.21.15.57:0/4029441992 >> v2:172.21.15.181:3300/0 conn(0x7fd1c00d0c60 0x7fd1c00b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.326+0000 7fd1be59c700 1 -- 172.21.15.57:0/4029441992 >> v1:172.21.15.57:6789/0 conn(0x7fd1c0160e70 legacy=0x7fd1c006f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.326+0000 7fd1be59c700 1 -- 172.21.15.57:0/4029441992 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c0160670 con 0x7fd1c006e400 2023-11-28T12:01:55.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.326+0000 7fd1bdd9b700 1 -- 172.21.15.57:0/4029441992 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420526832 0 0) 0x7fd1c00d1570 con 0x7fd1c0160e70 2023-11-28T12:01:55.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.326+0000 7fd1be59c700 1 --2- 172.21.15.57:0/4029441992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1c006e400 0x7fd1c00d1030 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0067340 tx=0x7fd1b4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9712e4e6d69ac648 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:55.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.326+0000 7fd1bdd9b700 1 -- 172.21.15.57:0/4029441992 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4011830 con 0x7fd1c006e400 2023-11-28T12:01:55.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.326+0000 7fd1bdd9b700 1 -- 172.21.15.57:0/4029441992 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b4011990 con 0x7fd1c006e400 2023-11-28T12:01:55.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.326+0000 7fd1bdd9b700 1 -- 172.21.15.57:0/4029441992 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4011d20 con 0x7fd1c006e400 2023-11-28T12:01:55.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.327+0000 7fd1c50a1700 1 -- 172.21.15.57:0/4029441992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1c006e400 msgr2=0x7fd1c00d1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.327+0000 7fd1c50a1700 1 --2- 172.21.15.57:0/4029441992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1c006e400 0x7fd1c00d1030 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0067340 tx=0x7fd1b4009b20 comp rx=0 tx=0).stop 2023-11-28T12:01:55.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.327+0000 7fd1c50a1700 1 -- 172.21.15.57:0/4029441992 shutdown_connections 2023-11-28T12:01:55.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.327+0000 7fd1c50a1700 1 --2- 172.21.15.57:0/4029441992 >> v2:172.21.15.181:3300/0 conn(0x7fd1c00d0c60 0x7fd1c00b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.327+0000 7fd1c50a1700 1 --2- 172.21.15.57:0/4029441992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1c006e400 0x7fd1c00d1030 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.327+0000 7fd1c50a1700 1 -- 172.21.15.57:0/4029441992 >> 172.21.15.57:0/4029441992 conn(0x7fd1c006a270 msgr2=0x7fd1c006a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:55.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.327+0000 7fd1c50a1700 1 -- 172.21.15.57:0/4029441992 shutdown_connections 2023-11-28T12:01:55.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.327+0000 7fd1c50a1700 1 -- 172.21.15.57:0/4029441992 wait complete. 2023-11-28T12:01:55.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.329+0000 7fd1c50a1700 1 Processor -- start 2023-11-28T12:01:55.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.329+0000 7fd1c50a1700 1 -- start start 2023-11-28T12:01:55.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1c50a1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1c006e400 0x7fd1c0148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1c50a1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1c00d0c60 0x7fd1c00b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1c50a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c0160e70 0x7fd1c01575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1c50a1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c00b08d0 con 0x7fd1c006e400 2023-11-28T12:01:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1c50a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c00b0b70 con 0x7fd1c0160e70 2023-11-28T12:01:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1c50a1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c00b0e50 con 0x7fd1c00d0c60 2023-11-28T12:01:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1bed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1c006e400 0x7fd1c0148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1bed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1c006e400 0x7fd1c0148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38836/0 (socket says 172.21.15.57:38836) 2023-11-28T12:01:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1bed9d700 1 -- 172.21.15.57:0/782844836 learned_addr learned my addr 172.21.15.57:0/782844836 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1bf59e700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c0160e70 0x7fd1c01575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1be59c700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1c00d0c60 0x7fd1c00b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1bed9d700 1 -- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1c00d0c60 msgr2=0x7fd1c00b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1bed9d700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1c00d0c60 0x7fd1c00b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1bed9d700 1 -- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c0160e70 msgr2=0x7fd1c01575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1bed9d700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c0160e70 0x7fd1c01575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.330+0000 7fd1bed9d700 1 -- 172.21.15.57:0/782844836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b40094a0 con 0x7fd1c006e400 2023-11-28T12:01:55.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.331+0000 7fd1bed9d700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1c006e400 0x7fd1c0148950 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000e9b0 tx=0x7fd1b000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:55.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.331+0000 7fd1a77fe700 1 -- 172.21.15.57:0/782844836 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0009710 con 0x7fd1c006e400 2023-11-28T12:01:55.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.331+0000 7fd1a77fe700 1 -- 172.21.15.57:0/782844836 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b000ee50 con 0x7fd1c006e400 2023-11-28T12:01:55.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.331+0000 7fd1c50a1700 1 -- 172.21.15.57:0/782844836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c00d0740 con 0x7fd1c006e400 2023-11-28T12:01:55.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.331+0000 7fd1a77fe700 1 -- 172.21.15.57:0/782844836 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b000ca50 con 0x7fd1c006e400 2023-11-28T12:01:55.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.331+0000 7fd1c50a1700 1 -- 172.21.15.57:0/782844836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c0160950 con 0x7fd1c006e400 2023-11-28T12:01:55.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.332+0000 7fd1c50a1700 1 -- 172.21.15.57:0/782844836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c015d980 con 0x7fd1c006e400 2023-11-28T12:01:55.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.333+0000 7fd1a77fe700 1 -- 172.21.15.57:0/782844836 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd1b0019030 con 0x7fd1c006e400 2023-11-28T12:01:55.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.334+0000 7fd1a77fe700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1ac068320 0x7fd1ac06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:55.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.334+0000 7fd1a77fe700 1 -- 172.21.15.57:0/782844836 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd1b0086210 con 0x7fd1c006e400 2023-11-28T12:01:55.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.334+0000 7fd1be59c700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1ac068320 0x7fd1ac06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:55.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.334+0000 7fd1be59c700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1ac068320 0x7fd1ac06a7d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd1c006e830 tx=0x7fd1b401c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:55.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.338+0000 7fd1a77fe700 1 -- 172.21.15.57:0/782844836 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd1b0056540 con 0x7fd1c006e400 2023-11-28T12:01:55.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.540+0000 7fd1c50a1700 1 -- 172.21.15.57:0/782844836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1c00743f0 con 0x7fd1c006e400 2023-11-28T12:01:55.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.541+0000 7fd1a77fe700 1 -- 172.21.15.57:0/782844836 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7fd1b0056130 con 0x7fd1c006e400 2023-11-28T12:01:55.542 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:55.542 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:01:55.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.543+0000 7fd1c50a1700 1 -- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1ac068320 msgr2=0x7fd1ac06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.543+0000 7fd1c50a1700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1ac068320 0x7fd1ac06a7d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd1c006e830 tx=0x7fd1b401c040 comp rx=0 tx=0).stop 2023-11-28T12:01:55.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.543+0000 7fd1c50a1700 1 -- 172.21.15.57:0/782844836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1c006e400 msgr2=0x7fd1c0148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.543+0000 7fd1c50a1700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1c006e400 0x7fd1c0148950 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000e9b0 tx=0x7fd1b000ecc0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.543+0000 7fd1c50a1700 1 -- 172.21.15.57:0/782844836 shutdown_connections 2023-11-28T12:01:55.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.543+0000 7fd1c50a1700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1ac068320 0x7fd1ac06a7d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.543+0000 7fd1c50a1700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c0160e70 0x7fd1c01575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.543+0000 7fd1c50a1700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1c006e400 0x7fd1c0148950 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.543+0000 7fd1c50a1700 1 --2- 172.21.15.57:0/782844836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1c00d0c60 0x7fd1c00b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.543+0000 7fd1c50a1700 1 -- 172.21.15.57:0/782844836 >> 172.21.15.57:0/782844836 conn(0x7fd1c006a270 msgr2=0x7fd1c006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:55.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.543+0000 7fd1c50a1700 1 -- 172.21.15.57:0/782844836 shutdown_connections 2023-11-28T12:01:55.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.543+0000 7fd1c50a1700 1 -- 172.21.15.57:0/782844836 wait complete. 2023-11-28T12:01:55.558 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:01:55.559 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:01:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.696+0000 7f06ab585700 1 Processor -- start 2023-11-28T12:01:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.696+0000 7f06ab585700 1 -- start start 2023-11-28T12:01:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.697+0000 7f06ab585700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06a415fe50 0x7f06a406d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.697+0000 7f06ab585700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a4161370 con 0x7f06a40b3b80 2023-11-28T12:01:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.697+0000 7f06ab585700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a40cf700 con 0x7f06a40b27d0 2023-11-28T12:01:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.697+0000 7f06ab585700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a4160220 con 0x7f06a415fe50 2023-11-28T12:01:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.697+0000 7f06a8b20700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f06a40b3b80 0x7f06a4160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41166/0 (socket says 172.21.15.57:41166) 2023-11-28T12:01:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.697+0000 7f06a8b20700 1 -- 172.21.15.57:0/3020868743 learned_addr learned my addr 172.21.15.57:0/3020868743 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.698+0000 7f06a9321700 1 --2- 172.21.15.57:0/3020868743 >> v2:172.21.15.181:3300/0 conn(0x7f06a415fe50 0x7f06a406d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.698+0000 7f0697fff700 1 -- 172.21.15.57:0/3020868743 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624627597 0 0) 0x7f06a4161370 con 0x7f06a40b3b80 2023-11-28T12:01:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.698+0000 7f0697fff700 1 -- 172.21.15.57:0/3020868743 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06900035b0 con 0x7f06a40b3b80 2023-11-28T12:01:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.698+0000 7f0697fff700 1 -- 172.21.15.57:0/3020868743 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241817173 0 0) 0x7f06a40cf700 con 0x7f06a40b27d0 2023-11-28T12:01:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.698+0000 7f0697fff700 1 -- 172.21.15.57:0/3020868743 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06a4161370 con 0x7f06a40b27d0 2023-11-28T12:01:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.698+0000 7f06a9321700 1 -- 172.21.15.57:0/3020868743 >> v1:172.21.15.181:6789/0 conn(0x7f06a40b27d0 legacy=0x7f06a40cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.698+0000 7f06a9321700 1 -- 172.21.15.57:0/3020868743 >> v1:172.21.15.57:6789/0 conn(0x7f06a40b3b80 legacy=0x7f06a4160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.698+0000 7f06a9321700 1 -- 172.21.15.57:0/3020868743 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a40b3fd0 con 0x7f06a415fe50 2023-11-28T12:01:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.698+0000 7f06a9321700 1 --2- 172.21.15.57:0/3020868743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a415fe50 0x7f06a406d170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0698009840 tx=0x7f0698009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2727dc5fb74b3d91 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.698+0000 7f0697fff700 1 -- 172.21.15.57:0/3020868743 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698011830 con 0x7f06a415fe50 2023-11-28T12:01:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.698+0000 7f0697fff700 1 -- 172.21.15.57:0/3020868743 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0698011990 con 0x7f06a415fe50 2023-11-28T12:01:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.698+0000 7f0697fff700 1 -- 172.21.15.57:0/3020868743 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698011d20 con 0x7f06a415fe50 2023-11-28T12:01:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.699+0000 7f06ab585700 1 -- 172.21.15.57:0/3020868743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a415fe50 msgr2=0x7f06a406d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.699+0000 7f06ab585700 1 --2- 172.21.15.57:0/3020868743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a415fe50 0x7f06a406d170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0698009840 tx=0x7f0698009b50 comp rx=0 tx=0).stop 2023-11-28T12:01:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.699+0000 7f06ab585700 1 -- 172.21.15.57:0/3020868743 shutdown_connections 2023-11-28T12:01:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.699+0000 7f06ab585700 1 --2- 172.21.15.57:0/3020868743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a415fe50 0x7f06a406d170 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.699+0000 7f06ab585700 1 -- 172.21.15.57:0/3020868743 >> 172.21.15.57:0/3020868743 conn(0x7f06a406ea80 msgr2=0x7f06a406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.699+0000 7f06ab585700 1 -- 172.21.15.57:0/3020868743 shutdown_connections 2023-11-28T12:01:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.699+0000 7f06ab585700 1 -- 172.21.15.57:0/3020868743 wait complete. 2023-11-28T12:01:55.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.700+0000 7f06ab585700 1 Processor -- start 2023-11-28T12:01:55.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.700+0000 7f06ab585700 1 -- start start 2023-11-28T12:01:55.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.701+0000 7f06ab585700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06a40b27d0 0x7f06a414b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:55.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.701+0000 7f06ab585700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06a40b3b80 0x7f06a40c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:55.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.701+0000 7f06ab585700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a415fe50 0x7f06a415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:55.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.701+0000 7f06ab585700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40c7e00 con 0x7f06a40b3b80 2023-11-28T12:01:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.701+0000 7f06ab585700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40c8050 con 0x7f06a415fe50 2023-11-28T12:01:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.701+0000 7f06ab585700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40c8330 con 0x7f06a40b27d0 2023-11-28T12:01:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.701+0000 7f06a8b20700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06a40b3b80 0x7f06a40c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.701+0000 7f06a8b20700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06a40b3b80 0x7f06a40c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38842/0 (socket says 172.21.15.57:38842) 2023-11-28T12:01:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.701+0000 7f06a8b20700 1 -- 172.21.15.57:0/3332382412 learned_addr learned my addr 172.21.15.57:0/3332382412 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.702+0000 7f06a9321700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06a40b27d0 0x7f06a414b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.702+0000 7f06a9b22700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a415fe50 0x7f06a415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.702+0000 7f06a8b20700 1 -- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06a40b27d0 msgr2=0x7f06a414b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.702+0000 7f06a8b20700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06a40b27d0 0x7f06a414b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.702+0000 7f06a8b20700 1 -- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a415fe50 msgr2=0x7f06a415a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.702+0000 7f06a8b20700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a415fe50 0x7f06a415a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.702+0000 7f06a8b20700 1 -- 172.21.15.57:0/3332382412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06980094a0 con 0x7f06a40b3b80 2023-11-28T12:01:55.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.702+0000 7f06a8b20700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06a40b3b80 0x7f06a40c8e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f06a000c700 tx=0x7f06a000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:55.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.702+0000 7f0695ffb700 1 -- 172.21.15.57:0/3332382412 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0005d70 con 0x7f06a40b3b80 2023-11-28T12:01:55.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.702+0000 7f0695ffb700 1 -- 172.21.15.57:0/3332382412 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a00070d0 con 0x7f06a40b3b80 2023-11-28T12:01:55.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.702+0000 7f0695ffb700 1 -- 172.21.15.57:0/3332382412 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0007480 con 0x7f06a40b3b80 2023-11-28T12:01:55.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.702+0000 7f06ab585700 1 -- 172.21.15.57:0/3332382412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a40cf0b0 con 0x7f06a40b3b80 2023-11-28T12:01:55.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.702+0000 7f06ab585700 1 -- 172.21.15.57:0/3332382412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a40cedd0 con 0x7f06a40b3b80 2023-11-28T12:01:55.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.703+0000 7f06ab585700 1 -- 172.21.15.57:0/3332382412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a40cf390 con 0x7f06a40b3b80 2023-11-28T12:01:55.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.704+0000 7f0695ffb700 1 -- 172.21.15.57:0/3332382412 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f06a0019030 con 0x7f06a40b3b80 2023-11-28T12:01:55.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.704+0000 7f0695ffb700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f06880682f0 0x7f068806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:55.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.704+0000 7f0695ffb700 1 -- 172.21.15.57:0/3332382412 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f06a00851e0 con 0x7f06a40b3b80 2023-11-28T12:01:55.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.704+0000 7f06a9321700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f06880682f0 0x7f068806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:55.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.705+0000 7f06a9321700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f06880682f0 0x7f068806a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0698009b90 tx=0x7f069801c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:55.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.709+0000 7f0695ffb700 1 -- 172.21.15.57:0/3332382412 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f06a00519c0 con 0x7f06a40b3b80 2023-11-28T12:01:55.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.911+0000 7f06ab585700 1 -- 172.21.15.57:0/3332382412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f06a40743f0 con 0x7f06a40b3b80 2023-11-28T12:01:55.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.912+0000 7f0695ffb700 1 -- 172.21.15.57:0/3332382412 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f06a0054fe0 con 0x7f06a40b3b80 2023-11-28T12:01:55.913 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:01:55.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.914+0000 7f06ab585700 1 -- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f06880682f0 msgr2=0x7f068806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.914+0000 7f06ab585700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f06880682f0 0x7f068806a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0698009b90 tx=0x7f069801c040 comp rx=0 tx=0).stop 2023-11-28T12:01:55.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.914+0000 7f06ab585700 1 -- 172.21.15.57:0/3332382412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06a40b3b80 msgr2=0x7f06a40c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:55.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.914+0000 7f06ab585700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06a40b3b80 0x7f06a40c8e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f06a000c700 tx=0x7f06a000cac0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.914+0000 7f06ab585700 1 -- 172.21.15.57:0/3332382412 shutdown_connections 2023-11-28T12:01:55.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.914+0000 7f06ab585700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f06880682f0 0x7f068806a7a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.914+0000 7f06ab585700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a415fe50 0x7f06a415a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.914+0000 7f06ab585700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06a40b27d0 0x7f06a414b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.914+0000 7f06ab585700 1 --2- 172.21.15.57:0/3332382412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06a40b3b80 0x7f06a40c8e10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:55.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.914+0000 7f06ab585700 1 -- 172.21.15.57:0/3332382412 >> 172.21.15.57:0/3332382412 conn(0x7f06a406ea80 msgr2=0x7f06a406c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:55.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.914+0000 7f06ab585700 1 -- 172.21.15.57:0/3332382412 shutdown_connections 2023-11-28T12:01:55.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:55.914+0000 7f06ab585700 1 -- 172.21.15.57:0/3332382412 wait complete. 2023-11-28T12:01:55.929 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:01:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.066+0000 7f9cb9248700 1 Processor -- start 2023-11-28T12:01:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.066+0000 7f9cb9248700 1 -- start start 2023-11-28T12:01:56.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb9248700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cb40cf710 0x7f9cb40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb9248700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9cb4071890 0x7f9cb4071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb9248700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9cb406a270 0x7f9cb406a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb9248700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb406ac30 con 0x7f9cb406a270 2023-11-28T12:01:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb9248700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb415cc10 con 0x7f9cb4071890 2023-11-28T12:01:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb9248700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb406c9b0 con 0x7f9cb40cf710 2023-11-28T12:01:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb359e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9cb406a270 0x7f9cb406a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb359e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9cb406a270 0x7f9cb406a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38844/0 (socket says 172.21.15.57:38844) 2023-11-28T12:01:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb359e700 1 -- 172.21.15.57:0/2109628227 learned_addr learned my addr 172.21.15.57:0/2109628227 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb259c700 1 --2- 172.21.15.57:0/2109628227 >> v2:172.21.15.181:3300/0 conn(0x7f9cb4071890 0x7f9cb4071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb2d9d700 1 --2- 172.21.15.57:0/2109628227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cb40cf710 0x7f9cb40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb359e700 1 -- 172.21.15.57:0/2109628227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cb40cf710 msgr2=0x7f9cb40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb359e700 1 --2- 172.21.15.57:0/2109628227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cb40cf710 0x7f9cb40ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb359e700 1 -- 172.21.15.57:0/2109628227 >> v2:172.21.15.181:3300/0 conn(0x7f9cb4071890 msgr2=0x7f9cb4071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:56.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb359e700 1 --2- 172.21.15.57:0/2109628227 >> v2:172.21.15.181:3300/0 conn(0x7f9cb4071890 0x7f9cb4071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb359e700 1 -- 172.21.15.57:0/2109628227 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb406cc90 con 0x7f9cb406a270 2023-11-28T12:01:56.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb359e700 1 --2- 172.21.15.57:0/2109628227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cb406a270 0x7f9cb406a6f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40b2780 tx=0x7f9cac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f730d64574f648 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:56.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb1d9b700 1 -- 172.21.15.57:0/2109628227 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac00e070 con 0x7f9cb406a270 2023-11-28T12:01:56.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb1d9b700 1 -- 172.21.15.57:0/2109628227 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cac00b910 con 0x7f9cb406a270 2023-11-28T12:01:56.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.067+0000 7f9cb1d9b700 1 -- 172.21.15.57:0/2109628227 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac00bcc0 con 0x7f9cb406a270 2023-11-28T12:01:56.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.068+0000 7f9cb9248700 1 -- 172.21.15.57:0/2109628227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cb406a270 msgr2=0x7f9cb406a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.068+0000 7f9cb9248700 1 --2- 172.21.15.57:0/2109628227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cb406a270 0x7f9cb406a6f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40b2780 tx=0x7f9cac00b330 comp rx=0 tx=0).stop 2023-11-28T12:01:56.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.068+0000 7f9cb9248700 1 -- 172.21.15.57:0/2109628227 shutdown_connections 2023-11-28T12:01:56.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.068+0000 7f9cb9248700 1 --2- 172.21.15.57:0/2109628227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cb40cf710 0x7f9cb40ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.068+0000 7f9cb9248700 1 --2- 172.21.15.57:0/2109628227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cb406a270 0x7f9cb406a6f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.068+0000 7f9cb9248700 1 --2- 172.21.15.57:0/2109628227 >> v2:172.21.15.181:3300/0 conn(0x7f9cb4071890 0x7f9cb4071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.068+0000 7f9cb9248700 1 -- 172.21.15.57:0/2109628227 >> 172.21.15.57:0/2109628227 conn(0x7f9cb406c010 msgr2=0x7f9cb40cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:56.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.068+0000 7f9cb9248700 1 -- 172.21.15.57:0/2109628227 shutdown_connections 2023-11-28T12:01:56.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.068+0000 7f9cb9248700 1 -- 172.21.15.57:0/2109628227 wait complete. 2023-11-28T12:01:56.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.070+0000 7f9cb9248700 1 Processor -- start 2023-11-28T12:01:56.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.070+0000 7f9cb9248700 1 -- start start 2023-11-28T12:01:56.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.070+0000 7f9cb9248700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cb406a270 0x7f9cb40c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.070+0000 7f9cb9248700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cb4071890 0x7f9cb40c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.070+0000 7f9cb9248700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb40cf710 0x7f9cb40ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb9248700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb4150550 con 0x7f9cb406a270 2023-11-28T12:01:56.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb9248700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40c7610 con 0x7f9cb40cf710 2023-11-28T12:01:56.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb9248700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40c78f0 con 0x7f9cb4071890 2023-11-28T12:01:56.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cb406a270 0x7f9cb40c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cb406a270 0x7f9cb40c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38858/0 (socket says 172.21.15.57:38858) 2023-11-28T12:01:56.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb2d9d700 1 -- 172.21.15.57:0/3165421091 learned_addr learned my addr 172.21.15.57:0/3165421091 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:56.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb359e700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb40cf710 0x7f9cb40ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb259c700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cb4071890 0x7f9cb40c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb2d9d700 1 -- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cb4071890 msgr2=0x7f9cb40c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb2d9d700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cb4071890 0x7f9cb40c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb2d9d700 1 -- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb40cf710 msgr2=0x7f9cb40ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb2d9d700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb40cf710 0x7f9cb40ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb2d9d700 1 -- 172.21.15.57:0/3165421091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac009b30 con 0x7f9cb406a270 2023-11-28T12:01:56.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb2d9d700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cb406a270 0x7f9cb40c86b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800e9b0 tx=0x7f9ca800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:56.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9c9b7fe700 1 -- 172.21.15.57:0/3165421091 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8009710 con 0x7f9cb406a270 2023-11-28T12:01:56.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9c9b7fe700 1 -- 172.21.15.57:0/3165421091 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca800ee50 con 0x7f9cb406a270 2023-11-28T12:01:56.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb9248700 1 -- 172.21.15.57:0/3165421091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb406d250 con 0x7f9cb406a270 2023-11-28T12:01:56.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9c9b7fe700 1 -- 172.21.15.57:0/3165421091 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca800ca50 con 0x7f9cb406a270 2023-11-28T12:01:56.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.071+0000 7f9cb9248700 1 -- 172.21.15.57:0/3165421091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb406cf70 con 0x7f9cb406a270 2023-11-28T12:01:56.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.073+0000 7f9cb9248700 1 -- 172.21.15.57:0/3165421091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb406d530 con 0x7f9cb406a270 2023-11-28T12:01:56.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.073+0000 7f9c9b7fe700 1 -- 172.21.15.57:0/3165421091 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9ca8019030 con 0x7f9cb406a270 2023-11-28T12:01:56.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.073+0000 7f9c9b7fe700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ca00682f0 0x7f9ca006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.073+0000 7f9c9b7fe700 1 -- 172.21.15.57:0/3165421091 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ca8085b10 con 0x7f9cb406a270 2023-11-28T12:01:56.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.074+0000 7f9cb259c700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ca00682f0 0x7f9ca006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.074+0000 7f9cb259c700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ca00682f0 0x7f9ca006a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9cb41f5780 tx=0x7f9ca4009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:56.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.078+0000 7f9c9b7fe700 1 -- 172.21.15.57:0/3165421091 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9ca8052370 con 0x7f9cb406a270 2023-11-28T12:01:56.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.276+0000 7f9cb9248700 1 -- 172.21.15.57:0/3165421091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9cb40c8500 con 0x7f9cb406a270 2023-11-28T12:01:56.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.276+0000 7f9c9b7fe700 1 -- 172.21.15.57:0/3165421091 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ca8055990 con 0x7f9cb406a270 2023-11-28T12:01:56.278 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:01:56.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.279+0000 7f9cb9248700 1 -- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ca00682f0 msgr2=0x7f9ca006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.279+0000 7f9cb9248700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ca00682f0 0x7f9ca006a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9cb41f5780 tx=0x7f9ca4009210 comp rx=0 tx=0).stop 2023-11-28T12:01:56.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.279+0000 7f9cb9248700 1 -- 172.21.15.57:0/3165421091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cb406a270 msgr2=0x7f9cb40c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.279+0000 7f9cb9248700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cb406a270 0x7f9cb40c86b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800e9b0 tx=0x7f9ca800ecc0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.279+0000 7f9cb9248700 1 -- 172.21.15.57:0/3165421091 shutdown_connections 2023-11-28T12:01:56.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.279+0000 7f9cb9248700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ca00682f0 0x7f9ca006a7a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.279+0000 7f9cb9248700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb40cf710 0x7f9cb40ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.279+0000 7f9cb9248700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cb406a270 0x7f9cb40c86b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.279+0000 7f9cb9248700 1 --2- 172.21.15.57:0/3165421091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cb4071890 0x7f9cb40c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.279+0000 7f9cb9248700 1 -- 172.21.15.57:0/3165421091 >> 172.21.15.57:0/3165421091 conn(0x7f9cb406c010 msgr2=0x7f9cb40b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:56.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.279+0000 7f9cb9248700 1 -- 172.21.15.57:0/3165421091 shutdown_connections 2023-11-28T12:01:56.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.279+0000 7f9cb9248700 1 -- 172.21.15.57:0/3165421091 wait complete. 2023-11-28T12:01:56.294 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:01:56.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.430+0000 7f4e72a5c700 1 Processor -- start 2023-11-28T12:01:56.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.430+0000 7f4e72a5c700 1 -- start start 2023-11-28T12:01:56.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.431+0000 7f4e72a5c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4e6c06b220 0x7f4e6c0d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.431+0000 7f4e72a5c700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c0d2d30 con 0x7f4e6c160cb0 2023-11-28T12:01:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.431+0000 7f4e72a5c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c0d0b40 con 0x7f4e6c15c8c0 2023-11-28T12:01:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.431+0000 7f4e72a5c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c0d2a50 con 0x7f4e6c06b220 2023-11-28T12:01:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.431+0000 7f4e6b7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4e6c160cb0 0x7f4e6c06ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41178/0 (socket says 172.21.15.57:41178) 2023-11-28T12:01:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.431+0000 7f4e6b7fe700 1 -- 172.21.15.57:0/1673899494 learned_addr learned my addr 172.21.15.57:0/1673899494 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e70ff9700 1 --2- 172.21.15.57:0/1673899494 >> v2:172.21.15.181:3300/0 conn(0x7f4e6c06b220 0x7f4e6c0d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003799619 0 0) 0x7f4e60001110 con 0x7f4e6c160cb0 2023-11-28T12:01:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e540035b0 con 0x7f4e6c160cb0 2023-11-28T12:01:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3389415005 0 0) 0x7f4e540035b0 con 0x7f4e6c160cb0 2023-11-28T12:01:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 >> v2:172.21.15.181:3300/0 conn(0x7f4e6c06b220 msgr2=0x7f4e6c0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 --2- 172.21.15.57:0/1673899494 >> v2:172.21.15.181:3300/0 conn(0x7f4e6c06b220 0x7f4e6c0d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 >> v1:172.21.15.181:6789/0 conn(0x7f4e6c15c8c0 legacy=0x7f4e6c0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c0cd760 con 0x7f4e6c160cb0 2023-11-28T12:01:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f4e60001800 con 0x7f4e6c160cb0 2023-11-28T12:01:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 >> v1:172.21.15.57:6789/0 conn(0x7f4e6c160cb0 legacy=0x7f4e6c06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 --2- 172.21.15.57:0/1673899494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e54003a00 0x7f4e54003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 --2- 172.21.15.57:0/1673899494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e54004850 0x7f4e54004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 --2- 172.21.15.57:0/1673899494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e54005200 0x7f4e5400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0d2a50 con 0x7f4e54003a00 2023-11-28T12:01:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5400c080 con 0x7f4e54005200 2023-11-28T12:01:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5400c240 con 0x7f4e54004850 2023-11-28T12:01:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6bfff700 1 --2- 172.21.15.57:0/1673899494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e54003a00 0x7f4e54003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6b7fe700 1 --2- 172.21.15.57:0/1673899494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e54004850 0x7f4e54004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e70ff9700 1 --2- 172.21.15.57:0/1673899494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e54005200 0x7f4e5400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6bfff700 1 -- 172.21.15.57:0/1673899494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e54004850 msgr2=0x7f4e54004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6bfff700 1 --2- 172.21.15.57:0/1673899494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e54004850 0x7f4e54004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6bfff700 1 -- 172.21.15.57:0/1673899494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e54005200 msgr2=0x7f4e5400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6bfff700 1 --2- 172.21.15.57:0/1673899494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e54005200 0x7f4e5400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e6bfff700 1 -- 172.21.15.57:0/1673899494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e540035b0 con 0x7f4e54003a00 2023-11-28T12:01:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.432+0000 7f4e70ff9700 1 -- 172.21.15.57:0/1673899494 reap_dead start 2023-11-28T12:01:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.433+0000 7f4e6bfff700 1 --2- 172.21.15.57:0/1673899494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e54003a00 0x7f4e54003e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0017c0 tx=0x7f4e54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed883f9d620ce36 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.433+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0052d0 con 0x7f4e54003a00 2023-11-28T12:01:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.433+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c005450 con 0x7f4e54003a00 2023-11-28T12:01:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.433+0000 7f4e6affd700 1 -- 172.21.15.57:0/1673899494 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0038c0 con 0x7f4e54003a00 2023-11-28T12:01:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.433+0000 7f4e72a5c700 1 -- 172.21.15.57:0/1673899494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e54003a00 msgr2=0x7f4e54003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.433+0000 7f4e72a5c700 1 --2- 172.21.15.57:0/1673899494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e54003a00 0x7f4e54003e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0017c0 tx=0x7f4e54005880 comp rx=0 tx=0).stop 2023-11-28T12:01:56.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.433+0000 7f4e72a5c700 1 -- 172.21.15.57:0/1673899494 shutdown_connections 2023-11-28T12:01:56.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.433+0000 7f4e72a5c700 1 --2- 172.21.15.57:0/1673899494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e54003a00 0x7f4e54003e10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.433+0000 7f4e72a5c700 1 -- 172.21.15.57:0/1673899494 >> 172.21.15.57:0/1673899494 conn(0x7f4e6c06c520 msgr2=0x7f4e6c06c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:56.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.434+0000 7f4e72a5c700 1 -- 172.21.15.57:0/1673899494 shutdown_connections 2023-11-28T12:01:56.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.434+0000 7f4e72a5c700 1 -- 172.21.15.57:0/1673899494 wait complete. 2023-11-28T12:01:56.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.435+0000 7f4e72a5c700 1 Processor -- start 2023-11-28T12:01:56.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.435+0000 7f4e72a5c700 1 -- start start 2023-11-28T12:01:56.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e72a5c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e54003a00 0x7f4e6c14ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e72a5c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e6c0b7cd0 0x7f4e6c0b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e72a5c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e6c15e1d0 0x7f4e6c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e72a5c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0b8680 con 0x7f4e54003a00 2023-11-28T12:01:56.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e72a5c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0ca040 con 0x7f4e6c15e1d0 2023-11-28T12:01:56.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e72a5c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0ca8e0 con 0x7f4e6c0b7cd0 2023-11-28T12:01:56.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e6bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e54003a00 0x7f4e6c14ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e6bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e54003a00 0x7f4e6c14ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38882/0 (socket says 172.21.15.57:38882) 2023-11-28T12:01:56.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e6bfff700 1 -- 172.21.15.57:0/2045137072 learned_addr learned my addr 172.21.15.57:0/2045137072 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:56.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e70ff9700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e6c15e1d0 0x7f4e6c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e6b7fe700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e6c0b7cd0 0x7f4e6c0b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e6bfff700 1 -- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e6c0b7cd0 msgr2=0x7f4e6c0b8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e6bfff700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e6c0b7cd0 0x7f4e6c0b8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e6bfff700 1 -- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e6c15e1d0 msgr2=0x7f4e6c15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e6bfff700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e6c15e1d0 0x7f4e6c15e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e6bfff700 1 -- 172.21.15.57:0/2045137072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c161120 con 0x7f4e54003a00 2023-11-28T12:01:56.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.436+0000 7f4e6bfff700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e54003a00 0x7f4e6c14ccf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c000c00 tx=0x7f4e5c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:56.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.437+0000 7f4e68ff9700 1 -- 172.21.15.57:0/2045137072 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c001290 con 0x7f4e54003a00 2023-11-28T12:01:56.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.437+0000 7f4e68ff9700 1 -- 172.21.15.57:0/2045137072 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c007040 con 0x7f4e54003a00 2023-11-28T12:01:56.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.437+0000 7f4e72a5c700 1 -- 172.21.15.57:0/2045137072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c0cda40 con 0x7f4e54003a00 2023-11-28T12:01:56.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.437+0000 7f4e68ff9700 1 -- 172.21.15.57:0/2045137072 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0073d0 con 0x7f4e54003a00 2023-11-28T12:01:56.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.437+0000 7f4e72a5c700 1 -- 172.21.15.57:0/2045137072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c0d2700 con 0x7f4e54003a00 2023-11-28T12:01:56.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.438+0000 7f4e72a5c700 1 -- 172.21.15.57:0/2045137072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6c067d70 con 0x7f4e54003a00 2023-11-28T12:01:56.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.438+0000 7f4e68ff9700 1 -- 172.21.15.57:0/2045137072 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e5c02d070 con 0x7f4e54003a00 2023-11-28T12:01:56.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.439+0000 7f4e68ff9700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4e580682f0 0x7f4e5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.439+0000 7f4e68ff9700 1 -- 172.21.15.57:0/2045137072 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e5c0862e0 con 0x7f4e54003a00 2023-11-28T12:01:56.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.439+0000 7f4e6b7fe700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4e580682f0 0x7f4e5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.439+0000 7f4e6b7fe700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4e580682f0 0x7f4e5806a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0cbe30 tx=0x7f4e60001d90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:56.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.443+0000 7f4e68ff9700 1 -- 172.21.15.57:0/2045137072 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e5c052b40 con 0x7f4e54003a00 2023-11-28T12:01:56.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.644+0000 7f4e72a5c700 1 -- 172.21.15.57:0/2045137072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e6c0cabc0 con 0x7f4e54003a00 2023-11-28T12:01:56.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.645+0000 7f4e68ff9700 1 -- 172.21.15.57:0/2045137072 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e5c056160 con 0x7f4e54003a00 2023-11-28T12:01:56.646 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:01:56.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.647+0000 7f4e72a5c700 1 -- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4e580682f0 msgr2=0x7f4e5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.647+0000 7f4e72a5c700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4e580682f0 0x7f4e5806a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0cbe30 tx=0x7f4e60001d90 comp rx=0 tx=0).stop 2023-11-28T12:01:56.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.647+0000 7f4e72a5c700 1 -- 172.21.15.57:0/2045137072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e54003a00 msgr2=0x7f4e6c14ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.647+0000 7f4e72a5c700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e54003a00 0x7f4e6c14ccf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c000c00 tx=0x7f4e5c001120 comp rx=0 tx=0).stop 2023-11-28T12:01:56.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.647+0000 7f4e72a5c700 1 -- 172.21.15.57:0/2045137072 shutdown_connections 2023-11-28T12:01:56.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.647+0000 7f4e72a5c700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4e580682f0 0x7f4e5806a7a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.647+0000 7f4e72a5c700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e6c15e1d0 0x7f4e6c15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.647+0000 7f4e72a5c700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e54003a00 0x7f4e6c14ccf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.647+0000 7f4e72a5c700 1 --2- 172.21.15.57:0/2045137072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e6c0b7cd0 0x7f4e6c0b8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.647+0000 7f4e72a5c700 1 -- 172.21.15.57:0/2045137072 >> 172.21.15.57:0/2045137072 conn(0x7f4e6c06c520 msgr2=0x7f4e6c0d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:56.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.647+0000 7f4e72a5c700 1 -- 172.21.15.57:0/2045137072 shutdown_connections 2023-11-28T12:01:56.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.647+0000 7f4e72a5c700 1 -- 172.21.15.57:0/2045137072 wait complete. 2023-11-28T12:01:56.662 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:01:56.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.799+0000 7fc03d77e700 1 Processor -- start 2023-11-28T12:01:56.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.799+0000 7fc03d77e700 1 -- start start 2023-11-28T12:01:56.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc03d77e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc03815c8c0 0x7fc038160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc03d77e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0380b5dc0 0x7fc03806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc03d77e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0380d2d30 con 0x7fc03806b050 2023-11-28T12:01:56.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc03d77e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03806b4d0 con 0x7fc03815c8c0 2023-11-28T12:01:56.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc03d77e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380d0b40 con 0x7fc0380b5dc0 2023-11-28T12:01:56.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc036ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc03815c8c0 0x7fc038160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc036ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc03815c8c0 0x7fc038160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38894/0 (socket says 172.21.15.57:38894) 2023-11-28T12:01:56.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc036ffd700 1 -- 172.21.15.57:0/1603971720 learned_addr learned my addr 172.21.15.57:0/1603971720 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:56.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc0367fc700 1 --2- 172.21.15.57:0/1603971720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0380b5dc0 0x7fc03806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc036ffd700 1 -- 172.21.15.57:0/1603971720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0380b5dc0 msgr2=0x7fc03806ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:56.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc036ffd700 1 --2- 172.21.15.57:0/1603971720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0380b5dc0 0x7fc03806ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc036ffd700 1 -- 172.21.15.57:0/1603971720 >> v1:172.21.15.181:6789/0 conn(0x7fc03806b050 legacy=0x7fc0380d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc036ffd700 1 -- 172.21.15.57:0/1603971720 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0381611f0 con 0x7fc03815c8c0 2023-11-28T12:01:56.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc036ffd700 1 --2- 172.21.15.57:0/1603971720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc03815c8c0 0x7fc038160cb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc0280098f0 tx=0x7fc028009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee7ff0766935fae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:56.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc035ffb700 1 -- 172.21.15.57:0/1603971720 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028011830 con 0x7fc03815c8c0 2023-11-28T12:01:56.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.800+0000 7fc035ffb700 1 -- 172.21.15.57:0/1603971720 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc028011990 con 0x7fc03815c8c0 2023-11-28T12:01:56.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.801+0000 7fc035ffb700 1 -- 172.21.15.57:0/1603971720 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028011d20 con 0x7fc03815c8c0 2023-11-28T12:01:56.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.801+0000 7fc03d77e700 1 -- 172.21.15.57:0/1603971720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc03815c8c0 msgr2=0x7fc038160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.801+0000 7fc03d77e700 1 --2- 172.21.15.57:0/1603971720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc03815c8c0 0x7fc038160cb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc0280098f0 tx=0x7fc028009c00 comp rx=0 tx=0).stop 2023-11-28T12:01:56.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.801+0000 7fc03d77e700 1 -- 172.21.15.57:0/1603971720 shutdown_connections 2023-11-28T12:01:56.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.801+0000 7fc03d77e700 1 --2- 172.21.15.57:0/1603971720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc03815c8c0 0x7fc038160cb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.801+0000 7fc03d77e700 1 --2- 172.21.15.57:0/1603971720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0380b5dc0 0x7fc03806ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.801+0000 7fc03d77e700 1 -- 172.21.15.57:0/1603971720 >> 172.21.15.57:0/1603971720 conn(0x7fc03806c520 msgr2=0x7fc03806c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.801+0000 7fc03d77e700 1 -- 172.21.15.57:0/1603971720 shutdown_connections 2023-11-28T12:01:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.801+0000 7fc03d77e700 1 -- 172.21.15.57:0/1603971720 wait complete. 2023-11-28T12:01:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.803+0000 7fc03d77e700 1 Processor -- start 2023-11-28T12:01:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.803+0000 7fc03d77e700 1 -- start start 2023-11-28T12:01:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.803+0000 7fc03d77e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03806b050 0x7fc038153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.803+0000 7fc03d77e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0380b5dc0 0x7fc038157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.803+0000 7fc03d77e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03815c8c0 0x7fc038157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.803+0000 7fc03d77e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380cc060 con 0x7fc0380b5dc0 2023-11-28T12:01:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.803+0000 7fc03d77e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03815eca0 con 0x7fc03806b050 2023-11-28T12:01:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.803+0000 7fc03d77e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380cbd80 con 0x7fc03815c8c0 2023-11-28T12:01:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.803+0000 7fc0367fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0380b5dc0 0x7fc038157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.803+0000 7fc0367fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0380b5dc0 0x7fc038157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38896/0 (socket says 172.21.15.57:38896) 2023-11-28T12:01:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.803+0000 7fc0367fc700 1 -- 172.21.15.57:0/1559649521 learned_addr learned my addr 172.21.15.57:0/1559649521 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc036ffd700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03806b050 0x7fc038153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc0377fe700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03815c8c0 0x7fc038157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc0367fc700 1 -- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03815c8c0 msgr2=0x7fc038157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc0367fc700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03815c8c0 0x7fc038157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc0367fc700 1 -- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03806b050 msgr2=0x7fc038153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc0367fc700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03806b050 0x7fc038153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc0367fc700 1 -- 172.21.15.57:0/1559649521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc028009550 con 0x7fc0380b5dc0 2023-11-28T12:01:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc0377fe700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03815c8c0 0x7fc038157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:01:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc0367fc700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0380b5dc0 0x7fc038157020 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00e9b0 tx=0x7fc02c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc01f7fe700 1 -- 172.21.15.57:0/1559649521 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c009710 con 0x7fc0380b5dc0 2023-11-28T12:01:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc01f7fe700 1 -- 172.21.15.57:0/1559649521 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c00c850 con 0x7fc0380b5dc0 2023-11-28T12:01:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc03d77e700 1 -- 172.21.15.57:0/1559649521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0380cd760 con 0x7fc0380b5dc0 2023-11-28T12:01:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc01f7fe700 1 -- 172.21.15.57:0/1559649521 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c00cbc0 con 0x7fc0380b5dc0 2023-11-28T12:01:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.804+0000 7fc03d77e700 1 -- 172.21.15.57:0/1559649521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0380d2a50 con 0x7fc0380b5dc0 2023-11-28T12:01:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.805+0000 7fc03d77e700 1 -- 172.21.15.57:0/1559649521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0380cda40 con 0x7fc0380b5dc0 2023-11-28T12:01:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.806+0000 7fc01f7fe700 1 -- 172.21.15.57:0/1559649521 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc02c019030 con 0x7fc0380b5dc0 2023-11-28T12:01:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.806+0000 7fc01f7fe700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0240682f0 0x7fc02406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.807+0000 7fc01f7fe700 1 -- 172.21.15.57:0/1559649521 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc02c0864c0 con 0x7fc0380b5dc0 2023-11-28T12:01:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.807+0000 7fc036ffd700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0240682f0 0x7fc02406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.807+0000 7fc036ffd700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0240682f0 0x7fc02406a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc028009c40 tx=0x7fc028008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:56.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:56.810+0000 7fc01f7fe700 1 -- 172.21.15.57:0/1559649521 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc02c052cb0 con 0x7fc0380b5dc0 2023-11-28T12:01:57.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.007+0000 7fc03d77e700 1 -- 172.21.15.57:0/1559649521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc038067d70 con 0x7fc0380b5dc0 2023-11-28T12:01:57.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.008+0000 7fc01f7fe700 1 -- 172.21.15.57:0/1559649521 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc02c0562d0 con 0x7fc0380b5dc0 2023-11-28T12:01:57.009 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:01:57.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.010+0000 7fc03d77e700 1 -- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0240682f0 msgr2=0x7fc02406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:57.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.010+0000 7fc03d77e700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0240682f0 0x7fc02406a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc028009c40 tx=0x7fc028008030 comp rx=0 tx=0).stop 2023-11-28T12:01:57.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.010+0000 7fc03d77e700 1 -- 172.21.15.57:0/1559649521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0380b5dc0 msgr2=0x7fc038157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:57.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.010+0000 7fc03d77e700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0380b5dc0 0x7fc038157020 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00e9b0 tx=0x7fc02c00ed70 comp rx=0 tx=0).stop 2023-11-28T12:01:57.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.010+0000 7fc03d77e700 1 -- 172.21.15.57:0/1559649521 shutdown_connections 2023-11-28T12:01:57.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.010+0000 7fc03d77e700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0240682f0 0x7fc02406a7a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.010+0000 7fc03d77e700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03815c8c0 0x7fc038157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.010+0000 7fc03d77e700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03806b050 0x7fc038153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.010+0000 7fc03d77e700 1 --2- 172.21.15.57:0/1559649521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0380b5dc0 0x7fc038157020 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.010+0000 7fc03d77e700 1 -- 172.21.15.57:0/1559649521 >> 172.21.15.57:0/1559649521 conn(0x7fc03806c520 msgr2=0x7fc0380b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:57.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.010+0000 7fc03d77e700 1 -- 172.21.15.57:0/1559649521 shutdown_connections 2023-11-28T12:01:57.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.010+0000 7fc03d77e700 1 -- 172.21.15.57:0/1559649521 wait complete. 2023-11-28T12:01:57.025 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:01:57.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.160+0000 7f26df87f700 1 Processor -- start 2023-11-28T12:01:57.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.160+0000 7f26df87f700 1 -- start start 2023-11-28T12:01:57.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.161+0000 7f26df87f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d8160fd0 0x7f26d8065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:57.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.161+0000 7f26df87f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f26d806ba00 0x7f26d80aca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:57.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.161+0000 7f26df87f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d80ac5f0 con 0x7f26d80c2420 2023-11-28T12:01:57.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.161+0000 7f26df87f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d80b4520 con 0x7f26d806ba00 2023-11-28T12:01:57.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.161+0000 7f26df87f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d80b4800 con 0x7f26d8160fd0 2023-11-28T12:01:57.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.161+0000 7f26dd61b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f26d80c2420 0x7f26d80c27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41188/0 (socket says 172.21.15.57:41188) 2023-11-28T12:01:57.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.161+0000 7f26dd61b700 1 -- 172.21.15.57:0/2364513660 learned_addr learned my addr 172.21.15.57:0/2364513660 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:57.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.161+0000 7f26dce1a700 1 --2- 172.21.15.57:0/2364513660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d8160fd0 0x7f26d8065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:57.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.161+0000 7f26dde1c700 1 --2- 172.21.15.57:0/2364513660 >> v2:172.21.15.181:3300/0 conn(0x7f26d806ba00 0x7f26d80aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:57.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.161+0000 7f26cffff700 1 -- 172.21.15.57:0/2364513660 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068567338 0 0) 0x7f26d80ac5f0 con 0x7f26d80c2420 2023-11-28T12:01:57.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.161+0000 7f26cffff700 1 -- 172.21.15.57:0/2364513660 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26c40035b0 con 0x7f26d80c2420 2023-11-28T12:01:57.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.162+0000 7f26cffff700 1 -- 172.21.15.57:0/2364513660 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3619911732 0 0) 0x7f26c40035b0 con 0x7f26d80c2420 2023-11-28T12:01:57.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.162+0000 7f26dde1c700 1 -- 172.21.15.57:0/2364513660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d8160fd0 msgr2=0x7f26d8065ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:57.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.162+0000 7f26dde1c700 1 --2- 172.21.15.57:0/2364513660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d8160fd0 0x7f26d8065ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.162+0000 7f26dde1c700 1 -- 172.21.15.57:0/2364513660 >> v1:172.21.15.57:6789/0 conn(0x7f26d80c2420 legacy=0x7f26d80c27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:57.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.162+0000 7f26dde1c700 1 -- 172.21.15.57:0/2364513660 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d80b4ae0 con 0x7f26d806ba00 2023-11-28T12:01:57.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.162+0000 7f26dce1a700 1 --2- 172.21.15.57:0/2364513660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d8160fd0 0x7f26d8065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:01:57.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.162+0000 7f26dde1c700 1 --2- 172.21.15.57:0/2364513660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d806ba00 0x7f26d80aca80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009d90 tx=0x7f26d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14e59b5f88e11f31 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:57.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.162+0000 7f26cffff700 1 -- 172.21.15.57:0/2364513660 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d000e070 con 0x7f26d806ba00 2023-11-28T12:01:57.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.162+0000 7f26cffff700 1 -- 172.21.15.57:0/2364513660 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d0003a20 con 0x7f26d806ba00 2023-11-28T12:01:57.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.162+0000 7f26cffff700 1 -- 172.21.15.57:0/2364513660 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8001670 con 0x7f26d806ba00 2023-11-28T12:01:57.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.163+0000 7f26df87f700 1 -- 172.21.15.57:0/2364513660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d806ba00 msgr2=0x7f26d80aca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:57.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.163+0000 7f26df87f700 1 --2- 172.21.15.57:0/2364513660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d806ba00 0x7f26d80aca80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009d90 tx=0x7f26d000b330 comp rx=0 tx=0).stop 2023-11-28T12:01:57.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.163+0000 7f26df87f700 1 -- 172.21.15.57:0/2364513660 shutdown_connections 2023-11-28T12:01:57.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.163+0000 7f26df87f700 1 --2- 172.21.15.57:0/2364513660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d806ba00 0x7f26d80aca80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.163+0000 7f26df87f700 1 --2- 172.21.15.57:0/2364513660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d8160fd0 0x7f26d8065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.163+0000 7f26df87f700 1 -- 172.21.15.57:0/2364513660 >> 172.21.15.57:0/2364513660 conn(0x7f26d80b2fc0 msgr2=0x7f26d8071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:57.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.163+0000 7f26df87f700 1 -- 172.21.15.57:0/2364513660 shutdown_connections 2023-11-28T12:01:57.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.163+0000 7f26df87f700 1 -- 172.21.15.57:0/2364513660 wait complete. 2023-11-28T12:01:57.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.164+0000 7f26df87f700 1 Processor -- start 2023-11-28T12:01:57.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.164+0000 7f26df87f700 1 -- start start 2023-11-28T12:01:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.165+0000 7f26df87f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d806ba00 0x7f26d80c3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.165+0000 7f26df87f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d80c2420 0x7f26d815c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.165+0000 7f26df87f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26d8160fd0 0x7f26d80af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.165+0000 7f26df87f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d80c4ad0 con 0x7f26d8160fd0 2023-11-28T12:01:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.165+0000 7f26df87f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d80c4db0 con 0x7f26d806ba00 2023-11-28T12:01:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.165+0000 7f26df87f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d80c5090 con 0x7f26d80c2420 2023-11-28T12:01:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26dde1c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26d8160fd0 0x7f26d80af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26dde1c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26d8160fd0 0x7f26d80af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38906/0 (socket says 172.21.15.57:38906) 2023-11-28T12:01:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26dde1c700 1 -- 172.21.15.57:0/1737683855 learned_addr learned my addr 172.21.15.57:0/1737683855 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26dce1a700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d80c2420 0x7f26d815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26dd61b700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d806ba00 0x7f26d80c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26dde1c700 1 -- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d80c2420 msgr2=0x7f26d815c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:57.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26dde1c700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d80c2420 0x7f26d815c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26dde1c700 1 -- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d806ba00 msgr2=0x7f26d80c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26dde1c700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d806ba00 0x7f26d80c3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26dde1c700 1 -- 172.21.15.57:0/1737683855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d40094a0 con 0x7f26d8160fd0 2023-11-28T12:01:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26dde1c700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26d8160fd0 0x7f26d80af3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f26d0004000 tx=0x7f26d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26cdffb700 1 -- 172.21.15.57:0/1737683855 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0004440 con 0x7f26d8160fd0 2023-11-28T12:01:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26cdffb700 1 -- 172.21.15.57:0/1737683855 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d00045c0 con 0x7f26d8160fd0 2023-11-28T12:01:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26df87f700 1 -- 172.21.15.57:0/1737683855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d0009a70 con 0x7f26d8160fd0 2023-11-28T12:01:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.166+0000 7f26cdffb700 1 -- 172.21.15.57:0/1737683855 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0004970 con 0x7f26d8160fd0 2023-11-28T12:01:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.167+0000 7f26df87f700 1 -- 172.21.15.57:0/1737683855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d8066430 con 0x7f26d8160fd0 2023-11-28T12:01:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.168+0000 7f26df87f700 1 -- 172.21.15.57:0/1737683855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d80ac280 con 0x7f26d8160fd0 2023-11-28T12:01:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.168+0000 7f26cdffb700 1 -- 172.21.15.57:0/1737683855 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f26d0012430 con 0x7f26d8160fd0 2023-11-28T12:01:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.169+0000 7f26cdffb700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f26bc0682f0 0x7f26bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:57.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.169+0000 7f26cdffb700 1 -- 172.21.15.57:0/1737683855 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f26d0087bc0 con 0x7f26d8160fd0 2023-11-28T12:01:57.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.169+0000 7f26dd61b700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f26bc0682f0 0x7f26bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:57.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.169+0000 7f26dd61b700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f26bc0682f0 0x7f26bc06a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f26c8001820 tx=0x7f26c8001230 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:57.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.173+0000 7f26cdffb700 1 -- 172.21.15.57:0/1737683855 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f26d00543c0 con 0x7f26d8160fd0 2023-11-28T12:01:57.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.418+0000 7f26df87f700 1 -- 172.21.15.57:0/1737683855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26d80665c0 con 0x7f26d8160fd0 2023-11-28T12:01:57.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.419+0000 7f26cdffb700 1 -- 172.21.15.57:0/1737683855 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4642 (secure 0 0 0) 0x7f26d002a090 con 0x7f26d8160fd0 2023-11-28T12:01:57.420 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:01:57.420 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/890913444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":890913444},{"type":"v1","addr":"172.21.15.181:6836","nonce":890913444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/847658150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":847658150},{"type":"v1","addr":"172.21.15.181:6837","nonce":847658150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1379013929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1379013929},{"type":"v1","addr":"172.21.15.57:6835","nonce":1379013929}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:01:43.919415+0000","modified":"2023-11-28T12:01:46.937696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6833/3776085595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3776085595},{"type":"v1","addr":"172.21.15.57:6833","nonce":3776085595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T12:01:57.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.421+0000 7f26df87f700 1 -- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f26bc0682f0 msgr2=0x7f26bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:57.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.421+0000 7f26df87f700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f26bc0682f0 0x7f26bc06a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f26c8001820 tx=0x7f26c8001230 comp rx=0 tx=0).stop 2023-11-28T12:01:57.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.421+0000 7f26df87f700 1 -- 172.21.15.57:0/1737683855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26d8160fd0 msgr2=0x7f26d80af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:57.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.421+0000 7f26df87f700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26d8160fd0 0x7f26d80af3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f26d0004000 tx=0x7f26d0004ad0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.421+0000 7f26df87f700 1 -- 172.21.15.57:0/1737683855 shutdown_connections 2023-11-28T12:01:57.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.421+0000 7f26df87f700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f26bc0682f0 0x7f26bc06a7a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.421+0000 7f26df87f700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26d8160fd0 0x7f26d80af3c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.421+0000 7f26df87f700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d806ba00 0x7f26d80c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.421+0000 7f26df87f700 1 --2- 172.21.15.57:0/1737683855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d80c2420 0x7f26d815c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.421+0000 7f26df87f700 1 -- 172.21.15.57:0/1737683855 >> 172.21.15.57:0/1737683855 conn(0x7f26d80b2fc0 msgr2=0x7f26d80cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:57.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.421+0000 7f26df87f700 1 -- 172.21.15.57:0/1737683855 shutdown_connections 2023-11-28T12:01:57.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.421+0000 7f26df87f700 1 -- 172.21.15.57:0/1737683855 wait complete. 2023-11-28T12:01:57.424 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 8 2023-11-28T12:01:57.838 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T12:01:43.919415+0000', 'modified': '2023-11-28T12:01:46.937696+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4276}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4276': {'gid': 4276, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.57:6833/3776085595', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6832', 'nonce': 3776085595}, {'type': 'v1', 'addr': '172.21.15.57:6833', 'nonce': 3776085595}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T12:01:57.839 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T12:01:57.839 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T12:01:57.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.984+0000 7f80ae69e700 1 Processor -- start 2023-11-28T12:01:57.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.984+0000 7f80ae69e700 1 -- start start 2023-11-28T12:01:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.985+0000 7f80ae69e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f80a815a590 0x7f80a806b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.985+0000 7f80ae69e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80a80b50d0 0x7f80a806c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.985+0000 7f80ae69e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a80cd480 con 0x7f80a815a960 2023-11-28T12:01:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.985+0000 7f80ae69e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a80cd760 con 0x7f80a815a590 2023-11-28T12:01:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.985+0000 7f80ae69e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80cda40 con 0x7f80a80b50d0 2023-11-28T12:01:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.985+0000 7f80a7fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f80a815a590 0x7f80a806b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.985+0000 7f80acc3b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f80a815a960 0x7f80a815e280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41190/0 (socket says 172.21.15.57:41190) 2023-11-28T12:01:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.985+0000 7f80acc3b700 1 -- 172.21.15.57:0/902640149 learned_addr learned my addr 172.21.15.57:0/902640149 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.985+0000 7f80a77fe700 1 --2- 172.21.15.57:0/902640149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80a80b50d0 0x7f80a806c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.986+0000 7f80a6ffd700 1 -- 172.21.15.57:0/902640149 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475477841 0 0) 0x7f80a80cd480 con 0x7f80a815a960 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.986+0000 7f80a6ffd700 1 -- 172.21.15.57:0/902640149 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80900035b0 con 0x7f80a815a960 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.986+0000 7f80a7fff700 1 -- 172.21.15.57:0/902640149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80a80b50d0 msgr2=0x7f80a806c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.986+0000 7f80a7fff700 1 --2- 172.21.15.57:0/902640149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80a80b50d0 0x7f80a806c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.986+0000 7f80a7fff700 1 -- 172.21.15.57:0/902640149 >> v1:172.21.15.57:6789/0 conn(0x7f80a815a960 legacy=0x7f80a815e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.986+0000 7f80a7fff700 1 -- 172.21.15.57:0/902640149 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a80cdd20 con 0x7f80a815a590 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.986+0000 7f80a7fff700 1 --2- 172.21.15.57:0/902640149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a815a590 0x7f80a806b5b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8098009840 tx=0x7f8098009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53d769e577b3302 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.986+0000 7f80a6ffd700 1 -- 172.21.15.57:0/902640149 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098011830 con 0x7f80a815a590 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.986+0000 7f80a6ffd700 1 -- 172.21.15.57:0/902640149 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8098011990 con 0x7f80a815a590 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.986+0000 7f80a6ffd700 1 -- 172.21.15.57:0/902640149 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098011d00 con 0x7f80a815a590 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.987+0000 7f80ae69e700 1 -- 172.21.15.57:0/902640149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a815a590 msgr2=0x7f80a806b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.987+0000 7f80ae69e700 1 --2- 172.21.15.57:0/902640149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a815a590 0x7f80a806b5b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8098009840 tx=0x7f8098009b50 comp rx=0 tx=0).stop 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.987+0000 7f80ae69e700 1 -- 172.21.15.57:0/902640149 shutdown_connections 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.987+0000 7f80ae69e700 1 --2- 172.21.15.57:0/902640149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a815a590 0x7f80a806b5b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.987+0000 7f80ae69e700 1 --2- 172.21.15.57:0/902640149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80a80b50d0 0x7f80a806c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.987+0000 7f80ae69e700 1 -- 172.21.15.57:0/902640149 >> 172.21.15.57:0/902640149 conn(0x7f80a80cf400 msgr2=0x7f80a80cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:57.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.987+0000 7f80ae69e700 1 -- 172.21.15.57:0/902640149 shutdown_connections 2023-11-28T12:01:57.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.987+0000 7f80ae69e700 1 -- 172.21.15.57:0/902640149 wait complete. 2023-11-28T12:01:57.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.988+0000 7f80ae69e700 1 Processor -- start 2023-11-28T12:01:57.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.988+0000 7f80ae69e700 1 -- start start 2023-11-28T12:01:57.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80ae69e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80a80b50d0 0x7f80a80c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:57.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80ae69e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a815a590 0x7f80a80c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:57.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80ae69e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80a815a960 0x7f80a8157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:57.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80ae69e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80c7b20 con 0x7f80a815a590 2023-11-28T12:01:57.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80ae69e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80c7e00 con 0x7f80a815a960 2023-11-28T12:01:57.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80ae69e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80c80e0 con 0x7f80a80b50d0 2023-11-28T12:01:57.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80a77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a815a590 0x7f80a80c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:57.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80a77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a815a590 0x7f80a80c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38920/0 (socket says 172.21.15.57:38920) 2023-11-28T12:01:57.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80a77fe700 1 -- 172.21.15.57:0/214978492 learned_addr learned my addr 172.21.15.57:0/214978492 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80acc3b700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80a815a960 0x7f80a8157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80a7fff700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80a80b50d0 0x7f80a80c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80a77fe700 1 -- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80a80b50d0 msgr2=0x7f80a80c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80a77fe700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80a80b50d0 0x7f80a80c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80a77fe700 1 -- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80a815a960 msgr2=0x7f80a8157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80a77fe700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80a815a960 0x7f80a8157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.989+0000 7f80a77fe700 1 -- 172.21.15.57:0/214978492 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80980094a0 con 0x7f80a815a590 2023-11-28T12:01:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.990+0000 7f80a77fe700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a815a590 0x7f80a80c8bc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f809c00e9b0 tx=0x7f809c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.990+0000 7f80a4ff9700 1 -- 172.21.15.57:0/214978492 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c009710 con 0x7f80a815a590 2023-11-28T12:01:57.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.990+0000 7f80a4ff9700 1 -- 172.21.15.57:0/214978492 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809c00c850 con 0x7f80a815a590 2023-11-28T12:01:57.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.990+0000 7f80ae69e700 1 -- 172.21.15.57:0/214978492 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a806baf0 con 0x7f80a815a590 2023-11-28T12:01:57.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.990+0000 7f80a4ff9700 1 -- 172.21.15.57:0/214978492 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c00cbc0 con 0x7f80a815a590 2023-11-28T12:01:57.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.990+0000 7f80ae69e700 1 -- 172.21.15.57:0/214978492 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a80ce000 con 0x7f80a815a590 2023-11-28T12:01:58.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.991+0000 7f80a4ff9700 1 -- 172.21.15.57:0/214978492 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f809c019030 con 0x7f80a815a590 2023-11-28T12:01:58.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.992+0000 7f80a4ff9700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f80940682f0 0x7f809406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.992+0000 7f80a4ff9700 1 -- 172.21.15.57:0/214978492 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f809c085da0 con 0x7f80a815a590 2023-11-28T12:01:58.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.992+0000 7f80a7fff700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f80940682f0 0x7f809406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.993+0000 7f80a7fff700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f80940682f0 0x7f809406a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8098009fd0 tx=0x7f8098008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:58.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.994+0000 7f80ba83db80 1 Processor -- start 2023-11-28T12:01:58.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.994+0000 7f80ba83db80 1 -- start start 2023-11-28T12:01:58.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80ba83db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f4590a0 0x56128f42a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80ba83db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f430690 0x56128f459470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80ba83db80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f42f5e0 0x56128f475a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80ba83db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128f3abde0 con 0x56128f4590a0 2023-11-28T12:01:58.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80ba83db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128f318140 con 0x56128f430690 2023-11-28T12:01:58.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80ba83db80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56128f386db0 con 0x56128f42f5e0 2023-11-28T12:01:58.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80a7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f4590a0 0x56128f42a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80a7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f4590a0 0x56128f42a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38924/0 (socket says 172.21.15.57:38924) 2023-11-28T12:01:58.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80a7fff700 1 -- 172.21.15.57:0/4246072205 learned_addr learned my addr 172.21.15.57:0/4246072205 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:58.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80acc3b700 1 --2- 172.21.15.57:0/4246072205 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f42f5e0 0x56128f475a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80a77fe700 1 --2- 172.21.15.57:0/4246072205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f430690 0x56128f459470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80a7fff700 1 -- 172.21.15.57:0/4246072205 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f42f5e0 msgr2=0x56128f475a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80a7fff700 1 --2- 172.21.15.57:0/4246072205 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f42f5e0 0x56128f475a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80a7fff700 1 -- 172.21.15.57:0/4246072205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f430690 msgr2=0x56128f459470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80a7fff700 1 --2- 172.21.15.57:0/4246072205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f430690 0x56128f459470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80a7fff700 1 -- 172.21.15.57:0/4246072205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56128f174790 con 0x56128f4590a0 2023-11-28T12:01:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.995+0000 7f80a7fff700 1 --2- 172.21.15.57:0/4246072205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f4590a0 0x56128f42a730 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f80a806fef0 tx=0x7f8098010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c96ec20fe758384 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.996+0000 7f808dffb700 1 -- 172.21.15.57:0/4246072205 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809801f070 con 0x56128f4590a0 2023-11-28T12:01:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.996+0000 7f808dffb700 1 -- 172.21.15.57:0/4246072205 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80980036a0 con 0x56128f4590a0 2023-11-28T12:01:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.996+0000 7f808dffb700 1 -- 172.21.15.57:0/4246072205 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098003a50 con 0x56128f4590a0 2023-11-28T12:01:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.996+0000 7f80ba83db80 1 -- 172.21.15.57:0/4246072205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f4590a0 msgr2=0x56128f42a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.996+0000 7f80ba83db80 1 --2- 172.21.15.57:0/4246072205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f4590a0 0x56128f42a730 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f80a806fef0 tx=0x7f8098010000 comp rx=0 tx=0).stop 2023-11-28T12:01:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.996+0000 7f80ba83db80 1 -- 172.21.15.57:0/4246072205 shutdown_connections 2023-11-28T12:01:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.996+0000 7f80ba83db80 1 --2- 172.21.15.57:0/4246072205 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f42f5e0 0x56128f475a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.996+0000 7f80ba83db80 1 --2- 172.21.15.57:0/4246072205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f4590a0 0x56128f42a730 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.996+0000 7f80ba83db80 1 --2- 172.21.15.57:0/4246072205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f430690 0x56128f459470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.996+0000 7f80ba83db80 1 -- 172.21.15.57:0/4246072205 >> 172.21.15.57:0/4246072205 conn(0x56128f423d20 msgr2=0x56128f42cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.996+0000 7f80ba83db80 1 -- 172.21.15.57:0/4246072205 shutdown_connections 2023-11-28T12:01:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.996+0000 7f80ba83db80 1 -- 172.21.15.57:0/4246072205 wait complete. 2023-11-28T12:01:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.997+0000 7f80ba83db80 1 -- start start 2023-11-28T12:01:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.997+0000 7f80ba83db80 1 Processor -- start 2023-11-28T12:01:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.998+0000 7f80ba83db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f42f5e0 0x56128f44bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.998+0000 7f80ba83db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f430690 0x56128f474080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.998+0000 7f80ba83db80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f4590a0 0x56128f44f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.998+0000 7f80ba83db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128f380120 con 0x56128f430690 2023-11-28T12:01:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.998+0000 7f80ba83db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128f175390 con 0x56128f42f5e0 2023-11-28T12:01:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.998+0000 7f80ba83db80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56128f318340 con 0x56128f4590a0 2023-11-28T12:01:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.998+0000 7f80ba83db80 10 client.0 _ll_register_callbacks cb 0x56128f31e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:01:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.998+0000 7f80a77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f430690 0x56128f474080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.998+0000 7f80a77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f430690 0x56128f474080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38938/0 (socket says 172.21.15.57:38938) 2023-11-28T12:01:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.998+0000 7f80a77fe700 1 -- 172.21.15.57:0/3173030793 learned_addr learned my addr 172.21.15.57:0/3173030793 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.998+0000 7f80a7fff700 1 --2- 172.21.15.57:0/3173030793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f42f5e0 0x56128f44bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.998+0000 7f80acc3b700 1 --2- 172.21.15.57:0/3173030793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f4590a0 0x56128f44f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f80a77fe700 1 -- 172.21.15.57:0/3173030793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f4590a0 msgr2=0x56128f44f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f80a77fe700 1 --2- 172.21.15.57:0/3173030793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f4590a0 0x56128f44f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f80a77fe700 1 -- 172.21.15.57:0/3173030793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f42f5e0 msgr2=0x56128f44bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f80a77fe700 1 --2- 172.21.15.57:0/3173030793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f42f5e0 0x56128f44bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f80a77fe700 1 -- 172.21.15.57:0/3173030793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a80cdd20 con 0x56128f430690 2023-11-28T12:01:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f80a77fe700 1 --2- 172.21.15.57:0/3173030793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f430690 0x56128f474080 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f809c006d70 tx=0x7f809c00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43072cec8c114d37 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f8086ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:01:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f8086ffd700 1 -- 172.21.15.57:0/3173030793 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c05b040 con 0x56128f430690 2023-11-28T12:01:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f8086ffd700 1 -- 172.21.15.57:0/3173030793 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f809c06d3c0 con 0x56128f430690 2023-11-28T12:01:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f80ba83db80 1 -- 172.21.15.57:0/3173030793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56128f384030 con 0x56128f430690 2023-11-28T12:01:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f8086ffd700 1 -- 172.21.15.57:0/3173030793 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f809c05e0a0 con 0x56128f430690 2023-11-28T12:01:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f80ba83db80 10 client.4566 fetch_fsmap learned FSMap version 8 2023-11-28T12:01:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f80ba83db80 1 -- 172.21.15.57:0/3173030793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56128f2610a0 con 0x56128f430690 2023-11-28T12:01:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f8086ffd700 1 -- 172.21.15.57:0/3173030793 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56128f384030 con 0x56128f430690 2023-11-28T12:01:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:57.999+0000 7f8086ffd700 1 -- 172.21.15.57:0/3173030793 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f809c051fc0 con 0x56128f430690 2023-11-28T12:01:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.000+0000 7f80ba83db80 10 client.4566 fetch_fsmap finished waiting for FSMap version 8 2023-11-28T12:01:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.000+0000 7f80ba83db80 10 client.4566 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T12:01:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.000+0000 7f80ba83db80 20 client.4566 populate_metadata read hostname 'smithi057' 2023-11-28T12:01:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.000+0000 7f80ba83db80 1 --2- 172.21.15.57:0/3173030793 >> [v2:172.21.15.57:6832/3776085595,v1:172.21.15.57:6833/3776085595] conn(0x56128f4580f0 0x56128f4584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.000+0000 7f80ba83db80 4 client.4566 mds_command: new command op to 4276 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T12:01:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.000+0000 7f80ba83db80 1 -- 172.21.15.57:0/3173030793 --> [v2:172.21.15.57:6832/3776085595,v1:172.21.15.57:6833/3776085595] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56128f1cfef0 con 0x56128f4580f0 2023-11-28T12:01:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.000+0000 7f80a7fff700 1 --2- 172.21.15.57:0/3173030793 >> [v2:172.21.15.57:6832/3776085595,v1:172.21.15.57:6833/3776085595] conn(0x56128f4580f0 0x56128f4584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.000+0000 7f8086ffd700 10 client.4566 ms_handle_connect on v2:172.21.15.57:6832/3776085595 2023-11-28T12:01:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.000+0000 7f80a7fff700 1 --2- 172.21.15.57:0/3173030793 >> [v2:172.21.15.57:6832/3776085595,v1:172.21.15.57:6833/3776085595] conn(0x56128f4580f0 0x56128f4584c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54088cc1aa81113a server_cookie=15a1b3747f92d504 in_seq=0 out_seq=0 2023-11-28T12:01:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f8086ffd700 1 -- 172.21.15.57:0/3173030793 <== mds.0 v2:172.21.15.57:6832/3776085595 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x56128f1cfef0 con 0x56128f4580f0 2023-11-28T12:01:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f8086ffd700 10 client.4566 handle_command_reply: tid=0 2023-11-28T12:01:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f80ba83db80 1 client.4566 shutdown 2023-11-28T12:01:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f80ba83db80 1 -- 172.21.15.57:0/3173030793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f430690 msgr2=0x56128f474080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f80ba83db80 1 --2- 172.21.15.57:0/3173030793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f430690 0x56128f474080 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f809c006d70 tx=0x7f809c00cda0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f80ba83db80 1 -- 172.21.15.57:0/3173030793 shutdown_connections 2023-11-28T12:01:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f80ba83db80 1 --2- 172.21.15.57:0/3173030793 >> [v2:172.21.15.57:6832/3776085595,v1:172.21.15.57:6833/3776085595] conn(0x56128f4580f0 0x56128f4584c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f80ba83db80 1 --2- 172.21.15.57:0/3173030793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f4590a0 0x56128f44f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f80ba83db80 1 --2- 172.21.15.57:0/3173030793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f42f5e0 0x56128f44bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f80ba83db80 1 --2- 172.21.15.57:0/3173030793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f430690 0x56128f474080 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f8086ffd700 0 client.4566 ms_handle_reset on v2:172.21.15.57:6832/3776085595 2023-11-28T12:01:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f80ba83db80 1 -- 172.21.15.57:0/3173030793 >> 172.21.15.57:0/3173030793 conn(0x56128f423d20 msgr2=0x56128f44ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f80ba83db80 1 -- 172.21.15.57:0/3173030793 shutdown_connections 2023-11-28T12:01:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.001+0000 7f80ba83db80 1 -- 172.21.15.57:0/3173030793 wait complete. 2023-11-28T12:01:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.002+0000 7f80ba83db80 20 client.4566 trim_cache size 0 max 16384 2023-11-28T12:01:58.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.020+0000 7f80ba83db80 1 Processor -- start 2023-11-28T12:01:58.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.020+0000 7f80ba83db80 1 -- start start 2023-11-28T12:01:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.020+0000 7f80ba83db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 0x56128f428700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.020+0000 7f80ba83db80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f430690 0x56128f44f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.020+0000 7f80ba83db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f4590a0 0x56128f45ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.020+0000 7f80ba83db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128f49a220 con 0x56128f42f5e0 2023-11-28T12:01:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.020+0000 7f80ba83db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128f473c00 con 0x56128f4590a0 2023-11-28T12:01:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.020+0000 7f80ba83db80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56128f476410 con 0x56128f430690 2023-11-28T12:01:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.021+0000 7f80a7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 0x56128f428700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.021+0000 7f80a7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 0x56128f428700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38944/0 (socket says 172.21.15.57:38944) 2023-11-28T12:01:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.021+0000 7f80a7fff700 1 -- 172.21.15.57:0/2889325331 learned_addr learned my addr 172.21.15.57:0/2889325331 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.021+0000 7f80acc3b700 1 --2- 172.21.15.57:0/2889325331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f4590a0 0x56128f45ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.021+0000 7f80a7fff700 1 -- 172.21.15.57:0/2889325331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f430690 msgr2=0x56128f44f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.021+0000 7f80a7fff700 1 --2- 172.21.15.57:0/2889325331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f430690 0x56128f44f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.021+0000 7f80a7fff700 1 -- 172.21.15.57:0/2889325331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f4590a0 msgr2=0x56128f45ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.021+0000 7f80a7fff700 1 --2- 172.21.15.57:0/2889325331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f4590a0 0x56128f45ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.021+0000 7f80a7fff700 1 -- 172.21.15.57:0/2889325331 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56128f3472e0 con 0x56128f42f5e0 2023-11-28T12:01:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.021+0000 7f80a7fff700 1 --2- 172.21.15.57:0/2889325331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 0x56128f428700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8098000c00 tx=0x7f809801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450f0570a6217174 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.021+0000 7f808e7fc700 1 -- 172.21.15.57:0/2889325331 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098035040 con 0x56128f42f5e0 2023-11-28T12:01:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.021+0000 7f808e7fc700 1 -- 172.21.15.57:0/2889325331 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8098036040 con 0x56128f42f5e0 2023-11-28T12:01:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.021+0000 7f808e7fc700 1 -- 172.21.15.57:0/2889325331 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809803e920 con 0x56128f42f5e0 2023-11-28T12:01:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.022+0000 7f80ba83db80 1 -- 172.21.15.57:0/2889325331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 msgr2=0x56128f428700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.022+0000 7f80ba83db80 1 --2- 172.21.15.57:0/2889325331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 0x56128f428700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8098000c00 tx=0x7f809801f000 comp rx=0 tx=0).stop 2023-11-28T12:01:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.022+0000 7f80ba83db80 1 -- 172.21.15.57:0/2889325331 shutdown_connections 2023-11-28T12:01:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.022+0000 7f80ba83db80 1 --2- 172.21.15.57:0/2889325331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f4590a0 0x56128f45ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.022+0000 7f80ba83db80 1 --2- 172.21.15.57:0/2889325331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 0x56128f428700 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.022+0000 7f80ba83db80 1 --2- 172.21.15.57:0/2889325331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f430690 0x56128f44f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.022+0000 7f80ba83db80 1 -- 172.21.15.57:0/2889325331 >> 172.21.15.57:0/2889325331 conn(0x56128f423d20 msgr2=0x56128f42a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.022+0000 7f80ba83db80 1 -- 172.21.15.57:0/2889325331 shutdown_connections 2023-11-28T12:01:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.022+0000 7f80ba83db80 1 -- 172.21.15.57:0/2889325331 wait complete. 2023-11-28T12:01:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.022+0000 7f80ba83db80 1 -- start start 2023-11-28T12:01:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.023+0000 7f80ba83db80 1 Processor -- start 2023-11-28T12:01:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.023+0000 7f80ba83db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 0x56128f4282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.023+0000 7f80ba83db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f430690 0x56128f45ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.023+0000 7f80ba83db80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f4590a0 0x56128f490ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.023+0000 7f80ba83db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128f4dd9f0 con 0x56128f42f5e0 2023-11-28T12:01:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.023+0000 7f80ba83db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128f490900 con 0x56128f430690 2023-11-28T12:01:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.023+0000 7f80ba83db80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56128f490aa0 con 0x56128f4590a0 2023-11-28T12:01:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80ba83db80 10 client.0 _ll_register_callbacks cb 0x56128f31e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:01:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80a7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 0x56128f4282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80a7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 0x56128f4282b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38948/0 (socket says 172.21.15.57:38948) 2023-11-28T12:01:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80a7fff700 1 -- 172.21.15.57:0/2888080926 learned_addr learned my addr 172.21.15.57:0/2888080926 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80acc3b700 1 --2- 172.21.15.57:0/2888080926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f4590a0 0x56128f490ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80a77fe700 1 --2- 172.21.15.57:0/2888080926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f430690 0x56128f45ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80a7fff700 1 -- 172.21.15.57:0/2888080926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f4590a0 msgr2=0x56128f490ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80a7fff700 1 --2- 172.21.15.57:0/2888080926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f4590a0 0x56128f490ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80a7fff700 1 -- 172.21.15.57:0/2888080926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f430690 msgr2=0x56128f45ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:01:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80a7fff700 1 --2- 172.21.15.57:0/2888080926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f430690 0x56128f45ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80a7fff700 1 -- 172.21.15.57:0/2888080926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56128f174790 con 0x56128f42f5e0 2023-11-28T12:01:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80a7fff700 1 --2- 172.21.15.57:0/2888080926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 0x56128f4282b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8098000c00 tx=0x7f80980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42dbe5b7da5a58d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f8086ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:01:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f8086ffd700 1 -- 172.21.15.57:0/2888080926 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809801f050 con 0x56128f42f5e0 2023-11-28T12:01:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f8086ffd700 1 -- 172.21.15.57:0/2888080926 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f809803eb20 con 0x56128f42f5e0 2023-11-28T12:01:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80ba83db80 1 -- 172.21.15.57:0/2888080926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80980036a0 con 0x56128f42f5e0 2023-11-28T12:01:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f8086ffd700 1 -- 172.21.15.57:0/2888080926 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8098035040 con 0x56128f42f5e0 2023-11-28T12:01:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80ba83db80 10 client.4572 fetch_fsmap learned FSMap version 8 2023-11-28T12:01:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.024+0000 7f80ba83db80 1 -- 172.21.15.57:0/2888080926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56128f1d51d0 con 0x56128f42f5e0 2023-11-28T12:01:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f8086ffd700 1 -- 172.21.15.57:0/2888080926 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80980036a0 con 0x56128f42f5e0 2023-11-28T12:01:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f8086ffd700 1 -- 172.21.15.57:0/2888080926 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f809803ed20 con 0x56128f42f5e0 2023-11-28T12:01:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f80ba83db80 10 client.4572 fetch_fsmap finished waiting for FSMap version 8 2023-11-28T12:01:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f80ba83db80 10 client.4572 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T12:01:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f80ba83db80 20 client.4572 populate_metadata read hostname 'smithi057' 2023-11-28T12:01:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f80ba83db80 1 --2- 172.21.15.57:0/2888080926 >> [v2:172.21.15.57:6832/3776085595,v1:172.21.15.57:6833/3776085595] conn(0x56128f4e0fe0 0x56128f4e13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f80ba83db80 4 client.4572 mds_command: new command op to 4276 tid=0[{"prefix": "status"}] 2023-11-28T12:01:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f80ba83db80 1 -- 172.21.15.57:0/2888080926 --> [v2:172.21.15.57:6832/3776085595,v1:172.21.15.57:6833/3776085595] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56128f205040 con 0x56128f4e0fe0 2023-11-28T12:01:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f80a77fe700 1 --2- 172.21.15.57:0/2888080926 >> [v2:172.21.15.57:6832/3776085595,v1:172.21.15.57:6833/3776085595] conn(0x56128f4e0fe0 0x56128f4e13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f8086ffd700 10 client.4572 ms_handle_connect on v2:172.21.15.57:6832/3776085595 2023-11-28T12:01:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f80a77fe700 1 --2- 172.21.15.57:0/2888080926 >> [v2:172.21.15.57:6832/3776085595,v1:172.21.15.57:6833/3776085595] conn(0x56128f4e0fe0 0x56128f4e13b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21fa7534fc61ad2 server_cookie=739f0b25ceaf69d4 in_seq=0 out_seq=0 2023-11-28T12:01:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f8086ffd700 1 -- 172.21.15.57:0/2888080926 <== mds.0 v2:172.21.15.57:6832/3776085595 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x56128f205040 con 0x56128f4e0fe0 2023-11-28T12:01:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f8086ffd700 10 client.4572 handle_command_reply: tid=0 2023-11-28T12:01:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f80ba83db80 1 client.4572 shutdown 2023-11-28T12:01:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f80ba83db80 1 -- 172.21.15.57:0/2888080926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 msgr2=0x56128f4282b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.025+0000 7f80ba83db80 1 --2- 172.21.15.57:0/2888080926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 0x56128f4282b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8098000c00 tx=0x7f80980047c0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ba83db80 1 -- 172.21.15.57:0/2888080926 shutdown_connections 2023-11-28T12:01:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ba83db80 1 --2- 172.21.15.57:0/2888080926 >> [v2:172.21.15.57:6832/3776085595,v1:172.21.15.57:6833/3776085595] conn(0x56128f4e0fe0 0x56128f4e13b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ba83db80 1 --2- 172.21.15.57:0/2888080926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56128f4590a0 0x56128f490ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ba83db80 1 --2- 172.21.15.57:0/2888080926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56128f42f5e0 0x56128f4282b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f8086ffd700 0 client.4572 ms_handle_reset on v2:172.21.15.57:6832/3776085595 2023-11-28T12:01:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ba83db80 1 --2- 172.21.15.57:0/2888080926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56128f430690 0x56128f45ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ba83db80 1 -- 172.21.15.57:0/2888080926 >> 172.21.15.57:0/2888080926 conn(0x56128f423d20 msgr2=0x56128f42a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ba83db80 1 -- 172.21.15.57:0/2888080926 shutdown_connections 2023-11-28T12:01:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ba83db80 1 -- 172.21.15.57:0/2888080926 wait complete. 2023-11-28T12:01:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ba83db80 20 client.4572 trim_cache size 0 max 16384 2023-11-28T12:01:58.037 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:01:58.037 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "0309b83f-6570-4eee-96fa-279822cd7766", 2023-11-28T12:01:58.037 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2023-11-28T12:01:58.037 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4276, 2023-11-28T12:01:58.037 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-11-28T12:01:58.037 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-11-28T12:01:58.037 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-11-28T12:01:58.037 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 14.102002697, 2023-11-28T12:01:58.037 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 8, 2023-11-28T12:01:58.037 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 21, 2023-11-28T12:01:58.037 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 21, 2023-11-28T12:01:58.038 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 19.673308633000001 2023-11-28T12:01:58.038 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:01:58.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ae69e700 1 -- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f80940682f0 msgr2=0x7f809406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:58.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ae69e700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f80940682f0 0x7f809406a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8098009fd0 tx=0x7f8098008030 comp rx=0 tx=0).stop 2023-11-28T12:01:58.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ae69e700 1 -- 172.21.15.57:0/214978492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a815a590 msgr2=0x7f80a80c8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:58.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ae69e700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a815a590 0x7f80a80c8bc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f809c00e9b0 tx=0x7f809c00ed70 comp rx=0 tx=0).stop 2023-11-28T12:01:58.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ae69e700 1 -- 172.21.15.57:0/214978492 shutdown_connections 2023-11-28T12:01:58.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ae69e700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f80940682f0 0x7f809406a7a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ae69e700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80a815a960 0x7f80a8157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ae69e700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80a80b50d0 0x7f80a80c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ae69e700 1 --2- 172.21.15.57:0/214978492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a815a590 0x7f80a80c8bc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ae69e700 1 -- 172.21.15.57:0/214978492 >> 172.21.15.57:0/214978492 conn(0x7f80a80cf400 msgr2=0x7f80a80afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:58.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ae69e700 1 -- 172.21.15.57:0/214978492 shutdown_connections 2023-11-28T12:01:58.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.026+0000 7f80ae69e700 1 -- 172.21.15.57:0/214978492 wait complete. 2023-11-28T12:01:58.041 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-28T12:01:58.065 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-28T12:01:58.065 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-28T12:01:58.065 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-28T12:01:58.066 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T12:01:58.066 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T12:01:58.066 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-28T12:01:58.066 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T12:01:58.066 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T12:01:58.066 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-28T12:01:58.067 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T12:01:58.067 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T12:01:58.067 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-28T12:01:58.067 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T12:01:58.067 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T12:01:58.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:58.067 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-11-28T12:01:58.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:58.089 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2023-11-28T12:01:58.166 INFO:teuthology.orchestra.run.smithi057.stderr:Cannot find device "ceph-brx" 2023-11-28T12:01:58.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:01:58.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:58.168 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-11-28T12:01:58.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:58.191 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-11-28T12:01:58.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:01:58.266 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2023-11-28T12:01:58.267 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-28T12:01:58.267 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T12:01:58.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.411+0000 7f968f380700 1 Processor -- start 2023-11-28T12:01:58.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.411+0000 7f968f380700 1 -- start start 2023-11-28T12:01:58.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f968f380700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9688158cd0 0x7f96880c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f968f380700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96880718e0 con 0x7f96880cd1c0 2023-11-28T12:01:58.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f968f380700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968806be90 con 0x7f96880ce290 2023-11-28T12:01:58.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f968f380700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96881590a0 con 0x7f9688158cd0 2023-11-28T12:01:58.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f968c91b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f96880cd1c0 0x7f96880cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41202/0 (socket says 172.21.15.57:41202) 2023-11-28T12:01:58.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f968c91b700 1 -- 172.21.15.57:0/4246155366 learned_addr learned my addr 172.21.15.57:0/4246155366 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:01:58.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f968d11c700 1 --2- 172.21.15.57:0/4246155366 >> v2:172.21.15.181:3300/0 conn(0x7f9688158cd0 0x7f96880c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f967ffff700 1 -- 172.21.15.57:0/4246155366 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195064574 0 0) 0x7f9684001110 con 0x7f96880cd1c0 2023-11-28T12:01:58.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f967ffff700 1 -- 172.21.15.57:0/4246155366 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96740035b0 con 0x7f96880cd1c0 2023-11-28T12:01:58.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f967ffff700 1 -- 172.21.15.57:0/4246155366 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868942654 0 0) 0x7f968806be90 con 0x7f96880ce290 2023-11-28T12:01:58.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f967ffff700 1 -- 172.21.15.57:0/4246155366 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9684001110 con 0x7f96880ce290 2023-11-28T12:01:58.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f968d11c700 1 -- 172.21.15.57:0/4246155366 >> v1:172.21.15.181:6789/0 conn(0x7f96880ce290 legacy=0x7f968806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:58.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f968d11c700 1 -- 172.21.15.57:0/4246155366 >> v1:172.21.15.57:6789/0 conn(0x7f96880cd1c0 legacy=0x7f96880cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:58.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.412+0000 7f968d11c700 1 -- 172.21.15.57:0/4246155366 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96880cad30 con 0x7f9688158cd0 2023-11-28T12:01:58.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.413+0000 7f968d11c700 1 --2- 172.21.15.57:0/4246155366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9688158cd0 0x7f96880c92c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9678009840 tx=0x7f9678009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae0eb1ebd57f2e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:58.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.413+0000 7f967ffff700 1 -- 172.21.15.57:0/4246155366 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678011830 con 0x7f9688158cd0 2023-11-28T12:01:58.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.413+0000 7f967ffff700 1 -- 172.21.15.57:0/4246155366 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9678011990 con 0x7f9688158cd0 2023-11-28T12:01:58.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.413+0000 7f967ffff700 1 -- 172.21.15.57:0/4246155366 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678011d20 con 0x7f9688158cd0 2023-11-28T12:01:58.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.413+0000 7f968f380700 1 -- 172.21.15.57:0/4246155366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9688158cd0 msgr2=0x7f96880c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:58.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.413+0000 7f968f380700 1 --2- 172.21.15.57:0/4246155366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9688158cd0 0x7f96880c92c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9678009840 tx=0x7f9678009b50 comp rx=0 tx=0).stop 2023-11-28T12:01:58.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.413+0000 7f968f380700 1 -- 172.21.15.57:0/4246155366 shutdown_connections 2023-11-28T12:01:58.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.413+0000 7f968f380700 1 --2- 172.21.15.57:0/4246155366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9688158cd0 0x7f96880c92c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.413+0000 7f968f380700 1 -- 172.21.15.57:0/4246155366 >> 172.21.15.57:0/4246155366 conn(0x7f96880cb660 msgr2=0x7f96880cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:58.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.414+0000 7f968f380700 1 -- 172.21.15.57:0/4246155366 shutdown_connections 2023-11-28T12:01:58.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.414+0000 7f968f380700 1 -- 172.21.15.57:0/4246155366 wait complete. 2023-11-28T12:01:58.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.415+0000 7f968f380700 1 Processor -- start 2023-11-28T12:01:58.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.415+0000 7f968f380700 1 -- start start 2023-11-28T12:01:58.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968f380700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96880cd1c0 0x7f96880c4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968f380700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96880ce290 0x7f96880c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968f380700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9688158cd0 0x7f968806ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968f380700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96880c4790 con 0x7f96880ce290 2023-11-28T12:01:58.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968f380700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96880c2060 con 0x7f96880cd1c0 2023-11-28T12:01:58.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968f380700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9688146a30 con 0x7f9688158cd0 2023-11-28T12:01:58.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968c91b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96880ce290 0x7f96880c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968c91b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96880ce290 0x7f96880c1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38952/0 (socket says 172.21.15.57:38952) 2023-11-28T12:01:58.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968c91b700 1 -- 172.21.15.57:0/702042657 learned_addr learned my addr 172.21.15.57:0/702042657 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:01:58.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968d11c700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96880cd1c0 0x7f96880c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968d91d700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9688158cd0 0x7f968806ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968c91b700 1 -- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9688158cd0 msgr2=0x7f968806ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:58.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968c91b700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9688158cd0 0x7f968806ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968c91b700 1 -- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96880cd1c0 msgr2=0x7f96880c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:58.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968c91b700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96880cd1c0 0x7f96880c4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968c91b700 1 -- 172.21.15.57:0/702042657 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96780094a0 con 0x7f96880ce290 2023-11-28T12:01:58.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968d91d700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9688158cd0 0x7f968806ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:01:58.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968c91b700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96880ce290 0x7f96880c1b20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9684001eb0 tx=0x7f968400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:58.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f967dffb700 1 -- 172.21.15.57:0/702042657 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684005d70 con 0x7f96880ce290 2023-11-28T12:01:58.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f967dffb700 1 -- 172.21.15.57:0/702042657 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9684007070 con 0x7f96880ce290 2023-11-28T12:01:58.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f968f380700 1 -- 172.21.15.57:0/702042657 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96880cb2f0 con 0x7f96880ce290 2023-11-28T12:01:58.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.416+0000 7f967dffb700 1 -- 172.21.15.57:0/702042657 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684007400 con 0x7f96880ce290 2023-11-28T12:01:58.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.417+0000 7f968f380700 1 -- 172.21.15.57:0/702042657 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96880cb010 con 0x7f96880ce290 2023-11-28T12:01:58.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.418+0000 7f968f380700 1 -- 172.21.15.57:0/702042657 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96880cd640 con 0x7f96880ce290 2023-11-28T12:01:58.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.418+0000 7f967dffb700 1 -- 172.21.15.57:0/702042657 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9684019030 con 0x7f96880ce290 2023-11-28T12:01:58.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.418+0000 7f967dffb700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f966c0682f0 0x7f966c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:01:58.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.419+0000 7f967dffb700 1 -- 172.21.15.57:0/702042657 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9684084f30 con 0x7f96880ce290 2023-11-28T12:01:58.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.419+0000 7f968d11c700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f966c0682f0 0x7f966c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:01:58.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.419+0000 7f968d11c700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f966c0682f0 0x7f966c06a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9678009fd0 tx=0x7f967801c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:01:58.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.423+0000 7f967dffb700 1 -- 172.21.15.57:0/702042657 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9684051790 con 0x7f96880ce290 2023-11-28T12:01:58.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.670+0000 7f968f380700 1 -- 172.21.15.57:0/702042657 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96880718e0 con 0x7f96880ce290 2023-11-28T12:01:58.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.671+0000 7f967dffb700 1 -- 172.21.15.57:0/702042657 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f96880718e0 con 0x7f96880ce290 2023-11-28T12:01:58.673 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T12:01:58.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.673+0000 7f968f380700 1 -- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f966c0682f0 msgr2=0x7f966c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:58.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.673+0000 7f968f380700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f966c0682f0 0x7f966c06a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9678009fd0 tx=0x7f967801c040 comp rx=0 tx=0).stop 2023-11-28T12:01:58.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.673+0000 7f968f380700 1 -- 172.21.15.57:0/702042657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96880ce290 msgr2=0x7f96880c1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:01:58.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.673+0000 7f968f380700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96880ce290 0x7f96880c1b20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9684001eb0 tx=0x7f968400c9e0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.673+0000 7f968f380700 1 -- 172.21.15.57:0/702042657 shutdown_connections 2023-11-28T12:01:58.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.673+0000 7f968f380700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f966c0682f0 0x7f966c06a7a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.673+0000 7f968f380700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9688158cd0 0x7f968806ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.673+0000 7f968f380700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96880cd1c0 0x7f96880c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.673+0000 7f968f380700 1 --2- 172.21.15.57:0/702042657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96880ce290 0x7f96880c1b20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:01:58.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.673+0000 7f968f380700 1 -- 172.21.15.57:0/702042657 >> 172.21.15.57:0/702042657 conn(0x7f96880cb660 msgr2=0x7f968815b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:01:58.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.673+0000 7f968f380700 1 -- 172.21.15.57:0/702042657 shutdown_connections 2023-11-28T12:01:58.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:01:58.673+0000 7f968f380700 1 -- 172.21.15.57:0/702042657 wait complete. 2023-11-28T12:01:58.688 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T12:01:58.688 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T12:01:58.688 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2023-11-28T12:01:58.688 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T12:01:58.688 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T12:01:58.688 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T12:01:58.688 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T12:01:58.688 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T12:01:58.688 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T12:01:58.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:58.689 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2023-11-28T12:01:58.709 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T12:01:58.709 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T12:01:58.709 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T12:01:58.709 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:01:58.709 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-11-28T12:01:58.709 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:01:58.710 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:01:58.710 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:01:58.710 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f0 2023-11-28T12:01:58.710 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:01:58.710 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:01:58.710 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f1 2023-11-28T12:01:58.710 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T12:01:58.710 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:01:58.710 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f0 2023-11-28T12:01:58.710 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T12:01:58.710 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 41941sec preferred_lft 41941sec 2023-11-28T12:01:58.711 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-11-28T12:01:58.711 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:01:58.711 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:01:58.711 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:01:58.711 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f1 2023-11-28T12:01:58.711 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T12:01:58.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:58.712 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-11-28T12:01:58.712 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-11-28T12:01:58.712 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add name ceph-brx type bridge 2023-11-28T12:01:58.712 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr flush dev ceph-brx 2023-11-28T12:01:58.712 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set ceph-brx up 2023-11-28T12:01:58.712 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T12:01:58.712 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-11-28T12:01:58.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:58.883 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T12:01:58.922 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-11-28T12:01:58.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:58.923 DEBUG:teuthology.orchestra.run.smithi057:> route 2023-11-28T12:01:59.009 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2023-11-28T12:01:59.010 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T12:01:59.010 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T12:01:59.010 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T12:01:59.010 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T12:01:59.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:59.011 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-11-28T12:01:59.011 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-11-28T12:01:59.011 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T12:01:59.011 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T12:01:59.011 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T12:01:59.011 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-11-28T12:01:59.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:59.288 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-11-28T12:01:59.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:59.312 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list-id 2023-11-28T12:01:59.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:59.376 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-11-28T12:01:59.377 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-11-28T12:01:59.377 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T12:01:59.377 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-28T12:01:59.377 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-11-28T12:01:59.487 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-28T12:01:59.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:59.487 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-11-28T12:01:59.488 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-11-28T12:01:59.488 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-28T12:01:59.488 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T12:01:59.488 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-28T12:01:59.488 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-28T12:01:59.488 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-28T12:01:59.488 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-11-28T12:01:59.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:01:59.713 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-11-28T12:01:59.713 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-11-28T12:01:59.713 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2023-11-28T12:01:59.713 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T12:01:59.713 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-11-28T12:01:59.824 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T12:01:59.824 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:01:59.825 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T12:01:59.844 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T12:01:59.845 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:01:59.845 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T12:01:59.906 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-11-28T12:01:59.982 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:01:59.982 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:02:00.057 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:02:00.058 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:02:00.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:00.058 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-11-28T12:02:00.122 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T12:02:00.123 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/) 2023-11-28T12:02:00.165 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-11-28T12:02:00.200 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:02:00.200 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:02:00.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:2023-11-28T12:02:00.268+0000 7f6b0a1175c0 -1 init, newargv = 0x564b64eda740 newargc=15 2023-11-28T12:02:00.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[129973]: starting ceph client 2023-11-28T12:02:00.271 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:02:00.272 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:02:00.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:00.273 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-11-28T12:02:00.275 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[129973]: starting fuse 2023-11-28T12:02:00.295 INFO:teuthology.orchestra.run.smithi057.stdout:47 2023-11-28T12:02:00.295 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T12:02:00.295 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:02:00.296 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2023-11-28T12:02:00.296 DEBUG:teuthology.orchestra.run.smithi057:> import re 2023-11-28T12:02:00.296 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-11-28T12:02:00.296 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2023-11-28T12:02:00.296 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:02:00.296 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2023-11-28T12:02:00.296 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T12:02:00.296 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2023-11-28T12:02:00.296 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T12:02:00.296 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:02:00.296 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2023-11-28T12:02:00.297 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2023-11-28T12:02:00.297 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2023-11-28T12:02:00.297 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2023-11-28T12:02:00.297 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:02:00.297 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2023-11-28T12:02:00.297 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T12:02:00.297 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T12:02:00.297 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T12:02:00.297 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2023-11-28T12:02:00.297 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2023-11-28T12:02:00.297 DEBUG:teuthology.orchestra.run.smithi057:> return f 2023-11-28T12:02:00.297 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T12:02:00.298 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:02:00.298 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.0")) 2023-11-28T12:02:00.298 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-11-28T12:02:00.432 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.0.129973.asok 2023-11-28T12:02:00.615 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.129973.asok 2023-11-28T12:02:00.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:00.615 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.129973.asok status 2023-11-28T12:02:00.795 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:02:00.796 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2023-11-28T12:02:00.796 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T12:02:00.796 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T12:02:00.796 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "0", 2023-11-28T12:02:00.797 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2023-11-28T12:02:00.797 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T12:02:00.797 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "129973", 2023-11-28T12:02:00.797 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2023-11-28T12:02:00.797 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:00.798 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2023-11-28T12:02:00.798 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2023-11-28T12:02:00.798 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4584, 2023-11-28T12:02:00.798 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2023-11-28T12:02:00.798 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2023-11-28T12:02:00.799 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2023-11-28T12:02:00.799 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 4584 2023-11-28T12:02:00.799 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:00.799 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-11-28T12:02:00.800 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-11-28T12:02:00.800 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-11-28T12:02:00.800 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1134032016 2023-11-28T12:02:00.800 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:00.801 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:00.801 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-11-28T12:02:00.801 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-11-28T12:02:00.801 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-11-28T12:02:00.801 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1134032016 2023-11-28T12:02:00.802 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:00.802 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.4584 192.168.0.1:0/1134032016", 2023-11-28T12:02:00.802 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.1:0/1134032016", 2023-11-28T12:02:00.802 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2023-11-28T12:02:00.802 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 8, 2023-11-28T12:02:00.803 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 21, 2023-11-28T12:02:00.803 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2023-11-28T12:02:00.803 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2023-11-28T12:02:00.803 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2023-11-28T12:02:00.803 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:02:00.807 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T12:02:00.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.940+0000 7f8d85867700 1 Processor -- start 2023-11-28T12:02:00.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.940+0000 7f8d85867700 1 -- start start 2023-11-28T12:02:00.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d85867700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8d8006ab10 0x7f8d8006aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:00.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d85867700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d800d0450 0x7f8d80160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:00.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d85867700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d800d1ee0 con 0x7f8d800cd5a0 2023-11-28T12:02:00.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d85867700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8006b420 con 0x7f8d8006ab10 2023-11-28T12:02:00.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d85867700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d800cdb10 con 0x7f8d800d0450 2023-11-28T12:02:00.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d7e7fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8d800cd5a0 0x7f8d80160300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41208/0 (socket says 172.21.15.57:41208) 2023-11-28T12:02:00.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d7e7fc700 1 -- 172.21.15.57:0/2158343093 learned_addr learned my addr 172.21.15.57:0/2158343093 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:00.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d7f7fe700 1 --2- 172.21.15.57:0/2158343093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d800d0450 0x7f8d80160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:00.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d7effd700 1 --2- 172.21.15.57:0/2158343093 >> v2:172.21.15.181:3300/0 conn(0x7f8d8006ab10 0x7f8d8006aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:00.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d7dffb700 1 -- 172.21.15.57:0/2158343093 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84879683 0 0) 0x7f8d800d1ee0 con 0x7f8d800cd5a0 2023-11-28T12:02:00.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d7dffb700 1 -- 172.21.15.57:0/2158343093 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d6c0035b0 con 0x7f8d800cd5a0 2023-11-28T12:02:00.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d7f7fe700 1 -- 172.21.15.57:0/2158343093 >> v2:172.21.15.181:3300/0 conn(0x7f8d8006ab10 msgr2=0x7f8d8006aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:00.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d7f7fe700 1 --2- 172.21.15.57:0/2158343093 >> v2:172.21.15.181:3300/0 conn(0x7f8d8006ab10 0x7f8d8006aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:00.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d7f7fe700 1 -- 172.21.15.57:0/2158343093 >> v1:172.21.15.57:6789/0 conn(0x7f8d800cd5a0 legacy=0x7f8d80160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:00.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d7f7fe700 1 -- 172.21.15.57:0/2158343093 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d800d1890 con 0x7f8d800d0450 2023-11-28T12:02:00.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.942+0000 7f8d7effd700 1 --2- 172.21.15.57:0/2158343093 >> v2:172.21.15.181:3300/0 conn(0x7f8d8006ab10 0x7f8d8006aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:00.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.943+0000 7f8d7f7fe700 1 --2- 172.21.15.57:0/2158343093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d800d0450 0x7f8d80160a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009d90 tx=0x7f8d7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4c2874a2b79b62d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:00.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.943+0000 7f8d7dffb700 1 -- 172.21.15.57:0/2158343093 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7800e070 con 0x7f8d800d0450 2023-11-28T12:02:00.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.943+0000 7f8d7dffb700 1 -- 172.21.15.57:0/2158343093 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7800bb20 con 0x7f8d800d0450 2023-11-28T12:02:00.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.943+0000 7f8d7dffb700 1 -- 172.21.15.57:0/2158343093 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d78012430 con 0x7f8d800d0450 2023-11-28T12:02:00.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.943+0000 7f8d85867700 1 -- 172.21.15.57:0/2158343093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d800d0450 msgr2=0x7f8d80160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:00.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.943+0000 7f8d85867700 1 --2- 172.21.15.57:0/2158343093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d800d0450 0x7f8d80160a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009d90 tx=0x7f8d7800b330 comp rx=0 tx=0).stop 2023-11-28T12:02:00.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.943+0000 7f8d85867700 1 -- 172.21.15.57:0/2158343093 shutdown_connections 2023-11-28T12:02:00.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.943+0000 7f8d85867700 1 --2- 172.21.15.57:0/2158343093 >> v2:172.21.15.181:3300/0 conn(0x7f8d8006ab10 0x7f8d8006aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:00.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.943+0000 7f8d85867700 1 --2- 172.21.15.57:0/2158343093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d800d0450 0x7f8d80160a10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:00.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.943+0000 7f8d85867700 1 -- 172.21.15.57:0/2158343093 >> 172.21.15.57:0/2158343093 conn(0x7f8d8006c520 msgr2=0x7f8d8006c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:00.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.943+0000 7f8d85867700 1 -- 172.21.15.57:0/2158343093 shutdown_connections 2023-11-28T12:02:00.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.943+0000 7f8d85867700 1 -- 172.21.15.57:0/2158343093 wait complete. 2023-11-28T12:02:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.945+0000 7f8d85867700 1 Processor -- start 2023-11-28T12:02:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.945+0000 7f8d85867700 1 -- start start 2023-11-28T12:02:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d85867700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d8006ab10 0x7f8d800c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d85867700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d800cd5a0 0x7f8d8015b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d85867700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d800d0450 0x7f8d801f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d85867700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d800c8d70 con 0x7f8d800cd5a0 2023-11-28T12:02:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d85867700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8015b950 con 0x7f8d8006ab10 2023-11-28T12:02:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d85867700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d800c8160 con 0x7f8d800d0450 2023-11-28T12:02:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d7e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d800cd5a0 0x7f8d8015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d7e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d800cd5a0 0x7f8d8015b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38958/0 (socket says 172.21.15.57:38958) 2023-11-28T12:02:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d7e7fc700 1 -- 172.21.15.57:0/3468647927 learned_addr learned my addr 172.21.15.57:0/3468647927 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d7f7fe700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d800d0450 0x7f8d801f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d7effd700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d8006ab10 0x7f8d800c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d7e7fc700 1 -- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d800d0450 msgr2=0x7f8d801f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d7e7fc700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d800d0450 0x7f8d801f6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d7e7fc700 1 -- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d8006ab10 msgr2=0x7f8d800c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d7e7fc700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d8006ab10 0x7f8d800c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d7e7fc700 1 -- 172.21.15.57:0/3468647927 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700094a0 con 0x7f8d800cd5a0 2023-11-28T12:02:00.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.946+0000 7f8d7e7fc700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d800cd5a0 0x7f8d8015b410 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8d68005910 tx=0x7f8d6800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:00.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.947+0000 7f8d77fff700 1 -- 172.21.15.57:0/3468647927 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68005d70 con 0x7f8d800cd5a0 2023-11-28T12:02:00.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.947+0000 7f8d77fff700 1 -- 172.21.15.57:0/3468647927 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d680070d0 con 0x7f8d800cd5a0 2023-11-28T12:02:00.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.947+0000 7f8d85867700 1 -- 172.21.15.57:0/3468647927 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d78009a70 con 0x7f8d800cd5a0 2023-11-28T12:02:00.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.947+0000 7f8d77fff700 1 -- 172.21.15.57:0/3468647927 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68007460 con 0x7f8d800cd5a0 2023-11-28T12:02:00.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.947+0000 7f8d85867700 1 -- 172.21.15.57:0/3468647927 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8015cad0 con 0x7f8d800cd5a0 2023-11-28T12:02:00.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.948+0000 7f8d85867700 1 -- 172.21.15.57:0/3468647927 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d800d1b70 con 0x7f8d800cd5a0 2023-11-28T12:02:00.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.949+0000 7f8d77fff700 1 -- 172.21.15.57:0/3468647927 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8d68019030 con 0x7f8d800cd5a0 2023-11-28T12:02:00.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.949+0000 7f8d77fff700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8d600682f0 0x7f8d6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:00.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.949+0000 7f8d77fff700 1 -- 172.21.15.57:0/3468647927 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8d680853c0 con 0x7f8d800cd5a0 2023-11-28T12:02:00.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.949+0000 7f8d7effd700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8d600682f0 0x7f8d6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:00.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.950+0000 7f8d7effd700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8d600682f0 0x7f8d6006a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009d70 tx=0x7f8d70009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:00.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:00.953+0000 7f8d77fff700 1 -- 172.21.15.57:0/3468647927 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8d68051cc0 con 0x7f8d800cd5a0 2023-11-28T12:02:01.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.205+0000 7f8d85867700 1 -- 172.21.15.57:0/3468647927 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8d800b1ec0 con 0x7f8d800cd5a0 2023-11-28T12:02:01.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.206+0000 7f8d77fff700 1 -- 172.21.15.57:0/3468647927 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8d680552e0 con 0x7f8d800cd5a0 2023-11-28T12:02:01.207 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T12:02:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.208+0000 7f8d85867700 1 -- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8d600682f0 msgr2=0x7f8d6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.208+0000 7f8d85867700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8d600682f0 0x7f8d6006a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009d70 tx=0x7f8d70009210 comp rx=0 tx=0).stop 2023-11-28T12:02:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.208+0000 7f8d85867700 1 -- 172.21.15.57:0/3468647927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d800cd5a0 msgr2=0x7f8d8015b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.208+0000 7f8d85867700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d800cd5a0 0x7f8d8015b410 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8d68005910 tx=0x7f8d6800c9e0 comp rx=0 tx=0).stop 2023-11-28T12:02:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.208+0000 7f8d85867700 1 -- 172.21.15.57:0/3468647927 shutdown_connections 2023-11-28T12:02:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.208+0000 7f8d85867700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8d600682f0 0x7f8d6006a7a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.208+0000 7f8d85867700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d800d0450 0x7f8d801f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.208+0000 7f8d85867700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d8006ab10 0x7f8d800c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.208+0000 7f8d85867700 1 --2- 172.21.15.57:0/3468647927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d800cd5a0 0x7f8d8015b410 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.208+0000 7f8d85867700 1 -- 172.21.15.57:0/3468647927 >> 172.21.15.57:0/3468647927 conn(0x7f8d8006c520 msgr2=0x7f8d800b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.208+0000 7f8d85867700 1 -- 172.21.15.57:0/3468647927 shutdown_connections 2023-11-28T12:02:01.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:01.208+0000 7f8d85867700 1 -- 172.21.15.57:0/3468647927 wait complete. 2023-11-28T12:02:01.222 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T12:02:01.223 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T12:02:01.223 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2023-11-28T12:02:01.223 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-28T12:02:01.223 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T12:02:01.223 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T12:02:01.223 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T12:02:01.223 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T12:02:01.223 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-28T12:02:01.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:01.225 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2023-11-28T12:02:01.245 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T12:02:01.245 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T12:02:01.245 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T12:02:01.245 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.246 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-11-28T12:02:01.246 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.246 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:02:01.246 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:01.246 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f0 2023-11-28T12:02:01.246 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:02:01.246 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:01.246 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f1 2023-11-28T12:02:01.246 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T12:02:01.246 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:01.246 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f0 2023-11-28T12:02:01.246 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T12:02:01.246 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 41938sec preferred_lft 41938sec 2023-11-28T12:02:01.247 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-11-28T12:02:01.247 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.247 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:02:01.247 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:01.247 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f1 2023-11-28T12:02:01.247 INFO:teuthology.orchestra.run.smithi057.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T12:02:01.247 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether f2:ef:c1:11:c5:fb brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:01.247 INFO:teuthology.orchestra.run.smithi057.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T12:02:01.247 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.247 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::3f:caff:fe00:648e/64 scope link 2023-11-28T12:02:01.247 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.247 INFO:teuthology.orchestra.run.smithi057.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T12:02:01.248 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether f2:ef:c1:11:c5:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T12:02:01.248 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::f0ef:c1ff:fe11:c5fb/64 scope link tentative 2023-11-28T12:02:01.248 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:01.248 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T12:02:01.317 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-11-28T12:02:01.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:01.318 DEBUG:teuthology.orchestra.run.smithi057:> route 2023-11-28T12:02:01.384 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2023-11-28T12:02:01.384 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T12:02:01.384 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T12:02:01.385 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T12:02:01.385 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T12:02:01.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:01.385 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-11-28T12:02:01.385 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-11-28T12:02:01.385 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T12:02:01.386 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T12:02:01.386 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T12:02:01.386 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-11-28T12:02:01.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:01.506 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-11-28T12:02:01.528 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T12:02:01.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:01.529 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list-id 2023-11-28T12:02:01.592 INFO:teuthology.orchestra.run.smithi057.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-28T12:02:01.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:01.593 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-11-28T12:02:01.593 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-11-28T12:02:01.593 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T12:02:01.593 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-28T12:02:01.593 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-11-28T12:02:01.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:01.701 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T12:02:01.742 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T12:02:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T12:02:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T12:02:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-11-28T12:02:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.743 INFO:teuthology.orchestra.run.smithi057.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T12:02:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 06:41:f7:f5:ab:c6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T12:02:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T12:02:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::441:f7ff:fef5:abc6/64 scope link tentative 2023-11-28T12:02:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:01.751 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T12:02:01.838 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T12:02:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T12:02:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T12:02:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-11-28T12:02:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.838 INFO:teuthology.orchestra.run.smithi057.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T12:02:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 06:41:f7:f5:ab:c6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T12:02:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T12:02:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::441:f7ff:fef5:abc6/64 scope link 2023-11-28T12:02:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:01.850 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-28T12:02:01.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:01.850 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-11-28T12:02:01.850 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-11-28T12:02:01.851 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-28T12:02:01.851 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T12:02:01.851 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-28T12:02:01.851 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-28T12:02:01.851 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-28T12:02:01.851 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-11-28T12:02:02.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:02.095 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-11-28T12:02:02.095 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-11-28T12:02:02.095 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.1 up 2023-11-28T12:02:02.095 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T12:02:02.095 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-11-28T12:02:02.207 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-28T12:02:02.207 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:02:02.207 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-28T12:02:02.227 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-28T12:02:02.227 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:02:02.228 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-28T12:02:02.289 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-11-28T12:02:02.366 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:02:02.366 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:02:02.441 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:02:02.442 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:02:02.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:02.443 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-11-28T12:02:02.509 INFO:teuthology.orchestra.run.smithi057.stdout:47 2023-11-28T12:02:02.510 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2023-11-28T12:02:02.510 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2 --client_mountpoint=/) 2023-11-28T12:02:02.553 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-11-28T12:02:02.587 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:02:02.588 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:02:02.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi057.stderr:2023-11-28T12:02:02.656+0000 7f1f70e825c0 -1 init, newargv = 0x555b687bcb70 newargc=15 2023-11-28T12:02:02.658 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi057.stderr:ceph-fuse[130697]: starting ceph client 2023-11-28T12:02:02.660 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:02:02.661 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:02:02.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:02.662 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-11-28T12:02:02.667 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi057.stderr:ceph-fuse[130697]: starting fuse 2023-11-28T12:02:02.687 INFO:teuthology.orchestra.run.smithi057.stdout:47 2023-11-28T12:02:02.687 INFO:teuthology.orchestra.run.smithi057.stdout:48 2023-11-28T12:02:02.687 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2023-11-28T12:02:02.687 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:02:02.688 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2023-11-28T12:02:02.688 DEBUG:teuthology.orchestra.run.smithi057:> import re 2023-11-28T12:02:02.688 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-11-28T12:02:02.688 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2023-11-28T12:02:02.688 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:02:02.688 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2023-11-28T12:02:02.688 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-28T12:02:02.688 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2023-11-28T12:02:02.688 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-28T12:02:02.688 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:02:02.689 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2023-11-28T12:02:02.689 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2023-11-28T12:02:02.689 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2023-11-28T12:02:02.689 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2023-11-28T12:02:02.689 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:02:02.689 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2023-11-28T12:02:02.689 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T12:02:02.689 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T12:02:02.689 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T12:02:02.689 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2023-11-28T12:02:02.689 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2023-11-28T12:02:02.689 DEBUG:teuthology.orchestra.run.smithi057:> return f 2023-11-28T12:02:02.690 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T12:02:02.690 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:02:02.690 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.2")) 2023-11-28T12:02:02.690 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-11-28T12:02:02.824 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.2.130697.asok 2023-11-28T12:02:03.005 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.130697.asok 2023-11-28T12:02:03.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:03.006 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.130697.asok status 2023-11-28T12:02:03.183 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:02:03.183 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2023-11-28T12:02:03.183 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T12:02:03.183 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T12:02:03.183 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "2", 2023-11-28T12:02:03.184 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2023-11-28T12:02:03.184 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-28T12:02:03.184 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "130697", 2023-11-28T12:02:03.184 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2023-11-28T12:02:03.184 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:03.184 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2023-11-28T12:02:03.184 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2023-11-28T12:02:03.184 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4596, 2023-11-28T12:02:03.184 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2023-11-28T12:02:03.184 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2023-11-28T12:02:03.184 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2023-11-28T12:02:03.184 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 4596 2023-11-28T12:02:03.184 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.2:0", 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3604186635 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.2:0", 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3604186635 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.4596 192.168.0.2:0/3604186635", 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.2:0/3604186635", 2023-11-28T12:02:03.185 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2023-11-28T12:02:03.186 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 8, 2023-11-28T12:02:03.186 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 21, 2023-11-28T12:02:03.186 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2023-11-28T12:02:03.186 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2023-11-28T12:02:03.186 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2023-11-28T12:02:03.186 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:02:03.195 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T12:02:03.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.330+0000 7f338fb35700 1 Processor -- start 2023-11-28T12:02:03.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.330+0000 7f338fb35700 1 -- start start 2023-11-28T12:02:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338fb35700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f338806e400 0x7f338806e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338fb35700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f338815e010 0x7f33880ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338fb35700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f338815af30 con 0x7f33880d15f0 2023-11-28T12:02:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338fb35700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338815a9d0 con 0x7f338806e400 2023-11-28T12:02:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338fb35700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33881584e0 con 0x7f338815e010 2023-11-28T12:02:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338d8d1700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f338806e400 0x7f338806e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338d0d0700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f33880d15f0 0x7f338815b3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41224/0 (socket says 172.21.15.57:41224) 2023-11-28T12:02:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338d0d0700 1 -- 172.21.15.57:0/2269052200 learned_addr learned my addr 172.21.15.57:0/2269052200 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338e0d2700 1 --2- 172.21.15.57:0/2269052200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f338815e010 0x7f33880ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338d8d1700 1 -- 172.21.15.57:0/2269052200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f338815e010 msgr2=0x7f33880ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338d8d1700 1 --2- 172.21.15.57:0/2269052200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f338815e010 0x7f33880ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:03.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338d8d1700 1 -- 172.21.15.57:0/2269052200 >> v1:172.21.15.57:6789/0 conn(0x7f33880d15f0 legacy=0x7f338815b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:03.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338d8d1700 1 -- 172.21.15.57:0/2269052200 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338815ef40 con 0x7f338806e400 2023-11-28T12:02:03.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.331+0000 7f338c8cf700 1 -- 172.21.15.57:0/2269052200 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476567982 0 0) 0x7f338815af30 con 0x7f33880d15f0 2023-11-28T12:02:03.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.332+0000 7f338d8d1700 1 --2- 172.21.15.57:0/2269052200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f338806e400 0x7f338806e7d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3384009840 tx=0x7f3384009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830654549cfdefde server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:03.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.332+0000 7f338c8cf700 1 -- 172.21.15.57:0/2269052200 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3384011830 con 0x7f338806e400 2023-11-28T12:02:03.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.332+0000 7f338c8cf700 1 -- 172.21.15.57:0/2269052200 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3384011990 con 0x7f338806e400 2023-11-28T12:02:03.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.332+0000 7f338c8cf700 1 -- 172.21.15.57:0/2269052200 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3384011d20 con 0x7f338806e400 2023-11-28T12:02:03.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.332+0000 7f338fb35700 1 -- 172.21.15.57:0/2269052200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f338806e400 msgr2=0x7f338806e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:03.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.332+0000 7f338fb35700 1 --2- 172.21.15.57:0/2269052200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f338806e400 0x7f338806e7d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3384009840 tx=0x7f3384009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:03.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.332+0000 7f338fb35700 1 -- 172.21.15.57:0/2269052200 shutdown_connections 2023-11-28T12:02:03.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.332+0000 7f338fb35700 1 --2- 172.21.15.57:0/2269052200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f338806e400 0x7f338806e7d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.332+0000 7f338fb35700 1 --2- 172.21.15.57:0/2269052200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f338815e010 0x7f33880ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.332+0000 7f338fb35700 1 -- 172.21.15.57:0/2269052200 >> 172.21.15.57:0/2269052200 conn(0x7f338806a270 msgr2=0x7f338806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.332+0000 7f338fb35700 1 -- 172.21.15.57:0/2269052200 shutdown_connections 2023-11-28T12:02:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.332+0000 7f338fb35700 1 -- 172.21.15.57:0/2269052200 wait complete. 2023-11-28T12:02:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.334+0000 7f338fb35700 1 Processor -- start 2023-11-28T12:02:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.334+0000 7f338fb35700 1 -- start start 2023-11-28T12:02:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.335+0000 7f338fb35700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f338806e400 0x7f33880b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.335+0000 7f338fb35700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33880d15f0 0x7f33881438e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.335+0000 7f338fb35700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f338815e010 0x7f3388152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.335+0000 7f338fb35700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338814dfc0 con 0x7f33880d15f0 2023-11-28T12:02:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.335+0000 7f338fb35700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33880b1180 con 0x7f338815e010 2023-11-28T12:02:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.335+0000 7f338fb35700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33880d0c40 con 0x7f338806e400 2023-11-28T12:02:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.335+0000 7f338d0d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33880d15f0 0x7f33881438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.335+0000 7f338d0d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33880d15f0 0x7f33881438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38970/0 (socket says 172.21.15.57:38970) 2023-11-28T12:02:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.335+0000 7f338d0d0700 1 -- 172.21.15.57:0/2415311462 learned_addr learned my addr 172.21.15.57:0/2415311462 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f338d8d1700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f338806e400 0x7f33880b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f338e0d2700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f338815e010 0x7f3388152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f338d0d0700 1 -- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f338806e400 msgr2=0x7f33880b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f338d0d0700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f338806e400 0x7f33880b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f338d0d0700 1 -- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f338815e010 msgr2=0x7f3388152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f338d0d0700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f338815e010 0x7f3388152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f338d0d0700 1 -- 172.21.15.57:0/2415311462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33840094a0 con 0x7f33880d15f0 2023-11-28T12:02:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f338d8d1700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f338806e400 0x7f33880b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f338d0d0700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33880d15f0 0x7f33881438e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f337800c570 tx=0x7f337800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f337e7fc700 1 -- 172.21.15.57:0/2415311462 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3378004c80 con 0x7f33880d15f0 2023-11-28T12:02:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f337e7fc700 1 -- 172.21.15.57:0/2415311462 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3378004e00 con 0x7f33880d15f0 2023-11-28T12:02:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f338fb35700 1 -- 172.21.15.57:0/2415311462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338815e980 con 0x7f33880d15f0 2023-11-28T12:02:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f337e7fc700 1 -- 172.21.15.57:0/2415311462 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3378007310 con 0x7f33880d15f0 2023-11-28T12:02:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.336+0000 7f338fb35700 1 -- 172.21.15.57:0/2415311462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338806ed10 con 0x7f33880d15f0 2023-11-28T12:02:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.337+0000 7f338fb35700 1 -- 172.21.15.57:0/2415311462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338815ec60 con 0x7f33880d15f0 2023-11-28T12:02:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.338+0000 7f337e7fc700 1 -- 172.21.15.57:0/2415311462 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f337801a020 con 0x7f33880d15f0 2023-11-28T12:02:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.338+0000 7f337e7fc700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f336c0682f0 0x7f336c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.338+0000 7f337e7fc700 1 -- 172.21.15.57:0/2415311462 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f33780858b0 con 0x7f33880d15f0 2023-11-28T12:02:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.338+0000 7f338d8d1700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f336c0682f0 0x7f336c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:03.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.339+0000 7f338d8d1700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f336c0682f0 0x7f336c06a7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33840090b0 tx=0x7f3384008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:03.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.343+0000 7f337e7fc700 1 -- 172.21.15.57:0/2415311462 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f33780521c0 con 0x7f33880d15f0 2023-11-28T12:02:03.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.588+0000 7f338fb35700 1 -- 172.21.15.57:0/2415311462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f33880743f0 con 0x7f33880d15f0 2023-11-28T12:02:03.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.589+0000 7f337e7fc700 1 -- 172.21.15.57:0/2415311462 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f33780557e0 con 0x7f33880d15f0 2023-11-28T12:02:03.590 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T12:02:03.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.591+0000 7f338fb35700 1 -- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f336c0682f0 msgr2=0x7f336c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:03.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.591+0000 7f338fb35700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f336c0682f0 0x7f336c06a7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33840090b0 tx=0x7f3384008030 comp rx=0 tx=0).stop 2023-11-28T12:02:03.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.591+0000 7f338fb35700 1 -- 172.21.15.57:0/2415311462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33880d15f0 msgr2=0x7f33881438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:03.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.591+0000 7f338fb35700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33880d15f0 0x7f33881438e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f337800c570 tx=0x7f337800c930 comp rx=0 tx=0).stop 2023-11-28T12:02:03.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.591+0000 7f338fb35700 1 -- 172.21.15.57:0/2415311462 shutdown_connections 2023-11-28T12:02:03.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.591+0000 7f338fb35700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f336c0682f0 0x7f336c06a7a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:03.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.591+0000 7f338fb35700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f338815e010 0x7f3388152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:03.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.591+0000 7f338fb35700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f338806e400 0x7f33880b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:03.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.591+0000 7f338fb35700 1 --2- 172.21.15.57:0/2415311462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33880d15f0 0x7f33881438e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:03.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.591+0000 7f338fb35700 1 -- 172.21.15.57:0/2415311462 >> 172.21.15.57:0/2415311462 conn(0x7f338806a270 msgr2=0x7f3388154320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:03.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.591+0000 7f338fb35700 1 -- 172.21.15.57:0/2415311462 shutdown_connections 2023-11-28T12:02:03.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:03.591+0000 7f338fb35700 1 -- 172.21.15.57:0/2415311462 wait complete. 2023-11-28T12:02:03.605 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T12:02:03.605 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T12:02:03.605 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-11-28T12:02:03.605 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-28T12:02:03.605 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T12:02:03.606 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T12:02:03.606 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T12:02:03.606 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T12:02:03.606 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-28T12:02:03.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:03.606 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-11-28T12:02:03.629 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T12:02:03.629 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T12:02:03.629 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T12:02:03.630 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:03.630 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-11-28T12:02:03.630 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:03.630 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:02:03.631 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:03.631 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-11-28T12:02:03.631 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:02:03.631 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:03.631 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-11-28T12:02:03.631 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:02:03.632 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:03.632 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T12:02:03.632 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:03.633 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T12:02:03.633 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42003sec preferred_lft 42003sec 2023-11-28T12:02:03.634 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-11-28T12:02:03.634 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:03.636 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T12:02:03.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:03.637 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-11-28T12:02:03.637 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-11-28T12:02:03.637 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add name ceph-brx type bridge 2023-11-28T12:02:03.637 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr flush dev ceph-brx 2023-11-28T12:02:03.638 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set ceph-brx up 2023-11-28T12:02:03.638 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T12:02:03.638 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-11-28T12:02:03.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:03.798 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T12:02:03.831 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-11-28T12:02:03.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:03.832 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-11-28T12:02:03.906 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-11-28T12:02:03.907 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T12:02:03.907 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T12:02:03.907 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T12:02:03.907 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T12:02:03.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:03.907 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-11-28T12:02:03.907 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-11-28T12:02:03.907 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T12:02:03.908 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T12:02:03.908 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T12:02:03.908 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-11-28T12:02:04.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:04.150 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-11-28T12:02:04.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:04.168 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2023-11-28T12:02:04.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:04.226 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-11-28T12:02:04.226 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-11-28T12:02:04.226 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T12:02:04.226 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-28T12:02:04.226 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-11-28T12:02:04.320 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-28T12:02:04.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:04.320 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-11-28T12:02:04.320 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-11-28T12:02:04.320 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-28T12:02:04.320 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T12:02:04.320 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-28T12:02:04.321 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-28T12:02:04.321 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-28T12:02:04.321 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-11-28T12:02:04.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:04.517 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-11-28T12:02:04.517 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-11-28T12:02:04.517 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-11-28T12:02:04.517 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T12:02:04.517 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-11-28T12:02:04.614 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-28T12:02:04.614 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:02:04.614 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-28T12:02:04.632 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-28T12:02:04.632 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:02:04.632 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-28T12:02:04.690 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2023-11-28T12:02:04.759 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:02:04.759 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:02:04.825 INFO:teuthology.orchestra.run.smithi181.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:02:04.826 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:02:04.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:04.827 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2023-11-28T12:02:04.884 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T12:02:04.884 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1 --client_mountpoint=/) 2023-11-28T12:02:04.927 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2023-11-28T12:02:04.953 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:02:04.953 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:02:05.013 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2023-11-28T12:02:05.012+0000 7f4c115fd5c0 -1 init, newargv = 0x55a73f9b9f40 newargc=15 2023-11-28T12:02:05.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[127806]: starting ceph client 2023-11-28T12:02:05.018 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[127806]: starting fuse 2023-11-28T12:02:05.020 INFO:teuthology.orchestra.run.smithi181.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:02:05.022 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:02:05.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:05.022 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2023-11-28T12:02:05.081 INFO:teuthology.orchestra.run.smithi181.stdout:46 2023-11-28T12:02:05.081 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-11-28T12:02:05.081 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:02:05.081 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2023-11-28T12:02:05.081 DEBUG:teuthology.orchestra.run.smithi181:> import re 2023-11-28T12:02:05.081 DEBUG:teuthology.orchestra.run.smithi181:> import os 2023-11-28T12:02:05.081 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2023-11-28T12:02:05.082 DEBUG:teuthology.orchestra.run.smithi181:> 2023-11-28T12:02:05.082 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2023-11-28T12:02:05.082 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-28T12:02:05.082 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2023-11-28T12:02:05.082 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-28T12:02:05.082 DEBUG:teuthology.orchestra.run.smithi181:> 2023-11-28T12:02:05.082 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2023-11-28T12:02:05.082 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2023-11-28T12:02:05.082 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2023-11-28T12:02:05.082 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2023-11-28T12:02:05.082 DEBUG:teuthology.orchestra.run.smithi181:> 2023-11-28T12:02:05.082 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2023-11-28T12:02:05.082 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T12:02:05.083 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T12:02:05.083 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T12:02:05.083 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2023-11-28T12:02:05.083 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2023-11-28T12:02:05.083 DEBUG:teuthology.orchestra.run.smithi181:> return f 2023-11-28T12:02:05.083 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T12:02:05.083 DEBUG:teuthology.orchestra.run.smithi181:> 2023-11-28T12:02:05.083 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2023-11-28T12:02:05.083 DEBUG:teuthology.orchestra.run.smithi181:> ' 2023-11-28T12:02:05.203 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.127806.asok 2023-11-28T12:02:05.385 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.127806.asok 2023-11-28T12:02:05.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:05.386 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.127806.asok status 2023-11-28T12:02:05.543 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-11-28T12:02:05.543 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2023-11-28T12:02:05.543 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T12:02:05.543 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T12:02:05.543 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2023-11-28T12:02:05.543 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2023-11-28T12:02:05.543 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-28T12:02:05.543 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "127806", 2023-11-28T12:02:05.543 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2023-11-28T12:02:05.544 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-11-28T12:02:05.544 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2023-11-28T12:02:05.544 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2023-11-28T12:02:05.544 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 4514, 2023-11-28T12:02:05.544 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2023-11-28T12:02:05.544 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2023-11-28T12:02:05.544 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2023-11-28T12:02:05.544 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 4514 2023-11-28T12:02:05.544 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-11-28T12:02:05.544 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2023-11-28T12:02:05.544 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2023-11-28T12:02:05.544 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2023-11-28T12:02:05.544 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2517850768 2023-11-28T12:02:05.545 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-11-28T12:02:05.545 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-11-28T12:02:05.545 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2023-11-28T12:02:05.545 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2023-11-28T12:02:05.545 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2023-11-28T12:02:05.545 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2517850768 2023-11-28T12:02:05.545 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-11-28T12:02:05.545 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.4514 192.168.0.1:0/2517850768", 2023-11-28T12:02:05.545 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/2517850768", 2023-11-28T12:02:05.545 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2023-11-28T12:02:05.545 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 8, 2023-11-28T12:02:05.545 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 21, 2023-11-28T12:02:05.545 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2023-11-28T12:02:05.546 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2023-11-28T12:02:05.546 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2023-11-28T12:02:05.546 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-11-28T12:02:05.553 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T12:02:05.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.696+0000 7fadc3f60700 1 Processor -- start 2023-11-28T12:02:05.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.696+0000 7fadc3f60700 1 -- start start 2023-11-28T12:02:05.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc3f60700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fadbc142db0 0x7fadbc13ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:05.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc3f60700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fadbc0b1770 0x7fadbc0b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:05.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc3f60700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadbc15e010 con 0x7fadbc0ce040 2023-11-28T12:02:05.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc3f60700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc13fc90 con 0x7fadbc142db0 2023-11-28T12:02:05.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc3f60700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc0b20d0 con 0x7fadbc0b1770 2023-11-28T12:02:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc14fb700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fadbc142db0 0x7fadbc13ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc14fb700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fadbc142db0 0x7fadbc13ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45550/0 (socket says 172.21.15.57:45550) 2023-11-28T12:02:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc14fb700 1 -- 172.21.15.57:0/2618350442 learned_addr learned my addr 172.21.15.57:0/2618350442 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc1cfc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fadbc0ce040 0x7fadbc15d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37580/0 (socket says 172.21.15.57:37580) 2023-11-28T12:02:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc24fd700 1 --2- 172.21.15.57:0/2618350442 >> v2:172.21.15.181:3300/0 conn(0x7fadbc0b1770 0x7fadbc0b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc14fb700 1 -- 172.21.15.57:0/2618350442 >> v2:172.21.15.181:3300/0 conn(0x7fadbc0b1770 msgr2=0x7fadbc0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc14fb700 1 --2- 172.21.15.57:0/2618350442 >> v2:172.21.15.181:3300/0 conn(0x7fadbc0b1770 0x7fadbc0b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc14fb700 1 -- 172.21.15.57:0/2618350442 >> v1:172.21.15.57:6789/0 conn(0x7fadbc0ce040 legacy=0x7fadbc15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc14fb700 1 -- 172.21.15.57:0/2618350442 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc13f6d0 con 0x7fadbc142db0 2023-11-28T12:02:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc24fd700 1 --2- 172.21.15.57:0/2618350442 >> v2:172.21.15.181:3300/0 conn(0x7fadbc0b1770 0x7fadbc0b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc14fb700 1 --2- 172.21.15.57:0/2618350442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadbc142db0 0x7fadbc13ed70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fadbc142cd0 tx=0x7fadac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb480640fe08e5ad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.697+0000 7fadc0cfa700 1 -- 172.21.15.57:0/2618350442 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac00b870 con 0x7fadbc142db0 2023-11-28T12:02:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.698+0000 7fadc0cfa700 1 -- 172.21.15.57:0/2618350442 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac00b9d0 con 0x7fadbc142db0 2023-11-28T12:02:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.698+0000 7fadc0cfa700 1 -- 172.21.15.57:0/2618350442 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac00bd60 con 0x7fadbc142db0 2023-11-28T12:02:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.698+0000 7fadc3f60700 1 -- 172.21.15.57:0/2618350442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadbc142db0 msgr2=0x7fadbc13ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.698+0000 7fadc3f60700 1 --2- 172.21.15.57:0/2618350442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadbc142db0 0x7fadbc13ed70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fadbc142cd0 tx=0x7fadac009bd0 comp rx=0 tx=0).stop 2023-11-28T12:02:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.698+0000 7fadc3f60700 1 -- 172.21.15.57:0/2618350442 shutdown_connections 2023-11-28T12:02:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.698+0000 7fadc3f60700 1 --2- 172.21.15.57:0/2618350442 >> v2:172.21.15.181:3300/0 conn(0x7fadbc0b1770 0x7fadbc0b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.698+0000 7fadc3f60700 1 --2- 172.21.15.57:0/2618350442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadbc142db0 0x7fadbc13ed70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.698+0000 7fadc3f60700 1 -- 172.21.15.57:0/2618350442 >> 172.21.15.57:0/2618350442 conn(0x7fadbc06c010 msgr2=0x7fadbc0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.698+0000 7fadc3f60700 1 -- 172.21.15.57:0/2618350442 shutdown_connections 2023-11-28T12:02:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.698+0000 7fadc3f60700 1 -- 172.21.15.57:0/2618350442 wait complete. 2023-11-28T12:02:05.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.699+0000 7fadc3f60700 1 Processor -- start 2023-11-28T12:02:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.700+0000 7fadc3f60700 1 -- start start 2023-11-28T12:02:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.700+0000 7fadc3f60700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadbc0b1770 0x7fadbc0c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.700+0000 7fadc3f60700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc0ce040 0x7fadbc0c2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.700+0000 7fadc3f60700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadbc142db0 0x7fadbc0cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.700+0000 7fadc3f60700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0c6690 con 0x7fadbc142db0 2023-11-28T12:02:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.700+0000 7fadc3f60700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc15c230 con 0x7fadbc0b1770 2023-11-28T12:02:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.700+0000 7fadc3f60700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0c37e0 con 0x7fadbc0ce040 2023-11-28T12:02:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.700+0000 7fadc24fd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadbc142db0 0x7fadbc0cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.700+0000 7fadc24fd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadbc142db0 0x7fadbc0cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45564/0 (socket says 172.21.15.57:45564) 2023-11-28T12:02:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.700+0000 7fadc24fd700 1 -- 172.21.15.57:0/3371041780 learned_addr learned my addr 172.21.15.57:0/3371041780 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.701+0000 7fadc14fb700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc0ce040 0x7fadbc0c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.701+0000 7fadc1cfc700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadbc0b1770 0x7fadbc0c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.701+0000 7fadc24fd700 1 -- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc0ce040 msgr2=0x7fadbc0c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.701+0000 7fadc24fd700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc0ce040 0x7fadbc0c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.701+0000 7fadc24fd700 1 -- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadbc0b1770 msgr2=0x7fadbc0c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.701+0000 7fadc24fd700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadbc0b1770 0x7fadbc0c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.701+0000 7fadc24fd700 1 -- 172.21.15.57:0/3371041780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac009550 con 0x7fadbc142db0 2023-11-28T12:02:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.701+0000 7fadc24fd700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadbc142db0 0x7fadbc0cb1d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fadb400cd50 tx=0x7fadb400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.701+0000 7fadb27fc700 1 -- 172.21.15.57:0/3371041780 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb400f480 con 0x7fadbc142db0 2023-11-28T12:02:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.701+0000 7fadb27fc700 1 -- 172.21.15.57:0/3371041780 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb400f600 con 0x7fadbc142db0 2023-11-28T12:02:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.701+0000 7fadc3f60700 1 -- 172.21.15.57:0/3371041780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc15d100 con 0x7fadbc142db0 2023-11-28T12:02:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.701+0000 7fadb27fc700 1 -- 172.21.15.57:0/3371041780 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb400f9b0 con 0x7fadbc142db0 2023-11-28T12:02:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.701+0000 7fadc3f60700 1 -- 172.21.15.57:0/3371041780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc13f9b0 con 0x7fadbc142db0 2023-11-28T12:02:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.702+0000 7fadc3f60700 1 -- 172.21.15.57:0/3371041780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc15d3e0 con 0x7fadbc142db0 2023-11-28T12:02:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.703+0000 7fadb27fc700 1 -- 172.21.15.57:0/3371041780 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fadb4009030 con 0x7fadbc142db0 2023-11-28T12:02:05.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.703+0000 7fadb27fc700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fada00682f0 0x7fada006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:05.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.703+0000 7fadb27fc700 1 -- 172.21.15.57:0/3371041780 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fadb40866e0 con 0x7fadbc142db0 2023-11-28T12:02:05.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.703+0000 7fadc1cfc700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fada00682f0 0x7fada006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:05.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.704+0000 7fadc1cfc700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fada00682f0 0x7fada006a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fadbc13ea00 tx=0x7fadb8007d70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:05.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.708+0000 7fadb27fc700 1 -- 172.21.15.57:0/3371041780 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fadb4052f40 con 0x7fadbc142db0 2023-11-28T12:02:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.954+0000 7fadc3f60700 1 -- 172.21.15.57:0/3371041780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fadbc13ff50 con 0x7fadbc142db0 2023-11-28T12:02:05.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.955+0000 7fadb27fc700 1 -- 172.21.15.57:0/3371041780 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fadbc13ff50 con 0x7fadbc142db0 2023-11-28T12:02:05.956 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T12:02:05.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.956+0000 7fadc3f60700 1 -- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fada00682f0 msgr2=0x7fada006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:05.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.956+0000 7fadc3f60700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fada00682f0 0x7fada006a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fadbc13ea00 tx=0x7fadb8007d70 comp rx=0 tx=0).stop 2023-11-28T12:02:05.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.956+0000 7fadc3f60700 1 -- 172.21.15.57:0/3371041780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadbc142db0 msgr2=0x7fadbc0cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:05.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.957+0000 7fadc3f60700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadbc142db0 0x7fadbc0cb1d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fadb400cd50 tx=0x7fadb400a560 comp rx=0 tx=0).stop 2023-11-28T12:02:05.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.957+0000 7fadc3f60700 1 -- 172.21.15.57:0/3371041780 shutdown_connections 2023-11-28T12:02:05.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.957+0000 7fadc3f60700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fada00682f0 0x7fada006a7a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:05.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.957+0000 7fadc3f60700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadbc142db0 0x7fadbc0cb1d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:05.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.957+0000 7fadc3f60700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadbc0b1770 0x7fadbc0c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:05.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.957+0000 7fadc3f60700 1 --2- 172.21.15.57:0/3371041780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc0ce040 0x7fadbc0c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:05.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.957+0000 7fadc3f60700 1 -- 172.21.15.57:0/3371041780 >> 172.21.15.57:0/3371041780 conn(0x7fadbc06c010 msgr2=0x7fadbc13eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:05.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.957+0000 7fadc3f60700 1 -- 172.21.15.57:0/3371041780 shutdown_connections 2023-11-28T12:02:05.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:05.957+0000 7fadc3f60700 1 -- 172.21.15.57:0/3371041780 wait complete. 2023-11-28T12:02:05.971 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T12:02:05.971 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T12:02:05.971 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-11-28T12:02:05.971 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-28T12:02:05.971 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T12:02:05.971 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T12:02:05.971 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T12:02:05.971 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T12:02:05.972 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-28T12:02:05.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:05.972 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-11-28T12:02:05.991 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T12:02:05.991 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T12:02:05.991 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T12:02:05.992 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:05.992 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-11-28T12:02:05.992 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:05.992 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:02:05.993 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:05.993 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-11-28T12:02:05.993 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:02:05.993 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:05.993 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-11-28T12:02:05.993 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:02:05.993 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:05.993 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T12:02:05.993 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:05.993 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T12:02:05.993 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42001sec preferred_lft 42001sec 2023-11-28T12:02:05.993 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-11-28T12:02:05.994 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:05.994 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T12:02:05.994 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether ee:8c:23:bf:6a:65 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:02:05.994 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T12:02:05.994 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:05.994 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::58ec:ecff:fe0d:6963/64 scope link 2023-11-28T12:02:05.994 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:05.994 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T12:02:05.994 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether ee:8c:23:bf:6a:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T12:02:05.994 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec8c:23ff:febf:6a65/64 scope link tentative 2023-11-28T12:02:05.994 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:05.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:05.995 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T12:02:06.057 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-11-28T12:02:06.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:06.059 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-11-28T12:02:06.119 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-11-28T12:02:06.119 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T12:02:06.119 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T12:02:06.119 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T12:02:06.119 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T12:02:06.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:06.120 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-11-28T12:02:06.120 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-11-28T12:02:06.120 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T12:02:06.120 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T12:02:06.120 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T12:02:06.120 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-11-28T12:02:06.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:06.224 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-11-28T12:02:06.241 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T12:02:06.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:06.242 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2023-11-28T12:02:06.301 INFO:teuthology.orchestra.run.smithi181.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-28T12:02:06.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:06.301 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-11-28T12:02:06.302 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-11-28T12:02:06.302 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T12:02:06.302 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-28T12:02:06.302 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-11-28T12:02:06.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:06.394 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T12:02:06.430 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T12:02:06.430 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T12:02:06.430 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T12:02:06.430 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:06.430 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-11-28T12:02:06.430 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:06.431 INFO:teuthology.orchestra.run.smithi181.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T12:02:06.431 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5a:63:30:72:ac:e7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T12:02:06.431 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T12:02:06.431 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:06.431 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5863:30ff:fe72:ace7/64 scope link tentative 2023-11-28T12:02:06.431 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:06.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:06.442 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T12:02:06.523 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T12:02:06.523 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T12:02:06.523 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T12:02:06.523 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:06.523 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-11-28T12:02:06.523 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:06.523 INFO:teuthology.orchestra.run.smithi181.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T12:02:06.524 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5a:63:30:72:ac:e7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T12:02:06.524 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T12:02:06.524 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:06.524 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5863:30ff:fe72:ace7/64 scope link tentative 2023-11-28T12:02:06.524 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:02:06.538 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-28T12:02:06.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:06.539 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-11-28T12:02:06.539 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-11-28T12:02:06.539 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-28T12:02:06.539 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T12:02:06.539 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-28T12:02:06.539 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-28T12:02:06.539 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-28T12:02:06.539 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-11-28T12:02:06.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:06.800 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-11-28T12:02:06.801 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-11-28T12:02:06.801 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.1 up 2023-11-28T12:02:06.801 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T12:02:06.801 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-11-28T12:02:06.896 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-28T12:02:06.897 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:02:06.897 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-28T12:02:06.914 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-28T12:02:06.915 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:02:06.915 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-28T12:02:06.972 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2023-11-28T12:02:07.038 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:02:07.039 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:02:07.105 INFO:teuthology.orchestra.run.smithi181.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:02:07.105 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:02:07.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:07.106 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2023-11-28T12:02:07.164 INFO:teuthology.orchestra.run.smithi181.stdout:46 2023-11-28T12:02:07.165 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [46] 2023-11-28T12:02:07.165 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3 --client_mountpoint=/) 2023-11-28T12:02:07.207 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2023-11-28T12:02:07.235 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:02:07.235 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:02:07.293 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi181.stderr:2023-11-28T12:02:07.292+0000 7f19fb95d5c0 -1 init, newargv = 0x55b050dc1950 newargc=15 2023-11-28T12:02:07.293 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi181.stderr:ceph-fuse[128481]: starting ceph client 2023-11-28T12:02:07.297 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi181.stderr:ceph-fuse[128481]: starting fuse 2023-11-28T12:02:07.305 INFO:teuthology.orchestra.run.smithi181.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:02:07.306 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:02:07.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:07.306 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2023-11-28T12:02:07.365 INFO:teuthology.orchestra.run.smithi181.stdout:46 2023-11-28T12:02:07.365 INFO:teuthology.orchestra.run.smithi181.stdout:48 2023-11-28T12:02:07.365 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46, 48] 2023-11-28T12:02:07.365 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:02:07.365 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2023-11-28T12:02:07.365 DEBUG:teuthology.orchestra.run.smithi181:> import re 2023-11-28T12:02:07.366 DEBUG:teuthology.orchestra.run.smithi181:> import os 2023-11-28T12:02:07.366 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2023-11-28T12:02:07.366 DEBUG:teuthology.orchestra.run.smithi181:> 2023-11-28T12:02:07.366 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2023-11-28T12:02:07.366 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-28T12:02:07.366 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2023-11-28T12:02:07.366 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-28T12:02:07.366 DEBUG:teuthology.orchestra.run.smithi181:> 2023-11-28T12:02:07.366 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2023-11-28T12:02:07.366 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2023-11-28T12:02:07.366 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2023-11-28T12:02:07.366 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2023-11-28T12:02:07.366 DEBUG:teuthology.orchestra.run.smithi181:> 2023-11-28T12:02:07.367 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2023-11-28T12:02:07.367 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T12:02:07.367 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T12:02:07.367 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T12:02:07.367 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2023-11-28T12:02:07.367 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2023-11-28T12:02:07.367 DEBUG:teuthology.orchestra.run.smithi181:> return f 2023-11-28T12:02:07.367 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T12:02:07.367 DEBUG:teuthology.orchestra.run.smithi181:> 2023-11-28T12:02:07.367 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.3")) 2023-11-28T12:02:07.367 DEBUG:teuthology.orchestra.run.smithi181:> ' 2023-11-28T12:02:07.486 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.3.128481.asok 2023-11-28T12:02:07.669 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.128481.asok 2023-11-28T12:02:07.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:07.669 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.128481.asok status 2023-11-28T12:02:07.825 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-11-28T12:02:07.825 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2023-11-28T12:02:07.826 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T12:02:07.826 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T12:02:07.826 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "3", 2023-11-28T12:02:07.826 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2023-11-28T12:02:07.826 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-28T12:02:07.826 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "128481", 2023-11-28T12:02:07.826 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2023-11-28T12:02:07.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-11-28T12:02:07.826 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2023-11-28T12:02:07.826 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2023-11-28T12:02:07.826 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 4523, 2023-11-28T12:02:07.827 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2023-11-28T12:02:07.827 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2023-11-28T12:02:07.827 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2023-11-28T12:02:07.827 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 4523 2023-11-28T12:02:07.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-11-28T12:02:07.827 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2023-11-28T12:02:07.827 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2023-11-28T12:02:07.827 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.2:0", 2023-11-28T12:02:07.827 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 1064428961 2023-11-28T12:02:07.827 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-11-28T12:02:07.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-11-28T12:02:07.827 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2023-11-28T12:02:07.827 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2023-11-28T12:02:07.828 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.2:0", 2023-11-28T12:02:07.828 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 1064428961 2023-11-28T12:02:07.828 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-11-28T12:02:07.828 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.4523 192.168.0.2:0/1064428961", 2023-11-28T12:02:07.828 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.2:0/1064428961", 2023-11-28T12:02:07.828 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2023-11-28T12:02:07.828 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 8, 2023-11-28T12:02:07.828 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 21, 2023-11-28T12:02:07.828 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2023-11-28T12:02:07.828 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2023-11-28T12:02:07.828 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2023-11-28T12:02:07.828 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-11-28T12:02:07.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:07.835 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2023-11-28T12:02:07.836 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T12:02:07.860 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2023-11-28T12:02:07.860 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T12:02:07.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:07.860 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T12:02:07.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:07.937 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2023-11-28T12:02:07.937 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-28T12:02:08.000 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2023-11-28T12:02:08.001 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-28T12:02:08.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:08.001 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-28T12:02:08.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:08.079 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2023-11-28T12:02:08.079 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-28T12:02:08.098 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2023-11-28T12:02:08.100 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-28T12:02:08.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:08.100 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-28T12:02:08.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:08.167 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2023-11-28T12:02:08.167 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-28T12:02:08.225 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2023-11-28T12:02:08.225 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-28T12:02:08.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:08.226 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-28T12:02:08.293 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-28T12:02:08.316 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:08.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.458+0000 7f938d343700 1 Processor -- start 2023-11-28T12:02:08.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.458+0000 7f938d343700 1 -- start start 2023-11-28T12:02:08.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.459+0000 7f938d343700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f938815e010 0x7f93880ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.459+0000 7f938d343700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938815af30 con 0x7f938806e400 2023-11-28T12:02:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.459+0000 7f938d343700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938815ef40 con 0x7f93880d15f0 2023-11-28T12:02:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.459+0000 7f938d343700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938815a9d0 con 0x7f938815e010 2023-11-28T12:02:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.459+0000 7f9386ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f938806e400 0x7f938806e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37588/0 (socket says 172.21.15.57:37588) 2023-11-28T12:02:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.459+0000 7f9386ffd700 1 -- 172.21.15.57:0/1598165636 learned_addr learned my addr 172.21.15.57:0/1598165636 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.460+0000 7f93877fe700 1 --2- 172.21.15.57:0/1598165636 >> v2:172.21.15.181:3300/0 conn(0x7f938815e010 0x7f93880ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.460+0000 7f9385ffb700 1 -- 172.21.15.57:0/1598165636 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524017443 0 0) 0x7f938815af30 con 0x7f938806e400 2023-11-28T12:02:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.460+0000 7f9385ffb700 1 -- 172.21.15.57:0/1598165636 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93700035b0 con 0x7f938806e400 2023-11-28T12:02:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.460+0000 7f9385ffb700 1 -- 172.21.15.57:0/1598165636 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3324337521 0 0) 0x7f938815ef40 con 0x7f93880d15f0 2023-11-28T12:02:08.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.460+0000 7f9385ffb700 1 -- 172.21.15.57:0/1598165636 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f938815af30 con 0x7f93880d15f0 2023-11-28T12:02:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.460+0000 7f93877fe700 1 -- 172.21.15.57:0/1598165636 >> v1:172.21.15.181:6789/0 conn(0x7f93880d15f0 legacy=0x7f938815b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.460+0000 7f93877fe700 1 -- 172.21.15.57:0/1598165636 >> v1:172.21.15.57:6789/0 conn(0x7f938806e400 legacy=0x7f938806e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.460+0000 7f93877fe700 1 -- 172.21.15.57:0/1598165636 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938815e980 con 0x7f938815e010 2023-11-28T12:02:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.460+0000 7f93877fe700 1 --2- 172.21.15.57:0/1598165636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f938815e010 0x7f93880ce970 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9380009d90 tx=0x7f938000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeb59224ea47ee18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.460+0000 7f9385ffb700 1 -- 172.21.15.57:0/1598165636 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938000e070 con 0x7f938815e010 2023-11-28T12:02:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.460+0000 7f9385ffb700 1 -- 172.21.15.57:0/1598165636 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938000bb20 con 0x7f938815e010 2023-11-28T12:02:08.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.460+0000 7f9385ffb700 1 -- 172.21.15.57:0/1598165636 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380012430 con 0x7f938815e010 2023-11-28T12:02:08.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.461+0000 7f938d343700 1 -- 172.21.15.57:0/1598165636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f938815e010 msgr2=0x7f93880ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:08.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.461+0000 7f938d343700 1 --2- 172.21.15.57:0/1598165636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f938815e010 0x7f93880ce970 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9380009d90 tx=0x7f938000b330 comp rx=0 tx=0).stop 2023-11-28T12:02:08.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.461+0000 7f938d343700 1 -- 172.21.15.57:0/1598165636 shutdown_connections 2023-11-28T12:02:08.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.461+0000 7f938d343700 1 --2- 172.21.15.57:0/1598165636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f938815e010 0x7f93880ce970 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:08.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.461+0000 7f938d343700 1 -- 172.21.15.57:0/1598165636 >> 172.21.15.57:0/1598165636 conn(0x7f938806a270 msgr2=0x7f938806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:08.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.461+0000 7f938d343700 1 -- 172.21.15.57:0/1598165636 shutdown_connections 2023-11-28T12:02:08.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.461+0000 7f938d343700 1 -- 172.21.15.57:0/1598165636 wait complete. 2023-11-28T12:02:08.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.463+0000 7f938d343700 1 Processor -- start 2023-11-28T12:02:08.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.463+0000 7f938d343700 1 -- start start 2023-11-28T12:02:08.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.463+0000 7f938d343700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f938806e400 0x7f93880b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:08.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.463+0000 7f938d343700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93880d15f0 0x7f9388149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:08.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.463+0000 7f938d343700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f938815e010 0x7f938815c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:08.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.463+0000 7f938d343700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93880b16b0 con 0x7f938815e010 2023-11-28T12:02:08.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.463+0000 7f938d343700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93880b1b10 con 0x7f93880d15f0 2023-11-28T12:02:08.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.463+0000 7f938d343700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9388151b30 con 0x7f938806e400 2023-11-28T12:02:08.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f93877fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f938815e010 0x7f938815c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:08.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f93877fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f938815e010 0x7f938815c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45570/0 (socket says 172.21.15.57:45570) 2023-11-28T12:02:08.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f93877fe700 1 -- 172.21.15.57:0/332744501 learned_addr learned my addr 172.21.15.57:0/332744501 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:08.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f93867fc700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93880d15f0 0x7f9388149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:08.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f9386ffd700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f938806e400 0x7f93880b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:08.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f93877fe700 1 -- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f938806e400 msgr2=0x7f93880b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:08.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f93877fe700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f938806e400 0x7f93880b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:08.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f93877fe700 1 -- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93880d15f0 msgr2=0x7f9388149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:08.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f93877fe700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93880d15f0 0x7f9388149740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:08.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f93877fe700 1 -- 172.21.15.57:0/332744501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9380009a70 con 0x7f938815e010 2023-11-28T12:02:08.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f9386ffd700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f938806e400 0x7f93880b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:08.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f93877fe700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f938815e010 0x7f938815c540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93880b6eb0 tx=0x7f9380004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:08.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f936f7fe700 1 -- 172.21.15.57:0/332744501 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938001b810 con 0x7f938815e010 2023-11-28T12:02:08.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f936f7fe700 1 -- 172.21.15.57:0/332744501 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9380020070 con 0x7f938815e010 2023-11-28T12:02:08.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f938d343700 1 -- 172.21.15.57:0/332744501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938815abc0 con 0x7f938815e010 2023-11-28T12:02:08.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f936f7fe700 1 -- 172.21.15.57:0/332744501 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938001bbb0 con 0x7f938815e010 2023-11-28T12:02:08.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.464+0000 7f938d343700 1 -- 172.21.15.57:0/332744501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938815ec60 con 0x7f938815e010 2023-11-28T12:02:08.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.465+0000 7f938d343700 1 -- 172.21.15.57:0/332744501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938815e490 con 0x7f938815e010 2023-11-28T12:02:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.466+0000 7f936f7fe700 1 -- 172.21.15.57:0/332744501 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f938001bd10 con 0x7f938815e010 2023-11-28T12:02:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.466+0000 7f936f7fe700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f93740682f0 0x7f937406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.466+0000 7f936f7fe700 1 -- 172.21.15.57:0/332744501 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9380022070 con 0x7f938815e010 2023-11-28T12:02:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.466+0000 7f9386ffd700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f93740682f0 0x7f937406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.467+0000 7f9386ffd700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f93740682f0 0x7f937406a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f937c000f60 tx=0x7f937c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.471+0000 7f936f7fe700 1 -- 172.21.15.57:0/332744501 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9380054ab0 con 0x7f938815e010 2023-11-28T12:02:08.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.674+0000 7f938d343700 1 -- 172.21.15.57:0/332744501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93880743f0 con 0x7f938815e010 2023-11-28T12:02:08.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.675+0000 7f936f7fe700 1 -- 172.21.15.57:0/332744501 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7f93800580d0 con 0x7f938815e010 2023-11-28T12:02:08.677 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:08.677 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:08.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.678+0000 7f938d343700 1 -- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f93740682f0 msgr2=0x7f937406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:08.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.678+0000 7f938d343700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f93740682f0 0x7f937406a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f937c000f60 tx=0x7f937c009040 comp rx=0 tx=0).stop 2023-11-28T12:02:08.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.678+0000 7f938d343700 1 -- 172.21.15.57:0/332744501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f938815e010 msgr2=0x7f938815c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:08.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.678+0000 7f938d343700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f938815e010 0x7f938815c540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93880b6eb0 tx=0x7f9380004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:08.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.678+0000 7f938d343700 1 -- 172.21.15.57:0/332744501 shutdown_connections 2023-11-28T12:02:08.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.678+0000 7f938d343700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f93740682f0 0x7f937406a7a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:08.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.678+0000 7f938d343700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f938815e010 0x7f938815c540 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:08.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.678+0000 7f938d343700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f938806e400 0x7f93880b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:08.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.678+0000 7f938d343700 1 --2- 172.21.15.57:0/332744501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93880d15f0 0x7f9388149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:08.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.678+0000 7f938d343700 1 -- 172.21.15.57:0/332744501 >> 172.21.15.57:0/332744501 conn(0x7f938806a270 msgr2=0x7f93880cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:08.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.678+0000 7f938d343700 1 -- 172.21.15.57:0/332744501 shutdown_connections 2023-11-28T12:02:08.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.678+0000 7f938d343700 1 -- 172.21.15.57:0/332744501 wait complete. 2023-11-28T12:02:08.693 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:08.693 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:08.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.832+0000 7f352ccec700 1 Processor -- start 2023-11-28T12:02:08.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.832+0000 7f352ccec700 1 -- start start 2023-11-28T12:02:08.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.833+0000 7f352ccec700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f352815fe50 0x7f352806d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:08.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.833+0000 7f352ccec700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3528161370 con 0x7f35280b3b80 2023-11-28T12:02:08.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.833+0000 7f352ccec700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35280cf700 con 0x7f35280b27d0 2023-11-28T12:02:08.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.833+0000 7f352ccec700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3528160220 con 0x7f352815fe50 2023-11-28T12:02:08.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.833+0000 7f352659c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f352815fe50 0x7f352806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:08.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.833+0000 7f3525d9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f35280b3b80 0x7f3528160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37602/0 (socket says 172.21.15.57:37602) 2023-11-28T12:02:08.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.833+0000 7f3525d9b700 1 -- 172.21.15.57:0/2236068940 learned_addr learned my addr 172.21.15.57:0/2236068940 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:08.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.834+0000 7f352659c700 1 -- 172.21.15.57:0/2236068940 >> v1:172.21.15.181:6789/0 conn(0x7f35280b27d0 legacy=0x7f35280cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:08.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.834+0000 7f352659c700 1 -- 172.21.15.57:0/2236068940 >> v1:172.21.15.57:6789/0 conn(0x7f35280b3b80 legacy=0x7f3528160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:08.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.834+0000 7f352559a700 1 -- 172.21.15.57:0/2236068940 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601516425 0 0) 0x7f3528161370 con 0x7f35280b3b80 2023-11-28T12:02:08.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.834+0000 7f352659c700 1 -- 172.21.15.57:0/2236068940 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35280b3fd0 con 0x7f352815fe50 2023-11-28T12:02:08.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.834+0000 7f352659c700 1 --2- 172.21.15.57:0/2236068940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f352815fe50 0x7f352806d170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3518009840 tx=0x7f3518009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118200df9606f2b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:08.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.834+0000 7f352559a700 1 -- 172.21.15.57:0/2236068940 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518004230 con 0x7f352815fe50 2023-11-28T12:02:08.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.834+0000 7f352559a700 1 -- 172.21.15.57:0/2236068940 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3518004390 con 0x7f352815fe50 2023-11-28T12:02:08.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.834+0000 7f352559a700 1 -- 172.21.15.57:0/2236068940 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180036a0 con 0x7f352815fe50 2023-11-28T12:02:08.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.835+0000 7f352ccec700 1 -- 172.21.15.57:0/2236068940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f352815fe50 msgr2=0x7f352806d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:08.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.835+0000 7f352ccec700 1 --2- 172.21.15.57:0/2236068940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f352815fe50 0x7f352806d170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3518009840 tx=0x7f3518009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:08.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.835+0000 7f352ccec700 1 -- 172.21.15.57:0/2236068940 shutdown_connections 2023-11-28T12:02:08.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.835+0000 7f352ccec700 1 --2- 172.21.15.57:0/2236068940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f352815fe50 0x7f352806d170 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:08.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.835+0000 7f352ccec700 1 -- 172.21.15.57:0/2236068940 >> 172.21.15.57:0/2236068940 conn(0x7f352806ea80 msgr2=0x7f352806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:08.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.835+0000 7f352ccec700 1 -- 172.21.15.57:0/2236068940 shutdown_connections 2023-11-28T12:02:08.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.835+0000 7f352ccec700 1 -- 172.21.15.57:0/2236068940 wait complete. 2023-11-28T12:02:08.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.836+0000 7f352ccec700 1 Processor -- start 2023-11-28T12:02:08.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.836+0000 7f352ccec700 1 -- start start 2023-11-28T12:02:08.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352ccec700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35280b27d0 0x7f35280b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:08.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352ccec700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35280b3b80 0x7f35281556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:08.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352ccec700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f352815fe50 0x7f35280c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:08.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352ccec700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3528159430 con 0x7f35280b27d0 2023-11-28T12:02:08.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352ccec700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352815b020 con 0x7f35280b3b80 2023-11-28T12:02:08.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352ccec700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35281f6370 con 0x7f352815fe50 2023-11-28T12:02:08.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35280b27d0 0x7f35280b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:08.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35280b27d0 0x7f35280b6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45592/0 (socket says 172.21.15.57:45592) 2023-11-28T12:02:08.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352659c700 1 -- 172.21.15.57:0/2749921352 learned_addr learned my addr 172.21.15.57:0/2749921352 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:08.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f3525d9b700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35280b3b80 0x7f35281556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:08.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f3526d9d700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f352815fe50 0x7f35280c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:08.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352659c700 1 -- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f352815fe50 msgr2=0x7f35280c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:08.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352659c700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f352815fe50 0x7f35280c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:08.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352659c700 1 -- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35280b3b80 msgr2=0x7f35281556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:08.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352659c700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35280b3b80 0x7f35281556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:08.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.837+0000 7f352659c700 1 -- 172.21.15.57:0/2749921352 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180094a0 con 0x7f35280b27d0 2023-11-28T12:02:08.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.838+0000 7f352659c700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35280b27d0 0x7f35280b6830 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f35180090b0 tx=0x7f3518004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:08.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.838+0000 7f351effd700 1 -- 172.21.15.57:0/2749921352 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518003ea0 con 0x7f35280b27d0 2023-11-28T12:02:08.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.838+0000 7f351effd700 1 -- 172.21.15.57:0/2749921352 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351801e070 con 0x7f35280b27d0 2023-11-28T12:02:08.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.838+0000 7f352ccec700 1 -- 172.21.15.57:0/2749921352 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35280cf0b0 con 0x7f35280b27d0 2023-11-28T12:02:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.838+0000 7f351effd700 1 -- 172.21.15.57:0/2749921352 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518011850 con 0x7f35280b27d0 2023-11-28T12:02:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.838+0000 7f352ccec700 1 -- 172.21.15.57:0/2749921352 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35280cedd0 con 0x7f35280b27d0 2023-11-28T12:02:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.839+0000 7f352ccec700 1 -- 172.21.15.57:0/2749921352 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35280cf390 con 0x7f35280b27d0 2023-11-28T12:02:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.839+0000 7f351effd700 1 -- 172.21.15.57:0/2749921352 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3518017020 con 0x7f35280b27d0 2023-11-28T12:02:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.840+0000 7f351effd700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f35080682f0 0x7f350806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.840+0000 7f351effd700 1 -- 172.21.15.57:0/2749921352 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3518086ef0 con 0x7f35280b27d0 2023-11-28T12:02:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.840+0000 7f3525d9b700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f35080682f0 0x7f350806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.840+0000 7f3525d9b700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f35080682f0 0x7f350806a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3510000f60 tx=0x7f3510007e70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:08.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:08.844+0000 7f351effd700 1 -- 172.21.15.57:0/2749921352 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3518053750 con 0x7f35280b27d0 2023-11-28T12:02:09.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.045+0000 7f352ccec700 1 -- 172.21.15.57:0/2749921352 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f35280743f0 con 0x7f35280b27d0 2023-11-28T12:02:09.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.045+0000 7f351effd700 1 -- 172.21.15.57:0/2749921352 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3518020020 con 0x7f35280b27d0 2023-11-28T12:02:09.046 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:09.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.047+0000 7f352ccec700 1 -- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f35080682f0 msgr2=0x7f350806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.047+0000 7f352ccec700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f35080682f0 0x7f350806a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3510000f60 tx=0x7f3510007e70 comp rx=0 tx=0).stop 2023-11-28T12:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.047+0000 7f352ccec700 1 -- 172.21.15.57:0/2749921352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35280b27d0 msgr2=0x7f35280b6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.047+0000 7f352ccec700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35280b27d0 0x7f35280b6830 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f35180090b0 tx=0x7f3518004750 comp rx=0 tx=0).stop 2023-11-28T12:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.047+0000 7f352ccec700 1 -- 172.21.15.57:0/2749921352 shutdown_connections 2023-11-28T12:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.047+0000 7f352ccec700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f35080682f0 0x7f350806a7a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.047+0000 7f352ccec700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f352815fe50 0x7f35280c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.047+0000 7f352ccec700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35280b27d0 0x7f35280b6830 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.047+0000 7f352ccec700 1 --2- 172.21.15.57:0/2749921352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35280b3b80 0x7f35281556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.047+0000 7f352ccec700 1 -- 172.21.15.57:0/2749921352 >> 172.21.15.57:0/2749921352 conn(0x7f352806ea80 msgr2=0x7f35280b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.047+0000 7f352ccec700 1 -- 172.21.15.57:0/2749921352 shutdown_connections 2023-11-28T12:02:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.047+0000 7f352ccec700 1 -- 172.21.15.57:0/2749921352 wait complete. 2023-11-28T12:02:09.062 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:09.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.198+0000 7f631e622700 1 Processor -- start 2023-11-28T12:02:09.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.199+0000 7f631e622700 1 -- start start 2023-11-28T12:02:09.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.199+0000 7f631e622700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63180b0540 0x7f63180c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.199+0000 7f631e622700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f631815e910 0x7f6318070940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.199+0000 7f631e622700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f631815a8c0 con 0x7f63180ca310 2023-11-28T12:02:09.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.199+0000 7f631e622700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63180ca780 con 0x7f631815e910 2023-11-28T12:02:09.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.199+0000 7f631e622700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63180caa10 con 0x7f63180b0540 2023-11-28T12:02:09.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.200+0000 7f631cbbf700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f631815e910 0x7f6318070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.200+0000 7f631cbbf700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f631815e910 0x7f6318070940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45600/0 (socket says 172.21.15.57:45600) 2023-11-28T12:02:09.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.200+0000 7f631cbbf700 1 -- 172.21.15.57:0/3228711421 learned_addr learned my addr 172.21.15.57:0/3228711421 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:09.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.200+0000 7f6317fff700 1 --2- 172.21.15.57:0/3228711421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63180b0540 0x7f63180c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.200+0000 7f631cbbf700 1 -- 172.21.15.57:0/3228711421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63180b0540 msgr2=0x7f63180c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.200+0000 7f631cbbf700 1 --2- 172.21.15.57:0/3228711421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63180b0540 0x7f63180c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.200+0000 7f631cbbf700 1 -- 172.21.15.57:0/3228711421 >> v1:172.21.15.181:6789/0 conn(0x7f63180ca310 legacy=0x7f63180b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.200+0000 7f631cbbf700 1 -- 172.21.15.57:0/3228711421 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63180cacf0 con 0x7f631815e910 2023-11-28T12:02:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.200+0000 7f631cbbf700 1 --2- 172.21.15.57:0/3228711421 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f631815e910 0x7f6318070940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6310009e50 tx=0x7f631000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8ca3b9386f9f0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.200+0000 7f6316ffd700 1 -- 172.21.15.57:0/3228711421 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631000e070 con 0x7f631815e910 2023-11-28T12:02:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.200+0000 7f6316ffd700 1 -- 172.21.15.57:0/3228711421 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631000b910 con 0x7f631815e910 2023-11-28T12:02:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.200+0000 7f6316ffd700 1 -- 172.21.15.57:0/3228711421 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631000bcc0 con 0x7f631815e910 2023-11-28T12:02:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.201+0000 7f631e622700 1 -- 172.21.15.57:0/3228711421 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f631815e910 msgr2=0x7f6318070940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.201+0000 7f631e622700 1 --2- 172.21.15.57:0/3228711421 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f631815e910 0x7f6318070940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6310009e50 tx=0x7f631000b330 comp rx=0 tx=0).stop 2023-11-28T12:02:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.201+0000 7f631e622700 1 -- 172.21.15.57:0/3228711421 shutdown_connections 2023-11-28T12:02:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.201+0000 7f631e622700 1 --2- 172.21.15.57:0/3228711421 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f631815e910 0x7f6318070940 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.201+0000 7f631e622700 1 --2- 172.21.15.57:0/3228711421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63180b0540 0x7f63180c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.201+0000 7f631e622700 1 -- 172.21.15.57:0/3228711421 >> 172.21.15.57:0/3228711421 conn(0x7f63180cb540 msgr2=0x7f63180cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:09.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.201+0000 7f631e622700 1 -- 172.21.15.57:0/3228711421 shutdown_connections 2023-11-28T12:02:09.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.201+0000 7f631e622700 1 -- 172.21.15.57:0/3228711421 wait complete. 2023-11-28T12:02:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.202+0000 7f631e622700 1 Processor -- start 2023-11-28T12:02:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.202+0000 7f631e622700 1 -- start start 2023-11-28T12:02:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.203+0000 7f631e622700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63180b0540 0x7f63180c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.203+0000 7f631e622700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63180ca310 0x7f63181e5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.203+0000 7f631e622700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631815e910 0x7f6318158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.203+0000 7f631e622700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63181e6080 con 0x7f63180ca310 2023-11-28T12:02:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.203+0000 7f631e622700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63181e4aa0 con 0x7f63180b0540 2023-11-28T12:02:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.203+0000 7f631e622700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63181e4d80 con 0x7f631815e910 2023-11-28T12:02:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.203+0000 7f63177fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63180ca310 0x7f63181e5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.203+0000 7f63177fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63180ca310 0x7f63181e5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45612/0 (socket says 172.21.15.57:45612) 2023-11-28T12:02:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.203+0000 7f63177fe700 1 -- 172.21.15.57:0/2636744908 learned_addr learned my addr 172.21.15.57:0/2636744908 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.203+0000 7f631cbbf700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631815e910 0x7f6318158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.203+0000 7f6317fff700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63180b0540 0x7f63180c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.204+0000 7f63177fe700 1 -- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631815e910 msgr2=0x7f6318158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.204+0000 7f63177fe700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631815e910 0x7f6318158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.204+0000 7f63177fe700 1 -- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63180b0540 msgr2=0x7f63180c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.204+0000 7f63177fe700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63180b0540 0x7f63180c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.204+0000 7f63177fe700 1 -- 172.21.15.57:0/2636744908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6310009b30 con 0x7f63180ca310 2023-11-28T12:02:09.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.204+0000 7f63177fe700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63180ca310 0x7f63181e5b40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f630c00c4e0 tx=0x7f630c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:09.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.204+0000 7f6314ff9700 1 -- 172.21.15.57:0/2636744908 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c0052d0 con 0x7f63180ca310 2023-11-28T12:02:09.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.204+0000 7f6314ff9700 1 -- 172.21.15.57:0/2636744908 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c005430 con 0x7f63180ca310 2023-11-28T12:02:09.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.204+0000 7f6314ff9700 1 -- 172.21.15.57:0/2636744908 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c0038c0 con 0x7f63180ca310 2023-11-28T12:02:09.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.204+0000 7f631e622700 1 -- 172.21.15.57:0/2636744908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63180c93f0 con 0x7f63180ca310 2023-11-28T12:02:09.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.204+0000 7f631e622700 1 -- 172.21.15.57:0/2636744908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63180c9110 con 0x7f63180ca310 2023-11-28T12:02:09.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.205+0000 7f631e622700 1 -- 172.21.15.57:0/2636744908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63180c96d0 con 0x7f63180ca310 2023-11-28T12:02:09.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.205+0000 7f6314ff9700 1 -- 172.21.15.57:0/2636744908 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f630c019030 con 0x7f63180ca310 2023-11-28T12:02:09.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.206+0000 7f6314ff9700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63040682f0 0x7f630406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.206+0000 7f6314ff9700 1 -- 172.21.15.57:0/2636744908 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f630c084eb0 con 0x7f63180ca310 2023-11-28T12:02:09.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.206+0000 7f6317fff700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63040682f0 0x7f630406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.207+0000 7f6317fff700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63040682f0 0x7f630406a7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6308009b40 tx=0x7f6308009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:09.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.210+0000 7f6314ff9700 1 -- 172.21.15.57:0/2636744908 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f630c051710 con 0x7f63180ca310 2023-11-28T12:02:09.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.411+0000 7f631e622700 1 -- 172.21.15.57:0/2636744908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63180743f0 con 0x7f63180ca310 2023-11-28T12:02:09.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.411+0000 7f6314ff9700 1 -- 172.21.15.57:0/2636744908 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f630c0170a0 con 0x7f63180ca310 2023-11-28T12:02:09.412 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:09.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.413+0000 7f631e622700 1 -- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63040682f0 msgr2=0x7f630406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.413+0000 7f631e622700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63040682f0 0x7f630406a7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6308009b40 tx=0x7f6308009210 comp rx=0 tx=0).stop 2023-11-28T12:02:09.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.414+0000 7f631e622700 1 -- 172.21.15.57:0/2636744908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63180ca310 msgr2=0x7f63181e5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.414+0000 7f631e622700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63180ca310 0x7f63181e5b40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f630c00c4e0 tx=0x7f630c00c7f0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.414+0000 7f631e622700 1 -- 172.21.15.57:0/2636744908 shutdown_connections 2023-11-28T12:02:09.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.414+0000 7f631e622700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63040682f0 0x7f630406a7a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.414+0000 7f631e622700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631815e910 0x7f6318158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.414+0000 7f631e622700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63180b0540 0x7f63180c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.414+0000 7f631e622700 1 --2- 172.21.15.57:0/2636744908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63180ca310 0x7f63181e5b40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.414+0000 7f631e622700 1 -- 172.21.15.57:0/2636744908 >> 172.21.15.57:0/2636744908 conn(0x7f63180cb540 msgr2=0x7f6318071190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:09.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.414+0000 7f631e622700 1 -- 172.21.15.57:0/2636744908 shutdown_connections 2023-11-28T12:02:09.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.414+0000 7f631e622700 1 -- 172.21.15.57:0/2636744908 wait complete. 2023-11-28T12:02:09.428 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:02:09.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.564+0000 7f43b7698700 1 Processor -- start 2023-11-28T12:02:09.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.564+0000 7f43b7698700 1 -- start start 2023-11-28T12:02:09.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.565+0000 7f43b7698700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b015a5f0 0x7f43b00d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.565+0000 7f43b7698700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f43b00cc200 0x7f43b00b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.565+0000 7f43b7698700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f43b00b2d10 0x7f43b015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.565+0000 7f43b7698700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b00b3190 con 0x7f43b00b2d10 2023-11-28T12:02:09.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.565+0000 7f43b7698700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b015a9c0 con 0x7f43b00cc200 2023-11-28T12:02:09.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.565+0000 7f43b7698700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00c7b80 con 0x7f43b015a5f0 2023-11-28T12:02:09.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.565+0000 7f43b5c35700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f43b00b2d10 0x7f43b015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.565+0000 7f43b5c35700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f43b00b2d10 0x7f43b015e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45628/0 (socket says 172.21.15.57:45628) 2023-11-28T12:02:09.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.565+0000 7f43b5c35700 1 -- 172.21.15.57:0/3381162612 learned_addr learned my addr 172.21.15.57:0/3381162612 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:09.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.566+0000 7f43b4c33700 1 --2- 172.21.15.57:0/3381162612 >> v2:172.21.15.181:3300/0 conn(0x7f43b00cc200 0x7f43b00b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.566+0000 7f43b5434700 1 --2- 172.21.15.57:0/3381162612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b015a5f0 0x7f43b00d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.566+0000 7f43b5c35700 1 -- 172.21.15.57:0/3381162612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b015a5f0 msgr2=0x7f43b00d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:09.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.566+0000 7f43b5c35700 1 --2- 172.21.15.57:0/3381162612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b015a5f0 0x7f43b00d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.566+0000 7f43b5c35700 1 -- 172.21.15.57:0/3381162612 >> v2:172.21.15.181:3300/0 conn(0x7f43b00cc200 msgr2=0x7f43b00b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:09.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.566+0000 7f43b5c35700 1 --2- 172.21.15.57:0/3381162612 >> v2:172.21.15.181:3300/0 conn(0x7f43b00cc200 0x7f43b00b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.566+0000 7f43b5c35700 1 -- 172.21.15.57:0/3381162612 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00cc990 con 0x7f43b00b2d10 2023-11-28T12:02:09.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.566+0000 7f43b5c35700 1 --2- 172.21.15.57:0/3381162612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b00b2d10 0x7f43b015e110 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f43b00719d0 tx=0x7f43a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20f45908906753a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:09.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.566+0000 7f43a3fff700 1 -- 172.21.15.57:0/3381162612 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800e070 con 0x7f43b00b2d10 2023-11-28T12:02:09.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.566+0000 7f43a3fff700 1 -- 172.21.15.57:0/3381162612 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a800b9c0 con 0x7f43b00b2d10 2023-11-28T12:02:09.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.566+0000 7f43a3fff700 1 -- 172.21.15.57:0/3381162612 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800bd70 con 0x7f43b00b2d10 2023-11-28T12:02:09.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.567+0000 7f43b7698700 1 -- 172.21.15.57:0/3381162612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b00b2d10 msgr2=0x7f43b015e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.567+0000 7f43b7698700 1 --2- 172.21.15.57:0/3381162612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b00b2d10 0x7f43b015e110 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f43b00719d0 tx=0x7f43a800b330 comp rx=0 tx=0).stop 2023-11-28T12:02:09.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.567+0000 7f43b7698700 1 -- 172.21.15.57:0/3381162612 shutdown_connections 2023-11-28T12:02:09.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.567+0000 7f43b7698700 1 --2- 172.21.15.57:0/3381162612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b015a5f0 0x7f43b00d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.567+0000 7f43b7698700 1 --2- 172.21.15.57:0/3381162612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b00b2d10 0x7f43b015e110 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.567+0000 7f43b7698700 1 --2- 172.21.15.57:0/3381162612 >> v2:172.21.15.181:3300/0 conn(0x7f43b00cc200 0x7f43b00b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.567+0000 7f43b7698700 1 -- 172.21.15.57:0/3381162612 >> 172.21.15.57:0/3381162612 conn(0x7f43b006ea80 msgr2=0x7f43b006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:09.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.567+0000 7f43b7698700 1 -- 172.21.15.57:0/3381162612 shutdown_connections 2023-11-28T12:02:09.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.567+0000 7f43b7698700 1 -- 172.21.15.57:0/3381162612 wait complete. 2023-11-28T12:02:09.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.568+0000 7f43b7698700 1 Processor -- start 2023-11-28T12:02:09.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.568+0000 7f43b7698700 1 -- start start 2023-11-28T12:02:09.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b7698700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b00b2d10 0x7f43b014cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b7698700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b00cc200 0x7f43b00c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b7698700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b015a5f0 0x7f43b00c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b7698700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00c5830 con 0x7f43b015a5f0 2023-11-28T12:02:09.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b7698700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00ca5a0 con 0x7f43b00b2d10 2023-11-28T12:02:09.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b7698700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00b4640 con 0x7f43b00cc200 2023-11-28T12:02:09.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b5c35700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b015a5f0 0x7f43b00c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b5c35700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b015a5f0 0x7f43b00c4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45630/0 (socket says 172.21.15.57:45630) 2023-11-28T12:02:09.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b5c35700 1 -- 172.21.15.57:0/1169998202 learned_addr learned my addr 172.21.15.57:0/1169998202 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:09.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b5434700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b00b2d10 0x7f43b014cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b4c33700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b00cc200 0x7f43b00c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b5c35700 1 -- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b00cc200 msgr2=0x7f43b00c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b5c35700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b00cc200 0x7f43b00c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b5c35700 1 -- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b00b2d10 msgr2=0x7f43b014cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b5c35700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b00b2d10 0x7f43b014cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.569+0000 7f43b5c35700 1 -- 172.21.15.57:0/1169998202 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a8009b30 con 0x7f43b015a5f0 2023-11-28T12:02:09.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.570+0000 7f43b5c35700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b015a5f0 0x7f43b00c4700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f43b006d5f0 tx=0x7f43a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:09.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.570+0000 7f43a1ffb700 1 -- 172.21.15.57:0/1169998202 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8004880 con 0x7f43b015a5f0 2023-11-28T12:02:09.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.570+0000 7f43a1ffb700 1 -- 172.21.15.57:0/1169998202 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a80099b0 con 0x7f43b015a5f0 2023-11-28T12:02:09.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.570+0000 7f43b7698700 1 -- 172.21.15.57:0/1169998202 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b00ccf50 con 0x7f43b015a5f0 2023-11-28T12:02:09.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.570+0000 7f43a1ffb700 1 -- 172.21.15.57:0/1169998202 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a801b670 con 0x7f43b015a5f0 2023-11-28T12:02:09.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.570+0000 7f43b7698700 1 -- 172.21.15.57:0/1169998202 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b00ccc70 con 0x7f43b015a5f0 2023-11-28T12:02:09.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.571+0000 7f43b7698700 1 -- 172.21.15.57:0/1169998202 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b00cd230 con 0x7f43b015a5f0 2023-11-28T12:02:09.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.572+0000 7f43a1ffb700 1 -- 172.21.15.57:0/1169998202 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f43a802b070 con 0x7f43b015a5f0 2023-11-28T12:02:09.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.572+0000 7f43a1ffb700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f43940682f0 0x7f439406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.572+0000 7f43a1ffb700 1 -- 172.21.15.57:0/1169998202 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f43a8087350 con 0x7f43b015a5f0 2023-11-28T12:02:09.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.572+0000 7f43b5434700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f43940682f0 0x7f439406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.573+0000 7f43b5434700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f43940682f0 0x7f439406a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f43ac006b20 tx=0x7f43ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:09.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.577+0000 7f43a1ffb700 1 -- 172.21.15.57:0/1169998202 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f43a8053bb0 con 0x7f43b015a5f0 2023-11-28T12:02:09.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.774+0000 7f43b7698700 1 -- 172.21.15.57:0/1169998202 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43b00743f0 con 0x7f43b015a5f0 2023-11-28T12:02:09.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.774+0000 7f43a1ffb700 1 -- 172.21.15.57:0/1169998202 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43a80571d0 con 0x7f43b015a5f0 2023-11-28T12:02:09.776 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:02:09.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.777+0000 7f43b7698700 1 -- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f43940682f0 msgr2=0x7f439406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.777+0000 7f43b7698700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f43940682f0 0x7f439406a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f43ac006b20 tx=0x7f43ac008040 comp rx=0 tx=0).stop 2023-11-28T12:02:09.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.777+0000 7f43b7698700 1 -- 172.21.15.57:0/1169998202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b015a5f0 msgr2=0x7f43b00c4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.777+0000 7f43b7698700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b015a5f0 0x7f43b00c4700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f43b006d5f0 tx=0x7f43a8004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.777+0000 7f43b7698700 1 -- 172.21.15.57:0/1169998202 shutdown_connections 2023-11-28T12:02:09.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.777+0000 7f43b7698700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f43940682f0 0x7f439406a7a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.777+0000 7f43b7698700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43b015a5f0 0x7f43b00c4700 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.777+0000 7f43b7698700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b00b2d10 0x7f43b014cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.777+0000 7f43b7698700 1 --2- 172.21.15.57:0/1169998202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b00cc200 0x7f43b00c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.777+0000 7f43b7698700 1 -- 172.21.15.57:0/1169998202 >> 172.21.15.57:0/1169998202 conn(0x7f43b006ea80 msgr2=0x7f43b006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:09.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.777+0000 7f43b7698700 1 -- 172.21.15.57:0/1169998202 shutdown_connections 2023-11-28T12:02:09.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.777+0000 7f43b7698700 1 -- 172.21.15.57:0/1169998202 wait complete. 2023-11-28T12:02:09.791 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:02:09.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.928+0000 7fbb8dac2700 1 Processor -- start 2023-11-28T12:02:09.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.928+0000 7fbb8dac2700 1 -- start start 2023-11-28T12:02:09.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.929+0000 7fbb8dac2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbb881599b0 0x7fbb8815d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.929+0000 7fbb8dac2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb880af0a0 0x7fbb8815a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.929+0000 7fbb8dac2700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb88159ec0 con 0x7fbb880b1650 2023-11-28T12:02:09.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.929+0000 7fbb8dac2700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8815d8a0 con 0x7fbb881599b0 2023-11-28T12:02:09.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.929+0000 7fbb8dac2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8815db80 con 0x7fbb880af0a0 2023-11-28T12:02:09.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.929+0000 7fbb86ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbb881599b0 0x7fbb8815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.929+0000 7fbb877fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fbb880b1650 0x7fbb880b1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37612/0 (socket says 172.21.15.57:37612) 2023-11-28T12:02:09.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.929+0000 7fbb86ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbb881599b0 0x7fbb8815d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45632/0 (socket says 172.21.15.57:45632) 2023-11-28T12:02:09.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.929+0000 7fbb877fe700 1 -- 172.21.15.57:0/3920173851 learned_addr learned my addr 172.21.15.57:0/3920173851 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:09.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.930+0000 7fbb87fff700 1 --2- 172.21.15.57:0/3920173851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb880af0a0 0x7fbb8815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.930+0000 7fbb86ffd700 1 -- 172.21.15.57:0/3920173851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb880af0a0 msgr2=0x7fbb8815a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:09.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.930+0000 7fbb86ffd700 1 --2- 172.21.15.57:0/3920173851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb880af0a0 0x7fbb8815a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.930+0000 7fbb86ffd700 1 -- 172.21.15.57:0/3920173851 >> v1:172.21.15.57:6789/0 conn(0x7fbb880b1650 legacy=0x7fbb880b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.930+0000 7fbb86ffd700 1 -- 172.21.15.57:0/3920173851 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8815de60 con 0x7fbb881599b0 2023-11-28T12:02:09.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.930+0000 7fbb87fff700 1 --2- 172.21.15.57:0/3920173851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb880af0a0 0x7fbb8815a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:09.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.930+0000 7fbb86ffd700 1 --2- 172.21.15.57:0/3920173851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb881599b0 0x7fbb8815d050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009840 tx=0x7fbb70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a715303024f8ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:09.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.930+0000 7fbb867fc700 1 -- 172.21.15.57:0/3920173851 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb70011830 con 0x7fbb881599b0 2023-11-28T12:02:09.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.930+0000 7fbb867fc700 1 -- 172.21.15.57:0/3920173851 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb70011990 con 0x7fbb881599b0 2023-11-28T12:02:09.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.930+0000 7fbb867fc700 1 -- 172.21.15.57:0/3920173851 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb70011d20 con 0x7fbb881599b0 2023-11-28T12:02:09.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.931+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3920173851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb881599b0 msgr2=0x7fbb8815d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.931+0000 7fbb8dac2700 1 --2- 172.21.15.57:0/3920173851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb881599b0 0x7fbb8815d050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009840 tx=0x7fbb70009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:09.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.931+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3920173851 shutdown_connections 2023-11-28T12:02:09.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.931+0000 7fbb8dac2700 1 --2- 172.21.15.57:0/3920173851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb880af0a0 0x7fbb8815a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.931+0000 7fbb8dac2700 1 --2- 172.21.15.57:0/3920173851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb881599b0 0x7fbb8815d050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.931+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3920173851 >> 172.21.15.57:0/3920173851 conn(0x7fbb8806c010 msgr2=0x7fbb880cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:09.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.931+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3920173851 shutdown_connections 2023-11-28T12:02:09.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.931+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3920173851 wait complete. 2023-11-28T12:02:09.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.932+0000 7fbb8dac2700 1 Processor -- start 2023-11-28T12:02:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.932+0000 7fbb8dac2700 1 -- start start 2023-11-28T12:02:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb8dac2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb880af0a0 0x7fbb88154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb8dac2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb880b1650 0x7fbb88151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb8dac2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb88150f30 0x7fbb880afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb8dac2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb88152170 con 0x7fbb88150f30 2023-11-28T12:02:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb8dac2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb880c5ce0 con 0x7fbb880af0a0 2023-11-28T12:02:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb8dac2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb880c5fc0 con 0x7fbb880b1650 2023-11-28T12:02:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb87fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb88150f30 0x7fbb880afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb87fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb88150f30 0x7fbb880afc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45634/0 (socket says 172.21.15.57:45634) 2023-11-28T12:02:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb87fff700 1 -- 172.21.15.57:0/3078934583 learned_addr learned my addr 172.21.15.57:0/3078934583 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb86ffd700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb880b1650 0x7fbb88151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb877fe700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb880af0a0 0x7fbb88154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb87fff700 1 -- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb880b1650 msgr2=0x7fbb88151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb87fff700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb880b1650 0x7fbb88151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb87fff700 1 -- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb880af0a0 msgr2=0x7fbb88154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:09.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb87fff700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb880af0a0 0x7fbb88154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:09.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb87fff700 1 -- 172.21.15.57:0/3078934583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb700094a0 con 0x7fbb88150f30 2023-11-28T12:02:09.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.933+0000 7fbb877fe700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb880af0a0 0x7fbb88154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:09.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.934+0000 7fbb87fff700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb88150f30 0x7fbb880afc40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbb80007df0 tx=0x7fbb8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:09.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.934+0000 7fbb7ffff700 1 -- 172.21.15.57:0/3078934583 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000ddd0 con 0x7fbb88150f30 2023-11-28T12:02:09.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.934+0000 7fbb7ffff700 1 -- 172.21.15.57:0/3078934583 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8000f040 con 0x7fbb88150f30 2023-11-28T12:02:09.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.934+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3078934583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8815a890 con 0x7fbb88150f30 2023-11-28T12:02:09.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.934+0000 7fbb7ffff700 1 -- 172.21.15.57:0/3078934583 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80003c60 con 0x7fbb88150f30 2023-11-28T12:02:09.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.934+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3078934583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8815d590 con 0x7fbb88150f30 2023-11-28T12:02:09.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.935+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3078934583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8815aa20 con 0x7fbb88150f30 2023-11-28T12:02:09.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.935+0000 7fbb7ffff700 1 -- 172.21.15.57:0/3078934583 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbb8001c030 con 0x7fbb88150f30 2023-11-28T12:02:09.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.936+0000 7fbb7ffff700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb680682f0 0x7fbb6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:09.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.936+0000 7fbb7ffff700 1 -- 172.21.15.57:0/3078934583 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbb80085b90 con 0x7fbb88150f30 2023-11-28T12:02:09.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.936+0000 7fbb877fe700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb680682f0 0x7fbb6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:09.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.937+0000 7fbb877fe700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb680682f0 0x7fbb6806a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbb78007fd0 tx=0x7fbb78009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:09.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:09.940+0000 7fbb7ffff700 1 -- 172.21.15.57:0/3078934583 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbb800524b0 con 0x7fbb88150f30 2023-11-28T12:02:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.136+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3078934583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb880c65e0 con 0x7fbb88150f30 2023-11-28T12:02:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.137+0000 7fbb7ffff700 1 -- 172.21.15.57:0/3078934583 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbb8001a0a0 con 0x7fbb88150f30 2023-11-28T12:02:10.139 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:02:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.139+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb680682f0 msgr2=0x7fbb6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.139+0000 7fbb8dac2700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb680682f0 0x7fbb6806a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbb78007fd0 tx=0x7fbb78009040 comp rx=0 tx=0).stop 2023-11-28T12:02:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.139+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3078934583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb88150f30 msgr2=0x7fbb880afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.139+0000 7fbb8dac2700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb88150f30 0x7fbb880afc40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbb80007df0 tx=0x7fbb8000d3b0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.139+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3078934583 shutdown_connections 2023-11-28T12:02:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.139+0000 7fbb8dac2700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb680682f0 0x7fbb6806a7a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.139+0000 7fbb8dac2700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb88150f30 0x7fbb880afc40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.139+0000 7fbb8dac2700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb880af0a0 0x7fbb88154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.139+0000 7fbb8dac2700 1 --2- 172.21.15.57:0/3078934583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb880b1650 0x7fbb88151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.139+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3078934583 >> 172.21.15.57:0/3078934583 conn(0x7fbb8806c010 msgr2=0x7fbb880cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.139+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3078934583 shutdown_connections 2023-11-28T12:02:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.139+0000 7fbb8dac2700 1 -- 172.21.15.57:0/3078934583 wait complete. 2023-11-28T12:02:10.154 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:10.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.288+0000 7f39c2d30700 1 Processor -- start 2023-11-28T12:02:10.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.288+0000 7f39c2d30700 1 -- start start 2023-11-28T12:02:10.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.289+0000 7f39c2d30700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f39bc0ce040 0x7f39bc15d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:10.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.289+0000 7f39c2d30700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39bc13fc90 con 0x7f39bc142db0 2023-11-28T12:02:10.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.289+0000 7f39c2d30700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39bc13f480 con 0x7f39bc0b1770 2023-11-28T12:02:10.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.289+0000 7f39c2d30700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39bc13f6d0 con 0x7f39bc0ce040 2023-11-28T12:02:10.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.289+0000 7f39bbfff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f39bc142db0 0x7f39bc13ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37626/0 (socket says 172.21.15.57:37626) 2023-11-28T12:02:10.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.289+0000 7f39bbfff700 1 -- 172.21.15.57:0/2451367286 learned_addr learned my addr 172.21.15.57:0/2451367286 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:10.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.289+0000 7f39c0acc700 1 --2- 172.21.15.57:0/2451367286 >> v2:172.21.15.181:3300/0 conn(0x7f39bc0ce040 0x7f39bc15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:10.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.289+0000 7f39bb7fe700 1 -- 172.21.15.57:0/2451367286 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75704233 0 0) 0x7f39bc13fc90 con 0x7f39bc142db0 2023-11-28T12:02:10.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.289+0000 7f39bb7fe700 1 -- 172.21.15.57:0/2451367286 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39a80035b0 con 0x7f39bc142db0 2023-11-28T12:02:10.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.289+0000 7f39bb7fe700 1 -- 172.21.15.57:0/2451367286 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4287363795 0 0) 0x7f39a80035b0 con 0x7f39bc142db0 2023-11-28T12:02:10.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.289+0000 7f39c0acc700 1 -- 172.21.15.57:0/2451367286 >> v1:172.21.15.181:6789/0 conn(0x7f39bc0b1770 legacy=0x7f39bc0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.289+0000 7f39c0acc700 1 -- 172.21.15.57:0/2451367286 >> v1:172.21.15.57:6789/0 conn(0x7f39bc142db0 legacy=0x7f39bc13ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.289+0000 7f39c0acc700 1 -- 172.21.15.57:0/2451367286 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc13f9b0 con 0x7f39bc0ce040 2023-11-28T12:02:10.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.290+0000 7f39c0acc700 1 --2- 172.21.15.57:0/2451367286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc0ce040 0x7f39bc15d900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009840 tx=0x7f39ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b92438720d5d720d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:10.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.290+0000 7f39bb7fe700 1 -- 172.21.15.57:0/2451367286 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4001670 con 0x7f39bc0ce040 2023-11-28T12:02:10.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.290+0000 7f39bb7fe700 1 -- 172.21.15.57:0/2451367286 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac004030 con 0x7f39bc0ce040 2023-11-28T12:02:10.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.290+0000 7f39bb7fe700 1 -- 172.21.15.57:0/2451367286 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0043c0 con 0x7f39bc0ce040 2023-11-28T12:02:10.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.290+0000 7f39c2d30700 1 -- 172.21.15.57:0/2451367286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc0ce040 msgr2=0x7f39bc15d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.290+0000 7f39c2d30700 1 --2- 172.21.15.57:0/2451367286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc0ce040 0x7f39bc15d900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009840 tx=0x7f39ac009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:10.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.290+0000 7f39c2d30700 1 -- 172.21.15.57:0/2451367286 shutdown_connections 2023-11-28T12:02:10.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.290+0000 7f39c2d30700 1 --2- 172.21.15.57:0/2451367286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc0ce040 0x7f39bc15d900 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.290+0000 7f39c2d30700 1 -- 172.21.15.57:0/2451367286 >> 172.21.15.57:0/2451367286 conn(0x7f39bc06c010 msgr2=0x7f39bc0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:10.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.290+0000 7f39c2d30700 1 -- 172.21.15.57:0/2451367286 shutdown_connections 2023-11-28T12:02:10.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.290+0000 7f39c2d30700 1 -- 172.21.15.57:0/2451367286 wait complete. 2023-11-28T12:02:10.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.292+0000 7f39c2d30700 1 Processor -- start 2023-11-28T12:02:10.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.292+0000 7f39c2d30700 1 -- start start 2023-11-28T12:02:10.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c2d30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc0b1770 0x7f39bc1519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:10.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c2d30700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39bc0ce040 0x7f39bc1513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:10.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c2d30700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc142db0 0x7f39bc0b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:10.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c2d30700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0b5b00 con 0x7f39bc142db0 2023-11-28T12:02:10.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c2d30700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0c42b0 con 0x7f39bc0b1770 2023-11-28T12:02:10.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c2d30700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0c3980 con 0x7f39bc0ce040 2023-11-28T12:02:10.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c12cd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc142db0 0x7f39bc0b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:10.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c0acc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc0b1770 0x7f39bc1519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:10.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c12cd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc142db0 0x7f39bc0b2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45638/0 (socket says 172.21.15.57:45638) 2023-11-28T12:02:10.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c12cd700 1 -- 172.21.15.57:0/956024335 learned_addr learned my addr 172.21.15.57:0/956024335 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:10.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39bbfff700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39bc0ce040 0x7f39bc1513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:10.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c12cd700 1 -- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39bc0ce040 msgr2=0x7f39bc1513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c12cd700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39bc0ce040 0x7f39bc1513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c12cd700 1 -- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc0b1770 msgr2=0x7f39bc1519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c12cd700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc0b1770 0x7f39bc1519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c12cd700 1 -- 172.21.15.57:0/956024335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac0094a0 con 0x7f39bc142db0 2023-11-28T12:02:10.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c0acc700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc0b1770 0x7f39bc1519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:10.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.293+0000 7f39c12cd700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc142db0 0x7f39bc0b2b20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f39b400c7f0 tx=0x7f39b400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:10.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.294+0000 7f39b97fa700 1 -- 172.21.15.57:0/956024335 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4003e30 con 0x7f39bc142db0 2023-11-28T12:02:10.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.294+0000 7f39b97fa700 1 -- 172.21.15.57:0/956024335 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b4005070 con 0x7f39bc142db0 2023-11-28T12:02:10.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.294+0000 7f39c2d30700 1 -- 172.21.15.57:0/956024335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc15d3e0 con 0x7f39bc142db0 2023-11-28T12:02:10.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.294+0000 7f39b97fa700 1 -- 172.21.15.57:0/956024335 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4005400 con 0x7f39bc142db0 2023-11-28T12:02:10.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.294+0000 7f39c2d30700 1 -- 172.21.15.57:0/956024335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc15d100 con 0x7f39bc142db0 2023-11-28T12:02:10.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.295+0000 7f39c2d30700 1 -- 172.21.15.57:0/956024335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc0ce410 con 0x7f39bc142db0 2023-11-28T12:02:10.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.295+0000 7f39b97fa700 1 -- 172.21.15.57:0/956024335 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f39b4010070 con 0x7f39bc142db0 2023-11-28T12:02:10.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.296+0000 7f39b97fa700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f399c0682f0 0x7f399c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:10.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.296+0000 7f39b97fa700 1 -- 172.21.15.57:0/956024335 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39b4085810 con 0x7f39bc142db0 2023-11-28T12:02:10.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.296+0000 7f39c0acc700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f399c0682f0 0x7f399c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:10.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.296+0000 7f39c0acc700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f399c0682f0 0x7f399c06a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009b90 tx=0x7f39ac008000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:10.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.300+0000 7f39b97fa700 1 -- 172.21.15.57:0/956024335 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f39b4051f80 con 0x7f39bc142db0 2023-11-28T12:02:10.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.502+0000 7f39c2d30700 1 -- 172.21.15.57:0/956024335 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39bc0cf610 con 0x7f39bc142db0 2023-11-28T12:02:10.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.503+0000 7f39b97fa700 1 -- 172.21.15.57:0/956024335 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7f39b40555a0 con 0x7f39bc142db0 2023-11-28T12:02:10.505 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:10.505 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:10.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.505+0000 7f39c2d30700 1 -- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f399c0682f0 msgr2=0x7f399c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.505+0000 7f39c2d30700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f399c0682f0 0x7f399c06a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009b90 tx=0x7f39ac008000 comp rx=0 tx=0).stop 2023-11-28T12:02:10.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.505+0000 7f39c2d30700 1 -- 172.21.15.57:0/956024335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc142db0 msgr2=0x7f39bc0b2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.505+0000 7f39c2d30700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc142db0 0x7f39bc0b2b20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f39b400c7f0 tx=0x7f39b400cb00 comp rx=0 tx=0).stop 2023-11-28T12:02:10.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.505+0000 7f39c2d30700 1 -- 172.21.15.57:0/956024335 shutdown_connections 2023-11-28T12:02:10.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.505+0000 7f39c2d30700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f399c0682f0 0x7f399c06a7a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.505+0000 7f39c2d30700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc142db0 0x7f39bc0b2b20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.505+0000 7f39c2d30700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc0b1770 0x7f39bc1519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.505+0000 7f39c2d30700 1 --2- 172.21.15.57:0/956024335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39bc0ce040 0x7f39bc1513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.505+0000 7f39c2d30700 1 -- 172.21.15.57:0/956024335 >> 172.21.15.57:0/956024335 conn(0x7f39bc06c010 msgr2=0x7f39bc06e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:10.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.505+0000 7f39c2d30700 1 -- 172.21.15.57:0/956024335 shutdown_connections 2023-11-28T12:02:10.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.505+0000 7f39c2d30700 1 -- 172.21.15.57:0/956024335 wait complete. 2023-11-28T12:02:10.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:10.521 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:10.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.658+0000 7f4d6c4d4700 1 Processor -- start 2023-11-28T12:02:10.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.658+0000 7f4d6c4d4700 1 -- start start 2023-11-28T12:02:10.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.659+0000 7f4d6c4d4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d6406b220 0x7f4d640d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:10.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.659+0000 7f4d6c4d4700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d640d2d30 con 0x7f4d64160cb0 2023-11-28T12:02:10.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.659+0000 7f4d6c4d4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d640d0b40 con 0x7f4d6415c8c0 2023-11-28T12:02:10.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.659+0000 7f4d6c4d4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640d2a50 con 0x7f4d6406b220 2023-11-28T12:02:10.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.659+0000 7f4d69a6f700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4d64160cb0 0x7f4d6406ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37634/0 (socket says 172.21.15.57:37634) 2023-11-28T12:02:10.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.659+0000 7f4d69a6f700 1 -- 172.21.15.57:0/4182425183 learned_addr learned my addr 172.21.15.57:0/4182425183 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:10.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6aa71700 1 --2- 172.21.15.57:0/4182425183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d6406b220 0x7f4d640d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:10.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917141382 0 0) 0x7f4d58001110 con 0x7f4d64160cb0 2023-11-28T12:02:10.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d500035b0 con 0x7f4d64160cb0 2023-11-28T12:02:10.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3196201356 0 0) 0x7f4d500035b0 con 0x7f4d64160cb0 2023-11-28T12:02:10.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d6406b220 msgr2=0x7f4d640d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:10.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 --2- 172.21.15.57:0/4182425183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d6406b220 0x7f4d640d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 >> v1:172.21.15.181:6789/0 conn(0x7f4d6415c8c0 legacy=0x7f4d640b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d640cd760 con 0x7f4d64160cb0 2023-11-28T12:02:10.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f4d58001800 con 0x7f4d64160cb0 2023-11-28T12:02:10.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 >> v1:172.21.15.57:6789/0 conn(0x7f4d64160cb0 legacy=0x7f4d6406ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 --2- 172.21.15.57:0/4182425183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d50003a30 0x7f4d50003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:10.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 --2- 172.21.15.57:0/4182425183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d6406b220 0x7f4d50004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:10.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 --2- 172.21.15.57:0/4182425183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d50004d90 0x7f4d50005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:10.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640d2a50 con 0x7f4d50003a30 2023-11-28T12:02:10.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5000bac0 con 0x7f4d50004d90 2023-11-28T12:02:10.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5000bc00 con 0x7f4d6406b220 2023-11-28T12:02:10.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6a270700 1 --2- 172.21.15.57:0/4182425183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d50003a30 0x7f4d50003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:10.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d69a6f700 1 --2- 172.21.15.57:0/4182425183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d6406b220 0x7f4d50004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:10.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6aa71700 1 --2- 172.21.15.57:0/4182425183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d50004d90 0x7f4d50005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:10.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6a270700 1 -- 172.21.15.57:0/4182425183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d6406b220 msgr2=0x7f4d50004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:10.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6a270700 1 --2- 172.21.15.57:0/4182425183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d6406b220 0x7f4d50004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6a270700 1 -- 172.21.15.57:0/4182425183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d50004d90 msgr2=0x7f4d50005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:10.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6a270700 1 --2- 172.21.15.57:0/4182425183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d50004d90 0x7f4d50005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.660+0000 7f4d6a270700 1 -- 172.21.15.57:0/4182425183 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d500035b0 con 0x7f4d50003a30 2023-11-28T12:02:10.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.661+0000 7f4d6a270700 1 --2- 172.21.15.57:0/4182425183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d50003a30 0x7f4d50003e00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d600018d0 tx=0x7f4d50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1e99ea159c8e44 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:10.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.661+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000c690 con 0x7f4d50003a30 2023-11-28T12:02:10.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.661+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6000c810 con 0x7f4d50003a30 2023-11-28T12:02:10.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.661+0000 7f4d6926e700 1 -- 172.21.15.57:0/4182425183 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000cbc0 con 0x7f4d50003a30 2023-11-28T12:02:10.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.661+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/4182425183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d50003a30 msgr2=0x7f4d50003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.661+0000 7f4d6c4d4700 1 --2- 172.21.15.57:0/4182425183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d50003a30 0x7f4d50003e00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d600018d0 tx=0x7f4d50005880 comp rx=0 tx=0).stop 2023-11-28T12:02:10.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.661+0000 7f4d6aa71700 1 -- 172.21.15.57:0/4182425183 reap_dead start 2023-11-28T12:02:10.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.662+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/4182425183 shutdown_connections 2023-11-28T12:02:10.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.662+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/4182425183 >> 172.21.15.57:0/4182425183 conn(0x7f4d6406c520 msgr2=0x7f4d6406c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:10.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.662+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/4182425183 shutdown_connections 2023-11-28T12:02:10.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.662+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/4182425183 wait complete. 2023-11-28T12:02:10.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.663+0000 7f4d6c4d4700 1 Processor -- start 2023-11-28T12:02:10.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.663+0000 7f4d6c4d4700 1 -- start start 2023-11-28T12:02:10.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6c4d4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d50003a30 0x7f4d6414d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:10.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6c4d4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d64159150 0x7f4d640cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:10.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6c4d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6415aed0 0x7f4d640b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:10.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6c4d4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640cb5d0 con 0x7f4d50003a30 2023-11-28T12:02:10.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6c4d4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640cb870 con 0x7f4d6415aed0 2023-11-28T12:02:10.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6c4d4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640cbb50 con 0x7f4d64159150 2023-11-28T12:02:10.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6a270700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d50003a30 0x7f4d6414d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:10.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6a270700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d50003a30 0x7f4d6414d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45664/0 (socket says 172.21.15.57:45664) 2023-11-28T12:02:10.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6a270700 1 -- 172.21.15.57:0/3057380789 learned_addr learned my addr 172.21.15.57:0/3057380789 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:10.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d69a6f700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d64159150 0x7f4d640cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:10.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6aa71700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6415aed0 0x7f4d640b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:10.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6a270700 1 -- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d64159150 msgr2=0x7f4d640cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6a270700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d64159150 0x7f4d640cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6a270700 1 -- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6415aed0 msgr2=0x7f4d640b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6a270700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6415aed0 0x7f4d640b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6a270700 1 -- 172.21.15.57:0/3057380789 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d64161120 con 0x7f4d50003a30 2023-11-28T12:02:10.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6aa71700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6415aed0 0x7f4d640b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:10.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.664+0000 7f4d6a270700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d50003a30 0x7f4d6414d5c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000c0e0 tx=0x7f4d60001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:10.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.665+0000 7f4d56ffd700 1 -- 172.21.15.57:0/3057380789 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000cb80 con 0x7f4d50003a30 2023-11-28T12:02:10.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.665+0000 7f4d56ffd700 1 -- 172.21.15.57:0/3057380789 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d60021070 con 0x7f4d50003a30 2023-11-28T12:02:10.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.665+0000 7f4d56ffd700 1 -- 172.21.15.57:0/3057380789 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000cee0 con 0x7f4d50003a30 2023-11-28T12:02:10.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.665+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/3057380789 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d640cda40 con 0x7f4d50003a30 2023-11-28T12:02:10.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.665+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/3057380789 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d640d2700 con 0x7f4d50003a30 2023-11-28T12:02:10.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.666+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/3057380789 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d64067d70 con 0x7f4d50003a30 2023-11-28T12:02:10.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.666+0000 7f4d56ffd700 1 -- 172.21.15.57:0/3057380789 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d6002c020 con 0x7f4d50003a30 2023-11-28T12:02:10.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.666+0000 7f4d56ffd700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4d480682f0 0x7f4d4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:10.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.667+0000 7f4d56ffd700 1 -- 172.21.15.57:0/3057380789 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d600870b0 con 0x7f4d50003a30 2023-11-28T12:02:10.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.667+0000 7f4d69a6f700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4d480682f0 0x7f4d4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:10.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.667+0000 7f4d69a6f700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4d480682f0 0x7f4d4806a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d640cf3f0 tx=0x7f4d58000fe0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:10.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.671+0000 7f4d56ffd700 1 -- 172.21.15.57:0/3057380789 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d60053890 con 0x7f4d50003a30 2023-11-28T12:02:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.869+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/3057380789 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4d640743f0 con 0x7f4d50003a30 2023-11-28T12:02:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.869+0000 7f4d56ffd700 1 -- 172.21.15.57:0/3057380789 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4d60056eb0 con 0x7f4d50003a30 2023-11-28T12:02:10.870 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:10.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.871+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4d480682f0 msgr2=0x7f4d4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.871+0000 7f4d6c4d4700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4d480682f0 0x7f4d4806a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d640cf3f0 tx=0x7f4d58000fe0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.871+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/3057380789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d50003a30 msgr2=0x7f4d6414d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.871+0000 7f4d6c4d4700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d50003a30 0x7f4d6414d5c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000c0e0 tx=0x7f4d60001230 comp rx=0 tx=0).stop 2023-11-28T12:02:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.871+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/3057380789 shutdown_connections 2023-11-28T12:02:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.871+0000 7f4d6c4d4700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4d480682f0 0x7f4d4806a7a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.871+0000 7f4d6c4d4700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6415aed0 0x7f4d640b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.871+0000 7f4d6c4d4700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d50003a30 0x7f4d6414d5c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.871+0000 7f4d6c4d4700 1 --2- 172.21.15.57:0/3057380789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d64159150 0x7f4d640cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.871+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/3057380789 >> 172.21.15.57:0/3057380789 conn(0x7f4d6406c520 msgr2=0x7f4d640d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.871+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/3057380789 shutdown_connections 2023-11-28T12:02:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:10.871+0000 7f4d6c4d4700 1 -- 172.21.15.57:0/3057380789 wait complete. 2023-11-28T12:02:10.888 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:11.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.025+0000 7f78e5641700 1 Processor -- start 2023-11-28T12:02:11.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.025+0000 7f78e5641700 1 -- start start 2023-11-28T12:02:11.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.026+0000 7f78e5641700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f78e0160cb0 0x7f78e006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.026+0000 7f78e5641700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e00d2d30 con 0x7f78e015c8c0 2023-11-28T12:02:11.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.026+0000 7f78e5641700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e00d0b40 con 0x7f78e006b050 2023-11-28T12:02:11.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.026+0000 7f78e5641700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e006b490 con 0x7f78e0160cb0 2023-11-28T12:02:11.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.026+0000 7f78de7fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f78e0160cb0 0x7f78e006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.026+0000 7f78deffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f78e015c8c0 0x7f78e00b5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37636/0 (socket says 172.21.15.57:37636) 2023-11-28T12:02:11.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.026+0000 7f78de7fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f78e0160cb0 0x7f78e006ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45670/0 (socket says 172.21.15.57:45670) 2023-11-28T12:02:11.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.026+0000 7f78deffd700 1 -- 172.21.15.57:0/2973101784 learned_addr learned my addr 172.21.15.57:0/2973101784 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:11.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.026+0000 7f78de7fc700 1 -- 172.21.15.57:0/2973101784 >> v1:172.21.15.181:6789/0 conn(0x7f78e006b050 legacy=0x7f78e00d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:11.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.026+0000 7f78de7fc700 1 -- 172.21.15.57:0/2973101784 >> v1:172.21.15.57:6789/0 conn(0x7f78e015c8c0 legacy=0x7f78e00b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.027+0000 7f78de7fc700 1 -- 172.21.15.57:0/2973101784 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e00d2a50 con 0x7f78e0160cb0 2023-11-28T12:02:11.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.027+0000 7f78ddffb700 1 -- 172.21.15.57:0/2973101784 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739884305 0 0) 0x7f78d0001110 con 0x7f78e015c8c0 2023-11-28T12:02:11.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.027+0000 7f78de7fc700 1 --2- 172.21.15.57:0/2973101784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e0160cb0 0x7f78e006ab10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009840 tx=0x7f78c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3014985cefec5680 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:11.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.027+0000 7f78ddffb700 1 -- 172.21.15.57:0/2973101784 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8004230 con 0x7f78e0160cb0 2023-11-28T12:02:11.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.027+0000 7f78ddffb700 1 -- 172.21.15.57:0/2973101784 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c8004390 con 0x7f78e0160cb0 2023-11-28T12:02:11.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.027+0000 7f78ddffb700 1 -- 172.21.15.57:0/2973101784 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c80036a0 con 0x7f78e0160cb0 2023-11-28T12:02:11.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.027+0000 7f78e5641700 1 -- 172.21.15.57:0/2973101784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e0160cb0 msgr2=0x7f78e006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.028+0000 7f78e5641700 1 --2- 172.21.15.57:0/2973101784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e0160cb0 0x7f78e006ab10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009840 tx=0x7f78c8009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:11.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.028+0000 7f78e5641700 1 -- 172.21.15.57:0/2973101784 shutdown_connections 2023-11-28T12:02:11.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.028+0000 7f78e5641700 1 --2- 172.21.15.57:0/2973101784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e0160cb0 0x7f78e006ab10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.028+0000 7f78e5641700 1 -- 172.21.15.57:0/2973101784 >> 172.21.15.57:0/2973101784 conn(0x7f78e006c520 msgr2=0x7f78e006c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:11.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.028+0000 7f78e5641700 1 -- 172.21.15.57:0/2973101784 shutdown_connections 2023-11-28T12:02:11.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.028+0000 7f78e5641700 1 -- 172.21.15.57:0/2973101784 wait complete. 2023-11-28T12:02:11.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.029+0000 7f78e5641700 1 Processor -- start 2023-11-28T12:02:11.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.029+0000 7f78e5641700 1 -- start start 2023-11-28T12:02:11.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78e5641700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e006b050 0x7f78e00c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78e5641700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78e015c8c0 0x7f78e00c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78e5641700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e0160cb0 0x7f78e014d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78e5641700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e014e240 con 0x7f78e0160cb0 2023-11-28T12:02:11.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78e5641700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e00c7a60 con 0x7f78e015c8c0 2023-11-28T12:02:11.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78e5641700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e00c8ea0 con 0x7f78e006b050 2023-11-28T12:02:11.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78df7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e0160cb0 0x7f78e014d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78df7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e0160cb0 0x7f78e014d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45686/0 (socket says 172.21.15.57:45686) 2023-11-28T12:02:11.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78df7fe700 1 -- 172.21.15.57:0/3465319210 learned_addr learned my addr 172.21.15.57:0/3465319210 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:11.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78deffd700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e006b050 0x7f78e00c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78de7fc700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78e015c8c0 0x7f78e00c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78df7fe700 1 -- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e006b050 msgr2=0x7f78e00c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78df7fe700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e006b050 0x7f78e00c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78df7fe700 1 -- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78e015c8c0 msgr2=0x7f78e00c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78df7fe700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78e015c8c0 0x7f78e00c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78df7fe700 1 -- 172.21.15.57:0/3465319210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c80094a0 con 0x7f78e0160cb0 2023-11-28T12:02:11.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.030+0000 7f78df7fe700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e0160cb0 0x7f78e014d640 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f78d800cc10 tx=0x7f78d800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:11.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.031+0000 7f78d77fe700 1 -- 172.21.15.57:0/3465319210 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d800aa60 con 0x7f78e0160cb0 2023-11-28T12:02:11.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.031+0000 7f78d77fe700 1 -- 172.21.15.57:0/3465319210 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d800abc0 con 0x7f78e0160cb0 2023-11-28T12:02:11.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.031+0000 7f78e5641700 1 -- 172.21.15.57:0/3465319210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e00cda40 con 0x7f78e0160cb0 2023-11-28T12:02:11.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.031+0000 7f78d77fe700 1 -- 172.21.15.57:0/3465319210 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8012420 con 0x7f78e0160cb0 2023-11-28T12:02:11.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.031+0000 7f78e5641700 1 -- 172.21.15.57:0/3465319210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e00cd760 con 0x7f78e0160cb0 2023-11-28T12:02:11.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.032+0000 7f78e5641700 1 -- 172.21.15.57:0/3465319210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e0161120 con 0x7f78e0160cb0 2023-11-28T12:02:11.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.032+0000 7f78d77fe700 1 -- 172.21.15.57:0/3465319210 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f78d8010050 con 0x7f78e0160cb0 2023-11-28T12:02:11.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.033+0000 7f78d77fe700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f78c00682f0 0x7f78c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.033+0000 7f78d77fe700 1 -- 172.21.15.57:0/3465319210 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f78d8085e20 con 0x7f78e0160cb0 2023-11-28T12:02:11.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.033+0000 7f78deffd700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f78c00682f0 0x7f78c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.033+0000 7f78deffd700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f78c00682f0 0x7f78c006a7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f78d0000f60 tx=0x7f78d0007c20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:11.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.037+0000 7f78d77fe700 1 -- 172.21.15.57:0/3465319210 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f78d80525a0 con 0x7f78e0160cb0 2023-11-28T12:02:11.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.235+0000 7f78e5641700 1 -- 172.21.15.57:0/3465319210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78e0067d70 con 0x7f78e0160cb0 2023-11-28T12:02:11.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.236+0000 7f78d77fe700 1 -- 172.21.15.57:0/3465319210 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f78d8055bc0 con 0x7f78e0160cb0 2023-11-28T12:02:11.237 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:11.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.238+0000 7f78e5641700 1 -- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f78c00682f0 msgr2=0x7f78c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.238+0000 7f78e5641700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f78c00682f0 0x7f78c006a7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f78d0000f60 tx=0x7f78d0007c20 comp rx=0 tx=0).stop 2023-11-28T12:02:11.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.238+0000 7f78e5641700 1 -- 172.21.15.57:0/3465319210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e0160cb0 msgr2=0x7f78e014d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.238+0000 7f78e5641700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e0160cb0 0x7f78e014d640 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f78d800cc10 tx=0x7f78d800cf20 comp rx=0 tx=0).stop 2023-11-28T12:02:11.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.238+0000 7f78e5641700 1 -- 172.21.15.57:0/3465319210 shutdown_connections 2023-11-28T12:02:11.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.238+0000 7f78e5641700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f78c00682f0 0x7f78c006a7a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.238+0000 7f78e5641700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78e0160cb0 0x7f78e014d640 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.238+0000 7f78e5641700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78e006b050 0x7f78e00c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.238+0000 7f78e5641700 1 --2- 172.21.15.57:0/3465319210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78e015c8c0 0x7f78e00c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.238+0000 7f78e5641700 1 -- 172.21.15.57:0/3465319210 >> 172.21.15.57:0/3465319210 conn(0x7f78e006c520 msgr2=0x7f78e00b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:11.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.238+0000 7f78e5641700 1 -- 172.21.15.57:0/3465319210 shutdown_connections 2023-11-28T12:02:11.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.238+0000 7f78e5641700 1 -- 172.21.15.57:0/3465319210 wait complete. 2023-11-28T12:02:11.253 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:02:11.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.390+0000 7fee96146700 1 Processor -- start 2023-11-28T12:02:11.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.390+0000 7fee96146700 1 -- start start 2023-11-28T12:02:11.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.391+0000 7fee96146700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee900af0a0 0x7fee9015a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.391+0000 7fee96146700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee90159ec0 con 0x7fee901599b0 2023-11-28T12:02:11.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.391+0000 7fee96146700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9015d8a0 con 0x7fee900b1650 2023-11-28T12:02:11.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.391+0000 7fee96146700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9015db80 con 0x7fee900af0a0 2023-11-28T12:02:11.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.391+0000 7fee8effd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fee901599b0 0x7fee9015d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37642/0 (socket says 172.21.15.57:37642) 2023-11-28T12:02:11.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.391+0000 7fee8effd700 1 -- 172.21.15.57:0/2465002671 learned_addr learned my addr 172.21.15.57:0/2465002671 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.392+0000 7fee8ffff700 1 --2- 172.21.15.57:0/2465002671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee900af0a0 0x7fee9015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.392+0000 7fee8e7fc700 1 -- 172.21.15.57:0/2465002671 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812051050 0 0) 0x7fee90159ec0 con 0x7fee901599b0 2023-11-28T12:02:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.392+0000 7fee8e7fc700 1 -- 172.21.15.57:0/2465002671 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee7c0035b0 con 0x7fee901599b0 2023-11-28T12:02:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.392+0000 7fee8e7fc700 1 -- 172.21.15.57:0/2465002671 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82753773 0 0) 0x7fee9015d8a0 con 0x7fee900b1650 2023-11-28T12:02:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.392+0000 7fee8e7fc700 1 -- 172.21.15.57:0/2465002671 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee90159ec0 con 0x7fee900b1650 2023-11-28T12:02:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.392+0000 7fee8ffff700 1 -- 172.21.15.57:0/2465002671 >> v1:172.21.15.181:6789/0 conn(0x7fee900b1650 legacy=0x7fee900b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.392+0000 7fee8ffff700 1 -- 172.21.15.57:0/2465002671 >> v1:172.21.15.57:6789/0 conn(0x7fee901599b0 legacy=0x7fee9015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.392+0000 7fee8ffff700 1 -- 172.21.15.57:0/2465002671 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9015de60 con 0x7fee900af0a0 2023-11-28T12:02:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.392+0000 7fee8ffff700 1 --2- 172.21.15.57:0/2465002671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee900af0a0 0x7fee9015a350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fee88009dc0 tx=0x7fee8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=886f9d150da77e90 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.392+0000 7fee8e7fc700 1 -- 172.21.15.57:0/2465002671 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8800e070 con 0x7fee900af0a0 2023-11-28T12:02:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.392+0000 7fee8e7fc700 1 -- 172.21.15.57:0/2465002671 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8800bb20 con 0x7fee900af0a0 2023-11-28T12:02:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.392+0000 7fee8e7fc700 1 -- 172.21.15.57:0/2465002671 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88012430 con 0x7fee900af0a0 2023-11-28T12:02:11.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.393+0000 7fee96146700 1 -- 172.21.15.57:0/2465002671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee900af0a0 msgr2=0x7fee9015a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.393+0000 7fee96146700 1 --2- 172.21.15.57:0/2465002671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee900af0a0 0x7fee9015a350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fee88009dc0 tx=0x7fee8800b330 comp rx=0 tx=0).stop 2023-11-28T12:02:11.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.393+0000 7fee96146700 1 -- 172.21.15.57:0/2465002671 shutdown_connections 2023-11-28T12:02:11.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.393+0000 7fee96146700 1 --2- 172.21.15.57:0/2465002671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee900af0a0 0x7fee9015a350 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.393+0000 7fee96146700 1 -- 172.21.15.57:0/2465002671 >> 172.21.15.57:0/2465002671 conn(0x7fee9006c010 msgr2=0x7fee900cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:11.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.393+0000 7fee96146700 1 -- 172.21.15.57:0/2465002671 shutdown_connections 2023-11-28T12:02:11.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.393+0000 7fee96146700 1 -- 172.21.15.57:0/2465002671 wait complete. 2023-11-28T12:02:11.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.394+0000 7fee96146700 1 Processor -- start 2023-11-28T12:02:11.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.395+0000 7fee96146700 1 -- start start 2023-11-28T12:02:11.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.395+0000 7fee96146700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee900af0a0 0x7fee9013fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.395+0000 7fee96146700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee900b1650 0x7fee900c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.395+0000 7fee96146700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee90155730 0x7fee90071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.395+0000 7fee96146700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee90142c50 con 0x7fee900b1650 2023-11-28T12:02:11.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.395+0000 7fee96146700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee90157050 con 0x7fee900af0a0 2023-11-28T12:02:11.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.395+0000 7fee96146700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee90155bb0 con 0x7fee90155730 2023-11-28T12:02:11.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.395+0000 7fee8effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee900b1650 0x7fee900c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee8effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee900b1650 0x7fee900c60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45702/0 (socket says 172.21.15.57:45702) 2023-11-28T12:02:11.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee8effd700 1 -- 172.21.15.57:0/4126093151 learned_addr learned my addr 172.21.15.57:0/4126093151 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:11.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee8ffff700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee90155730 0x7fee90071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee8f7fe700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee900af0a0 0x7fee9013fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee8effd700 1 -- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee90155730 msgr2=0x7fee90071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee8effd700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee90155730 0x7fee90071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee8effd700 1 -- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee900af0a0 msgr2=0x7fee9013fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee8effd700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee900af0a0 0x7fee9013fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee8effd700 1 -- 172.21.15.57:0/4126093151 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee88009a70 con 0x7fee900b1650 2023-11-28T12:02:11.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee8effd700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee900b1650 0x7fee900c60c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fee8000c450 tx=0x7fee8000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:11.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee87fff700 1 -- 172.21.15.57:0/4126093151 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80005d70 con 0x7fee900b1650 2023-11-28T12:02:11.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee87fff700 1 -- 172.21.15.57:0/4126093151 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8000ce60 con 0x7fee900b1650 2023-11-28T12:02:11.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee96146700 1 -- 172.21.15.57:0/4126093151 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9015aa20 con 0x7fee900b1650 2023-11-28T12:02:11.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee87fff700 1 -- 172.21.15.57:0/4126093151 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee800072f0 con 0x7fee900b1650 2023-11-28T12:02:11.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.396+0000 7fee96146700 1 -- 172.21.15.57:0/4126093151 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9015a890 con 0x7fee900b1650 2023-11-28T12:02:11.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.397+0000 7fee96146700 1 -- 172.21.15.57:0/4126093151 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee90157210 con 0x7fee900b1650 2023-11-28T12:02:11.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.398+0000 7fee87fff700 1 -- 172.21.15.57:0/4126093151 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fee80019030 con 0x7fee900b1650 2023-11-28T12:02:11.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.398+0000 7fee87fff700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee700682f0 0x7fee7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.398+0000 7fee87fff700 1 -- 172.21.15.57:0/4126093151 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fee80084fc0 con 0x7fee900b1650 2023-11-28T12:02:11.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.398+0000 7fee8f7fe700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee700682f0 0x7fee7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.399+0000 7fee8f7fe700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee700682f0 0x7fee7006a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fee78001220 tx=0x7fee78009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:11.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.403+0000 7fee87fff700 1 -- 172.21.15.57:0/4126093151 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fee800516b0 con 0x7fee900b1650 2023-11-28T12:02:11.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.601+0000 7fee96146700 1 -- 172.21.15.57:0/4126093151 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee900743f0 con 0x7fee900b1650 2023-11-28T12:02:11.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.602+0000 7fee87fff700 1 -- 172.21.15.57:0/4126093151 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee800170a0 con 0x7fee900b1650 2023-11-28T12:02:11.603 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:02:11.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.604+0000 7fee96146700 1 -- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee700682f0 msgr2=0x7fee7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.604+0000 7fee96146700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee700682f0 0x7fee7006a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fee78001220 tx=0x7fee78009040 comp rx=0 tx=0).stop 2023-11-28T12:02:11.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.604+0000 7fee96146700 1 -- 172.21.15.57:0/4126093151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee900b1650 msgr2=0x7fee900c60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.604+0000 7fee96146700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee900b1650 0x7fee900c60c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fee8000c450 tx=0x7fee8000c760 comp rx=0 tx=0).stop 2023-11-28T12:02:11.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.604+0000 7fee96146700 1 -- 172.21.15.57:0/4126093151 shutdown_connections 2023-11-28T12:02:11.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.604+0000 7fee96146700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee700682f0 0x7fee7006a7a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.604+0000 7fee96146700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee90155730 0x7fee90071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.604+0000 7fee96146700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee900af0a0 0x7fee9013fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.604+0000 7fee96146700 1 --2- 172.21.15.57:0/4126093151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee900b1650 0x7fee900c60c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.604+0000 7fee96146700 1 -- 172.21.15.57:0/4126093151 >> 172.21.15.57:0/4126093151 conn(0x7fee9006c010 msgr2=0x7fee9006a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:11.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.604+0000 7fee96146700 1 -- 172.21.15.57:0/4126093151 shutdown_connections 2023-11-28T12:02:11.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.604+0000 7fee96146700 1 -- 172.21.15.57:0/4126093151 wait complete. 2023-11-28T12:02:11.619 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:02:11.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.757+0000 7f868a951700 1 Processor -- start 2023-11-28T12:02:11.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.757+0000 7f868a951700 1 -- start start 2023-11-28T12:02:11.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.758+0000 7f868a951700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f868415fe50 0x7f868406d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.758+0000 7f868a951700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86840b3b80 0x7f8684160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.758+0000 7f868a951700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f86840b27d0 0x7f86840b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.758+0000 7f868a951700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86840b3190 con 0x7f86840b27d0 2023-11-28T12:02:11.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.758+0000 7f868a951700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8684160220 con 0x7f86840b3b80 2023-11-28T12:02:11.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.758+0000 7f868a951700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8684161370 con 0x7f868415fe50 2023-11-28T12:02:11.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.758+0000 7f8688eee700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f86840b27d0 0x7f86840b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.758+0000 7f8688eee700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f86840b27d0 0x7f86840b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45712/0 (socket says 172.21.15.57:45712) 2023-11-28T12:02:11.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.758+0000 7f8688eee700 1 -- 172.21.15.57:0/3053444351 learned_addr learned my addr 172.21.15.57:0/3053444351 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:11.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.758+0000 7f8683fff700 1 --2- 172.21.15.57:0/3053444351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f868415fe50 0x7f868406d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.758+0000 7f86837fe700 1 --2- 172.21.15.57:0/3053444351 >> v2:172.21.15.181:3300/0 conn(0x7f86840b3b80 0x7f8684160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.759+0000 7f8688eee700 1 -- 172.21.15.57:0/3053444351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f868415fe50 msgr2=0x7f868406d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:11.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.759+0000 7f8688eee700 1 --2- 172.21.15.57:0/3053444351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f868415fe50 0x7f868406d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.759+0000 7f8688eee700 1 -- 172.21.15.57:0/3053444351 >> v2:172.21.15.181:3300/0 conn(0x7f86840b3b80 msgr2=0x7f8684160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:11.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.759+0000 7f8688eee700 1 --2- 172.21.15.57:0/3053444351 >> v2:172.21.15.181:3300/0 conn(0x7f86840b3b80 0x7f8684160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.759+0000 7f8688eee700 1 -- 172.21.15.57:0/3053444351 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840cf700 con 0x7f86840b27d0 2023-11-28T12:02:11.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.759+0000 7f8688eee700 1 --2- 172.21.15.57:0/3053444351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86840b27d0 0x7f86840b2c50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f86840719d0 tx=0x7f867c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571eef37035baadc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:11.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.759+0000 7f8682ffd700 1 -- 172.21.15.57:0/3053444351 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00e070 con 0x7f86840b27d0 2023-11-28T12:02:11.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.759+0000 7f8682ffd700 1 -- 172.21.15.57:0/3053444351 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c00b980 con 0x7f86840b27d0 2023-11-28T12:02:11.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.759+0000 7f8682ffd700 1 -- 172.21.15.57:0/3053444351 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00bd30 con 0x7f86840b27d0 2023-11-28T12:02:11.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.759+0000 7f868a951700 1 -- 172.21.15.57:0/3053444351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86840b27d0 msgr2=0x7f86840b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.759+0000 7f868a951700 1 --2- 172.21.15.57:0/3053444351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86840b27d0 0x7f86840b2c50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f86840719d0 tx=0x7f867c00b330 comp rx=0 tx=0).stop 2023-11-28T12:02:11.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.760+0000 7f868a951700 1 -- 172.21.15.57:0/3053444351 shutdown_connections 2023-11-28T12:02:11.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.760+0000 7f868a951700 1 --2- 172.21.15.57:0/3053444351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f868415fe50 0x7f868406d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.760+0000 7f868a951700 1 --2- 172.21.15.57:0/3053444351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86840b27d0 0x7f86840b2c50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.760+0000 7f868a951700 1 --2- 172.21.15.57:0/3053444351 >> v2:172.21.15.181:3300/0 conn(0x7f86840b3b80 0x7f8684160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.760+0000 7f868a951700 1 -- 172.21.15.57:0/3053444351 >> 172.21.15.57:0/3053444351 conn(0x7f868406ea80 msgr2=0x7f868406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:11.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.760+0000 7f868a951700 1 -- 172.21.15.57:0/3053444351 shutdown_connections 2023-11-28T12:02:11.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.760+0000 7f868a951700 1 -- 172.21.15.57:0/3053444351 wait complete. 2023-11-28T12:02:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.761+0000 7f868a951700 1 Processor -- start 2023-11-28T12:02:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.761+0000 7f868a951700 1 -- start start 2023-11-28T12:02:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f868a951700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86840b27d0 0x7f86841f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f868a951700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86840b3b80 0x7f86840cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f868a951700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f868415fe50 0x7f86840b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f868a951700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86841f5e10 con 0x7f868415fe50 2023-11-28T12:02:11.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f868a951700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86841f60f0 con 0x7f86840b27d0 2023-11-28T12:02:11.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f868a951700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86841f63d0 con 0x7f86840b3b80 2023-11-28T12:02:11.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f8688eee700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f868415fe50 0x7f86840b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f8688eee700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f868415fe50 0x7f86840b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45724/0 (socket says 172.21.15.57:45724) 2023-11-28T12:02:11.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f8688eee700 1 -- 172.21.15.57:0/2953097467 learned_addr learned my addr 172.21.15.57:0/2953097467 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:11.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f8683fff700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86840b27d0 0x7f86841f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f86837fe700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86840b3b80 0x7f86840cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f8688eee700 1 -- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86840b3b80 msgr2=0x7f86840cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f8688eee700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86840b3b80 0x7f86840cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f8688eee700 1 -- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86840b27d0 msgr2=0x7f86841f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f8688eee700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86840b27d0 0x7f86841f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.762+0000 7f8688eee700 1 -- 172.21.15.57:0/2953097467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c009b30 con 0x7f868415fe50 2023-11-28T12:02:11.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.763+0000 7f8688eee700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f868415fe50 0x7f86840b15d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f868406dbd0 tx=0x7f867c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:11.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.763+0000 7f8680ff9700 1 -- 172.21.15.57:0/2953097467 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c0099b0 con 0x7f868415fe50 2023-11-28T12:02:11.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.763+0000 7f8680ff9700 1 -- 172.21.15.57:0/2953097467 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c004250 con 0x7f868415fe50 2023-11-28T12:02:11.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.763+0000 7f868a951700 1 -- 172.21.15.57:0/2953097467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86840cedd0 con 0x7f868415fe50 2023-11-28T12:02:11.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.763+0000 7f8680ff9700 1 -- 172.21.15.57:0/2953097467 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c0045e0 con 0x7f868415fe50 2023-11-28T12:02:11.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.763+0000 7f868a951700 1 -- 172.21.15.57:0/2953097467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86840b3fd0 con 0x7f868415fe50 2023-11-28T12:02:11.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.764+0000 7f868a951700 1 -- 172.21.15.57:0/2953097467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86840cf0b0 con 0x7f868415fe50 2023-11-28T12:02:11.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.764+0000 7f8680ff9700 1 -- 172.21.15.57:0/2953097467 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f867c02b070 con 0x7f868415fe50 2023-11-28T12:02:11.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.765+0000 7f8680ff9700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f86700682f0 0x7f867006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:11.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.765+0000 7f8680ff9700 1 -- 172.21.15.57:0/2953097467 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f867c086be0 con 0x7f868415fe50 2023-11-28T12:02:11.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.765+0000 7f8683fff700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f86700682f0 0x7f867006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:11.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.766+0000 7f8683fff700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f86700682f0 0x7f867006a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8674006b20 tx=0x7f8674008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:11.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.769+0000 7f8680ff9700 1 -- 172.21.15.57:0/2953097467 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f867c053440 con 0x7f868415fe50 2023-11-28T12:02:11.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.967+0000 7f868a951700 1 -- 172.21.15.57:0/2953097467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86840743f0 con 0x7f868415fe50 2023-11-28T12:02:11.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.967+0000 7f8680ff9700 1 -- 172.21.15.57:0/2953097467 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f867c01f020 con 0x7f868415fe50 2023-11-28T12:02:11.968 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:02:11.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.969+0000 7f868a951700 1 -- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f86700682f0 msgr2=0x7f867006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.969+0000 7f868a951700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f86700682f0 0x7f867006a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8674006b20 tx=0x7f8674008040 comp rx=0 tx=0).stop 2023-11-28T12:02:11.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.969+0000 7f868a951700 1 -- 172.21.15.57:0/2953097467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f868415fe50 msgr2=0x7f86840b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:11.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.969+0000 7f868a951700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f868415fe50 0x7f86840b15d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f868406dbd0 tx=0x7f867c004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.969+0000 7f868a951700 1 -- 172.21.15.57:0/2953097467 shutdown_connections 2023-11-28T12:02:11.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.969+0000 7f868a951700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f86700682f0 0x7f867006a7a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.969+0000 7f868a951700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f868415fe50 0x7f86840b15d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.969+0000 7f868a951700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86840b27d0 0x7f86841f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.969+0000 7f868a951700 1 --2- 172.21.15.57:0/2953097467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86840b3b80 0x7f86840cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:11.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.969+0000 7f868a951700 1 -- 172.21.15.57:0/2953097467 >> 172.21.15.57:0/2953097467 conn(0x7f868406ea80 msgr2=0x7f86840b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:11.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.969+0000 7f868a951700 1 -- 172.21.15.57:0/2953097467 shutdown_connections 2023-11-28T12:02:11.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:11.969+0000 7f868a951700 1 -- 172.21.15.57:0/2953097467 wait complete. 2023-11-28T12:02:11.984 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:12.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.118+0000 7f0fec5d0700 1 Processor -- start 2023-11-28T12:02:12.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.118+0000 7f0fec5d0700 1 -- start start 2023-11-28T12:02:12.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.119+0000 7f0fec5d0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0fe406a270 0x7f0fe406a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.119+0000 7f0fec5d0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe415cc80 0x7f0fe406de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.119+0000 7f0fec5d0700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe406ab80 con 0x7f0fe406e3a0 2023-11-28T12:02:12.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.119+0000 7f0fec5d0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe40b2490 con 0x7f0fe406a270 2023-11-28T12:02:12.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.119+0000 7f0fec5d0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe40b1810 con 0x7f0fe415cc80 2023-11-28T12:02:12.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.119+0000 7f0feab6d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f0fe406e3a0 0x7f0fe406ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37648/0 (socket says 172.21.15.57:37648) 2023-11-28T12:02:12.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.119+0000 7f0feab6d700 1 -- 172.21.15.57:0/3722076699 learned_addr learned my addr 172.21.15.57:0/3722076699 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:12.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.120+0000 7f0fe9b6b700 1 --2- 172.21.15.57:0/3722076699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe415cc80 0x7f0fe406de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.120+0000 7f0fea36c700 1 --2- 172.21.15.57:0/3722076699 >> v2:172.21.15.181:3300/0 conn(0x7f0fe406a270 0x7f0fe406a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.120+0000 7f0fe936a700 1 -- 172.21.15.57:0/3722076699 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132359984 0 0) 0x7f0fe406ab80 con 0x7f0fe406e3a0 2023-11-28T12:02:12.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.120+0000 7f0fe936a700 1 -- 172.21.15.57:0/3722076699 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd00035b0 con 0x7f0fe406e3a0 2023-11-28T12:02:12.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.120+0000 7f0fe9b6b700 1 -- 172.21.15.57:0/3722076699 >> v2:172.21.15.181:3300/0 conn(0x7f0fe406a270 msgr2=0x7f0fe406a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:12.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.120+0000 7f0fe9b6b700 1 --2- 172.21.15.57:0/3722076699 >> v2:172.21.15.181:3300/0 conn(0x7f0fe406a270 0x7f0fe406a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.120+0000 7f0fe9b6b700 1 -- 172.21.15.57:0/3722076699 >> v1:172.21.15.57:6789/0 conn(0x7f0fe406e3a0 legacy=0x7f0fe406ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.120+0000 7f0fe9b6b700 1 -- 172.21.15.57:0/3722076699 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe40b1af0 con 0x7f0fe415cc80 2023-11-28T12:02:12.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.120+0000 7f0fe9b6b700 1 --2- 172.21.15.57:0/3722076699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe415cc80 0x7f0fe406de60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009840 tx=0x7f0fd8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=168063f6b444a322 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:12.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.120+0000 7f0fe936a700 1 -- 172.21.15.57:0/3722076699 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8011830 con 0x7f0fe415cc80 2023-11-28T12:02:12.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.120+0000 7f0fe936a700 1 -- 172.21.15.57:0/3722076699 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd8011990 con 0x7f0fe415cc80 2023-11-28T12:02:12.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.120+0000 7f0fe936a700 1 -- 172.21.15.57:0/3722076699 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8011d20 con 0x7f0fe415cc80 2023-11-28T12:02:12.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.121+0000 7f0fec5d0700 1 -- 172.21.15.57:0/3722076699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe415cc80 msgr2=0x7f0fe406de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.121+0000 7f0fec5d0700 1 --2- 172.21.15.57:0/3722076699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe415cc80 0x7f0fe406de60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009840 tx=0x7f0fd8009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:12.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.121+0000 7f0fec5d0700 1 -- 172.21.15.57:0/3722076699 shutdown_connections 2023-11-28T12:02:12.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.121+0000 7f0fec5d0700 1 --2- 172.21.15.57:0/3722076699 >> v2:172.21.15.181:3300/0 conn(0x7f0fe406a270 0x7f0fe406a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.121+0000 7f0fec5d0700 1 --2- 172.21.15.57:0/3722076699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe415cc80 0x7f0fe406de60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.121+0000 7f0fec5d0700 1 -- 172.21.15.57:0/3722076699 >> 172.21.15.57:0/3722076699 conn(0x7f0fe406c010 msgr2=0x7f0fe40cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:12.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.121+0000 7f0fec5d0700 1 -- 172.21.15.57:0/3722076699 shutdown_connections 2023-11-28T12:02:12.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.121+0000 7f0fec5d0700 1 -- 172.21.15.57:0/3722076699 wait complete. 2023-11-28T12:02:12.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.122+0000 7f0fec5d0700 1 Processor -- start 2023-11-28T12:02:12.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.122+0000 7f0fec5d0700 1 -- start start 2023-11-28T12:02:12.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.123+0000 7f0fec5d0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe406a270 0x7f0fe40b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.123+0000 7f0fec5d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe406e3a0 0x7f0fe40b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.123+0000 7f0fec5d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe415cc80 0x7f0fe40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.123+0000 7f0fec5d0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe40b4090 con 0x7f0fe415cc80 2023-11-28T12:02:12.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.123+0000 7f0fec5d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415f3f0 con 0x7f0fe406e3a0 2023-11-28T12:02:12.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.123+0000 7f0fec5d0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415f6d0 con 0x7f0fe406a270 2023-11-28T12:02:12.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0feab6d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe415cc80 0x7f0fe40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0feab6d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe415cc80 0x7f0fe40b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45740/0 (socket says 172.21.15.57:45740) 2023-11-28T12:02:12.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0feab6d700 1 -- 172.21.15.57:0/1640966140 learned_addr learned my addr 172.21.15.57:0/1640966140 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:12.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0fea36c700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe406a270 0x7f0fe40b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0fe9b6b700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe406e3a0 0x7f0fe40b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0feab6d700 1 -- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe406a270 msgr2=0x7f0fe40b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0feab6d700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe406a270 0x7f0fe40b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0feab6d700 1 -- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe406e3a0 msgr2=0x7f0fe40b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0feab6d700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe406e3a0 0x7f0fe40b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0feab6d700 1 -- 172.21.15.57:0/1640966140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd80094a0 con 0x7f0fe415cc80 2023-11-28T12:02:12.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0fe9b6b700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe406e3a0 0x7f0fe40b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:12.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0feab6d700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe415cc80 0x7f0fe40b4f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00caa0 tx=0x7f0fdc00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:12.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0fd6ffd700 1 -- 172.21.15.57:0/1640966140 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00a9e0 con 0x7f0fe415cc80 2023-11-28T12:02:12.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0fd6ffd700 1 -- 172.21.15.57:0/1640966140 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc00ab40 con 0x7f0fe415cc80 2023-11-28T12:02:12.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0fd6ffd700 1 -- 172.21.15.57:0/1640966140 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc012400 con 0x7f0fe415cc80 2023-11-28T12:02:12.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0fec5d0700 1 -- 172.21.15.57:0/1640966140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe40b20b0 con 0x7f0fe415cc80 2023-11-28T12:02:12.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.124+0000 7f0fec5d0700 1 -- 172.21.15.57:0/1640966140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe40b1dd0 con 0x7f0fe415cc80 2023-11-28T12:02:12.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.125+0000 7f0fec5d0700 1 -- 172.21.15.57:0/1640966140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe406e7e0 con 0x7f0fe415cc80 2023-11-28T12:02:12.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.126+0000 7f0fd6ffd700 1 -- 172.21.15.57:0/1640966140 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0fdc019030 con 0x7f0fe415cc80 2023-11-28T12:02:12.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.126+0000 7f0fd6ffd700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0fc80682f0 0x7f0fc806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.126+0000 7f0fd6ffd700 1 -- 172.21.15.57:0/1640966140 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0fdc085910 con 0x7f0fe415cc80 2023-11-28T12:02:12.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.127+0000 7f0fea36c700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0fc80682f0 0x7f0fc806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.127+0000 7f0fea36c700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0fc80682f0 0x7f0fc806a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40b54c0 tx=0x7f0fe0009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:12.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.131+0000 7f0fd6ffd700 1 -- 172.21.15.57:0/1640966140 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0fdc052170 con 0x7f0fe415cc80 2023-11-28T12:02:12.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.331+0000 7f0fec5d0700 1 -- 172.21.15.57:0/1640966140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fe40743f0 con 0x7f0fe415cc80 2023-11-28T12:02:12.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.333+0000 7f0fd6ffd700 1 -- 172.21.15.57:0/1640966140 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7f0fdc055790 con 0x7f0fe415cc80 2023-11-28T12:02:12.335 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:12.335 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:12.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.335+0000 7f0fec5d0700 1 -- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0fc80682f0 msgr2=0x7f0fc806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.335+0000 7f0fec5d0700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0fc80682f0 0x7f0fc806a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40b54c0 tx=0x7f0fe0009210 comp rx=0 tx=0).stop 2023-11-28T12:02:12.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.335+0000 7f0fec5d0700 1 -- 172.21.15.57:0/1640966140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe415cc80 msgr2=0x7f0fe40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.335+0000 7f0fec5d0700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe415cc80 0x7f0fe40b4f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00caa0 tx=0x7f0fdc00cdb0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.335+0000 7f0fec5d0700 1 -- 172.21.15.57:0/1640966140 shutdown_connections 2023-11-28T12:02:12.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.335+0000 7f0fec5d0700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0fc80682f0 0x7f0fc806a7a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.335+0000 7f0fec5d0700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe415cc80 0x7f0fe40b4f30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.335+0000 7f0fec5d0700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe406a270 0x7f0fe40b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.335+0000 7f0fec5d0700 1 --2- 172.21.15.57:0/1640966140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe406e3a0 0x7f0fe40b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.335+0000 7f0fec5d0700 1 -- 172.21.15.57:0/1640966140 >> 172.21.15.57:0/1640966140 conn(0x7f0fe406c010 msgr2=0x7f0fe40acfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:12.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.335+0000 7f0fec5d0700 1 -- 172.21.15.57:0/1640966140 shutdown_connections 2023-11-28T12:02:12.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.335+0000 7f0fec5d0700 1 -- 172.21.15.57:0/1640966140 wait complete. 2023-11-28T12:02:12.350 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:12.351 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:12.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.488+0000 7f347d10b700 1 Processor -- start 2023-11-28T12:02:12.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.489+0000 7f347d10b700 1 -- start start 2023-11-28T12:02:12.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f347d10b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f347815a5f0 0x7f34780d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f347d10b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34780cc200 0x7f34780b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f347d10b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347815a9c0 con 0x7f34780b2d10 2023-11-28T12:02:12.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f347d10b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34780b3190 con 0x7f347815a5f0 2023-11-28T12:02:12.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f347d10b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34780c7b80 con 0x7f34780cc200 2023-11-28T12:02:12.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f3476d9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f347815a5f0 0x7f34780d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f3476d9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f347815a5f0 0x7f34780d0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45756/0 (socket says 172.21.15.57:45756) 2023-11-28T12:02:12.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f3476d9d700 1 -- 172.21.15.57:0/1090413839 learned_addr learned my addr 172.21.15.57:0/1090413839 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:12.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f347659c700 1 --2- 172.21.15.57:0/1090413839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34780cc200 0x7f34780b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f3476d9d700 1 -- 172.21.15.57:0/1090413839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34780cc200 msgr2=0x7f34780b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:12.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f3476d9d700 1 --2- 172.21.15.57:0/1090413839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34780cc200 0x7f34780b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f3476d9d700 1 -- 172.21.15.57:0/1090413839 >> v1:172.21.15.181:6789/0 conn(0x7f34780b2d10 legacy=0x7f347815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:12.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f3476d9d700 1 -- 172.21.15.57:0/1090413839 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780cc990 con 0x7f347815a5f0 2023-11-28T12:02:12.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.490+0000 7f3476d9d700 1 --2- 172.21.15.57:0/1090413839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f347815a5f0 0x7f34780d0ad0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f34680098f0 tx=0x7f3468009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaec5ce9dbe25cc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:12.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.491+0000 7f3475d9b700 1 -- 172.21.15.57:0/1090413839 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468004230 con 0x7f347815a5f0 2023-11-28T12:02:12.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.491+0000 7f3475d9b700 1 -- 172.21.15.57:0/1090413839 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3468004390 con 0x7f347815a5f0 2023-11-28T12:02:12.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.491+0000 7f3475d9b700 1 -- 172.21.15.57:0/1090413839 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34680036a0 con 0x7f347815a5f0 2023-11-28T12:02:12.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.491+0000 7f347d10b700 1 -- 172.21.15.57:0/1090413839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f347815a5f0 msgr2=0x7f34780d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.491+0000 7f347d10b700 1 --2- 172.21.15.57:0/1090413839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f347815a5f0 0x7f34780d0ad0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f34680098f0 tx=0x7f3468009c00 comp rx=0 tx=0).stop 2023-11-28T12:02:12.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.491+0000 7f347d10b700 1 -- 172.21.15.57:0/1090413839 shutdown_connections 2023-11-28T12:02:12.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.491+0000 7f347d10b700 1 --2- 172.21.15.57:0/1090413839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f347815a5f0 0x7f34780d0ad0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.491+0000 7f347d10b700 1 --2- 172.21.15.57:0/1090413839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34780cc200 0x7f34780b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.491+0000 7f347d10b700 1 -- 172.21.15.57:0/1090413839 >> 172.21.15.57:0/1090413839 conn(0x7f347806ea80 msgr2=0x7f347806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:12.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.491+0000 7f347d10b700 1 -- 172.21.15.57:0/1090413839 shutdown_connections 2023-11-28T12:02:12.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.491+0000 7f347d10b700 1 -- 172.21.15.57:0/1090413839 wait complete. 2023-11-28T12:02:12.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.493+0000 7f347d10b700 1 Processor -- start 2023-11-28T12:02:12.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.493+0000 7f347d10b700 1 -- start start 2023-11-28T12:02:12.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.494+0000 7f347d10b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34780b2d10 0x7f3478156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.494+0000 7f347d10b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34780cc200 0x7f34780c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.494+0000 7f347d10b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347815a5f0 0x7f34780c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.494+0000 7f347d10b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34780c7580 con 0x7f34780b2d10 2023-11-28T12:02:12.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.494+0000 7f347d10b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3478159970 con 0x7f347815a5f0 2023-11-28T12:02:12.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.494+0000 7f347d10b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3478159690 con 0x7f34780cc200 2023-11-28T12:02:12.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.494+0000 7f3476d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34780b2d10 0x7f3478156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f3476d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34780b2d10 0x7f3478156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45760/0 (socket says 172.21.15.57:45760) 2023-11-28T12:02:12.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f3476d9d700 1 -- 172.21.15.57:0/3505353552 learned_addr learned my addr 172.21.15.57:0/3505353552 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:12.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f347659c700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34780cc200 0x7f34780c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f347759e700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347815a5f0 0x7f34780c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f3476d9d700 1 -- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34780cc200 msgr2=0x7f34780c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f3476d9d700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34780cc200 0x7f34780c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f3476d9d700 1 -- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347815a5f0 msgr2=0x7f34780c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f3476d9d700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347815a5f0 0x7f34780c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f3476d9d700 1 -- 172.21.15.57:0/3505353552 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3468009550 con 0x7f34780b2d10 2023-11-28T12:02:12.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f347759e700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347815a5f0 0x7f34780c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:12.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f3476d9d700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34780b2d10 0x7f3478156210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3468009130 tx=0x7f3468004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:12.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f345f7fe700 1 -- 172.21.15.57:0/3505353552 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468003c00 con 0x7f34780b2d10 2023-11-28T12:02:12.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f345f7fe700 1 -- 172.21.15.57:0/3505353552 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346801f070 con 0x7f34780b2d10 2023-11-28T12:02:12.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f347d10b700 1 -- 172.21.15.57:0/3505353552 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34780ccf50 con 0x7f34780b2d10 2023-11-28T12:02:12.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f345f7fe700 1 -- 172.21.15.57:0/3505353552 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346800b660 con 0x7f34780b2d10 2023-11-28T12:02:12.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.495+0000 7f347d10b700 1 -- 172.21.15.57:0/3505353552 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34780ccc70 con 0x7f34780b2d10 2023-11-28T12:02:12.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.496+0000 7f347d10b700 1 -- 172.21.15.57:0/3505353552 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34780cd230 con 0x7f34780b2d10 2023-11-28T12:02:12.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.497+0000 7f345f7fe700 1 -- 172.21.15.57:0/3505353552 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f346802a020 con 0x7f34780b2d10 2023-11-28T12:02:12.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.497+0000 7f345f7fe700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3464068320 0x7f346406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.497+0000 7f345f7fe700 1 -- 172.21.15.57:0/3505353552 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3468087430 con 0x7f34780b2d10 2023-11-28T12:02:12.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.498+0000 7f347659c700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3464068320 0x7f346406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.498+0000 7f347659c700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3464068320 0x7f346406a7d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f347814fe40 tx=0x7f346c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:12.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.502+0000 7f345f7fe700 1 -- 172.21.15.57:0/3505353552 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3468053c90 con 0x7f34780b2d10 2023-11-28T12:02:12.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.698+0000 7f347d10b700 1 -- 172.21.15.57:0/3505353552 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f34780cd230 con 0x7f34780b2d10 2023-11-28T12:02:12.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.698+0000 7f345f7fe700 1 -- 172.21.15.57:0/3505353552 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3468010380 con 0x7f34780b2d10 2023-11-28T12:02:12.700 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:12.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.700+0000 7f347d10b700 1 -- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3464068320 msgr2=0x7f346406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.700+0000 7f347d10b700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3464068320 0x7f346406a7d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f347814fe40 tx=0x7f346c008040 comp rx=0 tx=0).stop 2023-11-28T12:02:12.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.700+0000 7f347d10b700 1 -- 172.21.15.57:0/3505353552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34780b2d10 msgr2=0x7f3478156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.700+0000 7f347d10b700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34780b2d10 0x7f3478156210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3468009130 tx=0x7f3468004750 comp rx=0 tx=0).stop 2023-11-28T12:02:12.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.700+0000 7f347d10b700 1 -- 172.21.15.57:0/3505353552 shutdown_connections 2023-11-28T12:02:12.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.700+0000 7f347d10b700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3464068320 0x7f346406a7d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.700+0000 7f347d10b700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347815a5f0 0x7f34780c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.700+0000 7f347d10b700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34780b2d10 0x7f3478156210 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.700+0000 7f347d10b700 1 --2- 172.21.15.57:0/3505353552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34780cc200 0x7f34780c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.700+0000 7f347d10b700 1 -- 172.21.15.57:0/3505353552 >> 172.21.15.57:0/3505353552 conn(0x7f347806ea80 msgr2=0x7f34780cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:12.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.700+0000 7f347d10b700 1 -- 172.21.15.57:0/3505353552 shutdown_connections 2023-11-28T12:02:12.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.700+0000 7f347d10b700 1 -- 172.21.15.57:0/3505353552 wait complete. 2023-11-28T12:02:12.715 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:12.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.853+0000 7f2754bcc700 1 Processor -- start 2023-11-28T12:02:12.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.853+0000 7f2754bcc700 1 -- start start 2023-11-28T12:02:12.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.854+0000 7f2754bcc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f27500cf710 0x7f27500ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.855+0000 7f2754bcc700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275015cc10 con 0x7f2750071890 2023-11-28T12:02:12.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.855+0000 7f2754bcc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275006c9b0 con 0x7f275006a980 2023-11-28T12:02:12.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.855+0000 7f2754bcc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275006cc90 con 0x7f27500cf710 2023-11-28T12:02:12.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.855+0000 7f274dd9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2750071890 0x7f275006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37652/0 (socket says 172.21.15.57:37652) 2023-11-28T12:02:12.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.855+0000 7f274dd9b700 1 -- 172.21.15.57:0/2691961101 learned_addr learned my addr 172.21.15.57:0/2691961101 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:12.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.855+0000 7f274e59c700 1 --2- 172.21.15.57:0/2691961101 >> v2:172.21.15.181:3300/0 conn(0x7f27500cf710 0x7f27500ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.855+0000 7f274d59a700 1 -- 172.21.15.57:0/2691961101 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358879199 0 0) 0x7f275015cc10 con 0x7f2750071890 2023-11-28T12:02:12.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.855+0000 7f274d59a700 1 -- 172.21.15.57:0/2691961101 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27380035b0 con 0x7f2750071890 2023-11-28T12:02:12.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.855+0000 7f274d59a700 1 -- 172.21.15.57:0/2691961101 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713121338 0 0) 0x7f275006c9b0 con 0x7f275006a980 2023-11-28T12:02:12.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.855+0000 7f274d59a700 1 -- 172.21.15.57:0/2691961101 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f275015cc10 con 0x7f275006a980 2023-11-28T12:02:12.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.855+0000 7f274e59c700 1 -- 172.21.15.57:0/2691961101 >> v1:172.21.15.181:6789/0 conn(0x7f275006a980 legacy=0x7f2750160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.855+0000 7f274e59c700 1 -- 172.21.15.57:0/2691961101 >> v1:172.21.15.57:6789/0 conn(0x7f2750071890 legacy=0x7f275006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.855+0000 7f274e59c700 1 -- 172.21.15.57:0/2691961101 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275006cf70 con 0x7f27500cf710 2023-11-28T12:02:12.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.856+0000 7f274e59c700 1 --2- 172.21.15.57:0/2691961101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27500cf710 0x7f27500ce740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2740009870 tx=0x7f2740009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca8ab4dc06a83bd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:12.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.856+0000 7f274d59a700 1 -- 172.21.15.57:0/2691961101 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740011830 con 0x7f27500cf710 2023-11-28T12:02:12.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.856+0000 7f274d59a700 1 -- 172.21.15.57:0/2691961101 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2740011990 con 0x7f27500cf710 2023-11-28T12:02:12.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.856+0000 7f274d59a700 1 -- 172.21.15.57:0/2691961101 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740011d20 con 0x7f27500cf710 2023-11-28T12:02:12.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.856+0000 7f2754bcc700 1 -- 172.21.15.57:0/2691961101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27500cf710 msgr2=0x7f27500ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.856+0000 7f2754bcc700 1 --2- 172.21.15.57:0/2691961101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27500cf710 0x7f27500ce740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2740009870 tx=0x7f2740009b80 comp rx=0 tx=0).stop 2023-11-28T12:02:12.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.856+0000 7f2754bcc700 1 -- 172.21.15.57:0/2691961101 shutdown_connections 2023-11-28T12:02:12.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.856+0000 7f2754bcc700 1 --2- 172.21.15.57:0/2691961101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27500cf710 0x7f27500ce740 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.856+0000 7f2754bcc700 1 -- 172.21.15.57:0/2691961101 >> 172.21.15.57:0/2691961101 conn(0x7f275006c010 msgr2=0x7f27500cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:12.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.856+0000 7f2754bcc700 1 -- 172.21.15.57:0/2691961101 shutdown_connections 2023-11-28T12:02:12.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.856+0000 7f2754bcc700 1 -- 172.21.15.57:0/2691961101 wait complete. 2023-11-28T12:02:12.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.858+0000 7f2754bcc700 1 Processor -- start 2023-11-28T12:02:12.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.858+0000 7f2754bcc700 1 -- start start 2023-11-28T12:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.859+0000 7f2754bcc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f275006a980 0x7f2750151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.859+0000 7f2754bcc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2750071890 0x7f275015d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.859+0000 7f2754bcc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27500cf710 0x7f27501f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.859+0000 7f2754bcc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275015ee60 con 0x7f2750071890 2023-11-28T12:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.859+0000 7f2754bcc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2750152dd0 con 0x7f27500cf710 2023-11-28T12:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.859+0000 7f2754bcc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27501530b0 con 0x7f275006a980 2023-11-28T12:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.859+0000 7f274dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2750071890 0x7f275015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f274dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2750071890 0x7f275015d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45764/0 (socket says 172.21.15.57:45764) 2023-11-28T12:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f274dd9b700 1 -- 172.21.15.57:0/3224550135 learned_addr learned my addr 172.21.15.57:0/3224550135 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f274e59c700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f275006a980 0x7f2750151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f274ed9d700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27500cf710 0x7f27501f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f274dd9b700 1 -- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f275006a980 msgr2=0x7f2750151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f274dd9b700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f275006a980 0x7f2750151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f274dd9b700 1 -- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27500cf710 msgr2=0x7f27501f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f274dd9b700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27500cf710 0x7f27501f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f274dd9b700 1 -- 172.21.15.57:0/3224550135 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27400094a0 con 0x7f2750071890 2023-11-28T12:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f274dd9b700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2750071890 0x7f275015d810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f274400c700 tx=0x7f274400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f273effd700 1 -- 172.21.15.57:0/3224550135 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744005d70 con 0x7f2750071890 2023-11-28T12:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f273effd700 1 -- 172.21.15.57:0/3224550135 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27440070d0 con 0x7f2750071890 2023-11-28T12:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f2754bcc700 1 -- 172.21.15.57:0/3224550135 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275006d530 con 0x7f2750071890 2023-11-28T12:02:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f273effd700 1 -- 172.21.15.57:0/3224550135 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744007460 con 0x7f2750071890 2023-11-28T12:02:12.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.860+0000 7f2754bcc700 1 -- 172.21.15.57:0/3224550135 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275006d250 con 0x7f2750071890 2023-11-28T12:02:12.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.861+0000 7f2754bcc700 1 -- 172.21.15.57:0/3224550135 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2750066260 con 0x7f2750071890 2023-11-28T12:02:12.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.862+0000 7f273effd700 1 -- 172.21.15.57:0/3224550135 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2744019030 con 0x7f2750071890 2023-11-28T12:02:12.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.863+0000 7f273effd700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f27300682f0 0x7f273006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:12.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.863+0000 7f273effd700 1 -- 172.21.15.57:0/3224550135 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2744085610 con 0x7f2750071890 2023-11-28T12:02:12.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.863+0000 7f274e59c700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f27300682f0 0x7f273006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:12.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.864+0000 7f274e59c700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f27300682f0 0x7f273006a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2740009bc0 tx=0x7f274001c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:12.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:12.867+0000 7f273effd700 1 -- 172.21.15.57:0/3224550135 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2744051df0 con 0x7f2750071890 2023-11-28T12:02:13.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.065+0000 7f2754bcc700 1 -- 172.21.15.57:0/3224550135 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27500743f0 con 0x7f2750071890 2023-11-28T12:02:13.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.066+0000 7f273effd700 1 -- 172.21.15.57:0/3224550135 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2744055410 con 0x7f2750071890 2023-11-28T12:02:13.067 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:13.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.068+0000 7f2754bcc700 1 -- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f27300682f0 msgr2=0x7f273006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.068+0000 7f2754bcc700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f27300682f0 0x7f273006a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2740009bc0 tx=0x7f274001c040 comp rx=0 tx=0).stop 2023-11-28T12:02:13.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.068+0000 7f2754bcc700 1 -- 172.21.15.57:0/3224550135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2750071890 msgr2=0x7f275015d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.068+0000 7f2754bcc700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2750071890 0x7f275015d810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f274400c700 tx=0x7f274400cac0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.068+0000 7f2754bcc700 1 -- 172.21.15.57:0/3224550135 shutdown_connections 2023-11-28T12:02:13.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.068+0000 7f2754bcc700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f27300682f0 0x7f273006a7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.068+0000 7f2754bcc700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27500cf710 0x7f27501f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.068+0000 7f2754bcc700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f275006a980 0x7f2750151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.068+0000 7f2754bcc700 1 --2- 172.21.15.57:0/3224550135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2750071890 0x7f275015d810 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.068+0000 7f2754bcc700 1 -- 172.21.15.57:0/3224550135 >> 172.21.15.57:0/3224550135 conn(0x7f275006c010 msgr2=0x7f27500aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:13.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.068+0000 7f2754bcc700 1 -- 172.21.15.57:0/3224550135 shutdown_connections 2023-11-28T12:02:13.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.068+0000 7f2754bcc700 1 -- 172.21.15.57:0/3224550135 wait complete. 2023-11-28T12:02:13.083 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:02:13.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.221+0000 7f5b81308700 1 Processor -- start 2023-11-28T12:02:13.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.221+0000 7f5b81308700 1 -- start start 2023-11-28T12:02:13.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.221+0000 7f5b81308700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b7c06a980 0x7f5b7c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.221+0000 7f5b81308700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7c15e0d0 con 0x7f5b7c0cdd50 2023-11-28T12:02:13.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.221+0000 7f5b81308700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7c0ce560 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.221+0000 7f5b81308700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c15e320 con 0x7f5b7c06a980 2023-11-28T12:02:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.222+0000 7f5b7b7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f5b7c0cdd50 0x7f5b7c0cb010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37664/0 (socket says 172.21.15.57:37664) 2023-11-28T12:02:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.222+0000 7f5b7b7fe700 1 -- 172.21.15.57:0/2145082316 learned_addr learned my addr 172.21.15.57:0/2145082316 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.222+0000 7f5b73fff700 1 --2- 172.21.15.57:0/2145082316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b7c06a980 0x7f5b7c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.222+0000 7f5b7a7fc700 1 -- 172.21.15.57:0/2145082316 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564963977 0 0) 0x7f5b7c15e0d0 con 0x7f5b7c0cdd50 2023-11-28T12:02:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.222+0000 7f5b7a7fc700 1 -- 172.21.15.57:0/2145082316 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b680035b0 con 0x7f5b7c0cdd50 2023-11-28T12:02:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.222+0000 7f5b7a7fc700 1 -- 172.21.15.57:0/2145082316 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426191901 0 0) 0x7f5b7c0ce560 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.222+0000 7f5b7a7fc700 1 -- 172.21.15.57:0/2145082316 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b7c15e0d0 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.222+0000 7f5b73fff700 1 -- 172.21.15.57:0/2145082316 >> v1:172.21.15.181:6789/0 conn(0x7f5b7c15a4d0 legacy=0x7f5b7c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.222+0000 7f5b73fff700 1 -- 172.21.15.57:0/2145082316 >> v1:172.21.15.57:6789/0 conn(0x7f5b7c0cdd50 legacy=0x7f5b7c0cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.222+0000 7f5b73fff700 1 -- 172.21.15.57:0/2145082316 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c15e600 con 0x7f5b7c06a980 2023-11-28T12:02:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.222+0000 7f5b73fff700 1 --2- 172.21.15.57:0/2145082316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b7c06a980 0x7f5b7c0cd810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009870 tx=0x7f5b64009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19e8015db5c3db66 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.223+0000 7f5b7a7fc700 1 -- 172.21.15.57:0/2145082316 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64011830 con 0x7f5b7c06a980 2023-11-28T12:02:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.223+0000 7f5b7a7fc700 1 -- 172.21.15.57:0/2145082316 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b64011990 con 0x7f5b7c06a980 2023-11-28T12:02:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.223+0000 7f5b7a7fc700 1 -- 172.21.15.57:0/2145082316 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64011d20 con 0x7f5b7c06a980 2023-11-28T12:02:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.223+0000 7f5b81308700 1 -- 172.21.15.57:0/2145082316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b7c06a980 msgr2=0x7f5b7c0cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.223+0000 7f5b81308700 1 --2- 172.21.15.57:0/2145082316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b7c06a980 0x7f5b7c0cd810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009870 tx=0x7f5b64009b80 comp rx=0 tx=0).stop 2023-11-28T12:02:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.223+0000 7f5b81308700 1 -- 172.21.15.57:0/2145082316 shutdown_connections 2023-11-28T12:02:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.223+0000 7f5b81308700 1 --2- 172.21.15.57:0/2145082316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b7c06a980 0x7f5b7c0cd810 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.223+0000 7f5b81308700 1 -- 172.21.15.57:0/2145082316 >> 172.21.15.57:0/2145082316 conn(0x7f5b7c06c010 msgr2=0x7f5b7c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.223+0000 7f5b81308700 1 -- 172.21.15.57:0/2145082316 shutdown_connections 2023-11-28T12:02:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.223+0000 7f5b81308700 1 -- 172.21.15.57:0/2145082316 wait complete. 2023-11-28T12:02:13.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.225+0000 7f5b81308700 1 Processor -- start 2023-11-28T12:02:13.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.225+0000 7f5b81308700 1 -- start start 2023-11-28T12:02:13.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.225+0000 7f5b81308700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b7c06a980 0x7f5b7c151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.225+0000 7f5b81308700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b7c0cdd50 0x7f5b7c158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.225+0000 7f5b81308700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b7c15a4d0 0x7f5b7c0b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.225+0000 7f5b81308700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c1591e0 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.225+0000 7f5b81308700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c1f0630 con 0x7f5b7c0cdd50 2023-11-28T12:02:13.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.225+0000 7f5b81308700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c158f00 con 0x7f5b7c06a980 2023-11-28T12:02:13.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.225+0000 7f5b7b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b7c15a4d0 0x7f5b7c0b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b7b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b7c15a4d0 0x7f5b7c0b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45778/0 (socket says 172.21.15.57:45778) 2023-11-28T12:02:13.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b7b7fe700 1 -- 172.21.15.57:0/4257206268 learned_addr learned my addr 172.21.15.57:0/4257206268 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:13.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b7affd700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b7c06a980 0x7f5b7c151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b73fff700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b7c0cdd50 0x7f5b7c158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b7b7fe700 1 -- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b7c06a980 msgr2=0x7f5b7c151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b7b7fe700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b7c06a980 0x7f5b7c151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b7b7fe700 1 -- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b7c0cdd50 msgr2=0x7f5b7c158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b7b7fe700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b7c0cdd50 0x7f5b7c158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b7b7fe700 1 -- 172.21.15.57:0/4257206268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b640094a0 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b7b7fe700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b7c15a4d0 0x7f5b7c0b4af0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400cba0 tx=0x7f5b7400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:13.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b737fe700 1 -- 172.21.15.57:0/4257206268 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400ab20 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b737fe700 1 -- 172.21.15.57:0/4257206268 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7400ac80 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b81308700 1 -- 172.21.15.57:0/4257206268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7c15a8a0 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b737fe700 1 -- 172.21.15.57:0/4257206268 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74012400 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.226+0000 7f5b81308700 1 -- 172.21.15.57:0/4257206268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7c15e8e0 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.227+0000 7f5b81308700 1 -- 172.21.15.57:0/4257206268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7c15aa30 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.228+0000 7f5b737fe700 1 -- 172.21.15.57:0/4257206268 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5b74019030 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.228+0000 7f5b737fe700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5b5c0682f0 0x7f5b5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.228+0000 7f5b737fe700 1 -- 172.21.15.57:0/4257206268 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5b740859c0 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.229+0000 7f5b7affd700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5b5c0682f0 0x7f5b5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.229+0000 7f5b7affd700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5b5c0682f0 0x7f5b5c06a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c001220 tx=0x7f5b6c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:13.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.233+0000 7f5b737fe700 1 -- 172.21.15.57:0/4257206268 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5b7c15aa30 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.430+0000 7f5b81308700 1 -- 172.21.15.57:0/4257206268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b7c15aa30 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.431+0000 7f5b737fe700 1 -- 172.21.15.57:0/4257206268 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b7c15aa30 con 0x7f5b7c15a4d0 2023-11-28T12:02:13.432 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:02:13.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.433+0000 7f5b81308700 1 -- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5b5c0682f0 msgr2=0x7f5b5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.433+0000 7f5b81308700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5b5c0682f0 0x7f5b5c06a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c001220 tx=0x7f5b6c009040 comp rx=0 tx=0).stop 2023-11-28T12:02:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.433+0000 7f5b81308700 1 -- 172.21.15.57:0/4257206268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b7c15a4d0 msgr2=0x7f5b7c0b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.433+0000 7f5b81308700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b7c15a4d0 0x7f5b7c0b4af0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400cba0 tx=0x7f5b7400cf60 comp rx=0 tx=0).stop 2023-11-28T12:02:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.433+0000 7f5b81308700 1 -- 172.21.15.57:0/4257206268 shutdown_connections 2023-11-28T12:02:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.433+0000 7f5b81308700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5b5c0682f0 0x7f5b5c06a7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.433+0000 7f5b81308700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b7c15a4d0 0x7f5b7c0b4af0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.433+0000 7f5b81308700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b7c06a980 0x7f5b7c151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.433+0000 7f5b81308700 1 --2- 172.21.15.57:0/4257206268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b7c0cdd50 0x7f5b7c158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.433+0000 7f5b81308700 1 -- 172.21.15.57:0/4257206268 >> 172.21.15.57:0/4257206268 conn(0x7f5b7c06c010 msgr2=0x7f5b7c0cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.433+0000 7f5b81308700 1 -- 172.21.15.57:0/4257206268 shutdown_connections 2023-11-28T12:02:13.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.433+0000 7f5b81308700 1 -- 172.21.15.57:0/4257206268 wait complete. 2023-11-28T12:02:13.448 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:02:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.585+0000 7fb93d37b700 1 Processor -- start 2023-11-28T12:02:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.585+0000 7fb93d37b700 1 -- start start 2023-11-28T12:02:13.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb93d37b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb93815c8c0 0x7fb938160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb93d37b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb93806b220 0x7fb9380d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb93d37b700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9380d2d30 con 0x7fb9380b5dc0 2023-11-28T12:02:13.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb93d37b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9380d0b40 con 0x7fb93815c8c0 2023-11-28T12:02:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb93d37b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9381611f0 con 0x7fb93806b220 2023-11-28T12:02:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb9367fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fb9380b5dc0 0x7fb93806ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37668/0 (socket says 172.21.15.57:37668) 2023-11-28T12:02:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb9367fc700 1 -- 172.21.15.57:0/2173900123 learned_addr learned my addr 172.21.15.57:0/2173900123 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb936ffd700 1 --2- 172.21.15.57:0/2173900123 >> v2:172.21.15.181:3300/0 conn(0x7fb93815c8c0 0x7fb938160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb9377fe700 1 --2- 172.21.15.57:0/2173900123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb93806b220 0x7fb9380d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb935ffb700 1 -- 172.21.15.57:0/2173900123 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310932269 0 0) 0x7fb92c001110 con 0x7fb9380b5dc0 2023-11-28T12:02:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb935ffb700 1 -- 172.21.15.57:0/2173900123 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9200035b0 con 0x7fb9380b5dc0 2023-11-28T12:02:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb936ffd700 1 -- 172.21.15.57:0/2173900123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb93806b220 msgr2=0x7fb9380d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb936ffd700 1 --2- 172.21.15.57:0/2173900123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb93806b220 0x7fb9380d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb936ffd700 1 -- 172.21.15.57:0/2173900123 >> v1:172.21.15.57:6789/0 conn(0x7fb9380b5dc0 legacy=0x7fb93806ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb936ffd700 1 -- 172.21.15.57:0/2173900123 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9380d2a50 con 0x7fb93815c8c0 2023-11-28T12:02:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb9377fe700 1 --2- 172.21.15.57:0/2173900123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb93806b220 0x7fb9380d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.586+0000 7fb936ffd700 1 --2- 172.21.15.57:0/2173900123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93815c8c0 0x7fb938160cb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb928009840 tx=0x7fb928009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72f4e5fa0c7b266d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.587+0000 7fb935ffb700 1 -- 172.21.15.57:0/2173900123 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928011830 con 0x7fb93815c8c0 2023-11-28T12:02:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.587+0000 7fb935ffb700 1 -- 172.21.15.57:0/2173900123 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb928011990 con 0x7fb93815c8c0 2023-11-28T12:02:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.587+0000 7fb935ffb700 1 -- 172.21.15.57:0/2173900123 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928011d20 con 0x7fb93815c8c0 2023-11-28T12:02:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.587+0000 7fb93d37b700 1 -- 172.21.15.57:0/2173900123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93815c8c0 msgr2=0x7fb938160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.587+0000 7fb93d37b700 1 --2- 172.21.15.57:0/2173900123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93815c8c0 0x7fb938160cb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb928009840 tx=0x7fb928009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.587+0000 7fb93d37b700 1 -- 172.21.15.57:0/2173900123 shutdown_connections 2023-11-28T12:02:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.587+0000 7fb93d37b700 1 --2- 172.21.15.57:0/2173900123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93815c8c0 0x7fb938160cb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.587+0000 7fb93d37b700 1 --2- 172.21.15.57:0/2173900123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb93806b220 0x7fb9380d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.587+0000 7fb93d37b700 1 -- 172.21.15.57:0/2173900123 >> 172.21.15.57:0/2173900123 conn(0x7fb93806c520 msgr2=0x7fb93806c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.587+0000 7fb93d37b700 1 -- 172.21.15.57:0/2173900123 shutdown_connections 2023-11-28T12:02:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.587+0000 7fb93d37b700 1 -- 172.21.15.57:0/2173900123 wait complete. 2023-11-28T12:02:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.589+0000 7fb93d37b700 1 Processor -- start 2023-11-28T12:02:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.589+0000 7fb93d37b700 1 -- start start 2023-11-28T12:02:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb93d37b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb93806b220 0x7fb93815cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb93d37b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9380b5dc0 0x7fb9380c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb93d37b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb93815c8c0 0x7fb9380c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb93d37b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93815dfa0 con 0x7fb93806b220 2023-11-28T12:02:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb93d37b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93815d320 con 0x7fb9380b5dc0 2023-11-28T12:02:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb93d37b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93815d600 con 0x7fb93815c8c0 2023-11-28T12:02:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb936ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb93806b220 0x7fb93815cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb936ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb93806b220 0x7fb93815cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45784/0 (socket says 172.21.15.57:45784) 2023-11-28T12:02:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb936ffd700 1 -- 172.21.15.57:0/1175916560 learned_addr learned my addr 172.21.15.57:0/1175916560 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb9377fe700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb93815c8c0 0x7fb9380c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb9367fc700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9380b5dc0 0x7fb9380c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb936ffd700 1 -- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb93815c8c0 msgr2=0x7fb9380c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb936ffd700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb93815c8c0 0x7fb9380c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb936ffd700 1 -- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9380b5dc0 msgr2=0x7fb9380c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb936ffd700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9380b5dc0 0x7fb9380c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb936ffd700 1 -- 172.21.15.57:0/1175916560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930009a70 con 0x7fb93806b220 2023-11-28T12:02:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb936ffd700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb93806b220 0x7fb93815cc90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb928009fd0 tx=0x7fb928004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb91f7fe700 1 -- 172.21.15.57:0/1175916560 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928004590 con 0x7fb93806b220 2023-11-28T12:02:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb91f7fe700 1 -- 172.21.15.57:0/1175916560 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92801e070 con 0x7fb93806b220 2023-11-28T12:02:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb91f7fe700 1 -- 172.21.15.57:0/1175916560 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9280038e0 con 0x7fb93806b220 2023-11-28T12:02:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.590+0000 7fb93d37b700 1 -- 172.21.15.57:0/1175916560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9280094a0 con 0x7fb93806b220 2023-11-28T12:02:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.591+0000 7fb93d37b700 1 -- 172.21.15.57:0/1175916560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9380cda40 con 0x7fb93806b220 2023-11-28T12:02:13.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.591+0000 7fb93d37b700 1 -- 172.21.15.57:0/1175916560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9380cd760 con 0x7fb93806b220 2023-11-28T12:02:13.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.592+0000 7fb91f7fe700 1 -- 172.21.15.57:0/1175916560 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb92800f050 con 0x7fb93806b220 2023-11-28T12:02:13.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.592+0000 7fb91f7fe700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9240682f0 0x7fb92406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.592+0000 7fb91f7fe700 1 -- 172.21.15.57:0/1175916560 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb928086b20 con 0x7fb93806b220 2023-11-28T12:02:13.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.593+0000 7fb9367fc700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9240682f0 0x7fb92406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.593+0000 7fb9367fc700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9240682f0 0x7fb92406a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb9380c6610 tx=0x7fb92c007c40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:13.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.597+0000 7fb91f7fe700 1 -- 172.21.15.57:0/1175916560 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb928053300 con 0x7fb93806b220 2023-11-28T12:02:13.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.796+0000 7fb93d37b700 1 -- 172.21.15.57:0/1175916560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb938067d70 con 0x7fb93806b220 2023-11-28T12:02:13.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.796+0000 7fb91f7fe700 1 -- 172.21.15.57:0/1175916560 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb9280150a0 con 0x7fb93806b220 2023-11-28T12:02:13.797 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:02:13.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.798+0000 7fb93d37b700 1 -- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9240682f0 msgr2=0x7fb92406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.798+0000 7fb93d37b700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9240682f0 0x7fb92406a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb9380c6610 tx=0x7fb92c007c40 comp rx=0 tx=0).stop 2023-11-28T12:02:13.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.798+0000 7fb93d37b700 1 -- 172.21.15.57:0/1175916560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb93806b220 msgr2=0x7fb93815cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.798+0000 7fb93d37b700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb93806b220 0x7fb93815cc90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb928009fd0 tx=0x7fb928004750 comp rx=0 tx=0).stop 2023-11-28T12:02:13.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.798+0000 7fb93d37b700 1 -- 172.21.15.57:0/1175916560 shutdown_connections 2023-11-28T12:02:13.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.798+0000 7fb93d37b700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9240682f0 0x7fb92406a7a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.798+0000 7fb93d37b700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb93815c8c0 0x7fb9380c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.798+0000 7fb93d37b700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb93806b220 0x7fb93815cc90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.798+0000 7fb93d37b700 1 --2- 172.21.15.57:0/1175916560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9380b5dc0 0x7fb9380c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.798+0000 7fb93d37b700 1 -- 172.21.15.57:0/1175916560 >> 172.21.15.57:0/1175916560 conn(0x7fb93806c520 msgr2=0x7fb93815ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:13.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.799+0000 7fb93d37b700 1 -- 172.21.15.57:0/1175916560 shutdown_connections 2023-11-28T12:02:13.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.799+0000 7fb93d37b700 1 -- 172.21.15.57:0/1175916560 wait complete. 2023-11-28T12:02:13.813 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:13.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.947+0000 7fe46cf5f700 1 Processor -- start 2023-11-28T12:02:13.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.947+0000 7fe46cf5f700 1 -- start start 2023-11-28T12:02:13.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.948+0000 7fe46cf5f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe46806a270 0x7fe46806a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.948+0000 7fe46cf5f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46806e570 0x7fe468070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.948+0000 7fe46cf5f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46806ab80 con 0x7fe46815cc80 2023-11-28T12:02:13.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.948+0000 7fe46cf5f700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4680b2490 con 0x7fe46806a270 2023-11-28T12:02:13.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.948+0000 7fe46cf5f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4680b1810 con 0x7fe46806e570 2023-11-28T12:02:13.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.948+0000 7fe46659c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe46806a270 0x7fe46806a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.948+0000 7fe465d9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe46815cc80 0x7fe46806de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37676/0 (socket says 172.21.15.57:37676) 2023-11-28T12:02:13.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.948+0000 7fe465d9b700 1 -- 172.21.15.57:0/1578767304 learned_addr learned my addr 172.21.15.57:0/1578767304 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:13.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.949+0000 7fe466d9d700 1 --2- 172.21.15.57:0/1578767304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46806e570 0x7fe468070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.949+0000 7fe46659c700 1 -- 172.21.15.57:0/1578767304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46806e570 msgr2=0x7fe468070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:13.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.949+0000 7fe46659c700 1 --2- 172.21.15.57:0/1578767304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46806e570 0x7fe468070d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.949+0000 7fe46659c700 1 -- 172.21.15.57:0/1578767304 >> v1:172.21.15.57:6789/0 conn(0x7fe46815cc80 legacy=0x7fe46806de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.949+0000 7fe46659c700 1 -- 172.21.15.57:0/1578767304 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4680b1af0 con 0x7fe46806a270 2023-11-28T12:02:13.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.949+0000 7fe46659c700 1 --2- 172.21.15.57:0/1578767304 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe46806a270 0x7fe46806a640 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe458009840 tx=0x7fe458009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87dd29bff24da868 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:13.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.949+0000 7fe46559a700 1 -- 172.21.15.57:0/1578767304 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458004230 con 0x7fe46806a270 2023-11-28T12:02:13.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.949+0000 7fe46559a700 1 -- 172.21.15.57:0/1578767304 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe458004390 con 0x7fe46806a270 2023-11-28T12:02:13.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.949+0000 7fe46559a700 1 -- 172.21.15.57:0/1578767304 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4580036a0 con 0x7fe46806a270 2023-11-28T12:02:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.950+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1578767304 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe46806a270 msgr2=0x7fe46806a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.950+0000 7fe46cf5f700 1 --2- 172.21.15.57:0/1578767304 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe46806a270 0x7fe46806a640 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe458009840 tx=0x7fe458009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.950+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1578767304 shutdown_connections 2023-11-28T12:02:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.950+0000 7fe46cf5f700 1 --2- 172.21.15.57:0/1578767304 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe46806a270 0x7fe46806a640 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.950+0000 7fe46cf5f700 1 --2- 172.21.15.57:0/1578767304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46806e570 0x7fe468070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.950+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1578767304 >> 172.21.15.57:0/1578767304 conn(0x7fe46806c010 msgr2=0x7fe4680cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.950+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1578767304 shutdown_connections 2023-11-28T12:02:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.950+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1578767304 wait complete. 2023-11-28T12:02:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.952+0000 7fe46cf5f700 1 Processor -- start 2023-11-28T12:02:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.952+0000 7fe46cf5f700 1 -- start start 2023-11-28T12:02:13.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46cf5f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe46806a270 0x7fe4680b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46cf5f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46806e570 0x7fe4681e6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46cf5f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe46815cc80 0x7fe4680c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46cf5f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4681e6770 con 0x7fe46806a270 2023-11-28T12:02:13.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46cf5f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4680af8f0 con 0x7fe46815cc80 2023-11-28T12:02:13.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46cf5f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4680afbd0 con 0x7fe46806e570 2023-11-28T12:02:13.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe46806a270 0x7fe4680b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe46806a270 0x7fe4680b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45804/0 (socket says 172.21.15.57:45804) 2023-11-28T12:02:13.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46659c700 1 -- 172.21.15.57:0/1785481175 learned_addr learned my addr 172.21.15.57:0/1785481175 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:13.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe466d9d700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe46815cc80 0x7fe4680c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe465d9b700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46806e570 0x7fe4681e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46659c700 1 -- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46806e570 msgr2=0x7fe4681e6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46659c700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46806e570 0x7fe4681e6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46659c700 1 -- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe46815cc80 msgr2=0x7fe4680c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:13.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46659c700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe46815cc80 0x7fe4680c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:13.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.953+0000 7fe46659c700 1 -- 172.21.15.57:0/1785481175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4580094a0 con 0x7fe46806a270 2023-11-28T12:02:13.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.954+0000 7fe46659c700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe46806a270 0x7fe4680b5120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe458000c00 tx=0x7fe458004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:13.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.954+0000 7fe45effd700 1 -- 172.21.15.57:0/1785481175 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458011830 con 0x7fe46806a270 2023-11-28T12:02:13.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.954+0000 7fe45effd700 1 -- 172.21.15.57:0/1785481175 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe458011990 con 0x7fe46806a270 2023-11-28T12:02:13.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.954+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1785481175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4680b20b0 con 0x7fe46806a270 2023-11-28T12:02:13.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.954+0000 7fe45effd700 1 -- 172.21.15.57:0/1785481175 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458011d20 con 0x7fe46806a270 2023-11-28T12:02:13.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.954+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1785481175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4680b1dd0 con 0x7fe46806a270 2023-11-28T12:02:13.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.955+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1785481175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46815daa0 con 0x7fe46806a270 2023-11-28T12:02:13.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.955+0000 7fe45effd700 1 -- 172.21.15.57:0/1785481175 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe458017020 con 0x7fe46806a270 2023-11-28T12:02:13.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.956+0000 7fe45effd700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe4480682f0 0x7fe44806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:13.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.956+0000 7fe45effd700 1 -- 172.21.15.57:0/1785481175 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe458087370 con 0x7fe46806a270 2023-11-28T12:02:13.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.956+0000 7fe465d9b700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe4480682f0 0x7fe44806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:13.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.956+0000 7fe465d9b700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe4480682f0 0x7fe44806a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe450007c20 tx=0x7fe450007d70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:13.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:13.960+0000 7fe45effd700 1 -- 172.21.15.57:0/1785481175 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe458053bd0 con 0x7fe46806a270 2023-11-28T12:02:14.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.161+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1785481175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4680743f0 con 0x7fe46806a270 2023-11-28T12:02:14.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.162+0000 7fe45effd700 1 -- 172.21.15.57:0/1785481175 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7fe4580571f0 con 0x7fe46806a270 2023-11-28T12:02:14.164 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:14.164 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:14.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.164+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe4480682f0 msgr2=0x7fe44806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.164+0000 7fe46cf5f700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe4480682f0 0x7fe44806a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe450007c20 tx=0x7fe450007d70 comp rx=0 tx=0).stop 2023-11-28T12:02:14.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.164+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1785481175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe46806a270 msgr2=0x7fe4680b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.164+0000 7fe46cf5f700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe46806a270 0x7fe4680b5120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe458000c00 tx=0x7fe458004750 comp rx=0 tx=0).stop 2023-11-28T12:02:14.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.164+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1785481175 shutdown_connections 2023-11-28T12:02:14.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.164+0000 7fe46cf5f700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe4480682f0 0x7fe44806a7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.164+0000 7fe46cf5f700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe46815cc80 0x7fe4680c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.164+0000 7fe46cf5f700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe46806a270 0x7fe4680b5120 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.164+0000 7fe46cf5f700 1 --2- 172.21.15.57:0/1785481175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46806e570 0x7fe4681e6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.164+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1785481175 >> 172.21.15.57:0/1785481175 conn(0x7fe46806c010 msgr2=0x7fe46815ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:14.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.164+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1785481175 shutdown_connections 2023-11-28T12:02:14.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.165+0000 7fe46cf5f700 1 -- 172.21.15.57:0/1785481175 wait complete. 2023-11-28T12:02:14.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:14.179 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:14.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.317+0000 7fbe5a08d700 1 Processor -- start 2023-11-28T12:02:14.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.317+0000 7fbe5a08d700 1 -- start start 2023-11-28T12:02:14.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.317+0000 7fbe5a08d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe54159710 0x7fbe54070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:14.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe5a08d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe5406a310 con 0x7fbe540c9a50 2023-11-28T12:02:14.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe5a08d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe540ce7e0 con 0x7fbe540cee40 2023-11-28T12:02:14.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe5a08d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe540cf2b0 con 0x7fbe54159710 2023-11-28T12:02:14.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe53fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fbe540c9a50 0x7fbe5406adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37680/0 (socket says 172.21.15.57:37680) 2023-11-28T12:02:14.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe53fff700 1 -- 172.21.15.57:0/3028661335 learned_addr learned my addr 172.21.15.57:0/3028661335 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:14.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe537fe700 1 --2- 172.21.15.57:0/3028661335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe54159710 0x7fbe54070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:14.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe527fc700 1 -- 172.21.15.57:0/3028661335 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283705871 0 0) 0x7fbe5406a310 con 0x7fbe540c9a50 2023-11-28T12:02:14.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe527fc700 1 -- 172.21.15.57:0/3028661335 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe400035b0 con 0x7fbe540c9a50 2023-11-28T12:02:14.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe527fc700 1 -- 172.21.15.57:0/3028661335 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886488809 0 0) 0x7fbe540ce7e0 con 0x7fbe540cee40 2023-11-28T12:02:14.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe527fc700 1 -- 172.21.15.57:0/3028661335 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe5406a310 con 0x7fbe540cee40 2023-11-28T12:02:14.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe527fc700 1 -- 172.21.15.57:0/3028661335 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3659048101 0 0) 0x7fbe400035b0 con 0x7fbe540c9a50 2023-11-28T12:02:14.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe537fe700 1 -- 172.21.15.57:0/3028661335 >> v1:172.21.15.181:6789/0 conn(0x7fbe540cee40 legacy=0x7fbe540cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe537fe700 1 -- 172.21.15.57:0/3028661335 >> v1:172.21.15.57:6789/0 conn(0x7fbe540c9a50 legacy=0x7fbe5406adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.318+0000 7fbe537fe700 1 -- 172.21.15.57:0/3028661335 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5406a5b0 con 0x7fbe54159710 2023-11-28T12:02:14.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.319+0000 7fbe537fe700 1 --2- 172.21.15.57:0/3028661335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe54159710 0x7fbe54070370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009870 tx=0x7fbe3c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1601393d7ca55c80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:14.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.319+0000 7fbe527fc700 1 -- 172.21.15.57:0/3028661335 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0019f0 con 0x7fbe54159710 2023-11-28T12:02:14.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.319+0000 7fbe527fc700 1 -- 172.21.15.57:0/3028661335 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c004030 con 0x7fbe54159710 2023-11-28T12:02:14.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.319+0000 7fbe527fc700 1 -- 172.21.15.57:0/3028661335 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c0043c0 con 0x7fbe54159710 2023-11-28T12:02:14.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.319+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3028661335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe54159710 msgr2=0x7fbe54070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.319+0000 7fbe5a08d700 1 --2- 172.21.15.57:0/3028661335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe54159710 0x7fbe54070370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009870 tx=0x7fbe3c009b80 comp rx=0 tx=0).stop 2023-11-28T12:02:14.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.319+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3028661335 shutdown_connections 2023-11-28T12:02:14.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.319+0000 7fbe5a08d700 1 --2- 172.21.15.57:0/3028661335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe54159710 0x7fbe54070370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.319+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3028661335 >> 172.21.15.57:0/3028661335 conn(0x7fbe540cf400 msgr2=0x7fbe540cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:14.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.319+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3028661335 shutdown_connections 2023-11-28T12:02:14.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.319+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3028661335 wait complete. 2023-11-28T12:02:14.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.321+0000 7fbe5a08d700 1 Processor -- start 2023-11-28T12:02:14.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.321+0000 7fbe5a08d700 1 -- start start 2023-11-28T12:02:14.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe5a08d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe540c9a50 0x7fbe5415c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:14.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe5a08d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe540cee40 0x7fbe540c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:14.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe5a08d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe54159710 0x7fbe540b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:14.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe5a08d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe540c4ce0 con 0x7fbe54159710 2023-11-28T12:02:14.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe5a08d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe54156910 con 0x7fbe540cee40 2023-11-28T12:02:14.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe5a08d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5415bb50 con 0x7fbe540c9a50 2023-11-28T12:02:14.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe53fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe54159710 0x7fbe540b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:14.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe53fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe54159710 0x7fbe540b3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45814/0 (socket says 172.21.15.57:45814) 2023-11-28T12:02:14.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe53fff700 1 -- 172.21.15.57:0/3308533410 learned_addr learned my addr 172.21.15.57:0/3308533410 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:14.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe537fe700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe540c9a50 0x7fbe5415c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:14.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe52ffd700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe540cee40 0x7fbe540c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:14.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe53fff700 1 -- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe540c9a50 msgr2=0x7fbe5415c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe53fff700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe540c9a50 0x7fbe5415c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe53fff700 1 -- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe540cee40 msgr2=0x7fbe540c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe53fff700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe540cee40 0x7fbe540c47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe53fff700 1 -- 172.21.15.57:0/3308533410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0094a0 con 0x7fbe54159710 2023-11-28T12:02:14.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.322+0000 7fbe53fff700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe54159710 0x7fbe540b3510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c001700 tx=0x7fbe4c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:14.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.323+0000 7fbe4bfff700 1 -- 172.21.15.57:0/3308533410 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00ab00 con 0x7fbe54159710 2023-11-28T12:02:14.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.323+0000 7fbe4bfff700 1 -- 172.21.15.57:0/3308533410 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c00ac60 con 0x7fbe54159710 2023-11-28T12:02:14.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.323+0000 7fbe4bfff700 1 -- 172.21.15.57:0/3308533410 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0070d0 con 0x7fbe54159710 2023-11-28T12:02:14.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.323+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3308533410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe540ceac0 con 0x7fbe54159710 2023-11-28T12:02:14.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.323+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3308533410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5406a890 con 0x7fbe54159710 2023-11-28T12:02:14.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.324+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3308533410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe540c9ed0 con 0x7fbe54159710 2023-11-28T12:02:14.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.324+0000 7fbe4bfff700 1 -- 172.21.15.57:0/3308533410 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbe4c010070 con 0x7fbe54159710 2023-11-28T12:02:14.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.325+0000 7fbe4bfff700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbe340682f0 0x7fbe3406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:14.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.325+0000 7fbe4bfff700 1 -- 172.21.15.57:0/3308533410 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe4c085660 con 0x7fbe54159710 2023-11-28T12:02:14.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.325+0000 7fbe537fe700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbe340682f0 0x7fbe3406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:14.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.325+0000 7fbe537fe700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbe340682f0 0x7fbe3406a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0097c0 tx=0x7fbe3c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:14.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.329+0000 7fbe4bfff700 1 -- 172.21.15.57:0/3308533410 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbe4c051e30 con 0x7fbe54159710 2023-11-28T12:02:14.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.532+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3308533410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbe540d0070 con 0x7fbe54159710 2023-11-28T12:02:14.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.532+0000 7fbe4bfff700 1 -- 172.21.15.57:0/3308533410 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbe4c018020 con 0x7fbe54159710 2023-11-28T12:02:14.534 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:14.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.534+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbe340682f0 msgr2=0x7fbe3406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.534+0000 7fbe5a08d700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbe340682f0 0x7fbe3406a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0097c0 tx=0x7fbe3c008030 comp rx=0 tx=0).stop 2023-11-28T12:02:14.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.535+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3308533410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe54159710 msgr2=0x7fbe540b3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.535+0000 7fbe5a08d700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe54159710 0x7fbe540b3510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c001700 tx=0x7fbe4c00cf70 comp rx=0 tx=0).stop 2023-11-28T12:02:14.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.535+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3308533410 shutdown_connections 2023-11-28T12:02:14.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.535+0000 7fbe5a08d700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbe340682f0 0x7fbe3406a7a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.535+0000 7fbe5a08d700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe54159710 0x7fbe540b3510 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.535+0000 7fbe5a08d700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe540c9a50 0x7fbe5415c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.535+0000 7fbe5a08d700 1 --2- 172.21.15.57:0/3308533410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe540cee40 0x7fbe540c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.535+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3308533410 >> 172.21.15.57:0/3308533410 conn(0x7fbe540cf400 msgr2=0x7fbe540aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:14.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.535+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3308533410 shutdown_connections 2023-11-28T12:02:14.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.535+0000 7fbe5a08d700 1 -- 172.21.15.57:0/3308533410 wait complete. 2023-11-28T12:02:14.549 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:14.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.687+0000 7fc5f67e7700 1 Processor -- start 2023-11-28T12:02:14.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.688+0000 7fc5f67e7700 1 -- start start 2023-11-28T12:02:14.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.688+0000 7fc5f67e7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f0159710 0x7fc5f0070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:14.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.688+0000 7fc5f67e7700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc5f00cee40 0x7fc5f00cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:14.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.688+0000 7fc5f67e7700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f006a310 con 0x7fc5f00c9a50 2023-11-28T12:02:14.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.688+0000 7fc5f67e7700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f00cf2b0 con 0x7fc5f00cee40 2023-11-28T12:02:14.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.688+0000 7fc5f67e7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00ce530 con 0x7fc5f0159710 2023-11-28T12:02:14.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5e7fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc5f00cee40 0x7fc5f00cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:14.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5e7fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc5f00cee40 0x7fc5f00cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58688/0 (socket says 172.21.15.57:58688) 2023-11-28T12:02:14.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5f4d84700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc5f00c9a50 0x7fc5f006adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45784/0 (socket says 172.21.15.57:45784) 2023-11-28T12:02:14.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5e7fff700 1 -- 172.21.15.57:0/4062648973 learned_addr learned my addr 172.21.15.57:0/4062648973 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:14.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5effff700 1 --2- 172.21.15.57:0/4062648973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f0159710 0x7fc5f0070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:14.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5e7fff700 1 -- 172.21.15.57:0/4062648973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f0159710 msgr2=0x7fc5f0070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:14.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5e7fff700 1 --2- 172.21.15.57:0/4062648973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f0159710 0x7fc5f0070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5e7fff700 1 -- 172.21.15.57:0/4062648973 >> v1:172.21.15.57:6789/0 conn(0x7fc5f00c9a50 legacy=0x7fc5f006adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5e7fff700 1 -- 172.21.15.57:0/4062648973 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f00ce7e0 con 0x7fc5f00cee40 2023-11-28T12:02:14.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5ef7fe700 1 -- 172.21.15.57:0/4062648973 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047621152 0 0) 0x7fc5f006a310 con 0x7fc5f00c9a50 2023-11-28T12:02:14.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5e7fff700 1 --2- 172.21.15.57:0/4062648973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5f00cee40 0x7fc5f00cdff0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80098f0 tx=0x7fc5d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8132b8834d16aeef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:14.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5ef7fe700 1 -- 172.21.15.57:0/4062648973 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800b8a0 con 0x7fc5f00cee40 2023-11-28T12:02:14.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5ef7fe700 1 -- 172.21.15.57:0/4062648973 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d800ba00 con 0x7fc5f00cee40 2023-11-28T12:02:14.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.689+0000 7fc5ef7fe700 1 -- 172.21.15.57:0/4062648973 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800bd90 con 0x7fc5f00cee40 2023-11-28T12:02:14.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.690+0000 7fc5f67e7700 1 -- 172.21.15.57:0/4062648973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5f00cee40 msgr2=0x7fc5f00cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.690+0000 7fc5f67e7700 1 --2- 172.21.15.57:0/4062648973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5f00cee40 0x7fc5f00cdff0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80098f0 tx=0x7fc5d8009c00 comp rx=0 tx=0).stop 2023-11-28T12:02:14.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.690+0000 7fc5f67e7700 1 -- 172.21.15.57:0/4062648973 shutdown_connections 2023-11-28T12:02:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.690+0000 7fc5f67e7700 1 --2- 172.21.15.57:0/4062648973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f0159710 0x7fc5f0070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.690+0000 7fc5f67e7700 1 --2- 172.21.15.57:0/4062648973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5f00cee40 0x7fc5f00cdff0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.690+0000 7fc5f67e7700 1 -- 172.21.15.57:0/4062648973 >> 172.21.15.57:0/4062648973 conn(0x7fc5f00cf400 msgr2=0x7fc5f00cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.690+0000 7fc5f67e7700 1 -- 172.21.15.57:0/4062648973 shutdown_connections 2023-11-28T12:02:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.690+0000 7fc5f67e7700 1 -- 172.21.15.57:0/4062648973 wait complete. 2023-11-28T12:02:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.691+0000 7fc5f67e7700 1 Processor -- start 2023-11-28T12:02:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.691+0000 7fc5f67e7700 1 -- start start 2023-11-28T12:02:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.692+0000 7fc5f67e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00c9a50 0x7fc5f00c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.692+0000 7fc5f67e7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5f00cee40 0x7fc5f01528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.692+0000 7fc5f67e7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f0159710 0x7fc5f00c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.692+0000 7fc5f67e7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f015abb0 con 0x7fc5f00cee40 2023-11-28T12:02:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.692+0000 7fc5f67e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00c8650 con 0x7fc5f00c9a50 2023-11-28T12:02:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.692+0000 7fc5f67e7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0152e10 con 0x7fc5f0159710 2023-11-28T12:02:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.692+0000 7fc5e7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5f00cee40 0x7fc5f01528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.692+0000 7fc5e7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5f00cee40 0x7fc5f01528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58704/0 (socket says 172.21.15.57:58704) 2023-11-28T12:02:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.692+0000 7fc5e7fff700 1 -- 172.21.15.57:0/1402162146 learned_addr learned my addr 172.21.15.57:0/1402162146 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.692+0000 7fc5f4d84700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f0159710 0x7fc5f00c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.693+0000 7fc5e7fff700 1 -- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f0159710 msgr2=0x7fc5f00c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.693+0000 7fc5e7fff700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f0159710 0x7fc5f00c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.693+0000 7fc5e7fff700 1 -- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00c9a50 msgr2=0x7fc5f00c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.693+0000 7fc5effff700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00c9a50 0x7fc5f00c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.693+0000 7fc5e7fff700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00c9a50 0x7fc5f00c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.693+0000 7fc5e7fff700 1 -- 172.21.15.57:0/1402162146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d8009550 con 0x7fc5f00cee40 2023-11-28T12:02:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.693+0000 7fc5e7fff700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5f00cee40 0x7fc5f01528d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009c40 tx=0x7fc5d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.693+0000 7fc5ed7fa700 1 -- 172.21.15.57:0/1402162146 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80036a0 con 0x7fc5f00cee40 2023-11-28T12:02:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.693+0000 7fc5ed7fa700 1 -- 172.21.15.57:0/1402162146 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d801e070 con 0x7fc5f00cee40 2023-11-28T12:02:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.693+0000 7fc5f67e7700 1 -- 172.21.15.57:0/1402162146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f006a890 con 0x7fc5f00cee40 2023-11-28T12:02:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.693+0000 7fc5ed7fa700 1 -- 172.21.15.57:0/1402162146 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8003a20 con 0x7fc5f00cee40 2023-11-28T12:02:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.693+0000 7fc5f67e7700 1 -- 172.21.15.57:0/1402162146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f006a5b0 con 0x7fc5f00cee40 2023-11-28T12:02:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.694+0000 7fc5f67e7700 1 -- 172.21.15.57:0/1402162146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f00ceac0 con 0x7fc5f00cee40 2023-11-28T12:02:14.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.695+0000 7fc5ed7fa700 1 -- 172.21.15.57:0/1402162146 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc5d800f050 con 0x7fc5f00cee40 2023-11-28T12:02:14.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.695+0000 7fc5ed7fa700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5d00682f0 0x7fc5d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:14.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.695+0000 7fc5ed7fa700 1 -- 172.21.15.57:0/1402162146 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc5d8020070 con 0x7fc5f00cee40 2023-11-28T12:02:14.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.695+0000 7fc5effff700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5d00682f0 0x7fc5d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:14.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.696+0000 7fc5effff700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5d00682f0 0x7fc5d006a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009770 tx=0x7fc5e0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.700+0000 7fc5ed7fa700 1 -- 172.21.15.57:0/1402162146 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc5d8053a10 con 0x7fc5f00cee40 2023-11-28T12:02:14.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.899+0000 7fc5f67e7700 1 -- 172.21.15.57:0/1402162146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5f00743f0 con 0x7fc5f00cee40 2023-11-28T12:02:14.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.900+0000 7fc5ed7fa700 1 -- 172.21.15.57:0/1402162146 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5d8057030 con 0x7fc5f00cee40 2023-11-28T12:02:14.901 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:14.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.902+0000 7fc5f67e7700 1 -- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5d00682f0 msgr2=0x7fc5d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.902+0000 7fc5f67e7700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5d00682f0 0x7fc5d006a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009770 tx=0x7fc5e0008040 comp rx=0 tx=0).stop 2023-11-28T12:02:14.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.902+0000 7fc5f67e7700 1 -- 172.21.15.57:0/1402162146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5f00cee40 msgr2=0x7fc5f01528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:14.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.902+0000 7fc5f67e7700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5f00cee40 0x7fc5f01528d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009c40 tx=0x7fc5d8004750 comp rx=0 tx=0).stop 2023-11-28T12:02:14.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.902+0000 7fc5f67e7700 1 -- 172.21.15.57:0/1402162146 shutdown_connections 2023-11-28T12:02:14.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.902+0000 7fc5f67e7700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5d00682f0 0x7fc5d006a7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.902+0000 7fc5f67e7700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f0159710 0x7fc5f00c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.902+0000 7fc5f67e7700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00c9a50 0x7fc5f00c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.902+0000 7fc5f67e7700 1 --2- 172.21.15.57:0/1402162146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5f00cee40 0x7fc5f01528d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:14.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.902+0000 7fc5f67e7700 1 -- 172.21.15.57:0/1402162146 >> 172.21.15.57:0/1402162146 conn(0x7fc5f00cf400 msgr2=0x7fc5f006ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:14.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.902+0000 7fc5f67e7700 1 -- 172.21.15.57:0/1402162146 shutdown_connections 2023-11-28T12:02:14.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:14.902+0000 7fc5f67e7700 1 -- 172.21.15.57:0/1402162146 wait complete. 2023-11-28T12:02:14.917 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:02:15.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.054+0000 7fdd92fce700 1 Processor -- start 2023-11-28T12:02:15.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.054+0000 7fdd92fce700 1 -- start start 2023-11-28T12:02:15.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.054+0000 7fdd92fce700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdd8c0cf710 0x7fdd8c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.054+0000 7fdd92fce700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdd8c06a980 0x7fdd8c071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.054+0000 7fdd92fce700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8c15cc10 con 0x7fdd8c071890 2023-11-28T12:02:15.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.054+0000 7fdd92fce700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd8c06c9b0 con 0x7fdd8c06a980 2023-11-28T12:02:15.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.054+0000 7fdd92fce700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd8c06cc90 con 0x7fdd8c0cf710 2023-11-28T12:02:15.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.055+0000 7fdd9156b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdd8c06a980 0x7fdd8c071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.055+0000 7fdd9156b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdd8c06a980 0x7fdd8c071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58720/0 (socket says 172.21.15.57:58720) 2023-11-28T12:02:15.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.055+0000 7fdd9156b700 1 -- 172.21.15.57:0/3373958000 learned_addr learned my addr 172.21.15.57:0/3373958000 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:15.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.055+0000 7fdd90d6a700 1 --2- 172.21.15.57:0/3373958000 >> v2:172.21.15.181:3300/0 conn(0x7fdd8c0cf710 0x7fdd8c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.055+0000 7fdd9156b700 1 -- 172.21.15.57:0/3373958000 >> v2:172.21.15.181:3300/0 conn(0x7fdd8c0cf710 msgr2=0x7fdd8c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:15.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.055+0000 7fdd9156b700 1 --2- 172.21.15.57:0/3373958000 >> v2:172.21.15.181:3300/0 conn(0x7fdd8c0cf710 0x7fdd8c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.055+0000 7fdd9156b700 1 -- 172.21.15.57:0/3373958000 >> v1:172.21.15.181:6789/0 conn(0x7fdd8c071890 legacy=0x7fdd8c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.055+0000 7fdd9156b700 1 -- 172.21.15.57:0/3373958000 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c06cf70 con 0x7fdd8c06a980 2023-11-28T12:02:15.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.055+0000 7fdd9156b700 1 --2- 172.21.15.57:0/3373958000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8c06a980 0x7fdd8c071220 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009e50 tx=0x7fdd8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f51e52375efd5ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:15.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.055+0000 7fdd8b7fe700 1 -- 172.21.15.57:0/3373958000 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400e070 con 0x7fdd8c06a980 2023-11-28T12:02:15.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.055+0000 7fdd8b7fe700 1 -- 172.21.15.57:0/3373958000 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8400b8d0 con 0x7fdd8c06a980 2023-11-28T12:02:15.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.055+0000 7fdd8b7fe700 1 -- 172.21.15.57:0/3373958000 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400bc80 con 0x7fdd8c06a980 2023-11-28T12:02:15.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.056+0000 7fdd92fce700 1 -- 172.21.15.57:0/3373958000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8c06a980 msgr2=0x7fdd8c071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.056+0000 7fdd92fce700 1 --2- 172.21.15.57:0/3373958000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8c06a980 0x7fdd8c071220 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009e50 tx=0x7fdd8400b330 comp rx=0 tx=0).stop 2023-11-28T12:02:15.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.056+0000 7fdd92fce700 1 -- 172.21.15.57:0/3373958000 shutdown_connections 2023-11-28T12:02:15.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.056+0000 7fdd92fce700 1 --2- 172.21.15.57:0/3373958000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8c06a980 0x7fdd8c071220 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.056+0000 7fdd92fce700 1 --2- 172.21.15.57:0/3373958000 >> v2:172.21.15.181:3300/0 conn(0x7fdd8c0cf710 0x7fdd8c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.056+0000 7fdd92fce700 1 -- 172.21.15.57:0/3373958000 >> 172.21.15.57:0/3373958000 conn(0x7fdd8c06c010 msgr2=0x7fdd8c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:15.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.056+0000 7fdd92fce700 1 -- 172.21.15.57:0/3373958000 shutdown_connections 2023-11-28T12:02:15.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.056+0000 7fdd92fce700 1 -- 172.21.15.57:0/3373958000 wait complete. 2023-11-28T12:02:15.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.057+0000 7fdd92fce700 1 Processor -- start 2023-11-28T12:02:15.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.057+0000 7fdd92fce700 1 -- start start 2023-11-28T12:02:15.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.058+0000 7fdd92fce700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd8c06a980 0x7fdd8c158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.058+0000 7fdd92fce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd8c071890 0x7fdd8c0b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.058+0000 7fdd92fce700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8c0cf710 0x7fdd8c0b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.058+0000 7fdd92fce700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c0c6660 con 0x7fdd8c0cf710 2023-11-28T12:02:15.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.058+0000 7fdd92fce700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c15e350 con 0x7fdd8c071890 2023-11-28T12:02:15.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.058+0000 7fdd92fce700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1586c0 con 0x7fdd8c06a980 2023-11-28T12:02:15.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd9156b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8c0cf710 0x7fdd8c0b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd9156b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8c0cf710 0x7fdd8c0b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58734/0 (socket says 172.21.15.57:58734) 2023-11-28T12:02:15.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd9156b700 1 -- 172.21.15.57:0/276763373 learned_addr learned my addr 172.21.15.57:0/276763373 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:15.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd90d6a700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd8c06a980 0x7fdd8c158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd8bfff700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd8c071890 0x7fdd8c0b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd9156b700 1 -- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd8c06a980 msgr2=0x7fdd8c158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd9156b700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd8c06a980 0x7fdd8c158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd9156b700 1 -- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd8c071890 msgr2=0x7fdd8c0b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd9156b700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd8c071890 0x7fdd8c0b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd9156b700 1 -- 172.21.15.57:0/276763373 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84009b30 con 0x7fdd8c0cf710 2023-11-28T12:02:15.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd9156b700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8c0cf710 0x7fdd8c0b0820 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c067450 tx=0x7fdd84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:15.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd897fa700 1 -- 172.21.15.57:0/276763373 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84004800 con 0x7fdd8c0cf710 2023-11-28T12:02:15.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd897fa700 1 -- 172.21.15.57:0/276763373 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd840092e0 con 0x7fdd8c0cf710 2023-11-28T12:02:15.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd92fce700 1 -- 172.21.15.57:0/276763373 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8c06d530 con 0x7fdd8c0cf710 2023-11-28T12:02:15.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd897fa700 1 -- 172.21.15.57:0/276763373 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8401c680 con 0x7fdd8c0cf710 2023-11-28T12:02:15.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.059+0000 7fdd92fce700 1 -- 172.21.15.57:0/276763373 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8c06d250 con 0x7fdd8c0cf710 2023-11-28T12:02:15.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.061+0000 7fdd92fce700 1 -- 172.21.15.57:0/276763373 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8c160480 con 0x7fdd8c0cf710 2023-11-28T12:02:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.061+0000 7fdd897fa700 1 -- 172.21.15.57:0/276763373 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdd8401c7e0 con 0x7fdd8c0cf710 2023-11-28T12:02:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.061+0000 7fdd897fa700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd6c0682f0 0x7fdd6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.061+0000 7fdd897fa700 1 -- 172.21.15.57:0/276763373 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdd84087570 con 0x7fdd8c0cf710 2023-11-28T12:02:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.062+0000 7fdd90d6a700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd6c0682f0 0x7fdd6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.062+0000 7fdd90d6a700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd6c0682f0 0x7fdd6c06a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009b40 tx=0x7fdd7c009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:15.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.066+0000 7fdd897fa700 1 -- 172.21.15.57:0/276763373 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdd84053dd0 con 0x7fdd8c0cf710 2023-11-28T12:02:15.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.262+0000 7fdd92fce700 1 -- 172.21.15.57:0/276763373 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd8c15e490 con 0x7fdd8c0cf710 2023-11-28T12:02:15.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.263+0000 7fdd897fa700 1 -- 172.21.15.57:0/276763373 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd8c15e490 con 0x7fdd8c0cf710 2023-11-28T12:02:15.265 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:02:15.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.265+0000 7fdd92fce700 1 -- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd6c0682f0 msgr2=0x7fdd6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.265+0000 7fdd92fce700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd6c0682f0 0x7fdd6c06a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009b40 tx=0x7fdd7c009210 comp rx=0 tx=0).stop 2023-11-28T12:02:15.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.265+0000 7fdd92fce700 1 -- 172.21.15.57:0/276763373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8c0cf710 msgr2=0x7fdd8c0b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.265+0000 7fdd92fce700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8c0cf710 0x7fdd8c0b0820 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c067450 tx=0x7fdd84004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.265+0000 7fdd92fce700 1 -- 172.21.15.57:0/276763373 shutdown_connections 2023-11-28T12:02:15.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.265+0000 7fdd92fce700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd6c0682f0 0x7fdd6c06a7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.265+0000 7fdd92fce700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8c0cf710 0x7fdd8c0b0820 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.265+0000 7fdd92fce700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd8c06a980 0x7fdd8c158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.265+0000 7fdd92fce700 1 --2- 172.21.15.57:0/276763373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd8c071890 0x7fdd8c0b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.265+0000 7fdd92fce700 1 -- 172.21.15.57:0/276763373 >> 172.21.15.57:0/276763373 conn(0x7fdd8c06c010 msgr2=0x7fdd8c06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:15.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.265+0000 7fdd92fce700 1 -- 172.21.15.57:0/276763373 shutdown_connections 2023-11-28T12:02:15.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.265+0000 7fdd92fce700 1 -- 172.21.15.57:0/276763373 wait complete. 2023-11-28T12:02:15.280 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:02:15.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.417+0000 7fc046f89700 1 Processor -- start 2023-11-28T12:02:15.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.417+0000 7fc046f89700 1 -- start start 2023-11-28T12:02:15.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.418+0000 7fc046f89700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc04015a5f0 0x7fc0400d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.418+0000 7fc046f89700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0400b2ee0 0x7fc0400ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.418+0000 7fc046f89700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04015a9c0 con 0x7fc0400cc200 2023-11-28T12:02:15.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.418+0000 7fc046f89700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0400c7b80 con 0x7fc04015a5f0 2023-11-28T12:02:15.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.418+0000 7fc046f89700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0400cc990 con 0x7fc0400b2ee0 2023-11-28T12:02:15.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.419+0000 7fc045526700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0400b2ee0 0x7fc0400ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.419+0000 7fc044d25700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc04015a5f0 0x7fc0400d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.419+0000 7fc045526700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0400b2ee0 0x7fc0400ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:45996/0 (socket says 172.21.15.57:45996) 2023-11-28T12:02:15.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.419+0000 7fc045526700 1 -- 172.21.15.57:0/4225668696 learned_addr learned my addr 172.21.15.57:0/4225668696 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:15.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.419+0000 7fc045526700 1 -- 172.21.15.57:0/4225668696 >> v2:172.21.15.181:3300/0 conn(0x7fc04015a5f0 msgr2=0x7fc0400d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:15.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.419+0000 7fc045526700 1 --2- 172.21.15.57:0/4225668696 >> v2:172.21.15.181:3300/0 conn(0x7fc04015a5f0 0x7fc0400d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.419+0000 7fc03f7fe700 1 -- 172.21.15.57:0/4225668696 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215758394 0 0) 0x7fc04015a9c0 con 0x7fc0400cc200 2023-11-28T12:02:15.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.419+0000 7fc045526700 1 -- 172.21.15.57:0/4225668696 >> v1:172.21.15.181:6789/0 conn(0x7fc0400cc200 legacy=0x7fc0400b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.419+0000 7fc045526700 1 -- 172.21.15.57:0/4225668696 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0400ccc70 con 0x7fc0400b2ee0 2023-11-28T12:02:15.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.419+0000 7fc045526700 1 --2- 172.21.15.57:0/4225668696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0400b2ee0 0x7fc0400ced30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc038009e50 tx=0x7fc03800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1887bad88c14221 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:15.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.419+0000 7fc03f7fe700 1 -- 172.21.15.57:0/4225668696 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03800e070 con 0x7fc0400b2ee0 2023-11-28T12:02:15.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.420+0000 7fc03f7fe700 1 -- 172.21.15.57:0/4225668696 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03800b910 con 0x7fc0400b2ee0 2023-11-28T12:02:15.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.420+0000 7fc03f7fe700 1 -- 172.21.15.57:0/4225668696 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03800bcc0 con 0x7fc0400b2ee0 2023-11-28T12:02:15.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.420+0000 7fc046f89700 1 -- 172.21.15.57:0/4225668696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0400b2ee0 msgr2=0x7fc0400ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.420+0000 7fc046f89700 1 --2- 172.21.15.57:0/4225668696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0400b2ee0 0x7fc0400ced30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc038009e50 tx=0x7fc03800b330 comp rx=0 tx=0).stop 2023-11-28T12:02:15.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.420+0000 7fc046f89700 1 -- 172.21.15.57:0/4225668696 shutdown_connections 2023-11-28T12:02:15.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.420+0000 7fc046f89700 1 --2- 172.21.15.57:0/4225668696 >> v2:172.21.15.181:3300/0 conn(0x7fc04015a5f0 0x7fc0400d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.420+0000 7fc046f89700 1 --2- 172.21.15.57:0/4225668696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0400b2ee0 0x7fc0400ced30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.420+0000 7fc046f89700 1 -- 172.21.15.57:0/4225668696 >> 172.21.15.57:0/4225668696 conn(0x7fc04006ea80 msgr2=0x7fc04006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:15.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.420+0000 7fc046f89700 1 -- 172.21.15.57:0/4225668696 shutdown_connections 2023-11-28T12:02:15.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.420+0000 7fc046f89700 1 -- 172.21.15.57:0/4225668696 wait complete. 2023-11-28T12:02:15.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.421+0000 7fc046f89700 1 Processor -- start 2023-11-28T12:02:15.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.421+0000 7fc046f89700 1 -- start start 2023-11-28T12:02:15.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.422+0000 7fc046f89700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0400b2ee0 0x7fc040150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.422+0000 7fc046f89700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0400cc200 0x7fc0400b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.422+0000 7fc046f89700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04015a5f0 0x7fc0400b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.422+0000 7fc046f89700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0401e3f50 con 0x7fc04015a5f0 2023-11-28T12:02:15.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.422+0000 7fc046f89700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0401e4230 con 0x7fc0400cc200 2023-11-28T12:02:15.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.422+0000 7fc046f89700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0400d0240 con 0x7fc0400b2ee0 2023-11-28T12:02:15.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc045526700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04015a5f0 0x7fc0400b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc045526700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04015a5f0 0x7fc0400b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58742/0 (socket says 172.21.15.57:58742) 2023-11-28T12:02:15.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc045526700 1 -- 172.21.15.57:0/1043039976 learned_addr learned my addr 172.21.15.57:0/1043039976 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:15.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc044d25700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0400b2ee0 0x7fc040150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc03ffff700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0400cc200 0x7fc0400b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc045526700 1 -- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0400b2ee0 msgr2=0x7fc040150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc045526700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0400b2ee0 0x7fc040150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc045526700 1 -- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0400cc200 msgr2=0x7fc0400b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc045526700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0400cc200 0x7fc0400b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc045526700 1 -- 172.21.15.57:0/1043039976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc038009b30 con 0x7fc04015a5f0 2023-11-28T12:02:15.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc045526700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04015a5f0 0x7fc0400b5600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc038012000 tx=0x7fc038004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:15.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc03d7fa700 1 -- 172.21.15.57:0/1043039976 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc038003ea0 con 0x7fc04015a5f0 2023-11-28T12:02:15.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc03d7fa700 1 -- 172.21.15.57:0/1043039976 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0380048e0 con 0x7fc04015a5f0 2023-11-28T12:02:15.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc046f89700 1 -- 172.21.15.57:0/1043039976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0400cd230 con 0x7fc04015a5f0 2023-11-28T12:02:15.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc03d7fa700 1 -- 172.21.15.57:0/1043039976 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03801b670 con 0x7fc04015a5f0 2023-11-28T12:02:15.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.423+0000 7fc046f89700 1 -- 172.21.15.57:0/1043039976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0400ccf50 con 0x7fc04015a5f0 2023-11-28T12:02:15.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.424+0000 7fc046f89700 1 -- 172.21.15.57:0/1043039976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0400cd510 con 0x7fc04015a5f0 2023-11-28T12:02:15.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.425+0000 7fc03d7fa700 1 -- 172.21.15.57:0/1043039976 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc03802b070 con 0x7fc04015a5f0 2023-11-28T12:02:15.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.425+0000 7fc03d7fa700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0200682f0 0x7fc02006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.425+0000 7fc03d7fa700 1 -- 172.21.15.57:0/1043039976 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc038087500 con 0x7fc04015a5f0 2023-11-28T12:02:15.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.426+0000 7fc044d25700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0200682f0 0x7fc02006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.426+0000 7fc044d25700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0200682f0 0x7fc02006a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc030009be0 tx=0x7fc030009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:15.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.430+0000 7fc03d7fa700 1 -- 172.21.15.57:0/1043039976 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc038053bf0 con 0x7fc04015a5f0 2023-11-28T12:02:15.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.630+0000 7fc046f89700 1 -- 172.21.15.57:0/1043039976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0400743f0 con 0x7fc04015a5f0 2023-11-28T12:02:15.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.630+0000 7fc03d7fa700 1 -- 172.21.15.57:0/1043039976 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc038057210 con 0x7fc04015a5f0 2023-11-28T12:02:15.632 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:02:15.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.633+0000 7fc046f89700 1 -- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0200682f0 msgr2=0x7fc02006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.633+0000 7fc046f89700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0200682f0 0x7fc02006a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc030009be0 tx=0x7fc030009210 comp rx=0 tx=0).stop 2023-11-28T12:02:15.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.633+0000 7fc046f89700 1 -- 172.21.15.57:0/1043039976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04015a5f0 msgr2=0x7fc0400b5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.633+0000 7fc046f89700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04015a5f0 0x7fc0400b5600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc038012000 tx=0x7fc038004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.633+0000 7fc046f89700 1 -- 172.21.15.57:0/1043039976 shutdown_connections 2023-11-28T12:02:15.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.633+0000 7fc046f89700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc0200682f0 0x7fc02006a7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.633+0000 7fc046f89700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04015a5f0 0x7fc0400b5600 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.633+0000 7fc046f89700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0400b2ee0 0x7fc040150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.633+0000 7fc046f89700 1 --2- 172.21.15.57:0/1043039976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0400cc200 0x7fc0400b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.633+0000 7fc046f89700 1 -- 172.21.15.57:0/1043039976 >> 172.21.15.57:0/1043039976 conn(0x7fc04006ea80 msgr2=0x7fc040154910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:15.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.633+0000 7fc046f89700 1 -- 172.21.15.57:0/1043039976 shutdown_connections 2023-11-28T12:02:15.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.633+0000 7fc046f89700 1 -- 172.21.15.57:0/1043039976 wait complete. 2023-11-28T12:02:15.655 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T12:02:15.656 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T12:02:15.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.790+0000 7f79ec959700 1 Processor -- start 2023-11-28T12:02:15.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.790+0000 7f79ec959700 1 -- start start 2023-11-28T12:02:15.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79ec959700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79e80ce040 0x7f79e815d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79ec959700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79e8142db0 0x7f79e813ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79ec959700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f79e80b1770 0x7f79e80b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79ec959700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e80b2130 con 0x7f79e80b1770 2023-11-28T12:02:15.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79ec959700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e813fc90 con 0x7f79e8142db0 2023-11-28T12:02:15.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79ec959700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e813f6d0 con 0x7f79e80ce040 2023-11-28T12:02:15.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79e6d9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f79e80b1770 0x7f79e80b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79e6d9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f79e80b1770 0x7f79e80b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58756/0 (socket says 172.21.15.57:58756) 2023-11-28T12:02:15.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79e6d9d700 1 -- 172.21.15.57:0/194923290 learned_addr learned my addr 172.21.15.57:0/194923290 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:15.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79e659c700 1 --2- 172.21.15.57:0/194923290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79e80ce040 0x7f79e815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79e5d9b700 1 --2- 172.21.15.57:0/194923290 >> v2:172.21.15.181:3300/0 conn(0x7f79e8142db0 0x7f79e813ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79e6d9d700 1 -- 172.21.15.57:0/194923290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79e80ce040 msgr2=0x7f79e815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:15.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79e6d9d700 1 --2- 172.21.15.57:0/194923290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79e80ce040 0x7f79e815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79e6d9d700 1 -- 172.21.15.57:0/194923290 >> v2:172.21.15.181:3300/0 conn(0x7f79e8142db0 msgr2=0x7f79e813ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:15.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79e6d9d700 1 --2- 172.21.15.57:0/194923290 >> v2:172.21.15.181:3300/0 conn(0x7f79e8142db0 0x7f79e813ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.792+0000 7f79e6d9d700 1 -- 172.21.15.57:0/194923290 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e813f9b0 con 0x7f79e80b1770 2023-11-28T12:02:15.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.793+0000 7f79e6d9d700 1 --2- 172.21.15.57:0/194923290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79e80b1770 0x7f79e80b1bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f79e806c6c0 tx=0x7f79e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb03e19baba0311 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:15.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.793+0000 7f79e559a700 1 -- 172.21.15.57:0/194923290 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000e070 con 0x7f79e80b1770 2023-11-28T12:02:15.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.793+0000 7f79e559a700 1 -- 172.21.15.57:0/194923290 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e000b9c0 con 0x7f79e80b1770 2023-11-28T12:02:15.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.793+0000 7f79e559a700 1 -- 172.21.15.57:0/194923290 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000bd70 con 0x7f79e80b1770 2023-11-28T12:02:15.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.793+0000 7f79ec959700 1 -- 172.21.15.57:0/194923290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79e80b1770 msgr2=0x7f79e80b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.793+0000 7f79ec959700 1 --2- 172.21.15.57:0/194923290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79e80b1770 0x7f79e80b1bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f79e806c6c0 tx=0x7f79e000b330 comp rx=0 tx=0).stop 2023-11-28T12:02:15.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.794+0000 7f79ec959700 1 -- 172.21.15.57:0/194923290 shutdown_connections 2023-11-28T12:02:15.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.794+0000 7f79ec959700 1 --2- 172.21.15.57:0/194923290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79e80ce040 0x7f79e815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.794+0000 7f79ec959700 1 --2- 172.21.15.57:0/194923290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79e80b1770 0x7f79e80b1bf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.794+0000 7f79ec959700 1 --2- 172.21.15.57:0/194923290 >> v2:172.21.15.181:3300/0 conn(0x7f79e8142db0 0x7f79e813ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.794+0000 7f79ec959700 1 -- 172.21.15.57:0/194923290 >> 172.21.15.57:0/194923290 conn(0x7f79e806c010 msgr2=0x7f79e80cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:15.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.794+0000 7f79ec959700 1 -- 172.21.15.57:0/194923290 shutdown_connections 2023-11-28T12:02:15.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.794+0000 7f79ec959700 1 -- 172.21.15.57:0/194923290 wait complete. 2023-11-28T12:02:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.795+0000 7f79ec959700 1 Processor -- start 2023-11-28T12:02:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.795+0000 7f79ec959700 1 -- start start 2023-11-28T12:02:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79ec959700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79e80b1770 0x7f79e80b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79ec959700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79e80ce040 0x7f79e81f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79ec959700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79e8142db0 0x7f79e81e4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79ec959700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e81e5170 con 0x7f79e80b1770 2023-11-28T12:02:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79ec959700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e81e5450 con 0x7f79e8142db0 2023-11-28T12:02:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79ec959700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e80b3b70 con 0x7f79e80ce040 2023-11-28T12:02:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79e659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79e80b1770 0x7f79e80b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79e659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79e80b1770 0x7f79e80b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58770/0 (socket says 172.21.15.57:58770) 2023-11-28T12:02:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79e659c700 1 -- 172.21.15.57:0/3839083741 learned_addr learned my addr 172.21.15.57:0/3839083741 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79e5d9b700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79e80ce040 0x7f79e81f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79e6d9d700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79e8142db0 0x7f79e81e4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79e659c700 1 -- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79e80ce040 msgr2=0x7f79e81f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79e659c700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79e80ce040 0x7f79e81f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79e659c700 1 -- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79e8142db0 msgr2=0x7f79e81e4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79e659c700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79e8142db0 0x7f79e81e4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79e659c700 1 -- 172.21.15.57:0/3839083741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e0009b30 con 0x7f79e80b1770 2023-11-28T12:02:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79e6d9d700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79e8142db0 0x7f79e81e4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.796+0000 7f79e659c700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79e80b1770 0x7f79e80b4650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e810 tx=0x7f79d800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.797+0000 7f79d6ffd700 1 -- 172.21.15.57:0/3839083741 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8013610 con 0x7f79e80b1770 2023-11-28T12:02:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.797+0000 7f79d6ffd700 1 -- 172.21.15.57:0/3839083741 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d8013770 con 0x7f79e80b1770 2023-11-28T12:02:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.797+0000 7f79ec959700 1 -- 172.21.15.57:0/3839083741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e815d3e0 con 0x7f79e80b1770 2023-11-28T12:02:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.797+0000 7f79d6ffd700 1 -- 172.21.15.57:0/3839083741 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8013b00 con 0x7f79e80b1770 2023-11-28T12:02:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.797+0000 7f79ec959700 1 -- 172.21.15.57:0/3839083741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e815d100 con 0x7f79e80b1770 2023-11-28T12:02:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.798+0000 7f79ec959700 1 -- 172.21.15.57:0/3839083741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e80ce410 con 0x7f79e80b1770 2023-11-28T12:02:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.798+0000 7f79d6ffd700 1 -- 172.21.15.57:0/3839083741 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f79d8023070 con 0x7f79e80b1770 2023-11-28T12:02:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.799+0000 7f79d6ffd700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f79c80682f0 0x7f79c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.799+0000 7f79d6ffd700 1 -- 172.21.15.57:0/3839083741 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f79d80858c0 con 0x7f79e80b1770 2023-11-28T12:02:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.799+0000 7f79e5d9b700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f79c80682f0 0x7f79c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.799+0000 7f79e5d9b700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f79c80682f0 0x7f79c806a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f79e80b1620 tx=0x7f79dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:15.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:15.803+0000 7f79d6ffd700 1 -- 172.21.15.57:0/3839083741 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f79d8052120 con 0x7f79e80b1770 2023-11-28T12:02:16.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.089+0000 7f79ec959700 1 -- 172.21.15.57:0/3839083741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f79e80b44a0 con 0x7f79e80b1770 2023-11-28T12:02:16.182 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:02:16.185 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:02:16.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.782+0000 7f79d6ffd700 1 -- 172.21.15.57:0/3839083741 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v48) v1 ==== 145+0+0 (secure 0 0 0) 0x7f79d8009440 con 0x7f79e80b1770 2023-11-28T12:02:16.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.785+0000 7f79ec959700 1 -- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f79c80682f0 msgr2=0x7f79c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:16.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.785+0000 7f79ec959700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f79c80682f0 0x7f79c806a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f79e80b1620 tx=0x7f79dc008040 comp rx=0 tx=0).stop 2023-11-28T12:02:16.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.785+0000 7f79ec959700 1 -- 172.21.15.57:0/3839083741 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79e80b1770 msgr2=0x7f79e80b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:16.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.785+0000 7f79ec959700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79e80b1770 0x7f79e80b4650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e810 tx=0x7f79d800ebd0 comp rx=0 tx=0).stop 2023-11-28T12:02:16.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.785+0000 7f79ec959700 1 -- 172.21.15.57:0/3839083741 shutdown_connections 2023-11-28T12:02:16.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.785+0000 7f79ec959700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f79c80682f0 0x7f79c806a7a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:16.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.785+0000 7f79ec959700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79e8142db0 0x7f79e81e4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:16.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.785+0000 7f79ec959700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79e80b1770 0x7f79e80b4650 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:16.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.785+0000 7f79ec959700 1 --2- 172.21.15.57:0/3839083741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79e80ce040 0x7f79e81f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:16.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.785+0000 7f79ec959700 1 -- 172.21.15.57:0/3839083741 >> 172.21.15.57:0/3839083741 conn(0x7f79e806c010 msgr2=0x7f79e8071630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:16.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.785+0000 7f79ec959700 1 -- 172.21.15.57:0/3839083741 shutdown_connections 2023-11-28T12:02:16.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.785+0000 7f79ec959700 1 -- 172.21.15.57:0/3839083741 wait complete. 2023-11-28T12:02:16.802 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T12:02:16.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.942+0000 7f8a1a883700 1 Processor -- start 2023-11-28T12:02:16.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.942+0000 7f8a1a883700 1 -- start start 2023-11-28T12:02:16.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.943+0000 7f8a1a883700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8a1415c8c0 0x7f8a14160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:16.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.943+0000 7f8a1a883700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a1406b220 0x7f8a140d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:16.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.943+0000 7f8a1a883700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a140d2d30 con 0x7f8a140b5dc0 2023-11-28T12:02:16.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.943+0000 7f8a1a883700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a140d0b40 con 0x7f8a1415c8c0 2023-11-28T12:02:16.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.943+0000 7f8a1a883700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141611f0 con 0x7f8a1406b220 2023-11-28T12:02:16.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.943+0000 7f8a13fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8a1415c8c0 0x7f8a14160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:16.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.943+0000 7f8a0bfff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8a140b5dc0 0x7f8a1406ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45786/0 (socket says 172.21.15.57:45786) 2023-11-28T12:02:16.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.943+0000 7f8a0bfff700 1 -- 172.21.15.57:0/518207441 learned_addr learned my addr 172.21.15.57:0/518207441 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:16.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.944+0000 7f8a18e20700 1 --2- 172.21.15.57:0/518207441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a1406b220 0x7f8a140d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:16.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.944+0000 7f8a13fff700 1 -- 172.21.15.57:0/518207441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a1406b220 msgr2=0x7f8a140d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:16.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.944+0000 7f8a137fe700 1 -- 172.21.15.57:0/518207441 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329309760 0 0) 0x7f89fc001110 con 0x7f8a140b5dc0 2023-11-28T12:02:16.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.944+0000 7f8a13fff700 1 --2- 172.21.15.57:0/518207441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a1406b220 0x7f8a140d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:16.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.944+0000 7f8a13fff700 1 -- 172.21.15.57:0/518207441 >> v1:172.21.15.57:6789/0 conn(0x7f8a140b5dc0 legacy=0x7f8a1406ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:16.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.944+0000 7f8a13fff700 1 -- 172.21.15.57:0/518207441 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a140d2a50 con 0x7f8a1415c8c0 2023-11-28T12:02:16.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.944+0000 7f8a13fff700 1 --2- 172.21.15.57:0/518207441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a1415c8c0 0x7f8a14160cb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009840 tx=0x7f8a04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257a2de9dad06bbc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:16.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.944+0000 7f8a137fe700 1 -- 172.21.15.57:0/518207441 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04011830 con 0x7f8a1415c8c0 2023-11-28T12:02:16.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.944+0000 7f8a137fe700 1 -- 172.21.15.57:0/518207441 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a04011990 con 0x7f8a1415c8c0 2023-11-28T12:02:16.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.944+0000 7f8a137fe700 1 -- 172.21.15.57:0/518207441 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04011d20 con 0x7f8a1415c8c0 2023-11-28T12:02:16.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.945+0000 7f8a1a883700 1 -- 172.21.15.57:0/518207441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a1415c8c0 msgr2=0x7f8a14160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:16.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.945+0000 7f8a1a883700 1 --2- 172.21.15.57:0/518207441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a1415c8c0 0x7f8a14160cb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009840 tx=0x7f8a04009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:16.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.945+0000 7f8a1a883700 1 -- 172.21.15.57:0/518207441 shutdown_connections 2023-11-28T12:02:16.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.945+0000 7f8a1a883700 1 --2- 172.21.15.57:0/518207441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a1415c8c0 0x7f8a14160cb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:16.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.945+0000 7f8a1a883700 1 --2- 172.21.15.57:0/518207441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a1406b220 0x7f8a140d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:16.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.945+0000 7f8a1a883700 1 -- 172.21.15.57:0/518207441 >> 172.21.15.57:0/518207441 conn(0x7f8a1406c520 msgr2=0x7f8a1406c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:16.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.945+0000 7f8a1a883700 1 -- 172.21.15.57:0/518207441 shutdown_connections 2023-11-28T12:02:16.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.945+0000 7f8a1a883700 1 -- 172.21.15.57:0/518207441 wait complete. 2023-11-28T12:02:16.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.947+0000 7f8a1a883700 1 Processor -- start 2023-11-28T12:02:16.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.947+0000 7f8a1a883700 1 -- start start 2023-11-28T12:02:16.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a1a883700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a1406b220 0x7f8a1415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:16.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a1a883700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a140b5dc0 0x7f8a1414d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:16.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a1a883700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a1415c8c0 0x7f8a140b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:16.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a1a883700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140cb0c0 con 0x7f8a140b5dc0 2023-11-28T12:02:16.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a1a883700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1414da80 con 0x7f8a1406b220 2023-11-28T12:02:16.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a1a883700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140ca4b0 con 0x7f8a1415c8c0 2023-11-28T12:02:16.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a0bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a140b5dc0 0x7f8a1414d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:16.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a0bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a140b5dc0 0x7f8a1414d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58784/0 (socket says 172.21.15.57:58784) 2023-11-28T12:02:16.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a0bfff700 1 -- 172.21.15.57:0/1331913972 learned_addr learned my addr 172.21.15.57:0/1331913972 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:16.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a13fff700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a1406b220 0x7f8a1415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:16.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a18e20700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a1415c8c0 0x7f8a140b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:16.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a0bfff700 1 -- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a1415c8c0 msgr2=0x7f8a140b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:16.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a0bfff700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a1415c8c0 0x7f8a140b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:16.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a0bfff700 1 -- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a1406b220 msgr2=0x7f8a1415e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:16.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a0bfff700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a1406b220 0x7f8a1415e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:16.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a0bfff700 1 -- 172.21.15.57:0/1331913972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a040094a0 con 0x7f8a140b5dc0 2023-11-28T12:02:16.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a18e20700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a1415c8c0 0x7f8a140b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:16.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.948+0000 7f8a0bfff700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a140b5dc0 0x7f8a1414d540 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c760 tx=0x7f89fc00ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:16.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.949+0000 7f8a117fa700 1 -- 172.21.15.57:0/1331913972 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc004c80 con 0x7f8a140b5dc0 2023-11-28T12:02:16.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.949+0000 7f8a117fa700 1 -- 172.21.15.57:0/1331913972 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc004de0 con 0x7f8a140b5dc0 2023-11-28T12:02:16.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.949+0000 7f8a117fa700 1 -- 172.21.15.57:0/1331913972 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc007310 con 0x7f8a140b5dc0 2023-11-28T12:02:16.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.949+0000 7f8a1a883700 1 -- 172.21.15.57:0/1331913972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a140cda40 con 0x7f8a140b5dc0 2023-11-28T12:02:16.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.949+0000 7f8a1a883700 1 -- 172.21.15.57:0/1331913972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a140cd760 con 0x7f8a140b5dc0 2023-11-28T12:02:16.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.950+0000 7f8a1a883700 1 -- 172.21.15.57:0/1331913972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a140d2700 con 0x7f8a140b5dc0 2023-11-28T12:02:16.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.950+0000 7f8a117fa700 1 -- 172.21.15.57:0/1331913972 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f89fc01a020 con 0x7f8a140b5dc0 2023-11-28T12:02:16.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.950+0000 7f8a117fa700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89f40682f0 0x7f89f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:16.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.951+0000 7f8a117fa700 1 -- 172.21.15.57:0/1331913972 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f89fc0856c0 con 0x7f8a140b5dc0 2023-11-28T12:02:16.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.951+0000 7f8a13fff700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89f40682f0 0x7f89f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:16.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.951+0000 7f8a13fff700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89f40682f0 0x7f89f406a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a040090b0 tx=0x7f8a0401c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:16.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:16.955+0000 7f8a117fa700 1 -- 172.21.15.57:0/1331913972 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f89fc051f20 con 0x7f8a140b5dc0 2023-11-28T12:02:17.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.156+0000 7f8a1a883700 1 -- 172.21.15.57:0/1331913972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8a14067d70 con 0x7f8a140b5dc0 2023-11-28T12:02:17.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.160+0000 7f8a117fa700 1 -- 172.21.15.57:0/1331913972 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f89fc055540 con 0x7f8a140b5dc0 2023-11-28T12:02:17.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.162+0000 7f8a1a883700 1 -- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89f40682f0 msgr2=0x7f89f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.162+0000 7f8a1a883700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89f40682f0 0x7f89f406a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a040090b0 tx=0x7f8a0401c040 comp rx=0 tx=0).stop 2023-11-28T12:02:17.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.162+0000 7f8a1a883700 1 -- 172.21.15.57:0/1331913972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a140b5dc0 msgr2=0x7f8a1414d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.162+0000 7f8a1a883700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a140b5dc0 0x7f8a1414d540 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c760 tx=0x7f89fc00ca70 comp rx=0 tx=0).stop 2023-11-28T12:02:17.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.162+0000 7f8a1a883700 1 -- 172.21.15.57:0/1331913972 shutdown_connections 2023-11-28T12:02:17.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.162+0000 7f8a1a883700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89f40682f0 0x7f89f406a7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.162+0000 7f8a1a883700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a1415c8c0 0x7f8a140b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.162+0000 7f8a1a883700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a1406b220 0x7f8a1415e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.162+0000 7f8a1a883700 1 --2- 172.21.15.57:0/1331913972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a140b5dc0 0x7f8a1414d540 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.162+0000 7f8a1a883700 1 -- 172.21.15.57:0/1331913972 >> 172.21.15.57:0/1331913972 conn(0x7f8a1406c520 msgr2=0x7f8a14157110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:17.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.162+0000 7f8a1a883700 1 -- 172.21.15.57:0/1331913972 shutdown_connections 2023-11-28T12:02:17.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.162+0000 7f8a1a883700 1 -- 172.21.15.57:0/1331913972 wait complete. 2023-11-28T12:02:17.177 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T12:02:17.177 DEBUG:teuthology.orchestra.run.smithi057:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T12:02:17.221 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T12:02:17.274 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T12:02:17.275 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:02:17.275 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:02:17.297 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:02:17.297 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:02:17.361 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:17.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.542+0000 7fc6fe60e700 1 Processor -- start 2023-11-28T12:02:17.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.542+0000 7fc6fe60e700 1 -- start start 2023-11-28T12:02:17.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.543+0000 7fc6fe60e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f8070d90 0x7fc6f80711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.543+0000 7fc6fe60e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f8151a50 con 0x7fc6f815e0b0 2023-11-28T12:02:17.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.543+0000 7fc6fe60e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f815b440 con 0x7fc6f806a270 2023-11-28T12:02:17.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.543+0000 7fc6fe60e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f8071ac0 con 0x7fc6f8070d90 2023-11-28T12:02:17.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.543+0000 7fc6fcbab700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc6f815e0b0 0x7fc6f815e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45802/0 (socket says 172.21.15.57:45802) 2023-11-28T12:02:17.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.543+0000 7fc6fcbab700 1 -- 172.21.15.57:0/3545636811 learned_addr learned my addr 172.21.15.57:0/3545636811 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:17.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f77fe700 1 --2- 172.21.15.57:0/3545636811 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f8070d90 0x7fc6f80711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510802599 0 0) 0x7fc6f8151a50 con 0x7fc6f815e0b0 2023-11-28T12:02:17.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6e00035b0 con 0x7fc6f815e0b0 2023-11-28T12:02:17.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370530231 0 0) 0x7fc6f815b440 con 0x7fc6f806a270 2023-11-28T12:02:17.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f8151a50 con 0x7fc6f806a270 2023-11-28T12:02:17.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 927878811 0 0) 0x7fc6e00035b0 con 0x7fc6f815e0b0 2023-11-28T12:02:17.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f8070d90 msgr2=0x7fc6f80711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:17.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 --2- 172.21.15.57:0/3545636811 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f8070d90 0x7fc6f80711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 >> v1:172.21.15.181:6789/0 conn(0x7fc6f806a270 legacy=0x7fc6f806a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f815adf0 con 0x7fc6f815e0b0 2023-11-28T12:02:17.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fc6f00019f0 con 0x7fc6f815e0b0 2023-11-28T12:02:17.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 >> v1:172.21.15.57:6789/0 conn(0x7fc6f815e0b0 legacy=0x7fc6f815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 --2- 172.21.15.57:0/3545636811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6e0003bc0 0x7fc6e0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 --2- 172.21.15.57:0/3545636811 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6e0004850 0x7fc6e0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 --2- 172.21.15.57:0/3545636811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e0005200 0x7fc6e000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f8071ac0 con 0x7fc6e0005200 2023-11-28T12:02:17.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e000c030 con 0x7fc6e0003bc0 2023-11-28T12:02:17.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e000c1f0 con 0x7fc6e0004850 2023-11-28T12:02:17.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f77fe700 1 --2- 172.21.15.57:0/3545636811 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f8070d90 0x7fc6f80711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:17.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6fcbab700 1 --2- 172.21.15.57:0/3545636811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e0005200 0x7fc6e000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f7fff700 1 --2- 172.21.15.57:0/3545636811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6e0003bc0 0x7fc6e0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.544+0000 7fc6f77fe700 1 --2- 172.21.15.57:0/3545636811 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6e0004850 0x7fc6e0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.545+0000 7fc6fcbab700 1 -- 172.21.15.57:0/3545636811 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6e0004850 msgr2=0x7fc6e0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:17.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.545+0000 7fc6fcbab700 1 --2- 172.21.15.57:0/3545636811 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6e0004850 0x7fc6e0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.545+0000 7fc6fcbab700 1 -- 172.21.15.57:0/3545636811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6e0003bc0 msgr2=0x7fc6e0003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:17.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.545+0000 7fc6fcbab700 1 --2- 172.21.15.57:0/3545636811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6e0003bc0 0x7fc6e0003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.545+0000 7fc6fcbab700 1 -- 172.21.15.57:0/3545636811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e00035b0 con 0x7fc6e0005200 2023-11-28T12:02:17.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.545+0000 7fc6fcbab700 1 --2- 172.21.15.57:0/3545636811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e0005200 0x7fc6e000ba10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000a710 tx=0x7fc6e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca1160254d55be5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.545+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f00072d0 con 0x7fc6e0005200 2023-11-28T12:02:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.545+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f0007430 con 0x7fc6e0005200 2023-11-28T12:02:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.545+0000 7fc6f6ffd700 1 -- 172.21.15.57:0/3545636811 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f00077e0 con 0x7fc6e0005200 2023-11-28T12:02:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.545+0000 7fc6fe60e700 1 -- 172.21.15.57:0/3545636811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e0005200 msgr2=0x7fc6e000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.545+0000 7fc6fe60e700 1 --2- 172.21.15.57:0/3545636811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e0005200 0x7fc6e000ba10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000a710 tx=0x7fc6e0005880 comp rx=0 tx=0).stop 2023-11-28T12:02:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.546+0000 7fc6fcbab700 1 -- 172.21.15.57:0/3545636811 reap_dead start 2023-11-28T12:02:17.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.546+0000 7fc6fe60e700 1 -- 172.21.15.57:0/3545636811 shutdown_connections 2023-11-28T12:02:17.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.546+0000 7fc6fe60e700 1 -- 172.21.15.57:0/3545636811 >> 172.21.15.57:0/3545636811 conn(0x7fc6f806c010 msgr2=0x7fc6f80cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:17.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.546+0000 7fc6fe60e700 1 -- 172.21.15.57:0/3545636811 shutdown_connections 2023-11-28T12:02:17.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.546+0000 7fc6fe60e700 1 -- 172.21.15.57:0/3545636811 wait complete. 2023-11-28T12:02:17.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.547+0000 7fc6fe60e700 1 Processor -- start 2023-11-28T12:02:17.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.547+0000 7fc6fe60e700 1 -- start start 2023-11-28T12:02:17.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fe60e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f814bd00 0x7fc6f814c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fe60e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f80b4b00 0x7fc6f80ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fe60e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f80b0120 0x7fc6f80b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fe60e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f80b0ae0 con 0x7fc6f80b0120 2023-11-28T12:02:17.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fe60e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f80ac8f0 con 0x7fc6f814bd00 2023-11-28T12:02:17.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fe60e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f80b5350 con 0x7fc6f80b4b00 2023-11-28T12:02:17.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fcbab700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f80b0120 0x7fc6f80b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fcbab700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f80b0120 0x7fc6f80b05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58810/0 (socket says 172.21.15.57:58810) 2023-11-28T12:02:17.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fcbab700 1 -- 172.21.15.57:0/1397081113 learned_addr learned my addr 172.21.15.57:0/1397081113 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:17.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6f77fe700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f80b4b00 0x7fc6f80ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6f7fff700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f814bd00 0x7fc6f814c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fcbab700 1 -- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f80b4b00 msgr2=0x7fc6f80ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fcbab700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f80b4b00 0x7fc6f80ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fcbab700 1 -- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f814bd00 msgr2=0x7fc6f814c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fcbab700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f814bd00 0x7fc6f814c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.548+0000 7fc6fcbab700 1 -- 172.21.15.57:0/1397081113 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f815aac0 con 0x7fc6f80b0120 2023-11-28T12:02:17.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.549+0000 7fc6fcbab700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f80b0120 0x7fc6f80b05a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000c000 tx=0x7fc6f0001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:17.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.549+0000 7fc6f4ff9700 1 -- 172.21.15.57:0/1397081113 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f00079b0 con 0x7fc6f80b0120 2023-11-28T12:02:17.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.549+0000 7fc6f4ff9700 1 -- 172.21.15.57:0/1397081113 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f0007b10 con 0x7fc6f80b0120 2023-11-28T12:02:17.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.549+0000 7fc6fe60e700 1 -- 172.21.15.57:0/1397081113 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f815b0d0 con 0x7fc6f80b0120 2023-11-28T12:02:17.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.549+0000 7fc6f4ff9700 1 -- 172.21.15.57:0/1397081113 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0012620 con 0x7fc6f80b0120 2023-11-28T12:02:17.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.549+0000 7fc6fe60e700 1 -- 172.21.15.57:0/1397081113 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f80b23e0 con 0x7fc6f80b0120 2023-11-28T12:02:17.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.550+0000 7fc6fe60e700 1 -- 172.21.15.57:0/1397081113 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f80743f0 con 0x7fc6f80b0120 2023-11-28T12:02:17.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.550+0000 7fc6f4ff9700 1 -- 172.21.15.57:0/1397081113 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc6f0010070 con 0x7fc6f80b0120 2023-11-28T12:02:17.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.551+0000 7fc6f4ff9700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc6e40682f0 0x7fc6e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.551+0000 7fc6f4ff9700 1 -- 172.21.15.57:0/1397081113 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6f0087590 con 0x7fc6f80b0120 2023-11-28T12:02:17.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.551+0000 7fc6f7fff700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc6e40682f0 0x7fc6e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.552+0000 7fc6f7fff700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc6e40682f0 0x7fc6e406a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8000c00 tx=0x7fc6e8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:17.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.555+0000 7fc6f4ff9700 1 -- 172.21.15.57:0/1397081113 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc6f0053eb0 con 0x7fc6f80b0120 2023-11-28T12:02:17.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.754+0000 7fc6fe60e700 1 -- 172.21.15.57:0/1397081113 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6f80cd520 con 0x7fc6f80b0120 2023-11-28T12:02:17.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.756+0000 7fc6f4ff9700 1 -- 172.21.15.57:0/1397081113 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7fc6f00574d0 con 0x7fc6f80b0120 2023-11-28T12:02:17.757 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:17.758 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:17.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.758+0000 7fc6fe60e700 1 -- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc6e40682f0 msgr2=0x7fc6e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.758+0000 7fc6fe60e700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc6e40682f0 0x7fc6e406a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8000c00 tx=0x7fc6e8009040 comp rx=0 tx=0).stop 2023-11-28T12:02:17.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.758+0000 7fc6fe60e700 1 -- 172.21.15.57:0/1397081113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f80b0120 msgr2=0x7fc6f80b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.758+0000 7fc6fe60e700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f80b0120 0x7fc6f80b05a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000c000 tx=0x7fc6f0001cb0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.758+0000 7fc6fe60e700 1 -- 172.21.15.57:0/1397081113 shutdown_connections 2023-11-28T12:02:17.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.758+0000 7fc6fe60e700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc6e40682f0 0x7fc6e406a7a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.758+0000 7fc6fe60e700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f80b0120 0x7fc6f80b05a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.758+0000 7fc6fe60e700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f814bd00 0x7fc6f814c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.758+0000 7fc6fe60e700 1 --2- 172.21.15.57:0/1397081113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f80b4b00 0x7fc6f80ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.758+0000 7fc6fe60e700 1 -- 172.21.15.57:0/1397081113 >> 172.21.15.57:0/1397081113 conn(0x7fc6f806c010 msgr2=0x7fc6f806db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:17.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.758+0000 7fc6fe60e700 1 -- 172.21.15.57:0/1397081113 shutdown_connections 2023-11-28T12:02:17.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.758+0000 7fc6fe60e700 1 -- 172.21.15.57:0/1397081113 wait complete. 2023-11-28T12:02:17.773 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:17.774 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:17.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.911+0000 7f89ace8f700 1 Processor -- start 2023-11-28T12:02:17.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.911+0000 7f89ace8f700 1 -- start start 2023-11-28T12:02:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89ace8f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f89a8160300 0x7f89a815d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89ace8f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a815ceb0 con 0x7f89a815d880 2023-11-28T12:02:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89ace8f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a81608c0 con 0x7f89a80cec00 2023-11-28T12:02:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89ace8f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a8160ba0 con 0x7f89a8160300 2023-11-28T12:02:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a6d9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f89a815d880 0x7f89a80cac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45810/0 (socket says 172.21.15.57:45810) 2023-11-28T12:02:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a6d9d700 1 -- 172.21.15.57:0/881325188 learned_addr learned my addr 172.21.15.57:0/881325188 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a5d9b700 1 --2- 172.21.15.57:0/881325188 >> v2:172.21.15.181:3300/0 conn(0x7f89a8160300 0x7f89a815d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161421051 0 0) 0x7f89a815ceb0 con 0x7f89a815d880 2023-11-28T12:02:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89900035b0 con 0x7f89a815d880 2023-11-28T12:02:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2741378249 0 0) 0x7f89900035b0 con 0x7f89a815d880 2023-11-28T12:02:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 >> v2:172.21.15.181:3300/0 conn(0x7f89a8160300 msgr2=0x7f89a815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a559a700 1 --2- 172.21.15.57:0/881325188 >> v2:172.21.15.181:3300/0 conn(0x7f89a8160300 0x7f89a815d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 >> v1:172.21.15.181:6789/0 conn(0x7f89a80cec00 legacy=0x7f89a8065fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a8160e80 con 0x7f89a815d880 2023-11-28T12:02:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f89a00019f0 con 0x7f89a815d880 2023-11-28T12:02:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 >> v1:172.21.15.57:6789/0 conn(0x7f89a815d880 legacy=0x7f89a80cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a559a700 1 --2- 172.21.15.57:0/881325188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8990003a00 0x7f8990003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a559a700 1 --2- 172.21.15.57:0/881325188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89900043f0 0x7f89900048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a559a700 1 --2- 172.21.15.57:0/881325188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89900050b0 0x7f899000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.912+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a8160ba0 con 0x7f8990003a00 2023-11-28T12:02:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899000c000 con 0x7f89900043f0 2023-11-28T12:02:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899000c1c0 con 0x7f89900050b0 2023-11-28T12:02:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a659c700 1 --2- 172.21.15.57:0/881325188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8990003a00 0x7f8990003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a5d9b700 1 --2- 172.21.15.57:0/881325188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89900043f0 0x7f89900048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a6d9d700 1 --2- 172.21.15.57:0/881325188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89900050b0 0x7f899000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a659c700 1 -- 172.21.15.57:0/881325188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89900050b0 msgr2=0x7f899000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a659c700 1 --2- 172.21.15.57:0/881325188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89900050b0 0x7f899000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a659c700 1 -- 172.21.15.57:0/881325188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89900043f0 msgr2=0x7f89900048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:17.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a659c700 1 --2- 172.21.15.57:0/881325188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89900043f0 0x7f89900048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a6d9d700 1 -- 172.21.15.57:0/881325188 reap_dead start 2023-11-28T12:02:17.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a659c700 1 -- 172.21.15.57:0/881325188 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89900035b0 con 0x7f8990003a00 2023-11-28T12:02:17.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a5d9b700 1 --2- 172.21.15.57:0/881325188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89900043f0 0x7f89900048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:17.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a659c700 1 --2- 172.21.15.57:0/881325188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8990003a00 0x7f8990003dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f89980018d0 tx=0x7f8990005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4067760c8f51cdc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:17.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89980052d0 con 0x7f8990003a00 2023-11-28T12:02:17.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8998005430 con 0x7f8990003a00 2023-11-28T12:02:17.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.913+0000 7f89a559a700 1 -- 172.21.15.57:0/881325188 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89980038c0 con 0x7f8990003a00 2023-11-28T12:02:17.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.914+0000 7f89ace8f700 1 -- 172.21.15.57:0/881325188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8990003a00 msgr2=0x7f8990003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.914+0000 7f89ace8f700 1 --2- 172.21.15.57:0/881325188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8990003a00 0x7f8990003dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f89980018d0 tx=0x7f8990005880 comp rx=0 tx=0).stop 2023-11-28T12:02:17.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.914+0000 7f89ace8f700 1 -- 172.21.15.57:0/881325188 shutdown_connections 2023-11-28T12:02:17.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.914+0000 7f89ace8f700 1 --2- 172.21.15.57:0/881325188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8990003a00 0x7f8990003dd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.914+0000 7f89ace8f700 1 -- 172.21.15.57:0/881325188 >> 172.21.15.57:0/881325188 conn(0x7f89a80cf400 msgr2=0x7f89a80cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:17.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.914+0000 7f89ace8f700 1 -- 172.21.15.57:0/881325188 shutdown_connections 2023-11-28T12:02:17.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.914+0000 7f89ace8f700 1 -- 172.21.15.57:0/881325188 wait complete. 2023-11-28T12:02:17.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.916+0000 7f89ace8f700 1 Processor -- start 2023-11-28T12:02:17.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.916+0000 7f89ace8f700 1 -- start start 2023-11-28T12:02:17.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.916+0000 7f89ace8f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8990003a00 0x7f89a815be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.916+0000 7f89ace8f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89a80c5a30 0x7f89a80712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89ace8f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89a8071800 0x7f89a80c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89ace8f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a8071c80 con 0x7f89a80c5a30 2023-11-28T12:02:17.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89ace8f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80c55a0 con 0x7f8990003a00 2023-11-28T12:02:17.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89ace8f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80c4990 con 0x7f89a8071800 2023-11-28T12:02:17.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89a5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89a80c5a30 0x7f89a80712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89a5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89a80c5a30 0x7f89a80712c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58814/0 (socket says 172.21.15.57:58814) 2023-11-28T12:02:17.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89a5d9b700 1 -- 172.21.15.57:0/270856944 learned_addr learned my addr 172.21.15.57:0/270856944 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:17.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89a659c700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8990003a00 0x7f89a815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89a6d9d700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89a8071800 0x7f89a80c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89a5d9b700 1 -- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89a8071800 msgr2=0x7f89a80c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89a5d9b700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89a8071800 0x7f89a80c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89a5d9b700 1 -- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8990003a00 msgr2=0x7f89a815be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:17.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89a5d9b700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8990003a00 0x7f89a815be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:17.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89a5d9b700 1 -- 172.21.15.57:0/270856944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a80cefd0 con 0x7f89a80c5a30 2023-11-28T12:02:17.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89a5d9b700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89a80c5a30 0x7f89a80712c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f899c011c00 tx=0x7f899c009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:17.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f8996ffd700 1 -- 172.21.15.57:0/270856944 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c00fab0 con 0x7f89a80c5a30 2023-11-28T12:02:17.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f8996ffd700 1 -- 172.21.15.57:0/270856944 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c00fc10 con 0x7f89a80c5a30 2023-11-28T12:02:17.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89ace8f700 1 -- 172.21.15.57:0/270856944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a8161160 con 0x7f89a80c5a30 2023-11-28T12:02:17.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f8996ffd700 1 -- 172.21.15.57:0/270856944 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c004020 con 0x7f89a80c5a30 2023-11-28T12:02:17.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.917+0000 7f89ace8f700 1 -- 172.21.15.57:0/270856944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a80cf160 con 0x7f89a80c5a30 2023-11-28T12:02:17.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.918+0000 7f89ace8f700 1 -- 172.21.15.57:0/270856944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a80c5730 con 0x7f89a80c5a30 2023-11-28T12:02:17.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.919+0000 7f8996ffd700 1 -- 172.21.15.57:0/270856944 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f899c024070 con 0x7f89a80c5a30 2023-11-28T12:02:17.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.919+0000 7f8996ffd700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89880682f0 0x7f898806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:17.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.919+0000 7f8996ffd700 1 -- 172.21.15.57:0/270856944 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f899c085f80 con 0x7f89a80c5a30 2023-11-28T12:02:17.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.920+0000 7f89a659c700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89880682f0 0x7f898806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:17.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.920+0000 7f89a659c700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89880682f0 0x7f898806a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8998015040 tx=0x7f8998005000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:17.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:17.924+0000 7f8996ffd700 1 -- 172.21.15.57:0/270856944 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f899c0526f0 con 0x7f89a80c5a30 2023-11-28T12:02:18.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.120+0000 7f89ace8f700 1 -- 172.21.15.57:0/270856944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f89a80aeec0 con 0x7f89a80c5a30 2023-11-28T12:02:18.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.121+0000 7f8996ffd700 1 -- 172.21.15.57:0/270856944 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f899c022070 con 0x7f89a80c5a30 2023-11-28T12:02:18.122 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:18.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.123+0000 7f89ace8f700 1 -- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89880682f0 msgr2=0x7f898806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.123+0000 7f89ace8f700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89880682f0 0x7f898806a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8998015040 tx=0x7f8998005000 comp rx=0 tx=0).stop 2023-11-28T12:02:18.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.123+0000 7f89ace8f700 1 -- 172.21.15.57:0/270856944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89a80c5a30 msgr2=0x7f89a80712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.123+0000 7f89ace8f700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89a80c5a30 0x7f89a80712c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f899c011c00 tx=0x7f899c009ef0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.123+0000 7f89ace8f700 1 -- 172.21.15.57:0/270856944 shutdown_connections 2023-11-28T12:02:18.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.123+0000 7f89ace8f700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89880682f0 0x7f898806a7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.123+0000 7f89ace8f700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89a8071800 0x7f89a80c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.123+0000 7f89ace8f700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8990003a00 0x7f89a815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.123+0000 7f89ace8f700 1 --2- 172.21.15.57:0/270856944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89a80c5a30 0x7f89a80712c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.123+0000 7f89ace8f700 1 -- 172.21.15.57:0/270856944 >> 172.21.15.57:0/270856944 conn(0x7f89a80cf400 msgr2=0x7f89a80af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:18.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.123+0000 7f89ace8f700 1 -- 172.21.15.57:0/270856944 shutdown_connections 2023-11-28T12:02:18.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.123+0000 7f89ace8f700 1 -- 172.21.15.57:0/270856944 wait complete. 2023-11-28T12:02:18.137 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:18.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.275+0000 7f654b3fa700 1 Processor -- start 2023-11-28T12:02:18.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.275+0000 7f654b3fa700 1 -- start start 2023-11-28T12:02:18.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.276+0000 7f654b3fa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65440cf710 0x7f65440ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:18.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.276+0000 7f654b3fa700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f654415cc10 con 0x7f6544071890 2023-11-28T12:02:18.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.276+0000 7f654b3fa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f654406c9b0 con 0x7f654406a980 2023-11-28T12:02:18.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.276+0000 7f654b3fa700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f654406cc90 con 0x7f65440cf710 2023-11-28T12:02:18.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.276+0000 7f6548995700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f6544071890 0x7f654406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45812/0 (socket says 172.21.15.57:45812) 2023-11-28T12:02:18.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.276+0000 7f6548995700 1 -- 172.21.15.57:0/2629442708 learned_addr learned my addr 172.21.15.57:0/2629442708 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:18.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.276+0000 7f6549196700 1 --2- 172.21.15.57:0/2629442708 >> v2:172.21.15.181:3300/0 conn(0x7f65440cf710 0x7f65440ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:18.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.276+0000 7f6537fff700 1 -- 172.21.15.57:0/2629442708 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18332660 0 0) 0x7f654415cc10 con 0x7f6544071890 2023-11-28T12:02:18.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.277+0000 7f6537fff700 1 -- 172.21.15.57:0/2629442708 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65300035b0 con 0x7f6544071890 2023-11-28T12:02:18.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.277+0000 7f6549196700 1 -- 172.21.15.57:0/2629442708 >> v1:172.21.15.181:6789/0 conn(0x7f654406a980 legacy=0x7f6544160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.277+0000 7f6549196700 1 -- 172.21.15.57:0/2629442708 >> v1:172.21.15.57:6789/0 conn(0x7f6544071890 legacy=0x7f654406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.277+0000 7f6549196700 1 -- 172.21.15.57:0/2629442708 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654406cf70 con 0x7f65440cf710 2023-11-28T12:02:18.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.277+0000 7f6549196700 1 --2- 172.21.15.57:0/2629442708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440cf710 0x7f65440ce740 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6540009870 tx=0x7f6540009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=641430e3c2dc3b29 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:18.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.277+0000 7f6537fff700 1 -- 172.21.15.57:0/2629442708 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540011830 con 0x7f65440cf710 2023-11-28T12:02:18.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.277+0000 7f6537fff700 1 -- 172.21.15.57:0/2629442708 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6540011990 con 0x7f65440cf710 2023-11-28T12:02:18.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.277+0000 7f6537fff700 1 -- 172.21.15.57:0/2629442708 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540011d20 con 0x7f65440cf710 2023-11-28T12:02:18.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.278+0000 7f654b3fa700 1 -- 172.21.15.57:0/2629442708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440cf710 msgr2=0x7f65440ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.278+0000 7f654b3fa700 1 --2- 172.21.15.57:0/2629442708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440cf710 0x7f65440ce740 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6540009870 tx=0x7f6540009b80 comp rx=0 tx=0).stop 2023-11-28T12:02:18.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.278+0000 7f654b3fa700 1 -- 172.21.15.57:0/2629442708 shutdown_connections 2023-11-28T12:02:18.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.278+0000 7f654b3fa700 1 --2- 172.21.15.57:0/2629442708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440cf710 0x7f65440ce740 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.278+0000 7f654b3fa700 1 -- 172.21.15.57:0/2629442708 >> 172.21.15.57:0/2629442708 conn(0x7f654406c010 msgr2=0x7f65440cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:18.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.278+0000 7f654b3fa700 1 -- 172.21.15.57:0/2629442708 shutdown_connections 2023-11-28T12:02:18.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.278+0000 7f654b3fa700 1 -- 172.21.15.57:0/2629442708 wait complete. 2023-11-28T12:02:18.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.279+0000 7f654b3fa700 1 Processor -- start 2023-11-28T12:02:18.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.279+0000 7f654b3fa700 1 -- start start 2023-11-28T12:02:18.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f654b3fa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654406a980 0x7f65440c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f654b3fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6544071890 0x7f65440c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f654b3fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440cf710 0x7f65441e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f654b3fa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6544148560 con 0x7f654406a980 2023-11-28T12:02:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f654b3fa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654414b1d0 con 0x7f65440cf710 2023-11-28T12:02:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f654b3fa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65440c7560 con 0x7f6544071890 2023-11-28T12:02:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f6549196700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654406a980 0x7f65440c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f6549196700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654406a980 0x7f65440c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58820/0 (socket says 172.21.15.57:58820) 2023-11-28T12:02:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f6549196700 1 -- 172.21.15.57:0/2657813472 learned_addr learned my addr 172.21.15.57:0/2657813472 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f6548995700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6544071890 0x7f65440c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f6549997700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440cf710 0x7f65441e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f6549196700 1 -- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6544071890 msgr2=0x7f65440c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f6549196700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6544071890 0x7f65440c6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f6549196700 1 -- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440cf710 msgr2=0x7f65441e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f6549196700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440cf710 0x7f65441e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.280+0000 7f6549196700 1 -- 172.21.15.57:0/2657813472 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65400094a0 con 0x7f654406a980 2023-11-28T12:02:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.281+0000 7f6549196700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654406a980 0x7f65440c6ca0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6540014040 tx=0x7f6540004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.281+0000 7f6535ffb700 1 -- 172.21.15.57:0/2657813472 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65400036a0 con 0x7f654406a980 2023-11-28T12:02:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.281+0000 7f6535ffb700 1 -- 172.21.15.57:0/2657813472 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654001e070 con 0x7f654406a980 2023-11-28T12:02:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.281+0000 7f654b3fa700 1 -- 172.21.15.57:0/2657813472 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654406d530 con 0x7f654406a980 2023-11-28T12:02:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.281+0000 7f6535ffb700 1 -- 172.21.15.57:0/2657813472 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540003a00 con 0x7f654406a980 2023-11-28T12:02:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.281+0000 7f654b3fa700 1 -- 172.21.15.57:0/2657813472 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654406d250 con 0x7f654406a980 2023-11-28T12:02:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.282+0000 7f654b3fa700 1 -- 172.21.15.57:0/2657813472 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65441486a0 con 0x7f654406a980 2023-11-28T12:02:18.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.283+0000 7f6535ffb700 1 -- 172.21.15.57:0/2657813472 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f654000f050 con 0x7f654406a980 2023-11-28T12:02:18.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.283+0000 7f6535ffb700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65280682f0 0x7f652806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:18.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.283+0000 7f6535ffb700 1 -- 172.21.15.57:0/2657813472 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6540086d20 con 0x7f654406a980 2023-11-28T12:02:18.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.283+0000 7f6548995700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65280682f0 0x7f652806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:18.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.284+0000 7f6548995700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65280682f0 0x7f652806a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f65441f5be0 tx=0x7f6538009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:18.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.288+0000 7f6535ffb700 1 -- 172.21.15.57:0/2657813472 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f65441486a0 con 0x7f654406a980 2023-11-28T12:02:18.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.490+0000 7f654b3fa700 1 -- 172.21.15.57:0/2657813472 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65441486a0 con 0x7f654406a980 2023-11-28T12:02:18.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.490+0000 7f6535ffb700 1 -- 172.21.15.57:0/2657813472 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65441486a0 con 0x7f654406a980 2023-11-28T12:02:18.492 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.493+0000 7f654b3fa700 1 -- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65280682f0 msgr2=0x7f652806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.493+0000 7f654b3fa700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65280682f0 0x7f652806a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f65441f5be0 tx=0x7f6538009040 comp rx=0 tx=0).stop 2023-11-28T12:02:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.493+0000 7f654b3fa700 1 -- 172.21.15.57:0/2657813472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654406a980 msgr2=0x7f65440c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.493+0000 7f654b3fa700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654406a980 0x7f65440c6ca0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6540014040 tx=0x7f6540004750 comp rx=0 tx=0).stop 2023-11-28T12:02:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.493+0000 7f654b3fa700 1 -- 172.21.15.57:0/2657813472 shutdown_connections 2023-11-28T12:02:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.493+0000 7f654b3fa700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65280682f0 0x7f652806a7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.493+0000 7f654b3fa700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65440cf710 0x7f65441e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.493+0000 7f654b3fa700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654406a980 0x7f65440c6ca0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.493+0000 7f654b3fa700 1 --2- 172.21.15.57:0/2657813472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6544071890 0x7f65440c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.493+0000 7f654b3fa700 1 -- 172.21.15.57:0/2657813472 >> 172.21.15.57:0/2657813472 conn(0x7f654406c010 msgr2=0x7f65440b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.493+0000 7f654b3fa700 1 -- 172.21.15.57:0/2657813472 shutdown_connections 2023-11-28T12:02:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.493+0000 7f654b3fa700 1 -- 172.21.15.57:0/2657813472 wait complete. 2023-11-28T12:02:18.508 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:02:18.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.647+0000 7f9ae4937700 1 Processor -- start 2023-11-28T12:02:18.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.647+0000 7f9ae4937700 1 -- start start 2023-11-28T12:02:18.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.648+0000 7f9ae4937700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9ae006d170 0x7f9ae00b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:18.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.648+0000 7f9ae4937700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9ae00b27d0 0x7f9ae00b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:18.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.648+0000 7f9ae4937700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae0161370 con 0x7f9ae015fe50 2023-11-28T12:02:18.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.648+0000 7f9ae4937700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae006d5c0 con 0x7f9ae006d170 2023-11-28T12:02:18.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.648+0000 7f9ae4937700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae0160220 con 0x7f9ae00b27d0 2023-11-28T12:02:18.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.649+0000 7f9addd9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9ae006d170 0x7f9ae00b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:18.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.649+0000 7f9addd9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9ae006d170 0x7f9ae00b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58836/0 (socket says 172.21.15.57:58836) 2023-11-28T12:02:18.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.649+0000 7f9addd9b700 1 -- 172.21.15.57:0/4214114703 learned_addr learned my addr 172.21.15.57:0/4214114703 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:18.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.649+0000 7f9aded9d700 1 --2- 172.21.15.57:0/4214114703 >> v2:172.21.15.181:3300/0 conn(0x7f9ae00b27d0 0x7f9ae00b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:18.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.649+0000 7f9addd9b700 1 -- 172.21.15.57:0/4214114703 >> v2:172.21.15.181:3300/0 conn(0x7f9ae00b27d0 msgr2=0x7f9ae00b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:18.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.649+0000 7f9addd9b700 1 --2- 172.21.15.57:0/4214114703 >> v2:172.21.15.181:3300/0 conn(0x7f9ae00b27d0 0x7f9ae00b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.649+0000 7f9addd9b700 1 -- 172.21.15.57:0/4214114703 >> v1:172.21.15.181:6789/0 conn(0x7f9ae015fe50 legacy=0x7f9ae0160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.649+0000 7f9addd9b700 1 -- 172.21.15.57:0/4214114703 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae00b3170 con 0x7f9ae006d170 2023-11-28T12:02:18.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.649+0000 7f9addd9b700 1 --2- 172.21.15.57:0/4214114703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ae006d170 0x7f9ae00b3b80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0071b30 tx=0x7f9ad4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d3ef80aed5666a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:18.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.649+0000 7f9add59a700 1 -- 172.21.15.57:0/4214114703 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400b860 con 0x7f9ae006d170 2023-11-28T12:02:18.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.649+0000 7f9add59a700 1 -- 172.21.15.57:0/4214114703 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad400b9c0 con 0x7f9ae006d170 2023-11-28T12:02:18.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.649+0000 7f9add59a700 1 -- 172.21.15.57:0/4214114703 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400bd50 con 0x7f9ae006d170 2023-11-28T12:02:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.650+0000 7f9ae4937700 1 -- 172.21.15.57:0/4214114703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ae006d170 msgr2=0x7f9ae00b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.650+0000 7f9ae4937700 1 --2- 172.21.15.57:0/4214114703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ae006d170 0x7f9ae00b3b80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0071b30 tx=0x7f9ad4009c00 comp rx=0 tx=0).stop 2023-11-28T12:02:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.650+0000 7f9ae4937700 1 -- 172.21.15.57:0/4214114703 shutdown_connections 2023-11-28T12:02:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.650+0000 7f9ae4937700 1 --2- 172.21.15.57:0/4214114703 >> v2:172.21.15.181:3300/0 conn(0x7f9ae00b27d0 0x7f9ae00b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.650+0000 7f9ae4937700 1 --2- 172.21.15.57:0/4214114703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ae006d170 0x7f9ae00b3b80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.650+0000 7f9ae4937700 1 -- 172.21.15.57:0/4214114703 >> 172.21.15.57:0/4214114703 conn(0x7f9ae006ea80 msgr2=0x7f9ae006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.650+0000 7f9ae4937700 1 -- 172.21.15.57:0/4214114703 shutdown_connections 2023-11-28T12:02:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.650+0000 7f9ae4937700 1 -- 172.21.15.57:0/4214114703 wait complete. 2023-11-28T12:02:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.652+0000 7f9ae4937700 1 Processor -- start 2023-11-28T12:02:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.652+0000 7f9ae4937700 1 -- start start 2023-11-28T12:02:18.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.652+0000 7f9ae4937700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae006d170 0x7f9ae015c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.652+0000 7f9ae4937700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ae00b27d0 0x7f9ae00c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.652+0000 7f9ae4937700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ae015fe50 0x7f9ae00b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.652+0000 7f9ae4937700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae015c990 con 0x7f9ae00b27d0 2023-11-28T12:02:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.652+0000 7f9ae4937700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae00c8ee0 con 0x7f9ae006d170 2023-11-28T12:02:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.652+0000 7f9ae4937700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae00c82d0 con 0x7f9ae015fe50 2023-11-28T12:02:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.652+0000 7f9addd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ae00b27d0 0x7f9ae00c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.652+0000 7f9addd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ae00b27d0 0x7f9ae00c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58850/0 (socket says 172.21.15.57:58850) 2023-11-28T12:02:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.652+0000 7f9addd9b700 1 -- 172.21.15.57:0/1942872412 learned_addr learned my addr 172.21.15.57:0/1942872412 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9ade59c700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae006d170 0x7f9ae015c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9aded9d700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ae015fe50 0x7f9ae00b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9addd9b700 1 -- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ae015fe50 msgr2=0x7f9ae00b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9addd9b700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ae015fe50 0x7f9ae00b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9addd9b700 1 -- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae006d170 msgr2=0x7f9ae015c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9addd9b700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae006d170 0x7f9ae015c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9addd9b700 1 -- 172.21.15.57:0/1942872412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4009550 con 0x7f9ae00b27d0 2023-11-28T12:02:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9aded9d700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ae015fe50 0x7f9ae00b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9ade59c700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae006d170 0x7f9ae015c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9addd9b700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ae00b27d0 0x7f9ae00c7150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009130 tx=0x7f9ad4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9aceffd700 1 -- 172.21.15.57:0/1942872412 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40045d0 con 0x7f9ae00b27d0 2023-11-28T12:02:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9aceffd700 1 -- 172.21.15.57:0/1942872412 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad401e070 con 0x7f9ae00b27d0 2023-11-28T12:02:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9ae4937700 1 -- 172.21.15.57:0/1942872412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae00cedd0 con 0x7f9ae00b27d0 2023-11-28T12:02:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9aceffd700 1 -- 172.21.15.57:0/1942872412 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40038a0 con 0x7f9ae00b27d0 2023-11-28T12:02:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.653+0000 7f9ae4937700 1 -- 172.21.15.57:0/1942872412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae00cf700 con 0x7f9ae00b27d0 2023-11-28T12:02:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.654+0000 7f9ae4937700 1 -- 172.21.15.57:0/1942872412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae00cf0b0 con 0x7f9ae00b27d0 2023-11-28T12:02:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.655+0000 7f9aceffd700 1 -- 172.21.15.57:0/1942872412 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9ad400f050 con 0x7f9ae00b27d0 2023-11-28T12:02:18.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.655+0000 7f9aceffd700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ac00682f0 0x7f9ac006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:18.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.655+0000 7f9aceffd700 1 -- 172.21.15.57:0/1942872412 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ad4020070 con 0x7f9ae00b27d0 2023-11-28T12:02:18.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.655+0000 7f9ade59c700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ac00682f0 0x7f9ac006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:18.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.656+0000 7f9ade59c700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ac00682f0 0x7f9ac006a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00d0520 tx=0x7f9ad0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:18.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.660+0000 7f9aceffd700 1 -- 172.21.15.57:0/1942872412 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9ad4053a20 con 0x7f9ae00b27d0 2023-11-28T12:02:18.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.860+0000 7f9ae4937700 1 -- 172.21.15.57:0/1942872412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ae00743f0 con 0x7f9ae00b27d0 2023-11-28T12:02:18.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.861+0000 7f9aceffd700 1 -- 172.21.15.57:0/1942872412 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ad4015020 con 0x7f9ae00b27d0 2023-11-28T12:02:18.865 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:02:18.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.863+0000 7f9ae4937700 1 -- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ac00682f0 msgr2=0x7f9ac006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.863+0000 7f9ae4937700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ac00682f0 0x7f9ac006a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00d0520 tx=0x7f9ad0009040 comp rx=0 tx=0).stop 2023-11-28T12:02:18.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.863+0000 7f9ae4937700 1 -- 172.21.15.57:0/1942872412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ae00b27d0 msgr2=0x7f9ae00c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:18.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.863+0000 7f9ae4937700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ae00b27d0 0x7f9ae00c7150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009130 tx=0x7f9ad4004750 comp rx=0 tx=0).stop 2023-11-28T12:02:18.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.863+0000 7f9ae4937700 1 -- 172.21.15.57:0/1942872412 shutdown_connections 2023-11-28T12:02:18.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.863+0000 7f9ae4937700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ac00682f0 0x7f9ac006a7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.863+0000 7f9ae4937700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ae015fe50 0x7f9ae00b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.863+0000 7f9ae4937700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae006d170 0x7f9ae015c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.863+0000 7f9ae4937700 1 --2- 172.21.15.57:0/1942872412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ae00b27d0 0x7f9ae00c7150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:18.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.863+0000 7f9ae4937700 1 -- 172.21.15.57:0/1942872412 >> 172.21.15.57:0/1942872412 conn(0x7f9ae006ea80 msgr2=0x7f9ae00cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:18.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.863+0000 7f9ae4937700 1 -- 172.21.15.57:0/1942872412 shutdown_connections 2023-11-28T12:02:18.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:18.863+0000 7f9ae4937700 1 -- 172.21.15.57:0/1942872412 wait complete. 2023-11-28T12:02:18.878 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:02:19.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.015+0000 7fd28367f700 1 Processor -- start 2023-11-28T12:02:19.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.016+0000 7fd28367f700 1 -- start start 2023-11-28T12:02:19.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.016+0000 7fd28367f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c06a980 0x7fd27c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.016+0000 7fd28367f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd27c15cc10 con 0x7fd27c0cf710 2023-11-28T12:02:19.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.016+0000 7fd28367f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd27c06c9b0 con 0x7fd27c071890 2023-11-28T12:02:19.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.016+0000 7fd28367f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c06cc90 con 0x7fd27c06a980 2023-11-28T12:02:19.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd28141b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd27c0cf710 0x7fd27c06a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45824/0 (socket says 172.21.15.57:45824) 2023-11-28T12:02:19.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd28141b700 1 -- 172.21.15.57:0/997380539 learned_addr learned my addr 172.21.15.57:0/997380539 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:19.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd280c1a700 1 --2- 172.21.15.57:0/997380539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c06a980 0x7fd27c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423423662 0 0) 0x7fd27c15cc10 con 0x7fd27c0cf710 2023-11-28T12:02:19.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2680035b0 con 0x7fd27c0cf710 2023-11-28T12:02:19.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2271651766 0 0) 0x7fd2680035b0 con 0x7fd27c0cf710 2023-11-28T12:02:19.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c06a980 msgr2=0x7fd27c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:19.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 --2- 172.21.15.57:0/997380539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c06a980 0x7fd27c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 >> v1:172.21.15.181:6789/0 conn(0x7fd27c071890 legacy=0x7fd27c160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c06cf70 con 0x7fd27c0cf710 2023-11-28T12:02:19.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fd278001670 con 0x7fd27c0cf710 2023-11-28T12:02:19.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 >> v1:172.21.15.57:6789/0 conn(0x7fd27c0cf710 legacy=0x7fd27c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 --2- 172.21.15.57:0/997380539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c06a980 0x7fd268003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 --2- 172.21.15.57:0/997380539 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd268003f70 0x7fd268004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 --2- 172.21.15.57:0/997380539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd268004d90 0x7fd268005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c06cc90 con 0x7fd268003f70 2023-11-28T12:02:19.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26800baf0 con 0x7fd268004d90 2023-11-28T12:02:19.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.017+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26800bcb0 con 0x7fd27c06a980 2023-11-28T12:02:19.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.018+0000 7fd280c1a700 1 --2- 172.21.15.57:0/997380539 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd268003f70 0x7fd268004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.018+0000 7fd28141b700 1 --2- 172.21.15.57:0/997380539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c06a980 0x7fd268003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.018+0000 7fd281c1c700 1 --2- 172.21.15.57:0/997380539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd268004d90 0x7fd268005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.018+0000 7fd280c1a700 1 -- 172.21.15.57:0/997380539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c06a980 msgr2=0x7fd268003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:19.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.018+0000 7fd280c1a700 1 --2- 172.21.15.57:0/997380539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c06a980 0x7fd268003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.018+0000 7fd280c1a700 1 -- 172.21.15.57:0/997380539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd268004d90 msgr2=0x7fd268005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:19.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.018+0000 7fd280c1a700 1 --2- 172.21.15.57:0/997380539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd268004d90 0x7fd268005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.018+0000 7fd280c1a700 1 -- 172.21.15.57:0/997380539 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2680035b0 con 0x7fd268003f70 2023-11-28T12:02:19.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.018+0000 7fd280c1a700 1 --2- 172.21.15.57:0/997380539 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd268003f70 0x7fd268004850 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd27c06ad70 tx=0x7fd268005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65375f1397aa2634 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:19.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.018+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00e6c0 con 0x7fd268003f70 2023-11-28T12:02:19.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.018+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26c00e820 con 0x7fd268003f70 2023-11-28T12:02:19.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.018+0000 7fd273fff700 1 -- 172.21.15.57:0/997380539 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00ebb0 con 0x7fd268003f70 2023-11-28T12:02:19.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.019+0000 7fd28367f700 1 -- 172.21.15.57:0/997380539 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd268003f70 msgr2=0x7fd268004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.019+0000 7fd28367f700 1 --2- 172.21.15.57:0/997380539 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd268003f70 0x7fd268004850 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd27c06ad70 tx=0x7fd268005880 comp rx=0 tx=0).stop 2023-11-28T12:02:19.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.019+0000 7fd281c1c700 1 -- 172.21.15.57:0/997380539 reap_dead start 2023-11-28T12:02:19.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.019+0000 7fd28367f700 1 -- 172.21.15.57:0/997380539 shutdown_connections 2023-11-28T12:02:19.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.019+0000 7fd28367f700 1 -- 172.21.15.57:0/997380539 >> 172.21.15.57:0/997380539 conn(0x7fd27c06c010 msgr2=0x7fd27c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:19.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.019+0000 7fd28367f700 1 -- 172.21.15.57:0/997380539 shutdown_connections 2023-11-28T12:02:19.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.019+0000 7fd28367f700 1 -- 172.21.15.57:0/997380539 wait complete. 2023-11-28T12:02:19.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.020+0000 7fd28367f700 1 Processor -- start 2023-11-28T12:02:19.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.020+0000 7fd28367f700 1 -- start start 2023-11-28T12:02:19.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28367f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd268003f70 0x7fd27c0ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28367f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c1f6110 0x7fd27c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28367f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd27c1f5500 0x7fd27c1f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28367f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c0c77a0 con 0x7fd268003f70 2023-11-28T12:02:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28367f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c1f5ee0 con 0x7fd27c1f5500 2023-11-28T12:02:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28367f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c15f240 con 0x7fd27c1f6110 2023-11-28T12:02:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28141b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd268003f70 0x7fd27c0ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28141b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd268003f70 0x7fd27c0ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58880/0 (socket says 172.21.15.57:58880) 2023-11-28T12:02:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28141b700 1 -- 172.21.15.57:0/2695811251 learned_addr learned my addr 172.21.15.57:0/2695811251 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd281c1c700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd27c1f5500 0x7fd27c1f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28141b700 1 -- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c1f6110 msgr2=0x7fd27c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd280c1a700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c1f6110 0x7fd27c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28141b700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c1f6110 0x7fd27c15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28141b700 1 -- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd27c1f5500 msgr2=0x7fd27c1f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28141b700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd27c1f5500 0x7fd27c1f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd28141b700 1 -- 172.21.15.57:0/2695811251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c06d530 con 0x7fd268003f70 2023-11-28T12:02:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd281c1c700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd27c1f5500 0x7fd27c1f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.021+0000 7fd280c1a700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c1f6110 0x7fd27c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.022+0000 7fd28141b700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd268003f70 0x7fd27c0ccaa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd278007290 tx=0x7fd278007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.022+0000 7fd271ffb700 1 -- 172.21.15.57:0/2695811251 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27800f6c0 con 0x7fd268003f70 2023-11-28T12:02:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.022+0000 7fd271ffb700 1 -- 172.21.15.57:0/2695811251 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27800f820 con 0x7fd268003f70 2023-11-28T12:02:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.022+0000 7fd28367f700 1 -- 172.21.15.57:0/2695811251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27c06d250 con 0x7fd268003f70 2023-11-28T12:02:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.022+0000 7fd271ffb700 1 -- 172.21.15.57:0/2695811251 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27800fbd0 con 0x7fd268003f70 2023-11-28T12:02:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.022+0000 7fd28367f700 1 -- 172.21.15.57:0/2695811251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c0ce740 con 0x7fd268003f70 2023-11-28T12:02:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.023+0000 7fd28367f700 1 -- 172.21.15.57:0/2695811251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27c0c78e0 con 0x7fd268003f70 2023-11-28T12:02:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.023+0000 7fd271ffb700 1 -- 172.21.15.57:0/2695811251 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd27800fd30 con 0x7fd268003f70 2023-11-28T12:02:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.024+0000 7fd271ffb700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd2600682f0 0x7fd26006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.024+0000 7fd271ffb700 1 -- 172.21.15.57:0/2695811251 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd278085d90 con 0x7fd268003f70 2023-11-28T12:02:19.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.024+0000 7fd280c1a700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd2600682f0 0x7fd26006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.025+0000 7fd280c1a700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd2600682f0 0x7fd26006a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd27c15fb70 tx=0x7fd26c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:19.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.028+0000 7fd271ffb700 1 -- 172.21.15.57:0/2695811251 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd27c0c78e0 con 0x7fd268003f70 2023-11-28T12:02:19.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.226+0000 7fd28367f700 1 -- 172.21.15.57:0/2695811251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd27c0c78e0 con 0x7fd268003f70 2023-11-28T12:02:19.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.227+0000 7fd271ffb700 1 -- 172.21.15.57:0/2695811251 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd27c0c78e0 con 0x7fd268003f70 2023-11-28T12:02:19.228 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:02:19.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.229+0000 7fd28367f700 1 -- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd2600682f0 msgr2=0x7fd26006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.229+0000 7fd28367f700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd2600682f0 0x7fd26006a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd27c15fb70 tx=0x7fd26c008030 comp rx=0 tx=0).stop 2023-11-28T12:02:19.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.229+0000 7fd28367f700 1 -- 172.21.15.57:0/2695811251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd268003f70 msgr2=0x7fd27c0ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.229+0000 7fd28367f700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd268003f70 0x7fd27c0ccaa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd278007290 tx=0x7fd278007650 comp rx=0 tx=0).stop 2023-11-28T12:02:19.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.229+0000 7fd28367f700 1 -- 172.21.15.57:0/2695811251 shutdown_connections 2023-11-28T12:02:19.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.229+0000 7fd28367f700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd2600682f0 0x7fd26006a7a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.229+0000 7fd28367f700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd27c1f5500 0x7fd27c1f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.229+0000 7fd28367f700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd268003f70 0x7fd27c0ccaa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.229+0000 7fd28367f700 1 --2- 172.21.15.57:0/2695811251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd27c1f6110 0x7fd27c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.229+0000 7fd28367f700 1 -- 172.21.15.57:0/2695811251 >> 172.21.15.57:0/2695811251 conn(0x7fd27c06c010 msgr2=0x7fd27c0cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:19.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.229+0000 7fd28367f700 1 -- 172.21.15.57:0/2695811251 shutdown_connections 2023-11-28T12:02:19.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.229+0000 7fd28367f700 1 -- 172.21.15.57:0/2695811251 wait complete. 2023-11-28T12:02:19.244 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T12:02:19.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.378+0000 7f503123c700 1 Processor -- start 2023-11-28T12:02:19.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.379+0000 7f503123c700 1 -- start start 2023-11-28T12:02:19.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.379+0000 7f503123c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f502c1599b0 0x7f502c15d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.379+0000 7f503123c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502c0af0a0 0x7f502c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.379+0000 7f503123c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502c159ec0 con 0x7f502c0b1650 2023-11-28T12:02:19.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.379+0000 7f503123c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502c15d8a0 con 0x7f502c1599b0 2023-11-28T12:02:19.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.379+0000 7f503123c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c15db80 con 0x7f502c0af0a0 2023-11-28T12:02:19.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.380+0000 7f502a59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f502c1599b0 0x7f502c15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.380+0000 7f502a59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f502c1599b0 0x7f502c15d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:33374/0 (socket says 172.21.15.57:33374) 2023-11-28T12:02:19.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.380+0000 7f502a59c700 1 -- 172.21.15.57:0/379875131 learned_addr learned my addr 172.21.15.57:0/379875131 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:19.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.380+0000 7f502b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502c0af0a0 0x7f502c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.380+0000 7f502a59c700 1 -- 172.21.15.57:0/379875131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502c0af0a0 msgr2=0x7f502c15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:19.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.380+0000 7f502a59c700 1 --2- 172.21.15.57:0/379875131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502c0af0a0 0x7f502c15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.380+0000 7f502a59c700 1 -- 172.21.15.57:0/379875131 >> v1:172.21.15.181:6789/0 conn(0x7f502c0b1650 legacy=0x7f502c0b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.380+0000 7f502a59c700 1 -- 172.21.15.57:0/379875131 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c15de60 con 0x7f502c1599b0 2023-11-28T12:02:19.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.380+0000 7f502a59c700 1 --2- 172.21.15.57:0/379875131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f502c1599b0 0x7f502c15d050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f50200098f0 tx=0x7f5020009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dacf16df38b70aa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:19.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.381+0000 7f5029d9b700 1 -- 172.21.15.57:0/379875131 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020004230 con 0x7f502c1599b0 2023-11-28T12:02:19.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.381+0000 7f5029d9b700 1 -- 172.21.15.57:0/379875131 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5020004390 con 0x7f502c1599b0 2023-11-28T12:02:19.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.381+0000 7f5029d9b700 1 -- 172.21.15.57:0/379875131 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50200036a0 con 0x7f502c1599b0 2023-11-28T12:02:19.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.381+0000 7f503123c700 1 -- 172.21.15.57:0/379875131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f502c1599b0 msgr2=0x7f502c15d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.381+0000 7f503123c700 1 --2- 172.21.15.57:0/379875131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f502c1599b0 0x7f502c15d050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f50200098f0 tx=0x7f5020009c00 comp rx=0 tx=0).stop 2023-11-28T12:02:19.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.381+0000 7f503123c700 1 -- 172.21.15.57:0/379875131 shutdown_connections 2023-11-28T12:02:19.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.381+0000 7f503123c700 1 --2- 172.21.15.57:0/379875131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502c0af0a0 0x7f502c15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.381+0000 7f503123c700 1 --2- 172.21.15.57:0/379875131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f502c1599b0 0x7f502c15d050 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.381+0000 7f503123c700 1 -- 172.21.15.57:0/379875131 >> 172.21.15.57:0/379875131 conn(0x7f502c06c010 msgr2=0x7f502c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:19.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.381+0000 7f503123c700 1 -- 172.21.15.57:0/379875131 shutdown_connections 2023-11-28T12:02:19.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.381+0000 7f503123c700 1 -- 172.21.15.57:0/379875131 wait complete. 2023-11-28T12:02:19.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.383+0000 7f503123c700 1 Processor -- start 2023-11-28T12:02:19.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.383+0000 7f503123c700 1 -- start start 2023-11-28T12:02:19.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.384+0000 7f503123c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f502c0af0a0 0x7f502c1501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.384+0000 7f503123c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502c0b1650 0x7f502c0c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.384+0000 7f503123c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f502c14f540 0x7f502c1f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.384+0000 7f503123c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c0c51e0 con 0x7f502c14f540 2023-11-28T12:02:19.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.384+0000 7f503123c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c14ff70 con 0x7f502c0af0a0 2023-11-28T12:02:19.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.384+0000 7f503123c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c14fc90 con 0x7f502c0b1650 2023-11-28T12:02:19.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.384+0000 7f502b59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f502c14f540 0x7f502c1f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.384+0000 7f502b59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f502c14f540 0x7f502c1f05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58896/0 (socket says 172.21.15.57:58896) 2023-11-28T12:02:19.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.384+0000 7f502b59e700 1 -- 172.21.15.57:0/4145215248 learned_addr learned my addr 172.21.15.57:0/4145215248 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.385+0000 7f502a59c700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502c0b1650 0x7f502c0c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.385+0000 7f502ad9d700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f502c0af0a0 0x7f502c1501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.385+0000 7f502b59e700 1 -- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502c0b1650 msgr2=0x7f502c0c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.385+0000 7f502b59e700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502c0b1650 0x7f502c0c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.385+0000 7f502b59e700 1 -- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f502c0af0a0 msgr2=0x7f502c1501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.385+0000 7f502b59e700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f502c0af0a0 0x7f502c1501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.385+0000 7f502b59e700 1 -- 172.21.15.57:0/4145215248 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5020009550 con 0x7f502c14f540 2023-11-28T12:02:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.385+0000 7f502b59e700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f502c14f540 0x7f502c1f05b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f502400e7d0 tx=0x7f502400eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.385+0000 7f50137fe700 1 -- 172.21.15.57:0/4145215248 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50240095b0 con 0x7f502c14f540 2023-11-28T12:02:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.385+0000 7f50137fe700 1 -- 172.21.15.57:0/4145215248 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5024013680 con 0x7f502c14f540 2023-11-28T12:02:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.385+0000 7f503123c700 1 -- 172.21.15.57:0/4145215248 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c15a890 con 0x7f502c14f540 2023-11-28T12:02:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.385+0000 7f50137fe700 1 -- 172.21.15.57:0/4145215248 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024013a10 con 0x7f502c14f540 2023-11-28T12:02:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.385+0000 7f503123c700 1 -- 172.21.15.57:0/4145215248 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c15d590 con 0x7f502c14f540 2023-11-28T12:02:19.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.386+0000 7f503123c700 1 -- 172.21.15.57:0/4145215248 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502c15aa20 con 0x7f502c14f540 2023-11-28T12:02:19.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.387+0000 7f50137fe700 1 -- 172.21.15.57:0/4145215248 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5024019030 con 0x7f502c14f540 2023-11-28T12:02:19.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.387+0000 7f50137fe700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50180682f0 0x7f501806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.387+0000 7f50137fe700 1 -- 172.21.15.57:0/4145215248 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5024085990 con 0x7f502c14f540 2023-11-28T12:02:19.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.388+0000 7f502ad9d700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50180682f0 0x7f501806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.388+0000 7f502ad9d700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50180682f0 0x7f501806a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f502c0b3b70 tx=0x7f501c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:19.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.392+0000 7f50137fe700 1 -- 172.21.15.57:0/4145215248 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f50240521f0 con 0x7f502c14f540 2023-11-28T12:02:19.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.605+0000 7f503123c700 1 -- 172.21.15.57:0/4145215248 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f502c0743f0 con 0x7f502c14f540 2023-11-28T12:02:19.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.606+0000 7f50137fe700 1 -- 172.21.15.57:0/4145215248 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5024055810 con 0x7f502c14f540 2023-11-28T12:02:19.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.608+0000 7f503123c700 1 -- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50180682f0 msgr2=0x7f501806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.608+0000 7f503123c700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50180682f0 0x7f501806a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f502c0b3b70 tx=0x7f501c009040 comp rx=0 tx=0).stop 2023-11-28T12:02:19.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.608+0000 7f503123c700 1 -- 172.21.15.57:0/4145215248 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f502c14f540 msgr2=0x7f502c1f05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.608+0000 7f503123c700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f502c14f540 0x7f502c1f05b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f502400e7d0 tx=0x7f502400eae0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.608+0000 7f503123c700 1 -- 172.21.15.57:0/4145215248 shutdown_connections 2023-11-28T12:02:19.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.608+0000 7f503123c700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50180682f0 0x7f501806a7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.608+0000 7f503123c700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f502c14f540 0x7f502c1f05b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.608+0000 7f503123c700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f502c0af0a0 0x7f502c1501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.608+0000 7f503123c700 1 --2- 172.21.15.57:0/4145215248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f502c0b1650 0x7f502c0c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.608+0000 7f503123c700 1 -- 172.21.15.57:0/4145215248 >> 172.21.15.57:0/4145215248 conn(0x7f502c06c010 msgr2=0x7f502c070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:19.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.608+0000 7f503123c700 1 -- 172.21.15.57:0/4145215248 shutdown_connections 2023-11-28T12:02:19.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.608+0000 7f503123c700 1 -- 172.21.15.57:0/4145215248 wait complete. 2023-11-28T12:02:19.611 INFO:teuthology.orchestra.run.smithi057.stderr:listed 0 entries 2023-11-28T12:02:19.623 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:02:19.623 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:02:19.643 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:19.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.823+0000 7f65a0594700 1 Processor -- start 2023-11-28T12:02:19.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.823+0000 7f65a0594700 1 -- start start 2023-11-28T12:02:19.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.824+0000 7f65a0594700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65980b1770 0x7f65980b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.824+0000 7f65a0594700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65980b2110 con 0x7f6598142db0 2023-11-28T12:02:19.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.824+0000 7f65a0594700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659815e010 con 0x7f65980ce040 2023-11-28T12:02:19.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.824+0000 7f65a0594700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659813fc90 con 0x7f65980b1770 2023-11-28T12:02:19.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.824+0000 7f659db2f700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f6598142db0 0x7f659813ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45828/0 (socket says 172.21.15.57:45828) 2023-11-28T12:02:19.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.824+0000 7f659db2f700 1 -- 172.21.15.57:0/833876158 learned_addr learned my addr 172.21.15.57:0/833876158 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:19.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.824+0000 7f659eb31700 1 --2- 172.21.15.57:0/833876158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65980b1770 0x7f65980b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.825+0000 7f659d32e700 1 -- 172.21.15.57:0/833876158 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106261803 0 0) 0x7f65980b2110 con 0x7f6598142db0 2023-11-28T12:02:19.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.825+0000 7f659d32e700 1 -- 172.21.15.57:0/833876158 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65840035b0 con 0x7f6598142db0 2023-11-28T12:02:19.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.825+0000 7f659d32e700 1 -- 172.21.15.57:0/833876158 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1187289466 0 0) 0x7f6594001110 con 0x7f65980ce040 2023-11-28T12:02:19.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.825+0000 7f659d32e700 1 -- 172.21.15.57:0/833876158 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65980b2110 con 0x7f65980ce040 2023-11-28T12:02:19.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.825+0000 7f659eb31700 1 -- 172.21.15.57:0/833876158 >> v1:172.21.15.181:6789/0 conn(0x7f65980ce040 legacy=0x7f659815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.825+0000 7f659eb31700 1 -- 172.21.15.57:0/833876158 >> v1:172.21.15.57:6789/0 conn(0x7f6598142db0 legacy=0x7f659813ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.825+0000 7f659eb31700 1 -- 172.21.15.57:0/833876158 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659813f480 con 0x7f65980b1770 2023-11-28T12:02:19.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.825+0000 7f659eb31700 1 --2- 172.21.15.57:0/833876158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65980b1770 0x7f65980b1bd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6590009dc0 tx=0x7f659000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41bcaafc0c09aadf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:19.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.825+0000 7f659d32e700 1 -- 172.21.15.57:0/833876158 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659000e070 con 0x7f65980b1770 2023-11-28T12:02:19.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.825+0000 7f659d32e700 1 -- 172.21.15.57:0/833876158 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659000bb20 con 0x7f65980b1770 2023-11-28T12:02:19.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.825+0000 7f659d32e700 1 -- 172.21.15.57:0/833876158 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590012430 con 0x7f65980b1770 2023-11-28T12:02:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.826+0000 7f65a0594700 1 -- 172.21.15.57:0/833876158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65980b1770 msgr2=0x7f65980b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.826+0000 7f65a0594700 1 --2- 172.21.15.57:0/833876158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65980b1770 0x7f65980b1bd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6590009dc0 tx=0x7f659000b330 comp rx=0 tx=0).stop 2023-11-28T12:02:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.826+0000 7f65a0594700 1 -- 172.21.15.57:0/833876158 shutdown_connections 2023-11-28T12:02:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.826+0000 7f65a0594700 1 --2- 172.21.15.57:0/833876158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65980b1770 0x7f65980b1bd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.826+0000 7f65a0594700 1 -- 172.21.15.57:0/833876158 >> 172.21.15.57:0/833876158 conn(0x7f659806c010 msgr2=0x7f65980cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.826+0000 7f65a0594700 1 -- 172.21.15.57:0/833876158 shutdown_connections 2023-11-28T12:02:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.826+0000 7f65a0594700 1 -- 172.21.15.57:0/833876158 wait complete. 2023-11-28T12:02:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.827+0000 7f65a0594700 1 Processor -- start 2023-11-28T12:02:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.827+0000 7f65a0594700 1 -- start start 2023-11-28T12:02:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.828+0000 7f65a0594700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65980b1770 0x7f6598141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.828+0000 7f65a0594700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65980ce040 0x7f659815b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.828+0000 7f65a0594700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6598142db0 0x7f65981f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.828+0000 7f65a0594700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659815c490 con 0x7f6598142db0 2023-11-28T12:02:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.828+0000 7f65a0594700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659815b5e0 con 0x7f65980b1770 2023-11-28T12:02:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.828+0000 7f65a0594700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659815b880 con 0x7f65980ce040 2023-11-28T12:02:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.828+0000 7f659eb31700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6598142db0 0x7f65981f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.828+0000 7f659eb31700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6598142db0 0x7f65981f1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58908/0 (socket says 172.21.15.57:58908) 2023-11-28T12:02:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.828+0000 7f659eb31700 1 -- 172.21.15.57:0/1533005173 learned_addr learned my addr 172.21.15.57:0/1533005173 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.828+0000 7f659db2f700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65980ce040 0x7f659815b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.828+0000 7f659e330700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65980b1770 0x7f6598141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.829+0000 7f659eb31700 1 -- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65980ce040 msgr2=0x7f659815b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.829+0000 7f659eb31700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65980ce040 0x7f659815b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.829+0000 7f659eb31700 1 -- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65980b1770 msgr2=0x7f6598141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.829+0000 7f659eb31700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65980b1770 0x7f6598141b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.829+0000 7f659eb31700 1 -- 172.21.15.57:0/1533005173 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6590009a70 con 0x7f6598142db0 2023-11-28T12:02:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.829+0000 7f659eb31700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6598142db0 0x7f65981f1fb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f659000b370 tx=0x7f6590004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.829+0000 7f658affd700 1 -- 172.21.15.57:0/1533005173 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659001b810 con 0x7f6598142db0 2023-11-28T12:02:19.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.829+0000 7f658affd700 1 -- 172.21.15.57:0/1533005173 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6590020070 con 0x7f6598142db0 2023-11-28T12:02:19.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.829+0000 7f65a0594700 1 -- 172.21.15.57:0/1533005173 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659813f9b0 con 0x7f6598142db0 2023-11-28T12:02:19.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.829+0000 7f658affd700 1 -- 172.21.15.57:0/1533005173 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659001bbf0 con 0x7f6598142db0 2023-11-28T12:02:19.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.829+0000 7f65a0594700 1 -- 172.21.15.57:0/1533005173 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659813f750 con 0x7f6598142db0 2023-11-28T12:02:19.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.830+0000 7f65a0594700 1 -- 172.21.15.57:0/1533005173 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659815d100 con 0x7f6598142db0 2023-11-28T12:02:19.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.830+0000 7f658affd700 1 -- 172.21.15.57:0/1533005173 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f659001bd50 con 0x7f6598142db0 2023-11-28T12:02:19.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.831+0000 7f658affd700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f657c068320 0x7f657c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:19.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.831+0000 7f658affd700 1 -- 172.21.15.57:0/1533005173 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6590022070 con 0x7f6598142db0 2023-11-28T12:02:19.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.831+0000 7f659e330700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f657c068320 0x7f657c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:19.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.831+0000 7f659e330700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f657c068320 0x7f657c06a7d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6594007fd0 tx=0x7f6594009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:19.835+0000 7f658affd700 1 -- 172.21.15.57:0/1533005173 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6590054b70 con 0x7f6598142db0 2023-11-28T12:02:20.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.035+0000 7f65a0594700 1 -- 172.21.15.57:0/1533005173 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65980743f0 con 0x7f6598142db0 2023-11-28T12:02:20.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.037+0000 7f658affd700 1 -- 172.21.15.57:0/1533005173 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7f6590058190 con 0x7f6598142db0 2023-11-28T12:02:20.038 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:20.039 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:20.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.039+0000 7f65a0594700 1 -- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f657c068320 msgr2=0x7f657c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.039+0000 7f65a0594700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f657c068320 0x7f657c06a7d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6594007fd0 tx=0x7f6594009040 comp rx=0 tx=0).stop 2023-11-28T12:02:20.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.039+0000 7f65a0594700 1 -- 172.21.15.57:0/1533005173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6598142db0 msgr2=0x7f65981f1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.039+0000 7f65a0594700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6598142db0 0x7f65981f1fb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f659000b370 tx=0x7f6590004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.039+0000 7f65a0594700 1 -- 172.21.15.57:0/1533005173 shutdown_connections 2023-11-28T12:02:20.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.039+0000 7f65a0594700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f657c068320 0x7f657c06a7d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.039+0000 7f65a0594700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6598142db0 0x7f65981f1fb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.039+0000 7f65a0594700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65980b1770 0x7f6598141b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.039+0000 7f65a0594700 1 --2- 172.21.15.57:0/1533005173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65980ce040 0x7f659815b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.039+0000 7f65a0594700 1 -- 172.21.15.57:0/1533005173 >> 172.21.15.57:0/1533005173 conn(0x7f659806c010 msgr2=0x7f65980cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:20.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.039+0000 7f65a0594700 1 -- 172.21.15.57:0/1533005173 shutdown_connections 2023-11-28T12:02:20.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.039+0000 7f65a0594700 1 -- 172.21.15.57:0/1533005173 wait complete. 2023-11-28T12:02:20.054 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:20.054 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:20.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.192+0000 7fe1d4d2a700 1 Processor -- start 2023-11-28T12:02:20.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.192+0000 7fe1d4d2a700 1 -- start start 2023-11-28T12:02:20.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.193+0000 7fe1d4d2a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe1d015fe50 0x7fe1d006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.193+0000 7fe1d4d2a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d00b3b80 0x7fe1d0160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1d4d2a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d0161370 con 0x7fe1d00b27d0 2023-11-28T12:02:20.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1d4d2a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d0160220 con 0x7fe1d015fe50 2023-11-28T12:02:20.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1d4d2a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d00cf700 con 0x7fe1d00b3b80 2023-11-28T12:02:20.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1ce59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe1d015fe50 0x7fe1d006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1ced9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe1d00b27d0 0x7fe1d00cfb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45842/0 (socket says 172.21.15.57:45842) 2023-11-28T12:02:20.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1ce59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe1d015fe50 0x7fe1d006d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58914/0 (socket says 172.21.15.57:58914) 2023-11-28T12:02:20.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1ced9d700 1 -- 172.21.15.57:0/4000396144 learned_addr learned my addr 172.21.15.57:0/4000396144 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:20.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1cdd9b700 1 --2- 172.21.15.57:0/4000396144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d00b3b80 0x7fe1d0160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1ce59c700 1 -- 172.21.15.57:0/4000396144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d00b3b80 msgr2=0x7fe1d0160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:20.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1ce59c700 1 --2- 172.21.15.57:0/4000396144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d00b3b80 0x7fe1d0160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1ce59c700 1 -- 172.21.15.57:0/4000396144 >> v1:172.21.15.57:6789/0 conn(0x7fe1d00b27d0 legacy=0x7fe1d00cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1ce59c700 1 -- 172.21.15.57:0/4000396144 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d00b3fd0 con 0x7fe1d015fe50 2023-11-28T12:02:20.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1cd59a700 1 -- 172.21.15.57:0/4000396144 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700560287 0 0) 0x7fe1d0161370 con 0x7fe1d00b27d0 2023-11-28T12:02:20.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1ce59c700 1 --2- 172.21.15.57:0/4000396144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d015fe50 0x7fe1d006d170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009840 tx=0x7fe1c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797ccee67bf586fd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:20.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1cd59a700 1 -- 172.21.15.57:0/4000396144 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0011830 con 0x7fe1d015fe50 2023-11-28T12:02:20.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.194+0000 7fe1cd59a700 1 -- 172.21.15.57:0/4000396144 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c0011990 con 0x7fe1d015fe50 2023-11-28T12:02:20.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.195+0000 7fe1cd59a700 1 -- 172.21.15.57:0/4000396144 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0011d20 con 0x7fe1d015fe50 2023-11-28T12:02:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.195+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4000396144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d015fe50 msgr2=0x7fe1d006d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.195+0000 7fe1d4d2a700 1 --2- 172.21.15.57:0/4000396144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d015fe50 0x7fe1d006d170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009840 tx=0x7fe1c0009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.195+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4000396144 shutdown_connections 2023-11-28T12:02:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.195+0000 7fe1d4d2a700 1 --2- 172.21.15.57:0/4000396144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d015fe50 0x7fe1d006d170 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.195+0000 7fe1d4d2a700 1 --2- 172.21.15.57:0/4000396144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d00b3b80 0x7fe1d0160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.195+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4000396144 >> 172.21.15.57:0/4000396144 conn(0x7fe1d006ea80 msgr2=0x7fe1d006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.195+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4000396144 shutdown_connections 2023-11-28T12:02:20.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.195+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4000396144 wait complete. 2023-11-28T12:02:20.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.197+0000 7fe1d4d2a700 1 Processor -- start 2023-11-28T12:02:20.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.197+0000 7fe1d4d2a700 1 -- start start 2023-11-28T12:02:20.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.198+0000 7fe1d4d2a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d00b27d0 0x7fe1d014fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.198+0000 7fe1d4d2a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d00b3b80 0x7fe1d01e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.198+0000 7fe1d4d2a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d015fe50 0x7fe1d015a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.198+0000 7fe1d4d2a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d0150fa0 con 0x7fe1d015fe50 2023-11-28T12:02:20.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.198+0000 7fe1d4d2a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d014faf0 con 0x7fe1d00b3b80 2023-11-28T12:02:20.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.198+0000 7fe1d4d2a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d00c7e00 con 0x7fe1d00b27d0 2023-11-28T12:02:20.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.198+0000 7fe1ced9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d015fe50 0x7fe1d015a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.198+0000 7fe1ced9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d015fe50 0x7fe1d015a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58924/0 (socket says 172.21.15.57:58924) 2023-11-28T12:02:20.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.198+0000 7fe1ced9d700 1 -- 172.21.15.57:0/4210804645 learned_addr learned my addr 172.21.15.57:0/4210804645 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:20.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.199+0000 7fe1cdd9b700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d00b3b80 0x7fe1d01e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.199+0000 7fe1ce59c700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d00b27d0 0x7fe1d014fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.199+0000 7fe1ced9d700 1 -- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d00b27d0 msgr2=0x7fe1d014fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.199+0000 7fe1ced9d700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d00b27d0 0x7fe1d014fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.199+0000 7fe1ced9d700 1 -- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d00b3b80 msgr2=0x7fe1d01e7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.199+0000 7fe1ced9d700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d00b3b80 0x7fe1d01e7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.199+0000 7fe1ced9d700 1 -- 172.21.15.57:0/4210804645 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c00094a0 con 0x7fe1d015fe50 2023-11-28T12:02:20.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.199+0000 7fe1ced9d700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d015fe50 0x7fe1d015a8b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800cbd0 tx=0x7fe1c800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:20.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.199+0000 7fe1c6ffd700 1 -- 172.21.15.57:0/4210804645 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c800aa60 con 0x7fe1d015fe50 2023-11-28T12:02:20.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.199+0000 7fe1c6ffd700 1 -- 172.21.15.57:0/4210804645 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c800abe0 con 0x7fe1d015fe50 2023-11-28T12:02:20.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.199+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4210804645 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d00cf0b0 con 0x7fe1d015fe50 2023-11-28T12:02:20.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.199+0000 7fe1c6ffd700 1 -- 172.21.15.57:0/4210804645 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80070d0 con 0x7fe1d015fe50 2023-11-28T12:02:20.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.199+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4210804645 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d00cedd0 con 0x7fe1d015fe50 2023-11-28T12:02:20.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.200+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4210804645 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d00cf390 con 0x7fe1d015fe50 2023-11-28T12:02:20.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.201+0000 7fe1c6ffd700 1 -- 172.21.15.57:0/4210804645 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe1c801a030 con 0x7fe1d015fe50 2023-11-28T12:02:20.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.201+0000 7fe1c6ffd700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1b00682f0 0x7fe1b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.201+0000 7fe1c6ffd700 1 -- 172.21.15.57:0/4210804645 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe1c8085870 con 0x7fe1d015fe50 2023-11-28T12:02:20.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.201+0000 7fe1ce59c700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1b00682f0 0x7fe1b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.202+0000 7fe1ce59c700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1b00682f0 0x7fe1b006a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00090b0 tx=0x7fe1c001c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:20.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.206+0000 7fe1c6ffd700 1 -- 172.21.15.57:0/4210804645 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe1c8052070 con 0x7fe1d015fe50 2023-11-28T12:02:20.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.405+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4210804645 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe1d01510e0 con 0x7fe1d015fe50 2023-11-28T12:02:20.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.405+0000 7fe1c6ffd700 1 -- 172.21.15.57:0/4210804645 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe1c80180a0 con 0x7fe1d015fe50 2023-11-28T12:02:20.407 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:20.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.408+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1b00682f0 msgr2=0x7fe1b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.408+0000 7fe1d4d2a700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1b00682f0 0x7fe1b006a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00090b0 tx=0x7fe1c001c040 comp rx=0 tx=0).stop 2023-11-28T12:02:20.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.408+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4210804645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d015fe50 msgr2=0x7fe1d015a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.408+0000 7fe1d4d2a700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d015fe50 0x7fe1d015a8b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800cbd0 tx=0x7fe1c800cee0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.408+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4210804645 shutdown_connections 2023-11-28T12:02:20.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.408+0000 7fe1d4d2a700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1b00682f0 0x7fe1b006a7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.408+0000 7fe1d4d2a700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d015fe50 0x7fe1d015a8b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.408+0000 7fe1d4d2a700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d00b27d0 0x7fe1d014fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.408+0000 7fe1d4d2a700 1 --2- 172.21.15.57:0/4210804645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d00b3b80 0x7fe1d01e7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.408+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4210804645 >> 172.21.15.57:0/4210804645 conn(0x7fe1d006ea80 msgr2=0x7fe1d006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:20.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.408+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4210804645 shutdown_connections 2023-11-28T12:02:20.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.408+0000 7fe1d4d2a700 1 -- 172.21.15.57:0/4210804645 wait complete. 2023-11-28T12:02:20.423 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:20.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.561+0000 7f75934b8700 1 Processor -- start 2023-11-28T12:02:20.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.561+0000 7f75934b8700 1 -- start start 2023-11-28T12:02:20.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f75934b8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f758c0718e0 0x7f758c06c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f75934b8700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f758c06cdc0 0x7f758c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f75934b8700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758c160a50 con 0x7f758c15c810 2023-11-28T12:02:20.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f75934b8700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c15cbe0 con 0x7f758c06cdc0 2023-11-28T12:02:20.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f75934b8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c160d30 con 0x7f758c0718e0 2023-11-28T12:02:20.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f7591a55700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f758c06cdc0 0x7f758c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f7591254700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f758c15c810 0x7f758c160260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45844/0 (socket says 172.21.15.57:45844) 2023-11-28T12:02:20.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f7591254700 1 -- 172.21.15.57:0/642849572 learned_addr learned my addr 172.21.15.57:0/642849572 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:20.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f7591a55700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f758c06cdc0 0x7f758c06a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58934/0 (socket says 172.21.15.57:58934) 2023-11-28T12:02:20.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f7590a53700 1 --2- 172.21.15.57:0/642849572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f758c0718e0 0x7f758c06c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f7591a55700 1 -- 172.21.15.57:0/642849572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f758c0718e0 msgr2=0x7f758c06c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:20.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f7591a55700 1 --2- 172.21.15.57:0/642849572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f758c0718e0 0x7f758c06c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f7591a55700 1 -- 172.21.15.57:0/642849572 >> v1:172.21.15.57:6789/0 conn(0x7f758c15c810 legacy=0x7f758c160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f7583fff700 1 -- 172.21.15.57:0/642849572 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942484850 0 0) 0x7f758c160a50 con 0x7f758c15c810 2023-11-28T12:02:20.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f7591a55700 1 -- 172.21.15.57:0/642849572 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c161010 con 0x7f758c06cdc0 2023-11-28T12:02:20.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.562+0000 7f7591a55700 1 --2- 172.21.15.57:0/642849572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f758c06cdc0 0x7f758c06a270 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7584009d90 tx=0x7f758400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22cdf4cd73f6527 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:20.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.563+0000 7f7583fff700 1 -- 172.21.15.57:0/642849572 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758400e070 con 0x7f758c06cdc0 2023-11-28T12:02:20.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.563+0000 7f7583fff700 1 -- 172.21.15.57:0/642849572 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758400bbd0 con 0x7f758c06cdc0 2023-11-28T12:02:20.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.563+0000 7f7583fff700 1 -- 172.21.15.57:0/642849572 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584012430 con 0x7f758c06cdc0 2023-11-28T12:02:20.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.563+0000 7f75934b8700 1 -- 172.21.15.57:0/642849572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f758c06cdc0 msgr2=0x7f758c06a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.563+0000 7f75934b8700 1 --2- 172.21.15.57:0/642849572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f758c06cdc0 0x7f758c06a270 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7584009d90 tx=0x7f758400b330 comp rx=0 tx=0).stop 2023-11-28T12:02:20.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.563+0000 7f75934b8700 1 -- 172.21.15.57:0/642849572 shutdown_connections 2023-11-28T12:02:20.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.563+0000 7f75934b8700 1 --2- 172.21.15.57:0/642849572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f758c06cdc0 0x7f758c06a270 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.563+0000 7f75934b8700 1 --2- 172.21.15.57:0/642849572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f758c0718e0 0x7f758c06c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.563+0000 7f75934b8700 1 -- 172.21.15.57:0/642849572 >> 172.21.15.57:0/642849572 conn(0x7f758c06c010 msgr2=0x7f758c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:20.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.563+0000 7f75934b8700 1 -- 172.21.15.57:0/642849572 shutdown_connections 2023-11-28T12:02:20.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.563+0000 7f75934b8700 1 -- 172.21.15.57:0/642849572 wait complete. 2023-11-28T12:02:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.565+0000 7f75934b8700 1 Processor -- start 2023-11-28T12:02:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.565+0000 7f75934b8700 1 -- start start 2023-11-28T12:02:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f75934b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f758c06cdc0 0x7f758c1f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f75934b8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f758c0718e0 0x7f758c0c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f75934b8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f758c15c810 0x7f758c0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f75934b8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c152b70 con 0x7f758c0718e0 2023-11-28T12:02:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f75934b8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c1f1ce0 con 0x7f758c06cdc0 2023-11-28T12:02:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f75934b8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0c9190 con 0x7f758c15c810 2023-11-28T12:02:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f7590a53700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f758c0718e0 0x7f758c0c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f7590a53700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f758c0718e0 0x7f758c0c4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58944/0 (socket says 172.21.15.57:58944) 2023-11-28T12:02:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f7590a53700 1 -- 172.21.15.57:0/2472032913 learned_addr learned my addr 172.21.15.57:0/2472032913 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f7591a55700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f758c15c810 0x7f758c0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f7591254700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f758c06cdc0 0x7f758c1f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f7590a53700 1 -- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f758c15c810 msgr2=0x7f758c0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f7590a53700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f758c15c810 0x7f758c0cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f7590a53700 1 -- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f758c06cdc0 msgr2=0x7f758c1f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f7590a53700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f758c06cdc0 0x7f758c1f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f7590a53700 1 -- 172.21.15.57:0/2472032913 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7584009a70 con 0x7f758c0718e0 2023-11-28T12:02:20.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f7591a55700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f758c15c810 0x7f758c0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:20.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.566+0000 7f7591254700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f758c06cdc0 0x7f758c1f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:20.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.567+0000 7f7590a53700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f758c0718e0 0x7f758c0c4010 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f758800e810 tx=0x7f758800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:20.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.567+0000 7f7581ffb700 1 -- 172.21.15.57:0/2472032913 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588013610 con 0x7f758c0718e0 2023-11-28T12:02:20.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.567+0000 7f7581ffb700 1 -- 172.21.15.57:0/2472032913 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7588013770 con 0x7f758c0718e0 2023-11-28T12:02:20.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.567+0000 7f75934b8700 1 -- 172.21.15.57:0/2472032913 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c0ce740 con 0x7f758c0718e0 2023-11-28T12:02:20.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.567+0000 7f7581ffb700 1 -- 172.21.15.57:0/2472032913 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588013b20 con 0x7f758c0718e0 2023-11-28T12:02:20.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.567+0000 7f75934b8700 1 -- 172.21.15.57:0/2472032913 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c0ceb20 con 0x7f758c0718e0 2023-11-28T12:02:20.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.568+0000 7f75934b8700 1 -- 172.21.15.57:0/2472032913 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c0ce8d0 con 0x7f758c0718e0 2023-11-28T12:02:20.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.569+0000 7f7581ffb700 1 -- 172.21.15.57:0/2472032913 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7588023070 con 0x7f758c0718e0 2023-11-28T12:02:20.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.569+0000 7f7581ffb700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f75700682f0 0x7f757006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.569+0000 7f7581ffb700 1 -- 172.21.15.57:0/2472032913 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f75880859b0 con 0x7f758c0718e0 2023-11-28T12:02:20.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.569+0000 7f7591254700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f75700682f0 0x7f757006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.570+0000 7f7591254700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f75700682f0 0x7f757006a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f757c007fd0 tx=0x7f757c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:20.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.574+0000 7f7581ffb700 1 -- 172.21.15.57:0/2472032913 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7588052190 con 0x7f758c0718e0 2023-11-28T12:02:20.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.777+0000 7f75934b8700 1 -- 172.21.15.57:0/2472032913 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f758c0743f0 con 0x7f758c0718e0 2023-11-28T12:02:20.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.777+0000 7f7581ffb700 1 -- 172.21.15.57:0/2472032913 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75880557b0 con 0x7f758c0718e0 2023-11-28T12:02:20.778 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:20.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.780+0000 7f75934b8700 1 -- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f75700682f0 msgr2=0x7f757006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.780+0000 7f75934b8700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f75700682f0 0x7f757006a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f757c007fd0 tx=0x7f757c009040 comp rx=0 tx=0).stop 2023-11-28T12:02:20.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.780+0000 7f75934b8700 1 -- 172.21.15.57:0/2472032913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f758c0718e0 msgr2=0x7f758c0c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.780+0000 7f75934b8700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f758c0718e0 0x7f758c0c4010 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f758800e810 tx=0x7f758800ebd0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.780+0000 7f75934b8700 1 -- 172.21.15.57:0/2472032913 shutdown_connections 2023-11-28T12:02:20.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.780+0000 7f75934b8700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f75700682f0 0x7f757006a7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.780+0000 7f75934b8700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f758c15c810 0x7f758c0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.780+0000 7f75934b8700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f758c06cdc0 0x7f758c1f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.780+0000 7f75934b8700 1 --2- 172.21.15.57:0/2472032913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f758c0718e0 0x7f758c0c4010 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.780+0000 7f75934b8700 1 -- 172.21.15.57:0/2472032913 >> 172.21.15.57:0/2472032913 conn(0x7f758c06c010 msgr2=0x7f758c158070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:20.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.780+0000 7f75934b8700 1 -- 172.21.15.57:0/2472032913 shutdown_connections 2023-11-28T12:02:20.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.780+0000 7f75934b8700 1 -- 172.21.15.57:0/2472032913 wait complete. 2023-11-28T12:02:20.796 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:02:20.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.933+0000 7f1acbe6e700 1 Processor -- start 2023-11-28T12:02:20.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.933+0000 7f1acbe6e700 1 -- start start 2023-11-28T12:02:20.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.934+0000 7f1acbe6e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ac40cf710 0x7f1ac40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.934+0000 7f1acbe6e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1ac4071890 0x7f1ac4071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.934+0000 7f1acbe6e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac415cc10 con 0x7f1ac406a270 2023-11-28T12:02:20.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.934+0000 7f1acbe6e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac406c9b0 con 0x7f1ac4071890 2023-11-28T12:02:20.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.934+0000 7f1acbe6e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac406cc90 con 0x7f1ac40cf710 2023-11-28T12:02:20.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.934+0000 7f1aca40b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f1ac406a270 0x7f1ac40b1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45850/0 (socket says 172.21.15.57:45850) 2023-11-28T12:02:20.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.934+0000 7f1aca40b700 1 -- 172.21.15.57:0/1095997447 learned_addr learned my addr 172.21.15.57:0/1095997447 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:20.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac9409700 1 --2- 172.21.15.57:0/1095997447 >> v2:172.21.15.181:3300/0 conn(0x7f1ac4071890 0x7f1ac4071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac9c0a700 1 --2- 172.21.15.57:0/1095997447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ac40cf710 0x7f1ac40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189343371 0 0) 0x7f1ac415cc10 con 0x7f1ac406a270 2023-11-28T12:02:20.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ab00035b0 con 0x7f1ac406a270 2023-11-28T12:02:20.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 54551920 0 0) 0x7f1ab00035b0 con 0x7f1ac406a270 2023-11-28T12:02:20.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ac40cf710 msgr2=0x7f1ac40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:20.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 --2- 172.21.15.57:0/1095997447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ac40cf710 0x7f1ac40ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 >> v2:172.21.15.181:3300/0 conn(0x7f1ac4071890 msgr2=0x7f1ac4071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:20.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 --2- 172.21.15.57:0/1095997447 >> v2:172.21.15.181:3300/0 conn(0x7f1ac4071890 0x7f1ac4071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac406cf70 con 0x7f1ac406a270 2023-11-28T12:02:20.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f1abc0019f0 con 0x7f1ac406a270 2023-11-28T12:02:20.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 >> v1:172.21.15.57:6789/0 conn(0x7f1ac406a270 legacy=0x7f1ac40b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 --2- 172.21.15.57:0/1095997447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ac40cf710 0x7f1ab0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 --2- 172.21.15.57:0/1095997447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ab0003f80 0x7f1ab0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 --2- 172.21.15.57:0/1095997447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ab0004d90 0x7f1ab0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac406c9b0 con 0x7f1ab0003f80 2023-11-28T12:02:20.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac406cc90 con 0x7f1ab0004d90 2023-11-28T12:02:20.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab000ba40 con 0x7f1ac40cf710 2023-11-28T12:02:20.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac9409700 1 --2- 172.21.15.57:0/1095997447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ab0003f80 0x7f1ab0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1ac9c0a700 1 --2- 172.21.15.57:0/1095997447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ac40cf710 0x7f1ab0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.935+0000 7f1aca40b700 1 --2- 172.21.15.57:0/1095997447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ab0004d90 0x7f1ab0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.936+0000 7f1ac9409700 1 -- 172.21.15.57:0/1095997447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ac40cf710 msgr2=0x7f1ab0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:20.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.936+0000 7f1ac9409700 1 --2- 172.21.15.57:0/1095997447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ac40cf710 0x7f1ab0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.936+0000 7f1ac9409700 1 -- 172.21.15.57:0/1095997447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ab0004d90 msgr2=0x7f1ab0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:20.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.936+0000 7f1ac9409700 1 --2- 172.21.15.57:0/1095997447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ab0004d90 0x7f1ab0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.936+0000 7f1ac9409700 1 -- 172.21.15.57:0/1095997447 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab00035b0 con 0x7f1ab0003f80 2023-11-28T12:02:20.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.936+0000 7f1ac9409700 1 --2- 172.21.15.57:0/1095997447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ab0003f80 0x7f1ab0004850 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009b60 tx=0x7f1ab0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a714ca95f37233 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:20.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.936+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac000e660 con 0x7f1ab0003f80 2023-11-28T12:02:20.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.936+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac000e7c0 con 0x7f1ab0003f80 2023-11-28T12:02:20.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.936+0000 7f1ac8c08700 1 -- 172.21.15.57:0/1095997447 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac000eb30 con 0x7f1ab0003f80 2023-11-28T12:02:20.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.936+0000 7f1acbe6e700 1 -- 172.21.15.57:0/1095997447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ab0003f80 msgr2=0x7f1ab0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.936+0000 7f1acbe6e700 1 --2- 172.21.15.57:0/1095997447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ab0003f80 0x7f1ab0004850 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009b60 tx=0x7f1ab0005880 comp rx=0 tx=0).stop 2023-11-28T12:02:20.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.937+0000 7f1aca40b700 1 -- 172.21.15.57:0/1095997447 reap_dead start 2023-11-28T12:02:20.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.937+0000 7f1acbe6e700 1 -- 172.21.15.57:0/1095997447 shutdown_connections 2023-11-28T12:02:20.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.937+0000 7f1acbe6e700 1 -- 172.21.15.57:0/1095997447 >> 172.21.15.57:0/1095997447 conn(0x7f1ac406c010 msgr2=0x7f1ac40cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:20.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.937+0000 7f1acbe6e700 1 -- 172.21.15.57:0/1095997447 shutdown_connections 2023-11-28T12:02:20.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.937+0000 7f1acbe6e700 1 -- 172.21.15.57:0/1095997447 wait complete. 2023-11-28T12:02:20.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.938+0000 7f1acbe6e700 1 Processor -- start 2023-11-28T12:02:20.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.938+0000 7f1acbe6e700 1 -- start start 2023-11-28T12:02:20.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.939+0000 7f1acbe6e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ab0003f80 0x7f1ac40c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.939+0000 7f1acbe6e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ac4154b50 0x7f1ac4154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.939+0000 7f1acbe6e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac40aec90 0x7f1ac40af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.939+0000 7f1acbe6e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac4155500 con 0x7f1ac40aec90 2023-11-28T12:02:20.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.939+0000 7f1acbe6e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac40b05b0 con 0x7f1ac4154b50 2023-11-28T12:02:20.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.939+0000 7f1acbe6e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac40c5ea0 con 0x7f1ab0003f80 2023-11-28T12:02:20.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1aca40b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac40aec90 0x7f1ac40af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1aca40b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac40aec90 0x7f1ac40af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58972/0 (socket says 172.21.15.57:58972) 2023-11-28T12:02:20.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1aca40b700 1 -- 172.21.15.57:0/3193081295 learned_addr learned my addr 172.21.15.57:0/3193081295 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:20.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1ac9c0a700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ab0003f80 0x7f1ac40c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1ac9409700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ac4154b50 0x7f1ac4154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1aca40b700 1 -- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ab0003f80 msgr2=0x7f1ac40c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1aca40b700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ab0003f80 0x7f1ac40c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1aca40b700 1 -- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ac4154b50 msgr2=0x7f1ac4154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:20.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1aca40b700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ac4154b50 0x7f1ac4154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:20.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1aca40b700 1 -- 172.21.15.57:0/3193081295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac406d530 con 0x7f1ac40aec90 2023-11-28T12:02:20.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1ac9409700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ac4154b50 0x7f1ac4154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:20.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1ac9c0a700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ab0003f80 0x7f1ac40c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:20.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1aca40b700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac40aec90 0x7f1ac40af100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1ac406b2b0 tx=0x7f1abc00a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:20.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1ab67fc700 1 -- 172.21.15.57:0/3193081295 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc00ae40 con 0x7f1ac40aec90 2023-11-28T12:02:20.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1ab67fc700 1 -- 172.21.15.57:0/3193081295 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1abc0133f0 con 0x7f1ac40aec90 2023-11-28T12:02:20.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1ab67fc700 1 -- 172.21.15.57:0/3193081295 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0137a0 con 0x7f1ac40aec90 2023-11-28T12:02:20.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1acbe6e700 1 -- 172.21.15.57:0/3193081295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac406d250 con 0x7f1ac40aec90 2023-11-28T12:02:20.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.940+0000 7f1acbe6e700 1 -- 172.21.15.57:0/3193081295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac40ce740 con 0x7f1ac40aec90 2023-11-28T12:02:20.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.941+0000 7f1acbe6e700 1 -- 172.21.15.57:0/3193081295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac40743f0 con 0x7f1ac40aec90 2023-11-28T12:02:20.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.942+0000 7f1ab67fc700 1 -- 172.21.15.57:0/3193081295 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1abc013900 con 0x7f1ac40aec90 2023-11-28T12:02:20.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.942+0000 7f1ab67fc700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1aa80682f0 0x7f1aa806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:20.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.942+0000 7f1ab67fc700 1 -- 172.21.15.57:0/3193081295 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1abc0867c0 con 0x7f1ac40aec90 2023-11-28T12:02:20.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.943+0000 7f1ac9c0a700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1aa80682f0 0x7f1aa806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:20.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.943+0000 7f1ac9c0a700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1aa80682f0 0x7f1aa806a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009b80 tx=0x7f1ab8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:20.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:20.947+0000 7f1ab67fc700 1 -- 172.21.15.57:0/3193081295 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1abc053020 con 0x7f1ac40aec90 2023-11-28T12:02:21.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.146+0000 7f1acbe6e700 1 -- 172.21.15.57:0/3193081295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ac40b5b50 con 0x7f1ac40aec90 2023-11-28T12:02:21.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.146+0000 7f1ab67fc700 1 -- 172.21.15.57:0/3193081295 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1abc056640 con 0x7f1ac40aec90 2023-11-28T12:02:21.148 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:02:21.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.148+0000 7f1acbe6e700 1 -- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1aa80682f0 msgr2=0x7f1aa806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.148+0000 7f1acbe6e700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1aa80682f0 0x7f1aa806a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009b80 tx=0x7f1ab8008040 comp rx=0 tx=0).stop 2023-11-28T12:02:21.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.148+0000 7f1acbe6e700 1 -- 172.21.15.57:0/3193081295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac40aec90 msgr2=0x7f1ac40af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.148+0000 7f1acbe6e700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac40aec90 0x7f1ac40af100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1ac406b2b0 tx=0x7f1abc00a640 comp rx=0 tx=0).stop 2023-11-28T12:02:21.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.148+0000 7f1acbe6e700 1 -- 172.21.15.57:0/3193081295 shutdown_connections 2023-11-28T12:02:21.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.148+0000 7f1acbe6e700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1aa80682f0 0x7f1aa806a7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.148+0000 7f1acbe6e700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ac40aec90 0x7f1ac40af100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.148+0000 7f1acbe6e700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ab0003f80 0x7f1ac40c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.148+0000 7f1acbe6e700 1 --2- 172.21.15.57:0/3193081295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ac4154b50 0x7f1ac4154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.148+0000 7f1acbe6e700 1 -- 172.21.15.57:0/3193081295 >> 172.21.15.57:0/3193081295 conn(0x7f1ac406c010 msgr2=0x7f1ac415be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:21.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.148+0000 7f1acbe6e700 1 -- 172.21.15.57:0/3193081295 shutdown_connections 2023-11-28T12:02:21.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.148+0000 7f1acbe6e700 1 -- 172.21.15.57:0/3193081295 wait complete. 2023-11-28T12:02:21.163 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:02:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.300+0000 7f23768e7700 1 Processor -- start 2023-11-28T12:02:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.300+0000 7f23768e7700 1 -- start start 2023-11-28T12:02:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.301+0000 7f23768e7700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f237006a980 0x7f23700ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.301+0000 7f23768e7700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237015cc10 con 0x7f2370071890 2023-11-28T12:02:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.301+0000 7f23768e7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237006c9b0 con 0x7f23700cf710 2023-11-28T12:02:21.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.301+0000 7f23768e7700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237006cc90 con 0x7f237006a980 2023-11-28T12:02:21.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.301+0000 7f236f7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f237006a980 0x7f23700ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:21.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.301+0000 7f2374e84700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2370071890 0x7f2370160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45862/0 (socket says 172.21.15.57:45862) 2023-11-28T12:02:21.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.301+0000 7f2374e84700 1 -- 172.21.15.57:0/640815017 learned_addr learned my addr 172.21.15.57:0/640815017 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:21.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.302+0000 7f236f7fe700 1 -- 172.21.15.57:0/640815017 >> v1:172.21.15.181:6789/0 conn(0x7f23700cf710 legacy=0x7f237006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.302+0000 7f236f7fe700 1 -- 172.21.15.57:0/640815017 >> v1:172.21.15.57:6789/0 conn(0x7f2370071890 legacy=0x7f2370160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.302+0000 7f236f7fe700 1 -- 172.21.15.57:0/640815017 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237006cf70 con 0x7f237006a980 2023-11-28T12:02:21.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.302+0000 7f236effd700 1 -- 172.21.15.57:0/640815017 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310148305 0 0) 0x7f237015cc10 con 0x7f2370071890 2023-11-28T12:02:21.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.302+0000 7f236f7fe700 1 --2- 172.21.15.57:0/640815017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f237006a980 0x7f23700ce740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2364009840 tx=0x7f2364009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df82e7e71885f52d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:21.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.302+0000 7f236effd700 1 -- 172.21.15.57:0/640815017 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364011830 con 0x7f237006a980 2023-11-28T12:02:21.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.302+0000 7f236effd700 1 -- 172.21.15.57:0/640815017 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364011990 con 0x7f237006a980 2023-11-28T12:02:21.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.302+0000 7f236effd700 1 -- 172.21.15.57:0/640815017 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364011d20 con 0x7f237006a980 2023-11-28T12:02:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.303+0000 7f23768e7700 1 -- 172.21.15.57:0/640815017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f237006a980 msgr2=0x7f23700ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.303+0000 7f23768e7700 1 --2- 172.21.15.57:0/640815017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f237006a980 0x7f23700ce740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2364009840 tx=0x7f2364009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.303+0000 7f23768e7700 1 -- 172.21.15.57:0/640815017 shutdown_connections 2023-11-28T12:02:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.303+0000 7f23768e7700 1 --2- 172.21.15.57:0/640815017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f237006a980 0x7f23700ce740 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.303+0000 7f23768e7700 1 -- 172.21.15.57:0/640815017 >> 172.21.15.57:0/640815017 conn(0x7f237006c010 msgr2=0x7f23700cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.303+0000 7f23768e7700 1 -- 172.21.15.57:0/640815017 shutdown_connections 2023-11-28T12:02:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.303+0000 7f23768e7700 1 -- 172.21.15.57:0/640815017 wait complete. 2023-11-28T12:02:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.304+0000 7f23768e7700 1 Processor -- start 2023-11-28T12:02:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.304+0000 7f23768e7700 1 -- start start 2023-11-28T12:02:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.305+0000 7f23768e7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f237006a980 0x7f23701f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.305+0000 7f23768e7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2370071890 0x7f23701f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.305+0000 7f23768e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23700cf710 0x7f23700b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.305+0000 7f23768e7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23700b2a70 con 0x7f237006a980 2023-11-28T12:02:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.305+0000 7f23768e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23701f2ea0 con 0x7f23700cf710 2023-11-28T12:02:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.305+0000 7f23768e7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237015ebd0 con 0x7f2370071890 2023-11-28T12:02:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.305+0000 7f236ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f237006a980 0x7f23701f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.305+0000 7f236ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f237006a980 0x7f23701f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58982/0 (socket says 172.21.15.57:58982) 2023-11-28T12:02:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.305+0000 7f236ffff700 1 -- 172.21.15.57:0/1212966432 learned_addr learned my addr 172.21.15.57:0/1212966432 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.305+0000 7f2374e84700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23700cf710 0x7f23700b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.305+0000 7f236f7fe700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2370071890 0x7f23701f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.305+0000 7f236ffff700 1 -- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2370071890 msgr2=0x7f23701f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.306+0000 7f236ffff700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2370071890 0x7f23701f17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.306+0000 7f236ffff700 1 -- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23700cf710 msgr2=0x7f23700b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.306+0000 7f236ffff700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23700cf710 0x7f23700b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.306+0000 7f236ffff700 1 -- 172.21.15.57:0/1212966432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23640094a0 con 0x7f237006a980 2023-11-28T12:02:21.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.306+0000 7f2374e84700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23700cf710 0x7f23700b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:21.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.306+0000 7f236ffff700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f237006a980 0x7f23701f2960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f236000c450 tx=0x7f236000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.306+0000 7f236cff9700 1 -- 172.21.15.57:0/1212966432 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600052d0 con 0x7f237006a980 2023-11-28T12:02:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.306+0000 7f236cff9700 1 -- 172.21.15.57:0/1212966432 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360005430 con 0x7f237006a980 2023-11-28T12:02:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.306+0000 7f23768e7700 1 -- 172.21.15.57:0/1212966432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237006d530 con 0x7f237006a980 2023-11-28T12:02:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.306+0000 7f236cff9700 1 -- 172.21.15.57:0/1212966432 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600038c0 con 0x7f237006a980 2023-11-28T12:02:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.306+0000 7f23768e7700 1 -- 172.21.15.57:0/1212966432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237006d250 con 0x7f237006a980 2023-11-28T12:02:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.307+0000 7f23768e7700 1 -- 172.21.15.57:0/1212966432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23700743f0 con 0x7f237006a980 2023-11-28T12:02:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.307+0000 7f236cff9700 1 -- 172.21.15.57:0/1212966432 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2360019030 con 0x7f237006a980 2023-11-28T12:02:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.308+0000 7f236cff9700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f235c0682f0 0x7f235c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.308+0000 7f236cff9700 1 -- 172.21.15.57:0/1212966432 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2360084fc0 con 0x7f237006a980 2023-11-28T12:02:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.308+0000 7f236f7fe700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f235c0682f0 0x7f235c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.309+0000 7f236f7fe700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f235c0682f0 0x7f235c06a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f23700af380 tx=0x7f236401c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:21.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.312+0000 7f236cff9700 1 -- 172.21.15.57:0/1212966432 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2360051820 con 0x7f237006a980 2023-11-28T12:02:21.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.512+0000 7f23768e7700 1 -- 172.21.15.57:0/1212966432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23700711a0 con 0x7f237006a980 2023-11-28T12:02:21.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.513+0000 7f236cff9700 1 -- 172.21.15.57:0/1212966432 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2360054e40 con 0x7f237006a980 2023-11-28T12:02:21.514 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:02:21.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.515+0000 7f23768e7700 1 -- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f235c0682f0 msgr2=0x7f235c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.515+0000 7f23768e7700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f235c0682f0 0x7f235c06a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f23700af380 tx=0x7f236401c040 comp rx=0 tx=0).stop 2023-11-28T12:02:21.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.515+0000 7f23768e7700 1 -- 172.21.15.57:0/1212966432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f237006a980 msgr2=0x7f23701f2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.515+0000 7f23768e7700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f237006a980 0x7f23701f2960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f236000c450 tx=0x7f236000c810 comp rx=0 tx=0).stop 2023-11-28T12:02:21.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.515+0000 7f23768e7700 1 -- 172.21.15.57:0/1212966432 shutdown_connections 2023-11-28T12:02:21.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.515+0000 7f23768e7700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f235c0682f0 0x7f235c06a7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.515+0000 7f23768e7700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23700cf710 0x7f23700b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.515+0000 7f23768e7700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f237006a980 0x7f23701f2960 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.515+0000 7f23768e7700 1 --2- 172.21.15.57:0/1212966432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2370071890 0x7f23701f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.515+0000 7f23768e7700 1 -- 172.21.15.57:0/1212966432 >> 172.21.15.57:0/1212966432 conn(0x7f237006c010 msgr2=0x7f237015c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:21.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.515+0000 7f23768e7700 1 -- 172.21.15.57:0/1212966432 shutdown_connections 2023-11-28T12:02:21.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.515+0000 7f23768e7700 1 -- 172.21.15.57:0/1212966432 wait complete. 2023-11-28T12:02:21.530 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T12:02:21.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.665+0000 7fadbcdd1700 1 Processor -- start 2023-11-28T12:02:21.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.665+0000 7fadbcdd1700 1 -- start start 2023-11-28T12:02:21.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.666+0000 7fadbcdd1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fadb80af0a0 0x7fadb815a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:21.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.666+0000 7fadbcdd1700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb8159ec0 con 0x7fadb80b1650 2023-11-28T12:02:21.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.666+0000 7fadbcdd1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb815d8a0 con 0x7fadb81599b0 2023-11-28T12:02:21.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.666+0000 7fadbcdd1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb815db80 con 0x7fadb80af0a0 2023-11-28T12:02:21.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb659c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fadb80b1650 0x7fadb80b1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45876/0 (socket says 172.21.15.57:45876) 2023-11-28T12:02:21.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb659c700 1 -- 172.21.15.57:0/2502032 learned_addr learned my addr 172.21.15.57:0/2502032 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:21.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb6d9d700 1 --2- 172.21.15.57:0/2502032 >> v2:172.21.15.181:3300/0 conn(0x7fadb80af0a0 0x7fadb815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:21.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148411703 0 0) 0x7fadb8159ec0 con 0x7fadb80b1650 2023-11-28T12:02:21.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fada00035b0 con 0x7fadb80b1650 2023-11-28T12:02:21.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1302693714 0 0) 0x7fada00035b0 con 0x7fadb80b1650 2023-11-28T12:02:21.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 >> v2:172.21.15.181:3300/0 conn(0x7fadb80af0a0 msgr2=0x7fadb815a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:21.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 --2- 172.21.15.57:0/2502032 >> v2:172.21.15.181:3300/0 conn(0x7fadb80af0a0 0x7fadb815a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 >> v1:172.21.15.181:6789/0 conn(0x7fadb81599b0 legacy=0x7fadb815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb815de60 con 0x7fadb80b1650 2023-11-28T12:02:21.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fada8001670 con 0x7fadb80b1650 2023-11-28T12:02:21.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb6d9d700 1 --2- 172.21.15.57:0/2502032 >> v2:172.21.15.181:3300/0 conn(0x7fadb80af0a0 0x7fadb815a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:21.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 >> v1:172.21.15.57:6789/0 conn(0x7fadb80b1650 legacy=0x7fadb80b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 --2- 172.21.15.57:0/2502032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fada0003a00 0x7fada0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:21.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 --2- 172.21.15.57:0/2502032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0004850 0x7fada0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:21.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 --2- 172.21.15.57:0/2502032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fada0005200 0x7fada000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:21.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb815db80 con 0x7fada0003a00 2023-11-28T12:02:21.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada000c080 con 0x7fada0004850 2023-11-28T12:02:21.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada000c240 con 0x7fada0005200 2023-11-28T12:02:21.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.667+0000 7fadb659c700 1 --2- 172.21.15.57:0/2502032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fada0003a00 0x7fada0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:21.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.668+0000 7fadb5d9b700 1 --2- 172.21.15.57:0/2502032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0004850 0x7fada0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:21.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.668+0000 7fadb6d9d700 1 --2- 172.21.15.57:0/2502032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fada0005200 0x7fada000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:21.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.668+0000 7fadb5d9b700 1 -- 172.21.15.57:0/2502032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fada0005200 msgr2=0x7fada000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:21.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.668+0000 7fadb5d9b700 1 --2- 172.21.15.57:0/2502032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fada0005200 0x7fada000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.668+0000 7fadb5d9b700 1 -- 172.21.15.57:0/2502032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fada0003a00 msgr2=0x7fada0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:21.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.668+0000 7fadb5d9b700 1 --2- 172.21.15.57:0/2502032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fada0003a00 0x7fada0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.668+0000 7fadb5d9b700 1 -- 172.21.15.57:0/2502032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada00035b0 con 0x7fada0004850 2023-11-28T12:02:21.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.668+0000 7fadb6d9d700 1 -- 172.21.15.57:0/2502032 reap_dead start 2023-11-28T12:02:21.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.668+0000 7fadb659c700 1 --2- 172.21.15.57:0/2502032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fada0003a00 0x7fada0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:21.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.668+0000 7fadb5d9b700 1 --2- 172.21.15.57:0/2502032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0004850 0x7fada0004cc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fadb806eb00 tx=0x7fada0005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba184548b67b2e2e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:21.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.668+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac0052d0 con 0x7fada0004850 2023-11-28T12:02:21.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.668+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac005430 con 0x7fada0004850 2023-11-28T12:02:21.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.668+0000 7fadb559a700 1 -- 172.21.15.57:0/2502032 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac0038c0 con 0x7fada0004850 2023-11-28T12:02:21.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.669+0000 7fadbcdd1700 1 -- 172.21.15.57:0/2502032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0004850 msgr2=0x7fada0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.669+0000 7fadbcdd1700 1 --2- 172.21.15.57:0/2502032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0004850 0x7fada0004cc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fadb806eb00 tx=0x7fada0005880 comp rx=0 tx=0).stop 2023-11-28T12:02:21.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.669+0000 7fadbcdd1700 1 -- 172.21.15.57:0/2502032 shutdown_connections 2023-11-28T12:02:21.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.669+0000 7fadbcdd1700 1 --2- 172.21.15.57:0/2502032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0004850 0x7fada0004cc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.669+0000 7fadbcdd1700 1 -- 172.21.15.57:0/2502032 >> 172.21.15.57:0/2502032 conn(0x7fadb806c010 msgr2=0x7fadb80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:21.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.669+0000 7fadbcdd1700 1 -- 172.21.15.57:0/2502032 shutdown_connections 2023-11-28T12:02:21.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.669+0000 7fadbcdd1700 1 -- 172.21.15.57:0/2502032 wait complete. 2023-11-28T12:02:21.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.670+0000 7fadbcdd1700 1 Processor -- start 2023-11-28T12:02:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.670+0000 7fadbcdd1700 1 -- start start 2023-11-28T12:02:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.671+0000 7fadbcdd1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0004850 0x7fadb80b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.671+0000 7fadbcdd1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadb80b4510 0x7fadb81f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.671+0000 7fadbcdd1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadb80b4980 0x7fadb81f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.671+0000 7fadbcdd1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80c6830 con 0x7fadb80b4980 2023-11-28T12:02:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.671+0000 7fadbcdd1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80b2c30 con 0x7fada0004850 2023-11-28T12:02:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.671+0000 7fadbcdd1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80b2f10 con 0x7fadb80b4510 2023-11-28T12:02:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.671+0000 7fadb6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadb80b4980 0x7fadb81f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.671+0000 7fadb6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadb80b4980 0x7fadb81f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59004/0 (socket says 172.21.15.57:59004) 2023-11-28T12:02:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.671+0000 7fadb6d9d700 1 -- 172.21.15.57:0/308785841 learned_addr learned my addr 172.21.15.57:0/308785841 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:21.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.672+0000 7fadb659c700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0004850 0x7fadb80b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:21.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.672+0000 7fadb5d9b700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadb80b4510 0x7fadb81f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:21.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.672+0000 7fadb6d9d700 1 -- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadb80b4510 msgr2=0x7fadb81f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.672+0000 7fadb6d9d700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadb80b4510 0x7fadb81f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.672+0000 7fadb6d9d700 1 -- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0004850 msgr2=0x7fadb80b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.672+0000 7fadb6d9d700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0004850 0x7fadb80b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.672+0000 7fadb6d9d700 1 -- 172.21.15.57:0/308785841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb80b1bb0 con 0x7fadb80b4980 2023-11-28T12:02:21.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.672+0000 7fadb6d9d700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadb80b4980 0x7fadb81f1ff0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fadb80b2620 tx=0x7fadb000f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:21.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.672+0000 7fada6ffd700 1 -- 172.21.15.57:0/308785841 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb001c070 con 0x7fadb80b4980 2023-11-28T12:02:21.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.672+0000 7fada6ffd700 1 -- 172.21.15.57:0/308785841 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb000f040 con 0x7fadb80b4980 2023-11-28T12:02:21.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.672+0000 7fadbcdd1700 1 -- 172.21.15.57:0/308785841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb80b1a20 con 0x7fadb80b4980 2023-11-28T12:02:21.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.672+0000 7fada6ffd700 1 -- 172.21.15.57:0/308785841 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb000e040 con 0x7fadb80b4980 2023-11-28T12:02:21.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.672+0000 7fadbcdd1700 1 -- 172.21.15.57:0/308785841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb80b1d40 con 0x7fadb80b4980 2023-11-28T12:02:21.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.673+0000 7fadbcdd1700 1 -- 172.21.15.57:0/308785841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb80b2d70 con 0x7fadb80b4980 2023-11-28T12:02:21.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.674+0000 7fada6ffd700 1 -- 172.21.15.57:0/308785841 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fadb000fd90 con 0x7fadb80b4980 2023-11-28T12:02:21.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.674+0000 7fada6ffd700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fad980682f0 0x7fad9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:21.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.674+0000 7fada6ffd700 1 -- 172.21.15.57:0/308785841 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fadb0085b60 con 0x7fadb80b4980 2023-11-28T12:02:21.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.674+0000 7fadb659c700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fad980682f0 0x7fad9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:21.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.675+0000 7fadb659c700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fad980682f0 0x7fad9806a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fada8000c00 tx=0x7fada80011c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:21.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.679+0000 7fada6ffd700 1 -- 172.21.15.57:0/308785841 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fadb80b2d70 con 0x7fadb80b4980 2023-11-28T12:02:21.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.902+0000 7fadbcdd1700 1 -- 172.21.15.57:0/308785841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fadb80b2d70 con 0x7fadb80b4980 2023-11-28T12:02:21.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.902+0000 7fada6ffd700 1 -- 172.21.15.57:0/308785841 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fadb80b2d70 con 0x7fadb80b4980 2023-11-28T12:02:21.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.905+0000 7fadbcdd1700 1 -- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fad980682f0 msgr2=0x7fad9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.905+0000 7fadbcdd1700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fad980682f0 0x7fad9806a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fada8000c00 tx=0x7fada80011c0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.905+0000 7fadbcdd1700 1 -- 172.21.15.57:0/308785841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadb80b4980 msgr2=0x7fadb81f1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:21.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.905+0000 7fadbcdd1700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadb80b4980 0x7fadb81f1ff0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fadb80b2620 tx=0x7fadb000f490 comp rx=0 tx=0).stop 2023-11-28T12:02:21.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.905+0000 7fadbcdd1700 1 -- 172.21.15.57:0/308785841 shutdown_connections 2023-11-28T12:02:21.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.905+0000 7fadbcdd1700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fad980682f0 0x7fad9806a7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.905+0000 7fadbcdd1700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fadb80b4980 0x7fadb81f1ff0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.905+0000 7fadbcdd1700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fada0004850 0x7fadb80b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.905+0000 7fadbcdd1700 1 --2- 172.21.15.57:0/308785841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadb80b4510 0x7fadb81f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:21.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.905+0000 7fadbcdd1700 1 -- 172.21.15.57:0/308785841 >> 172.21.15.57:0/308785841 conn(0x7fadb806c010 msgr2=0x7fadb80b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:21.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.905+0000 7fadbcdd1700 1 -- 172.21.15.57:0/308785841 shutdown_connections 2023-11-28T12:02:21.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:21.905+0000 7fadbcdd1700 1 -- 172.21.15.57:0/308785841 wait complete. 2023-11-28T12:02:21.907 INFO:teuthology.orchestra.run.smithi057.stderr:listed 0 entries 2023-11-28T12:02:21.920 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2023-11-28T12:02:21.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:21.920 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T12:02:21.963 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:02:21.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:02:21.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:21.964 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T12:02:22.032 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:02:22.033 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:02:22.058 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:02:22.058 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:02:22.120 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:22.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.262+0000 7f81c4f99700 1 Processor -- start 2023-11-28T12:02:22.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.262+0000 7f81c4f99700 1 -- start start 2023-11-28T12:02:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.263+0000 7f81c4f99700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f81c006a270 0x7f81c006a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.263+0000 7f81c4f99700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f81c015cc80 0x7f81c006de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.263+0000 7f81c4f99700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c006ab80 con 0x7f81c006e3a0 2023-11-28T12:02:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.263+0000 7f81c4f99700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c00b2490 con 0x7f81c015cc80 2023-11-28T12:02:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.263+0000 7f81c4f99700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c00b1810 con 0x7f81c006a270 2023-11-28T12:02:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.263+0000 7f81bdd9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f81c015cc80 0x7f81c006de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.263+0000 7f81bdd9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f81c015cc80 0x7f81c006de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59018/0 (socket says 172.21.15.57:59018) 2023-11-28T12:02:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.263+0000 7f81bdd9b700 1 -- 172.21.15.57:0/3384735273 learned_addr learned my addr 172.21.15.57:0/3384735273 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.264+0000 7f81bdd9b700 1 -- 172.21.15.57:0/3384735273 >> v2:172.21.15.181:3300/0 conn(0x7f81c006a270 msgr2=0x7f81c006a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.264+0000 7f81be59c700 1 --2- 172.21.15.57:0/3384735273 >> v2:172.21.15.181:3300/0 conn(0x7f81c006a270 0x7f81c006a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.264+0000 7f81bdd9b700 1 --2- 172.21.15.57:0/3384735273 >> v2:172.21.15.181:3300/0 conn(0x7f81c006a270 0x7f81c006a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.264+0000 7f81bdd9b700 1 -- 172.21.15.57:0/3384735273 >> v1:172.21.15.181:6789/0 conn(0x7f81c006e3a0 legacy=0x7f81c006ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.264+0000 7f81bdd9b700 1 -- 172.21.15.57:0/3384735273 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c00b1af0 con 0x7f81c015cc80 2023-11-28T12:02:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.264+0000 7f81bdd9b700 1 --2- 172.21.15.57:0/3384735273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81c015cc80 0x7f81c006de60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81c0067380 tx=0x7f81b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ccafec5731ea5c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.264+0000 7f81bd59a700 1 -- 172.21.15.57:0/3384735273 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b0004230 con 0x7f81c015cc80 2023-11-28T12:02:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.264+0000 7f81bd59a700 1 -- 172.21.15.57:0/3384735273 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b0004390 con 0x7f81c015cc80 2023-11-28T12:02:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.264+0000 7f81bd59a700 1 -- 172.21.15.57:0/3384735273 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b00036a0 con 0x7f81c015cc80 2023-11-28T12:02:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.265+0000 7f81c4f99700 1 -- 172.21.15.57:0/3384735273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81c015cc80 msgr2=0x7f81c006de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.265+0000 7f81c4f99700 1 --2- 172.21.15.57:0/3384735273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81c015cc80 0x7f81c006de60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81c0067380 tx=0x7f81b0009bd0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.265+0000 7f81c4f99700 1 -- 172.21.15.57:0/3384735273 shutdown_connections 2023-11-28T12:02:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.265+0000 7f81c4f99700 1 --2- 172.21.15.57:0/3384735273 >> v2:172.21.15.181:3300/0 conn(0x7f81c006a270 0x7f81c006a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.265+0000 7f81c4f99700 1 --2- 172.21.15.57:0/3384735273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81c015cc80 0x7f81c006de60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.265+0000 7f81c4f99700 1 -- 172.21.15.57:0/3384735273 >> 172.21.15.57:0/3384735273 conn(0x7f81c006c010 msgr2=0x7f81c00cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:22.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.265+0000 7f81c4f99700 1 -- 172.21.15.57:0/3384735273 shutdown_connections 2023-11-28T12:02:22.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.265+0000 7f81c4f99700 1 -- 172.21.15.57:0/3384735273 wait complete. 2023-11-28T12:02:22.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.266+0000 7f81c4f99700 1 Processor -- start 2023-11-28T12:02:22.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.266+0000 7f81c4f99700 1 -- start start 2023-11-28T12:02:22.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81c4f99700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c006a270 0x7f81c00c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81c4f99700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81c006e3a0 0x7f81c00c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81c4f99700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81c015cc80 0x7f81c015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81c4f99700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00c5210 con 0x7f81c015cc80 2023-11-28T12:02:22.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81c4f99700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00c3c30 con 0x7f81c006a270 2023-11-28T12:02:22.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81c4f99700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00c3f10 con 0x7f81c006e3a0 2023-11-28T12:02:22.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81bed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81c015cc80 0x7f81c015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81bed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81c015cc80 0x7f81c015ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59034/0 (socket says 172.21.15.57:59034) 2023-11-28T12:02:22.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81bed9d700 1 -- 172.21.15.57:0/2027236444 learned_addr learned my addr 172.21.15.57:0/2027236444 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:22.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81bdd9b700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81c006e3a0 0x7f81c00c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81be59c700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c006a270 0x7f81c00c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81bed9d700 1 -- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81c006e3a0 msgr2=0x7f81c00c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:22.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81bed9d700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81c006e3a0 0x7f81c00c4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81bed9d700 1 -- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c006a270 msgr2=0x7f81c00c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:22.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81bed9d700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c006a270 0x7f81c00c3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.267+0000 7f81bed9d700 1 -- 172.21.15.57:0/2027236444 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b0009550 con 0x7f81c015cc80 2023-11-28T12:02:22.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.268+0000 7f81bed9d700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81c015cc80 0x7f81c015ee40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f81b800b970 tx=0x7f81b800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:22.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.268+0000 7f81b6ffd700 1 -- 172.21.15.57:0/2027236444 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b800c9b0 con 0x7f81c015cc80 2023-11-28T12:02:22.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.268+0000 7f81b6ffd700 1 -- 172.21.15.57:0/2027236444 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b800cb10 con 0x7f81c015cc80 2023-11-28T12:02:22.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.268+0000 7f81c4f99700 1 -- 172.21.15.57:0/2027236444 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c00b20b0 con 0x7f81c015cc80 2023-11-28T12:02:22.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.268+0000 7f81b6ffd700 1 -- 172.21.15.57:0/2027236444 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b8014440 con 0x7f81c015cc80 2023-11-28T12:02:22.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.268+0000 7f81c4f99700 1 -- 172.21.15.57:0/2027236444 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c00b1dd0 con 0x7f81c015cc80 2023-11-28T12:02:22.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.269+0000 7f81c4f99700 1 -- 172.21.15.57:0/2027236444 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c006e7c0 con 0x7f81c015cc80 2023-11-28T12:02:22.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.269+0000 7f81b6ffd700 1 -- 172.21.15.57:0/2027236444 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f81b801b030 con 0x7f81c015cc80 2023-11-28T12:02:22.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.270+0000 7f81b6ffd700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f81a00682f0 0x7f81a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.270+0000 7f81b6ffd700 1 -- 172.21.15.57:0/2027236444 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f81b8085780 con 0x7f81c015cc80 2023-11-28T12:02:22.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.270+0000 7f81be59c700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f81a00682f0 0x7f81a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.270+0000 7f81be59c700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f81a00682f0 0x7f81a006a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f81a800b840 tx=0x7f81a8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:22.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.274+0000 7f81b6ffd700 1 -- 172.21.15.57:0/2027236444 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f81b8051fe0 con 0x7f81c015cc80 2023-11-28T12:02:22.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.477+0000 7f81c4f99700 1 -- 172.21.15.57:0/2027236444 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81c00743f0 con 0x7f81c015cc80 2023-11-28T12:02:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.478+0000 7f81b6ffd700 1 -- 172.21.15.57:0/2027236444 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7f81b8055600 con 0x7f81c015cc80 2023-11-28T12:02:22.480 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:22.480 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:22.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.480+0000 7f81c4f99700 1 -- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f81a00682f0 msgr2=0x7f81a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:22.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.480+0000 7f81c4f99700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f81a00682f0 0x7f81a006a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f81a800b840 tx=0x7f81a8008040 comp rx=0 tx=0).stop 2023-11-28T12:02:22.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.481+0000 7f81c4f99700 1 -- 172.21.15.57:0/2027236444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81c015cc80 msgr2=0x7f81c015ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:22.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.481+0000 7f81c4f99700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81c015cc80 0x7f81c015ee40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f81b800b970 tx=0x7f81b800bd30 comp rx=0 tx=0).stop 2023-11-28T12:02:22.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.481+0000 7f81c4f99700 1 -- 172.21.15.57:0/2027236444 shutdown_connections 2023-11-28T12:02:22.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.481+0000 7f81c4f99700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f81a00682f0 0x7f81a006a7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.481+0000 7f81c4f99700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81c015cc80 0x7f81c015ee40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.481+0000 7f81c4f99700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c006a270 0x7f81c00c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.481+0000 7f81c4f99700 1 --2- 172.21.15.57:0/2027236444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81c006e3a0 0x7f81c00c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.481+0000 7f81c4f99700 1 -- 172.21.15.57:0/2027236444 >> 172.21.15.57:0/2027236444 conn(0x7f81c006c010 msgr2=0x7f81c00ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:22.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.481+0000 7f81c4f99700 1 -- 172.21.15.57:0/2027236444 shutdown_connections 2023-11-28T12:02:22.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.481+0000 7f81c4f99700 1 -- 172.21.15.57:0/2027236444 wait complete. 2023-11-28T12:02:22.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:22.496 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:22.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.633+0000 7ff72465f700 1 Processor -- start 2023-11-28T12:02:22.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.633+0000 7ff72465f700 1 -- start start 2023-11-28T12:02:22.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.634+0000 7ff72465f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff71c0b05b0 0x7ff71c0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.634+0000 7ff72465f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff71c0cdf00 con 0x7ff71c0adba0 2023-11-28T12:02:22.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.634+0000 7ff72465f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff71c0c9110 con 0x7ff71c06d4e0 2023-11-28T12:02:22.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.634+0000 7ff72465f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff71c06d170 con 0x7ff71c0b05b0 2023-11-28T12:02:22.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.634+0000 7ff721bfa700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff71c0adba0 0x7ff71c158980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45892/0 (socket says 172.21.15.57:45892) 2023-11-28T12:02:22.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.634+0000 7ff721bfa700 1 -- 172.21.15.57:0/224033938 learned_addr learned my addr 172.21.15.57:0/224033938 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:22.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.634+0000 7ff722bfc700 1 --2- 172.21.15.57:0/224033938 >> v2:172.21.15.181:3300/0 conn(0x7ff71c0b05b0 0x7ff71c0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.634+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572615048 0 0) 0x7ff71c0cdf00 con 0x7ff71c0adba0 2023-11-28T12:02:22.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7080035b0 con 0x7ff71c0adba0 2023-11-28T12:02:22.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475088932 0 0) 0x7ff71c0c9110 con 0x7ff71c06d4e0 2023-11-28T12:02:22.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff71c0cdf00 con 0x7ff71c06d4e0 2023-11-28T12:02:22.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1984148906 0 0) 0x7ff7080035b0 con 0x7ff71c0adba0 2023-11-28T12:02:22.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 >> v2:172.21.15.181:3300/0 conn(0x7ff71c0b05b0 msgr2=0x7ff71c0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:22.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 --2- 172.21.15.57:0/224033938 >> v2:172.21.15.181:3300/0 conn(0x7ff71c0b05b0 0x7ff71c0cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 >> v1:172.21.15.181:6789/0 conn(0x7ff71c06d4e0 legacy=0x7ff71c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:22.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c0c93f0 con 0x7ff71c0adba0 2023-11-28T12:02:22.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7ff718001670 con 0x7ff71c0adba0 2023-11-28T12:02:22.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff722bfc700 1 --2- 172.21.15.57:0/224033938 >> v2:172.21.15.181:3300/0 conn(0x7ff71c0b05b0 0x7ff71c0cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:22.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 >> v1:172.21.15.57:6789/0 conn(0x7ff71c0adba0 legacy=0x7ff71c158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:22.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 --2- 172.21.15.57:0/224033938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff708003bc0 0x7ff708003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 --2- 172.21.15.57:0/224033938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff708004850 0x7ff708004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 --2- 172.21.15.57:0/224033938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff708005200 0x7ff70800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c06d170 con 0x7ff708003bc0 2023-11-28T12:02:22.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70800e2a0 con 0x7ff708005200 2023-11-28T12:02:22.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70800e460 con 0x7ff708004850 2023-11-28T12:02:22.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7223fb700 1 --2- 172.21.15.57:0/224033938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff708003bc0 0x7ff708003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff721bfa700 1 --2- 172.21.15.57:0/224033938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff708004850 0x7ff708004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff722bfc700 1 --2- 172.21.15.57:0/224033938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff708005200 0x7ff70800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7223fb700 1 -- 172.21.15.57:0/224033938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff708004850 msgr2=0x7ff708004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:22.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7223fb700 1 --2- 172.21.15.57:0/224033938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff708004850 0x7ff708004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7223fb700 1 -- 172.21.15.57:0/224033938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff708005200 msgr2=0x7ff70800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:22.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7223fb700 1 --2- 172.21.15.57:0/224033938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff708005200 0x7ff70800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff7223fb700 1 -- 172.21.15.57:0/224033938 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7080035b0 con 0x7ff708003bc0 2023-11-28T12:02:22.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.635+0000 7ff722bfc700 1 --2- 172.21.15.57:0/224033938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff708005200 0x7ff70800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:22.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.636+0000 7ff722bfc700 1 -- 172.21.15.57:0/224033938 reap_dead start 2023-11-28T12:02:22.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.636+0000 7ff7223fb700 1 --2- 172.21.15.57:0/224033938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff708003bc0 0x7ff708003fb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff710001940 tx=0x7ff708005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edb6fa2d5906813 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:22.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.636+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710001430 con 0x7ff708003bc0 2023-11-28T12:02:22.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.636+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff710001d70 con 0x7ff708003bc0 2023-11-28T12:02:22.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.636+0000 7ff7213f9700 1 -- 172.21.15.57:0/224033938 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710005510 con 0x7ff708003bc0 2023-11-28T12:02:22.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.636+0000 7ff72465f700 1 -- 172.21.15.57:0/224033938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff708003bc0 msgr2=0x7ff708003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:22.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.636+0000 7ff72465f700 1 --2- 172.21.15.57:0/224033938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff708003bc0 0x7ff708003fb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff710001940 tx=0x7ff708005880 comp rx=0 tx=0).stop 2023-11-28T12:02:22.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.637+0000 7ff72465f700 1 -- 172.21.15.57:0/224033938 shutdown_connections 2023-11-28T12:02:22.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.637+0000 7ff72465f700 1 --2- 172.21.15.57:0/224033938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff708003bc0 0x7ff708003fb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.637+0000 7ff72465f700 1 -- 172.21.15.57:0/224033938 >> 172.21.15.57:0/224033938 conn(0x7ff71c0cb540 msgr2=0x7ff71c0cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:22.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.637+0000 7ff72465f700 1 -- 172.21.15.57:0/224033938 shutdown_connections 2023-11-28T12:02:22.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.637+0000 7ff72465f700 1 -- 172.21.15.57:0/224033938 wait complete. 2023-11-28T12:02:22.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.638+0000 7ff72465f700 1 Processor -- start 2023-11-28T12:02:22.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.638+0000 7ff72465f700 1 -- start start 2023-11-28T12:02:22.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff72465f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff708003bc0 0x7ff71c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff72465f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c0c5b90 0x7ff71c148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff72465f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff71c0c73a0 0x7ff71c0c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff72465f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c149440 con 0x7ff71c0c5b90 2023-11-28T12:02:22.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff72465f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0c6300 con 0x7ff708003bc0 2023-11-28T12:02:22.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff72465f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0c65e0 con 0x7ff71c0c73a0 2023-11-28T12:02:22.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff721bfa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c0c5b90 0x7ff71c148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff721bfa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c0c5b90 0x7ff71c148f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59044/0 (socket says 172.21.15.57:59044) 2023-11-28T12:02:22.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff721bfa700 1 -- 172.21.15.57:0/2468547009 learned_addr learned my addr 172.21.15.57:0/2468547009 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:22.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff7223fb700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff708003bc0 0x7ff71c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff722bfc700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff71c0c73a0 0x7ff71c0c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff721bfa700 1 -- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff71c0c73a0 msgr2=0x7ff71c0c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:22.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff721bfa700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff71c0c73a0 0x7ff71c0c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff721bfa700 1 -- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff708003bc0 msgr2=0x7ff71c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:22.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff721bfa700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff708003bc0 0x7ff71c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff721bfa700 1 -- 172.21.15.57:0/2468547009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c0cdb90 con 0x7ff71c0c5b90 2023-11-28T12:02:22.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.639+0000 7ff721bfa700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c0c5b90 0x7ff71c148f00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff718007310 tx=0x7ff7180076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:22.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.640+0000 7ff70effd700 1 -- 172.21.15.57:0/2468547009 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7180077f0 con 0x7ff71c0c5b90 2023-11-28T12:02:22.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.640+0000 7ff70effd700 1 -- 172.21.15.57:0/2468547009 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71800f870 con 0x7ff71c0c5b90 2023-11-28T12:02:22.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.640+0000 7ff72465f700 1 -- 172.21.15.57:0/2468547009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71c0cd8b0 con 0x7ff71c0c5b90 2023-11-28T12:02:22.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.640+0000 7ff70effd700 1 -- 172.21.15.57:0/2468547009 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71800fbe0 con 0x7ff71c0c5b90 2023-11-28T12:02:22.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.640+0000 7ff72465f700 1 -- 172.21.15.57:0/2468547009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71c0cd350 con 0x7ff71c0c5b90 2023-11-28T12:02:22.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.641+0000 7ff72465f700 1 -- 172.21.15.57:0/2468547009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff71c0743f0 con 0x7ff71c0c5b90 2023-11-28T12:02:22.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.641+0000 7ff70effd700 1 -- 172.21.15.57:0/2468547009 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff71800fd40 con 0x7ff71c0c5b90 2023-11-28T12:02:22.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.642+0000 7ff70effd700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff7000682f0 0x7ff70006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:22.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.642+0000 7ff70effd700 1 -- 172.21.15.57:0/2468547009 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff718085e40 con 0x7ff71c0c5b90 2023-11-28T12:02:22.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.642+0000 7ff7223fb700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff7000682f0 0x7ff70006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:22.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.642+0000 7ff7223fb700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff7000682f0 0x7ff70006a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff710005040 tx=0x7ff71000a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:22.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.646+0000 7ff70effd700 1 -- 172.21.15.57:0/2468547009 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff7180526a0 con 0x7ff71c0c5b90 2023-11-28T12:02:22.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.845+0000 7ff72465f700 1 -- 172.21.15.57:0/2468547009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff71c0b4a40 con 0x7ff71c0c5b90 2023-11-28T12:02:22.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.846+0000 7ff70effd700 1 -- 172.21.15.57:0/2468547009 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff718055cc0 con 0x7ff71c0c5b90 2023-11-28T12:02:22.847 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:22.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.848+0000 7ff72465f700 1 -- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff7000682f0 msgr2=0x7ff70006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:22.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.848+0000 7ff72465f700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff7000682f0 0x7ff70006a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff710005040 tx=0x7ff71000a000 comp rx=0 tx=0).stop 2023-11-28T12:02:22.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.848+0000 7ff72465f700 1 -- 172.21.15.57:0/2468547009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c0c5b90 msgr2=0x7ff71c148f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:22.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.848+0000 7ff72465f700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c0c5b90 0x7ff71c148f00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff718007310 tx=0x7ff7180076d0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.848+0000 7ff72465f700 1 -- 172.21.15.57:0/2468547009 shutdown_connections 2023-11-28T12:02:22.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.848+0000 7ff72465f700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff7000682f0 0x7ff70006a7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.848+0000 7ff72465f700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff71c0c73a0 0x7ff71c0c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.848+0000 7ff72465f700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff708003bc0 0x7ff71c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.848+0000 7ff72465f700 1 --2- 172.21.15.57:0/2468547009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff71c0c5b90 0x7ff71c148f00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:22.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.848+0000 7ff72465f700 1 -- 172.21.15.57:0/2468547009 >> 172.21.15.57:0/2468547009 conn(0x7ff71c0cb540 msgr2=0x7ff71c0af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:22.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.848+0000 7ff72465f700 1 -- 172.21.15.57:0/2468547009 shutdown_connections 2023-11-28T12:02:22.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:22.848+0000 7ff72465f700 1 -- 172.21.15.57:0/2468547009 wait complete. 2023-11-28T12:02:22.863 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:23.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.000+0000 7ff56631f700 1 Processor -- start 2023-11-28T12:02:23.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.000+0000 7ff56631f700 1 -- start start 2023-11-28T12:02:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.001+0000 7ff56631f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff560071890 0x7ff5600ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.001+0000 7ff56631f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff56015cc10 con 0x7ff56006a980 2023-11-28T12:02:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.001+0000 7ff56631f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff56006c9b0 con 0x7ff5600cf710 2023-11-28T12:02:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.001+0000 7ff56631f700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff56006cc90 con 0x7ff560071890 2023-11-28T12:02:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.001+0000 7ff5648bc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff560071890 0x7ff5600ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.001+0000 7ff5648bc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff560071890 0x7ff5600ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59056/0 (socket says 172.21.15.57:59056) 2023-11-28T12:02:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.001+0000 7ff55f7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff56006a980 0x7ff560160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45896/0 (socket says 172.21.15.57:45896) 2023-11-28T12:02:23.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.001+0000 7ff5648bc700 1 -- 172.21.15.57:0/167925729 learned_addr learned my addr 172.21.15.57:0/167925729 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:23.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.002+0000 7ff5648bc700 1 -- 172.21.15.57:0/167925729 >> v1:172.21.15.181:6789/0 conn(0x7ff5600cf710 legacy=0x7ff56006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.002+0000 7ff5648bc700 1 -- 172.21.15.57:0/167925729 >> v1:172.21.15.57:6789/0 conn(0x7ff56006a980 legacy=0x7ff560160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.002+0000 7ff5648bc700 1 -- 172.21.15.57:0/167925729 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56006cf70 con 0x7ff560071890 2023-11-28T12:02:23.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.002+0000 7ff5648bc700 1 --2- 172.21.15.57:0/167925729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff560071890 0x7ff5600ce740 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff558009e50 tx=0x7ff55800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55854a77eddec4e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:23.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.002+0000 7ff55effd700 1 -- 172.21.15.57:0/167925729 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55800e070 con 0x7ff560071890 2023-11-28T12:02:23.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.002+0000 7ff55effd700 1 -- 172.21.15.57:0/167925729 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55800b9c0 con 0x7ff560071890 2023-11-28T12:02:23.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.002+0000 7ff55effd700 1 -- 172.21.15.57:0/167925729 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55800bd70 con 0x7ff560071890 2023-11-28T12:02:23.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.002+0000 7ff56631f700 1 -- 172.21.15.57:0/167925729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff560071890 msgr2=0x7ff5600ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.002+0000 7ff56631f700 1 --2- 172.21.15.57:0/167925729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff560071890 0x7ff5600ce740 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff558009e50 tx=0x7ff55800b330 comp rx=0 tx=0).stop 2023-11-28T12:02:23.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.003+0000 7ff56631f700 1 -- 172.21.15.57:0/167925729 shutdown_connections 2023-11-28T12:02:23.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.003+0000 7ff56631f700 1 --2- 172.21.15.57:0/167925729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff560071890 0x7ff5600ce740 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.003+0000 7ff56631f700 1 -- 172.21.15.57:0/167925729 >> 172.21.15.57:0/167925729 conn(0x7ff56006c010 msgr2=0x7ff5600cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:23.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.003+0000 7ff56631f700 1 -- 172.21.15.57:0/167925729 shutdown_connections 2023-11-28T12:02:23.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.003+0000 7ff56631f700 1 -- 172.21.15.57:0/167925729 wait complete. 2023-11-28T12:02:23.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.004+0000 7ff56631f700 1 Processor -- start 2023-11-28T12:02:23.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.004+0000 7ff56631f700 1 -- start start 2023-11-28T12:02:23.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff56631f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff56006a980 0x7ff5601561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff56631f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff560071890 0x7ff560155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff56631f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5600cf710 0x7ff5600c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff56631f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5600c35c0 con 0x7ff5600cf710 2023-11-28T12:02:23.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff56631f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5600c4140 con 0x7ff560071890 2023-11-28T12:02:23.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff56631f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5600c3810 con 0x7ff56006a980 2023-11-28T12:02:23.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff5648bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5600cf710 0x7ff5600c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff5648bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5600cf710 0x7ff5600c45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59060/0 (socket says 172.21.15.57:59060) 2023-11-28T12:02:23.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff5648bc700 1 -- 172.21.15.57:0/2007573919 learned_addr learned my addr 172.21.15.57:0/2007573919 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:23.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff55f7fe700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff560071890 0x7ff560155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff55ffff700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff56006a980 0x7ff5601561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff5648bc700 1 -- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff56006a980 msgr2=0x7ff5601561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff5648bc700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff56006a980 0x7ff5601561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff5648bc700 1 -- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff560071890 msgr2=0x7ff560155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff5648bc700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff560071890 0x7ff560155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff5648bc700 1 -- 172.21.15.57:0/2007573919 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff558009b30 con 0x7ff5600cf710 2023-11-28T12:02:23.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff55f7fe700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff560071890 0x7ff560155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:23.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.005+0000 7ff5648bc700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5600cf710 0x7ff5600c45d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff55800b730 tx=0x7ff558004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:23.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.006+0000 7ff55cff9700 1 -- 172.21.15.57:0/2007573919 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5580048b0 con 0x7ff5600cf710 2023-11-28T12:02:23.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.006+0000 7ff55cff9700 1 -- 172.21.15.57:0/2007573919 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5580099b0 con 0x7ff5600cf710 2023-11-28T12:02:23.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.006+0000 7ff56631f700 1 -- 172.21.15.57:0/2007573919 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56006d530 con 0x7ff5600cf710 2023-11-28T12:02:23.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.006+0000 7ff55cff9700 1 -- 172.21.15.57:0/2007573919 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55801c680 con 0x7ff5600cf710 2023-11-28T12:02:23.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.006+0000 7ff56631f700 1 -- 172.21.15.57:0/2007573919 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56006d250 con 0x7ff5600cf710 2023-11-28T12:02:23.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.007+0000 7ff56631f700 1 -- 172.21.15.57:0/2007573919 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5600c42d0 con 0x7ff5600cf710 2023-11-28T12:02:23.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.007+0000 7ff55cff9700 1 -- 172.21.15.57:0/2007573919 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff55802c070 con 0x7ff5600cf710 2023-11-28T12:02:23.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.008+0000 7ff55cff9700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff54c0682f0 0x7ff54c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.008+0000 7ff55cff9700 1 -- 172.21.15.57:0/2007573919 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff558087940 con 0x7ff5600cf710 2023-11-28T12:02:23.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.008+0000 7ff55ffff700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff54c0682f0 0x7ff54c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.008+0000 7ff55ffff700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff54c0682f0 0x7ff54c06a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5600b0fa0 tx=0x7ff550009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:23.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.012+0000 7ff55cff9700 1 -- 172.21.15.57:0/2007573919 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff558054140 con 0x7ff5600cf710 2023-11-28T12:02:23.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.211+0000 7ff56631f700 1 -- 172.21.15.57:0/2007573919 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5600743f0 con 0x7ff5600cf710 2023-11-28T12:02:23.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.212+0000 7ff55cff9700 1 -- 172.21.15.57:0/2007573919 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff558057760 con 0x7ff5600cf710 2023-11-28T12:02:23.213 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:23.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.214+0000 7ff56631f700 1 -- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff54c0682f0 msgr2=0x7ff54c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.214+0000 7ff56631f700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff54c0682f0 0x7ff54c06a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5600b0fa0 tx=0x7ff550009040 comp rx=0 tx=0).stop 2023-11-28T12:02:23.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.214+0000 7ff56631f700 1 -- 172.21.15.57:0/2007573919 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5600cf710 msgr2=0x7ff5600c45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.214+0000 7ff56631f700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5600cf710 0x7ff5600c45d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff55800b730 tx=0x7ff558004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.214+0000 7ff56631f700 1 -- 172.21.15.57:0/2007573919 shutdown_connections 2023-11-28T12:02:23.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.214+0000 7ff56631f700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff54c0682f0 0x7ff54c06a7a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.214+0000 7ff56631f700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5600cf710 0x7ff5600c45d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.214+0000 7ff56631f700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff56006a980 0x7ff5601561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.214+0000 7ff56631f700 1 --2- 172.21.15.57:0/2007573919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff560071890 0x7ff560155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.214+0000 7ff56631f700 1 -- 172.21.15.57:0/2007573919 >> 172.21.15.57:0/2007573919 conn(0x7ff56006c010 msgr2=0x7ff56006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:23.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.214+0000 7ff56631f700 1 -- 172.21.15.57:0/2007573919 shutdown_connections 2023-11-28T12:02:23.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.214+0000 7ff56631f700 1 -- 172.21.15.57:0/2007573919 wait complete. 2023-11-28T12:02:23.229 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:02:23.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.366+0000 7f92ad9cb700 1 Processor -- start 2023-11-28T12:02:23.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.366+0000 7f92ad9cb700 1 -- start start 2023-11-28T12:02:23.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.367+0000 7f92ad9cb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a80b1650 0x7f92a80b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.367+0000 7f92ad9cb700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a8159ec0 con 0x7f92a80af0a0 2023-11-28T12:02:23.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.367+0000 7f92ad9cb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a815aa60 con 0x7f92a81599b0 2023-11-28T12:02:23.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.367+0000 7f92ad9cb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a80b1f60 con 0x7f92a80b1650 2023-11-28T12:02:23.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.367+0000 7f92a77fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f92a80af0a0 0x7f92a815a350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45906/0 (socket says 172.21.15.57:45906) 2023-11-28T12:02:23.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.367+0000 7f92a77fe700 1 -- 172.21.15.57:0/867311537 learned_addr learned my addr 172.21.15.57:0/867311537 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177619182 0 0) 0x7f92a8159ec0 con 0x7f92a80af0a0 2023-11-28T12:02:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a6ffd700 1 --2- 172.21.15.57:0/867311537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a80b1650 0x7f92a80b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92900035b0 con 0x7f92a80af0a0 2023-11-28T12:02:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1806514910 0 0) 0x7f92900035b0 con 0x7f92a80af0a0 2023-11-28T12:02:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a80b1650 msgr2=0x7f92a80b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 --2- 172.21.15.57:0/867311537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a80b1650 0x7f92a80b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 >> v1:172.21.15.181:6789/0 conn(0x7f92a81599b0 legacy=0x7f92a815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a815d8a0 con 0x7f92a80af0a0 2023-11-28T12:02:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f92a00019f0 con 0x7f92a80af0a0 2023-11-28T12:02:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a6ffd700 1 --2- 172.21.15.57:0/867311537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a80b1650 0x7f92a80b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 >> v1:172.21.15.57:6789/0 conn(0x7f92a80af0a0 legacy=0x7f92a815a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 --2- 172.21.15.57:0/867311537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9290003a30 0x7f9290003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 --2- 172.21.15.57:0/867311537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a80b1650 0x7f9290004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 --2- 172.21.15.57:0/867311537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9290004d90 0x7f9290005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a80b1f60 con 0x7f9290003a30 2023-11-28T12:02:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929000ba10 con 0x7f9290004d90 2023-11-28T12:02:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929000bbd0 con 0x7f92a80b1650 2023-11-28T12:02:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a6ffd700 1 --2- 172.21.15.57:0/867311537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9290003a30 0x7f9290003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a67fc700 1 --2- 172.21.15.57:0/867311537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a80b1650 0x7f9290004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a77fe700 1 --2- 172.21.15.57:0/867311537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9290004d90 0x7f9290005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.368+0000 7f92a6ffd700 1 -- 172.21.15.57:0/867311537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a80b1650 msgr2=0x7f9290004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.369+0000 7f92a6ffd700 1 --2- 172.21.15.57:0/867311537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a80b1650 0x7f9290004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.369+0000 7f92a6ffd700 1 -- 172.21.15.57:0/867311537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9290004d90 msgr2=0x7f9290005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.369+0000 7f92a6ffd700 1 --2- 172.21.15.57:0/867311537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9290004d90 0x7f9290005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.369+0000 7f92a6ffd700 1 -- 172.21.15.57:0/867311537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929800b740 con 0x7f9290003a30 2023-11-28T12:02:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.369+0000 7f92a6ffd700 1 --2- 172.21.15.57:0/867311537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9290003a30 0x7f9290003e00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f929800bfd0 tx=0x7f9290005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a02f0d9862d06b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.369+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9298010e10 con 0x7f9290003a30 2023-11-28T12:02:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.369+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9298011450 con 0x7f9290003a30 2023-11-28T12:02:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.369+0000 7f92a5ffb700 1 -- 172.21.15.57:0/867311537 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92980117e0 con 0x7f9290003a30 2023-11-28T12:02:23.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.369+0000 7f92ad9cb700 1 -- 172.21.15.57:0/867311537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9290003a30 msgr2=0x7f9290003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.369+0000 7f92ad9cb700 1 --2- 172.21.15.57:0/867311537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9290003a30 0x7f9290003e00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f929800bfd0 tx=0x7f9290005880 comp rx=0 tx=0).stop 2023-11-28T12:02:23.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.369+0000 7f92a77fe700 1 -- 172.21.15.57:0/867311537 reap_dead start 2023-11-28T12:02:23.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.369+0000 7f92ad9cb700 1 -- 172.21.15.57:0/867311537 shutdown_connections 2023-11-28T12:02:23.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.369+0000 7f92ad9cb700 1 -- 172.21.15.57:0/867311537 >> 172.21.15.57:0/867311537 conn(0x7f92a806c010 msgr2=0x7f92a80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:23.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.370+0000 7f92ad9cb700 1 -- 172.21.15.57:0/867311537 shutdown_connections 2023-11-28T12:02:23.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.370+0000 7f92ad9cb700 1 -- 172.21.15.57:0/867311537 wait complete. 2023-11-28T12:02:23.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.371+0000 7f92ad9cb700 1 Processor -- start 2023-11-28T12:02:23.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.371+0000 7f92ad9cb700 1 -- start start 2023-11-28T12:02:23.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92ad9cb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9290003a30 0x7f92a80b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92ad9cb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92a814b2b0 0x7f92a8070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92ad9cb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a81ee2b0 0x7f92a80710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92ad9cb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a80c3700 con 0x7f9290003a30 2023-11-28T12:02:23.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92ad9cb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a81ef4b0 con 0x7f92a814b2b0 2023-11-28T12:02:23.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92ad9cb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a81ee820 con 0x7f92a81ee2b0 2023-11-28T12:02:23.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92a6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9290003a30 0x7f92a80b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92a6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9290003a30 0x7f92a80b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59066/0 (socket says 172.21.15.57:59066) 2023-11-28T12:02:23.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92a6ffd700 1 -- 172.21.15.57:0/2247762413 learned_addr learned my addr 172.21.15.57:0/2247762413 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:23.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92a67fc700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92a814b2b0 0x7f92a8070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92a77fe700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a81ee2b0 0x7f92a80710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92a6ffd700 1 -- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a81ee2b0 msgr2=0x7f92a80710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92a6ffd700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a81ee2b0 0x7f92a80710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92a6ffd700 1 -- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92a814b2b0 msgr2=0x7f92a8070bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92a6ffd700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92a814b2b0 0x7f92a8070bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92a6ffd700 1 -- 172.21.15.57:0/2247762413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a815de60 con 0x7f9290003a30 2023-11-28T12:02:23.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92a77fe700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a81ee2b0 0x7f92a80710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:23.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.372+0000 7f92a6ffd700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9290003a30 0x7f92a80b4460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f929800b2c0 tx=0x7f92980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:23.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.373+0000 7f928ffff700 1 -- 172.21.15.57:0/2247762413 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9298003ab0 con 0x7f9290003a30 2023-11-28T12:02:23.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.373+0000 7f928ffff700 1 -- 172.21.15.57:0/2247762413 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9298003c10 con 0x7f9290003a30 2023-11-28T12:02:23.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.373+0000 7f92ad9cb700 1 -- 172.21.15.57:0/2247762413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a815db80 con 0x7f9290003a30 2023-11-28T12:02:23.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.373+0000 7f928ffff700 1 -- 172.21.15.57:0/2247762413 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92980115c0 con 0x7f9290003a30 2023-11-28T12:02:23.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.373+0000 7f92ad9cb700 1 -- 172.21.15.57:0/2247762413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a80b1a20 con 0x7f9290003a30 2023-11-28T12:02:23.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.374+0000 7f92ad9cb700 1 -- 172.21.15.57:0/2247762413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a80743f0 con 0x7f9290003a30 2023-11-28T12:02:23.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.374+0000 7f928ffff700 1 -- 172.21.15.57:0/2247762413 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9298011720 con 0x7f9290003a30 2023-11-28T12:02:23.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.375+0000 7f928ffff700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f92940682f0 0x7f929406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.375+0000 7f928ffff700 1 -- 172.21.15.57:0/2247762413 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f929808a2d0 con 0x7f9290003a30 2023-11-28T12:02:23.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.375+0000 7f92a67fc700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f92940682f0 0x7f929406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.375+0000 7f92a67fc700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f92940682f0 0x7f929406a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f92a80c5090 tx=0x7f929c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:23.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.379+0000 7f928ffff700 1 -- 172.21.15.57:0/2247762413 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9298056b30 con 0x7f9290003a30 2023-11-28T12:02:23.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.582+0000 7f92ad9cb700 1 -- 172.21.15.57:0/2247762413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f92a806e620 con 0x7f9290003a30 2023-11-28T12:02:23.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.583+0000 7f928ffff700 1 -- 172.21.15.57:0/2247762413 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f929805a150 con 0x7f9290003a30 2023-11-28T12:02:23.584 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:02:23.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.585+0000 7f92ad9cb700 1 -- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f92940682f0 msgr2=0x7f929406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.585+0000 7f92ad9cb700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f92940682f0 0x7f929406a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f92a80c5090 tx=0x7f929c009040 comp rx=0 tx=0).stop 2023-11-28T12:02:23.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.585+0000 7f92ad9cb700 1 -- 172.21.15.57:0/2247762413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9290003a30 msgr2=0x7f92a80b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.585+0000 7f92ad9cb700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9290003a30 0x7f92a80b4460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f929800b2c0 tx=0x7f92980047c0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.585+0000 7f92ad9cb700 1 -- 172.21.15.57:0/2247762413 shutdown_connections 2023-11-28T12:02:23.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.585+0000 7f92ad9cb700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f92940682f0 0x7f929406a7a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.585+0000 7f92ad9cb700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92a81ee2b0 0x7f92a80710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.585+0000 7f92ad9cb700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9290003a30 0x7f92a80b4460 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.585+0000 7f92ad9cb700 1 --2- 172.21.15.57:0/2247762413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92a814b2b0 0x7f92a8070bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.585+0000 7f92ad9cb700 1 -- 172.21.15.57:0/2247762413 >> 172.21.15.57:0/2247762413 conn(0x7f92a806c010 msgr2=0x7f92a815d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:23.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.585+0000 7f92ad9cb700 1 -- 172.21.15.57:0/2247762413 shutdown_connections 2023-11-28T12:02:23.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.585+0000 7f92ad9cb700 1 -- 172.21.15.57:0/2247762413 wait complete. 2023-11-28T12:02:23.600 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:02:23.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.736+0000 7fc9f7521700 1 Processor -- start 2023-11-28T12:02:23.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.736+0000 7fc9f7521700 1 -- start start 2023-11-28T12:02:23.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.737+0000 7fc9f7521700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc9f015a5f0 0x7fc9f00d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.737+0000 7fc9f7521700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9f00b2ee0 0x7fc9f00ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.737+0000 7fc9f7521700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f015a9c0 con 0x7fc9f00cc200 2023-11-28T12:02:23.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.737+0000 7fc9f7521700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f00c7b80 con 0x7fc9f015a5f0 2023-11-28T12:02:23.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.737+0000 7fc9f7521700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f00cc990 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.737+0000 7fc9f4abc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc9f00cc200 0x7fc9f00b27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45922/0 (socket says 172.21.15.57:45922) 2023-11-28T12:02:23.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.737+0000 7fc9f4abc700 1 -- 172.21.15.57:0/6422565 learned_addr learned my addr 172.21.15.57:0/6422565 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:23.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.738+0000 7fc9f52bd700 1 --2- 172.21.15.57:0/6422565 >> v2:172.21.15.181:3300/0 conn(0x7fc9f015a5f0 0x7fc9f00d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.738+0000 7fc9f5abe700 1 --2- 172.21.15.57:0/6422565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9f00b2ee0 0x7fc9f00ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.738+0000 7fc9e3fff700 1 -- 172.21.15.57:0/6422565 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924050649 0 0) 0x7fc9f015a9c0 con 0x7fc9f00cc200 2023-11-28T12:02:23.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.738+0000 7fc9e3fff700 1 -- 172.21.15.57:0/6422565 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9dc0035b0 con 0x7fc9f00cc200 2023-11-28T12:02:23.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.738+0000 7fc9f52bd700 1 -- 172.21.15.57:0/6422565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9f00b2ee0 msgr2=0x7fc9f00ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:23.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.738+0000 7fc9e3fff700 1 -- 172.21.15.57:0/6422565 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1143534063 0 0) 0x7fc9dc0035b0 con 0x7fc9f00cc200 2023-11-28T12:02:23.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.738+0000 7fc9f52bd700 1 --2- 172.21.15.57:0/6422565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9f00b2ee0 0x7fc9f00ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.738+0000 7fc9f52bd700 1 -- 172.21.15.57:0/6422565 >> v1:172.21.15.57:6789/0 conn(0x7fc9f00cc200 legacy=0x7fc9f00b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.738+0000 7fc9f52bd700 1 -- 172.21.15.57:0/6422565 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f00ccc70 con 0x7fc9f015a5f0 2023-11-28T12:02:23.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.738+0000 7fc9f5abe700 1 --2- 172.21.15.57:0/6422565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9f00b2ee0 0x7fc9f00ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:23.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.738+0000 7fc9f52bd700 1 --2- 172.21.15.57:0/6422565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f015a5f0 0x7fc9f00d0ad0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009840 tx=0x7fc9e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9185841c6335268d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:23.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.738+0000 7fc9e3fff700 1 -- 172.21.15.57:0/6422565 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec001670 con 0x7fc9f015a5f0 2023-11-28T12:02:23.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.739+0000 7fc9e3fff700 1 -- 172.21.15.57:0/6422565 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e4004030 con 0x7fc9f015a5f0 2023-11-28T12:02:23.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.739+0000 7fc9e3fff700 1 -- 172.21.15.57:0/6422565 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e40043a0 con 0x7fc9f015a5f0 2023-11-28T12:02:23.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.739+0000 7fc9f7521700 1 -- 172.21.15.57:0/6422565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f015a5f0 msgr2=0x7fc9f00d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.739+0000 7fc9f7521700 1 --2- 172.21.15.57:0/6422565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f015a5f0 0x7fc9f00d0ad0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009840 tx=0x7fc9e4009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:23.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.739+0000 7fc9f7521700 1 -- 172.21.15.57:0/6422565 shutdown_connections 2023-11-28T12:02:23.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.739+0000 7fc9f7521700 1 --2- 172.21.15.57:0/6422565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f015a5f0 0x7fc9f00d0ad0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.739+0000 7fc9f7521700 1 --2- 172.21.15.57:0/6422565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9f00b2ee0 0x7fc9f00ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.739+0000 7fc9f7521700 1 -- 172.21.15.57:0/6422565 >> 172.21.15.57:0/6422565 conn(0x7fc9f006ea80 msgr2=0x7fc9f006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:23.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.739+0000 7fc9f7521700 1 -- 172.21.15.57:0/6422565 shutdown_connections 2023-11-28T12:02:23.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.739+0000 7fc9f7521700 1 -- 172.21.15.57:0/6422565 wait complete. 2023-11-28T12:02:23.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.740+0000 7fc9f7521700 1 Processor -- start 2023-11-28T12:02:23.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.740+0000 7fc9f7521700 1 -- start start 2023-11-28T12:02:23.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.741+0000 7fc9f7521700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9f00b2ee0 0x7fc9f00c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.741+0000 7fc9f7521700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9f00cc200 0x7fc9f014aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.741+0000 7fc9f7521700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f015a5f0 0x7fc9f01f11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.741+0000 7fc9f7521700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f014d5c0 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.741+0000 7fc9f7521700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f014b030 con 0x7fc9f015a5f0 2023-11-28T12:02:23.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.741+0000 7fc9f7521700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f014b960 con 0x7fc9f00cc200 2023-11-28T12:02:23.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.741+0000 7fc9f52bd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9f00b2ee0 0x7fc9f00c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.741+0000 7fc9f52bd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9f00b2ee0 0x7fc9f00c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59082/0 (socket says 172.21.15.57:59082) 2023-11-28T12:02:23.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.741+0000 7fc9f52bd700 1 -- 172.21.15.57:0/1685630579 learned_addr learned my addr 172.21.15.57:0/1685630579 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:23.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.742+0000 7fc9f4abc700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9f00cc200 0x7fc9f014aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.742+0000 7fc9f5abe700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f015a5f0 0x7fc9f01f11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.742+0000 7fc9f52bd700 1 -- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9f00cc200 msgr2=0x7fc9f014aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.742+0000 7fc9f52bd700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9f00cc200 0x7fc9f014aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.742+0000 7fc9f52bd700 1 -- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f015a5f0 msgr2=0x7fc9f01f11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.742+0000 7fc9f52bd700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f015a5f0 0x7fc9f01f11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.742+0000 7fc9f52bd700 1 -- 172.21.15.57:0/1685630579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e8009a70 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.742+0000 7fc9f52bd700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9f00b2ee0 0x7fc9f00c8330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4000c00 tx=0x7fc9e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:23.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.742+0000 7fc9e1ffb700 1 -- 172.21.15.57:0/1685630579 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4004590 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.742+0000 7fc9e1ffb700 1 -- 172.21.15.57:0/1685630579 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e401f070 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.742+0000 7fc9f7521700 1 -- 172.21.15.57:0/1685630579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e40094a0 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.742+0000 7fc9e1ffb700 1 -- 172.21.15.57:0/1685630579 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4011870 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.742+0000 7fc9f7521700 1 -- 172.21.15.57:0/1685630579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f00cd230 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.743+0000 7fc9f7521700 1 -- 172.21.15.57:0/1685630579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f00ccf50 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.744+0000 7fc9e1ffb700 1 -- 172.21.15.57:0/1685630579 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc9e4017020 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.744+0000 7fc9e1ffb700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc9d40682f0 0x7fc9d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:23.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.745+0000 7fc9e1ffb700 1 -- 172.21.15.57:0/1685630579 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc9e4021070 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.745+0000 7fc9f4abc700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc9d40682f0 0x7fc9d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:23.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.745+0000 7fc9f4abc700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc9d40682f0 0x7fc9d406a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc9f01523d0 tx=0x7fc9ec001530 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:23.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.749+0000 7fc9e1ffb700 1 -- 172.21.15.57:0/1685630579 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc9e4053e50 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.951+0000 7fc9f7521700 1 -- 172.21.15.57:0/1685630579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc9f00cd510 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.951+0000 7fc9e1ffb700 1 -- 172.21.15.57:0/1685630579 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc9e4057470 con 0x7fc9f00b2ee0 2023-11-28T12:02:23.952 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:02:23.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.954+0000 7fc9f7521700 1 -- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc9d40682f0 msgr2=0x7fc9d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.954+0000 7fc9f7521700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc9d40682f0 0x7fc9d406a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc9f01523d0 tx=0x7fc9ec001530 comp rx=0 tx=0).stop 2023-11-28T12:02:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.954+0000 7fc9f7521700 1 -- 172.21.15.57:0/1685630579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9f00b2ee0 msgr2=0x7fc9f00c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.954+0000 7fc9f7521700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9f00b2ee0 0x7fc9f00c8330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4000c00 tx=0x7fc9e4004750 comp rx=0 tx=0).stop 2023-11-28T12:02:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.954+0000 7fc9f7521700 1 -- 172.21.15.57:0/1685630579 shutdown_connections 2023-11-28T12:02:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.954+0000 7fc9f7521700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc9d40682f0 0x7fc9d406a7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.954+0000 7fc9f7521700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9f015a5f0 0x7fc9f01f11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.954+0000 7fc9f7521700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9f00b2ee0 0x7fc9f00c8330 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.954+0000 7fc9f7521700 1 --2- 172.21.15.57:0/1685630579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9f00cc200 0x7fc9f014aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.954+0000 7fc9f7521700 1 -- 172.21.15.57:0/1685630579 >> 172.21.15.57:0/1685630579 conn(0x7fc9f006ea80 msgr2=0x7fc9f0152420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.954+0000 7fc9f7521700 1 -- 172.21.15.57:0/1685630579 shutdown_connections 2023-11-28T12:02:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:23.954+0000 7fc9f7521700 1 -- 172.21.15.57:0/1685630579 wait complete. 2023-11-28T12:02:23.970 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T12:02:24.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.105+0000 7ff034bc7700 1 Processor -- start 2023-11-28T12:02:24.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.105+0000 7ff034bc7700 1 -- start start 2023-11-28T12:02:24.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.106+0000 7ff034bc7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff030071890 0x7ff0300ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.106+0000 7ff034bc7700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff03015cc10 con 0x7ff03006a980 2023-11-28T12:02:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.106+0000 7ff034bc7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff03006c9b0 con 0x7ff0300cf710 2023-11-28T12:02:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.106+0000 7ff034bc7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03006cc90 con 0x7ff030071890 2023-11-28T12:02:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.106+0000 7ff02dd9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff03006a980 0x7ff030160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45936/0 (socket says 172.21.15.57:45936) 2023-11-28T12:02:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02dd9b700 1 -- 172.21.15.57:0/3712355353 learned_addr learned my addr 172.21.15.57:0/3712355353 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02ed9d700 1 --2- 172.21.15.57:0/3712355353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff030071890 0x7ff0300ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261090838 0 0) 0x7ff03015cc10 con 0x7ff03006a980 2023-11-28T12:02:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0180035b0 con 0x7ff03006a980 2023-11-28T12:02:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906852569 0 0) 0x7ff03006c9b0 con 0x7ff0300cf710 2023-11-28T12:02:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff03015cc10 con 0x7ff0300cf710 2023-11-28T12:02:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3226354959 0 0) 0x7ff0180035b0 con 0x7ff03006a980 2023-11-28T12:02:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff030071890 msgr2=0x7ff0300ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 --2- 172.21.15.57:0/3712355353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff030071890 0x7ff0300ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 >> v1:172.21.15.181:6789/0 conn(0x7ff0300cf710 legacy=0x7ff03006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03006cf70 con 0x7ff03006a980 2023-11-28T12:02:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7ff024001670 con 0x7ff03006a980 2023-11-28T12:02:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02ed9d700 1 --2- 172.21.15.57:0/3712355353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff030071890 0x7ff0300ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 >> v1:172.21.15.57:6789/0 conn(0x7ff03006a980 legacy=0x7ff030160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 --2- 172.21.15.57:0/3712355353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff030071890 0x7ff018003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 --2- 172.21.15.57:0/3712355353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff018004100 0x7ff018004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 --2- 172.21.15.57:0/3712355353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff018004d90 0x7ff018005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03006cc90 con 0x7ff018004100 2023-11-28T12:02:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01800bb50 con 0x7ff018004d90 2023-11-28T12:02:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01800bd10 con 0x7ff030071890 2023-11-28T12:02:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.107+0000 7ff02dd9b700 1 --2- 172.21.15.57:0/3712355353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff018004100 0x7ff018004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.108+0000 7ff02e59c700 1 --2- 172.21.15.57:0/3712355353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff030071890 0x7ff018003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.108+0000 7ff02ed9d700 1 --2- 172.21.15.57:0/3712355353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff018004d90 0x7ff018005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.108+0000 7ff02dd9b700 1 -- 172.21.15.57:0/3712355353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff030071890 msgr2=0x7ff018003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:24.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.108+0000 7ff02dd9b700 1 --2- 172.21.15.57:0/3712355353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff030071890 0x7ff018003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.108+0000 7ff02dd9b700 1 -- 172.21.15.57:0/3712355353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff018004d90 msgr2=0x7ff018005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:24.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.108+0000 7ff02dd9b700 1 --2- 172.21.15.57:0/3712355353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff018004d90 0x7ff018005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.108+0000 7ff02dd9b700 1 -- 172.21.15.57:0/3712355353 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff028009a70 con 0x7ff018004100 2023-11-28T12:02:24.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.108+0000 7ff02dd9b700 1 --2- 172.21.15.57:0/3712355353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff018004100 0x7ff018004850 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff024001eb0 tx=0x7ff018005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5f8460e05c4085 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:24.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.108+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02400cc60 con 0x7ff018004100 2023-11-28T12:02:24.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.108+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02400cde0 con 0x7ff018004100 2023-11-28T12:02:24.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.108+0000 7ff02d59a700 1 -- 172.21.15.57:0/3712355353 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024005ba0 con 0x7ff018004100 2023-11-28T12:02:24.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.109+0000 7ff034bc7700 1 -- 172.21.15.57:0/3712355353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff018004100 msgr2=0x7ff018004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.109+0000 7ff034bc7700 1 --2- 172.21.15.57:0/3712355353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff018004100 0x7ff018004850 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff024001eb0 tx=0x7ff018005880 comp rx=0 tx=0).stop 2023-11-28T12:02:24.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.109+0000 7ff02ed9d700 1 -- 172.21.15.57:0/3712355353 reap_dead start 2023-11-28T12:02:24.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.109+0000 7ff034bc7700 1 -- 172.21.15.57:0/3712355353 shutdown_connections 2023-11-28T12:02:24.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.109+0000 7ff034bc7700 1 -- 172.21.15.57:0/3712355353 >> 172.21.15.57:0/3712355353 conn(0x7ff03006c010 msgr2=0x7ff0300cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:24.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.109+0000 7ff034bc7700 1 -- 172.21.15.57:0/3712355353 shutdown_connections 2023-11-28T12:02:24.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.109+0000 7ff034bc7700 1 -- 172.21.15.57:0/3712355353 wait complete. 2023-11-28T12:02:24.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.110+0000 7ff034bc7700 1 Processor -- start 2023-11-28T12:02:24.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.110+0000 7ff034bc7700 1 -- start start 2023-11-28T12:02:24.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff034bc7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff018004100 0x7ff030151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff034bc7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff030154060 0x7ff0301550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff034bc7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff030156a90 0x7ff0300c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff034bc7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff030156600 con 0x7ff030156a90 2023-11-28T12:02:24.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff034bc7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff030154840 con 0x7ff030154060 2023-11-28T12:02:24.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff034bc7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff030155ff0 con 0x7ff018004100 2023-11-28T12:02:24.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff02ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff030156a90 0x7ff0300c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff02ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff030156a90 0x7ff0300c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59094/0 (socket says 172.21.15.57:59094) 2023-11-28T12:02:24.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff02ed9d700 1 -- 172.21.15.57:0/583989883 learned_addr learned my addr 172.21.15.57:0/583989883 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:24.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff02e59c700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff018004100 0x7ff030151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff02dd9b700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff030154060 0x7ff0301550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff02ed9d700 1 -- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff018004100 msgr2=0x7ff030151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff02ed9d700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff018004100 0x7ff030151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.111+0000 7ff02ed9d700 1 -- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff030154060 msgr2=0x7ff0301550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.112+0000 7ff02ed9d700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff030154060 0x7ff0301550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.112+0000 7ff02ed9d700 1 -- 172.21.15.57:0/583989883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03006d530 con 0x7ff030156a90 2023-11-28T12:02:24.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.112+0000 7ff02ed9d700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff030156a90 0x7ff0300c5130 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff0300672e0 tx=0x7ff02800f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:24.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.112+0000 7ff01effd700 1 -- 172.21.15.57:0/583989883 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02800e040 con 0x7ff030156a90 2023-11-28T12:02:24.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.112+0000 7ff01effd700 1 -- 172.21.15.57:0/583989883 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02800f040 con 0x7ff030156a90 2023-11-28T12:02:24.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.112+0000 7ff034bc7700 1 -- 172.21.15.57:0/583989883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03006d250 con 0x7ff030156a90 2023-11-28T12:02:24.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.112+0000 7ff01effd700 1 -- 172.21.15.57:0/583989883 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028003a20 con 0x7ff030156a90 2023-11-28T12:02:24.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.112+0000 7ff034bc7700 1 -- 172.21.15.57:0/583989883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0300ce740 con 0x7ff030156a90 2023-11-28T12:02:24.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.113+0000 7ff034bc7700 1 -- 172.21.15.57:0/583989883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0300743f0 con 0x7ff030156a90 2023-11-28T12:02:24.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.113+0000 7ff01effd700 1 -- 172.21.15.57:0/583989883 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff028024070 con 0x7ff030156a90 2023-11-28T12:02:24.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.114+0000 7ff01effd700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0100682f0 0x7ff01006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.114+0000 7ff01effd700 1 -- 172.21.15.57:0/583989883 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff028086160 con 0x7ff030156a90 2023-11-28T12:02:24.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.114+0000 7ff02e59c700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0100682f0 0x7ff01006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.115+0000 7ff02e59c700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0100682f0 0x7ff01006a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff020007fd0 tx=0x7ff020009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:24.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.118+0000 7ff01effd700 1 -- 172.21.15.57:0/583989883 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff0280529c0 con 0x7ff030156a90 2023-11-28T12:02:24.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.335+0000 7ff034bc7700 1 -- 172.21.15.57:0/583989883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff0300b5b50 con 0x7ff030156a90 2023-11-28T12:02:24.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.336+0000 7ff01effd700 1 -- 172.21.15.57:0/583989883 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff028055fe0 con 0x7ff030156a90 2023-11-28T12:02:24.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.338+0000 7ff034bc7700 1 -- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0100682f0 msgr2=0x7ff01006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.338+0000 7ff034bc7700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0100682f0 0x7ff01006a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff020007fd0 tx=0x7ff020009040 comp rx=0 tx=0).stop 2023-11-28T12:02:24.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.338+0000 7ff034bc7700 1 -- 172.21.15.57:0/583989883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff030156a90 msgr2=0x7ff0300c5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.338+0000 7ff034bc7700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff030156a90 0x7ff0300c5130 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff0300672e0 tx=0x7ff02800f490 comp rx=0 tx=0).stop 2023-11-28T12:02:24.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.338+0000 7ff034bc7700 1 -- 172.21.15.57:0/583989883 shutdown_connections 2023-11-28T12:02:24.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.338+0000 7ff034bc7700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0100682f0 0x7ff01006a7a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.338+0000 7ff034bc7700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff030156a90 0x7ff0300c5130 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.338+0000 7ff034bc7700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff018004100 0x7ff030151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.338+0000 7ff034bc7700 1 --2- 172.21.15.57:0/583989883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff030154060 0x7ff0301550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.338+0000 7ff034bc7700 1 -- 172.21.15.57:0/583989883 >> 172.21.15.57:0/583989883 conn(0x7ff03006c010 msgr2=0x7ff0300b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:24.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.338+0000 7ff034bc7700 1 -- 172.21.15.57:0/583989883 shutdown_connections 2023-11-28T12:02:24.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.338+0000 7ff034bc7700 1 -- 172.21.15.57:0/583989883 wait complete. 2023-11-28T12:02:24.341 INFO:teuthology.orchestra.run.smithi057.stderr:listed 0 entries 2023-11-28T12:02:24.353 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:02:24.353 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:02:24.376 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:24.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.511+0000 7f665b640700 1 Processor -- start 2023-11-28T12:02:24.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.511+0000 7f665b640700 1 -- start start 2023-11-28T12:02:24.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.512+0000 7f665b640700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f665406a270 0x7f6654164690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.512+0000 7f665b640700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f665406a640 0x7f665406df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.512+0000 7f665b640700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66540b1910 con 0x7f665406e450 2023-11-28T12:02:24.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.512+0000 7f665b640700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66540b2590 con 0x7f665406a270 2023-11-28T12:02:24.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.512+0000 7f665b640700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66540b1bf0 con 0x7f665406a640 2023-11-28T12:02:24.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.512+0000 7f66593dc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f665406a270 0x7f6654164690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.512+0000 7f6659bdd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f665406e450 0x7f66541612c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40190/0 (socket says 172.21.15.57:40190) 2023-11-28T12:02:24.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.512+0000 7f6659bdd700 1 -- 172.21.15.57:0/3325302307 learned_addr learned my addr 172.21.15.57:0/3325302307 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.512+0000 7f66593dc700 1 --2- 172.21.15.57:0/3325302307 >> v2:172.21.15.57:3300/0 conn(0x7f665406a270 0x7f6654164690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53526/0 (socket says 172.21.15.57:53526) 2023-11-28T12:02:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.513+0000 7f6658bdb700 1 --2- 172.21.15.57:0/3325302307 >> v2:172.21.15.181:3300/0 conn(0x7f665406a640 0x7f665406df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.513+0000 7f66593dc700 1 -- 172.21.15.57:0/3325302307 >> v2:172.21.15.181:3300/0 conn(0x7f665406a640 msgr2=0x7f665406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.513+0000 7f66593dc700 1 --2- 172.21.15.57:0/3325302307 >> v2:172.21.15.181:3300/0 conn(0x7f665406a640 0x7f665406df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.513+0000 7f66593dc700 1 -- 172.21.15.57:0/3325302307 >> v1:172.21.15.57:6789/0 conn(0x7f665406e450 legacy=0x7f66541612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.513+0000 7f66593dc700 1 -- 172.21.15.57:0/3325302307 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66540b1ed0 con 0x7f665406a270 2023-11-28T12:02:24.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.513+0000 7f6647fff700 1 -- 172.21.15.57:0/3325302307 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319876697 0 0) 0x7f66540b1910 con 0x7f665406e450 2023-11-28T12:02:24.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.513+0000 7f66593dc700 1 --2- 172.21.15.57:0/3325302307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f665406a270 0x7f6654164690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f66540707d0 tx=0x7f6650009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cbe2861fdab9e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:24.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.513+0000 7f6647fff700 1 -- 172.21.15.57:0/3325302307 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6650011830 con 0x7f665406a270 2023-11-28T12:02:24.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.513+0000 7f6647fff700 1 -- 172.21.15.57:0/3325302307 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6650011990 con 0x7f665406a270 2023-11-28T12:02:24.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.513+0000 7f6647fff700 1 -- 172.21.15.57:0/3325302307 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6650011d20 con 0x7f665406a270 2023-11-28T12:02:24.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.514+0000 7f665b640700 1 -- 172.21.15.57:0/3325302307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f665406a270 msgr2=0x7f6654164690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.514+0000 7f665b640700 1 --2- 172.21.15.57:0/3325302307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f665406a270 0x7f6654164690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f66540707d0 tx=0x7f6650009b20 comp rx=0 tx=0).stop 2023-11-28T12:02:24.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.514+0000 7f665b640700 1 -- 172.21.15.57:0/3325302307 shutdown_connections 2023-11-28T12:02:24.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.514+0000 7f665b640700 1 --2- 172.21.15.57:0/3325302307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f665406a270 0x7f6654164690 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.514+0000 7f665b640700 1 --2- 172.21.15.57:0/3325302307 >> v2:172.21.15.181:3300/0 conn(0x7f665406a640 0x7f665406df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.514+0000 7f665b640700 1 -- 172.21.15.57:0/3325302307 >> 172.21.15.57:0/3325302307 conn(0x7f665406c010 msgr2=0x7f66540cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:24.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.514+0000 7f665b640700 1 -- 172.21.15.57:0/3325302307 shutdown_connections 2023-11-28T12:02:24.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.514+0000 7f665b640700 1 -- 172.21.15.57:0/3325302307 wait complete. 2023-11-28T12:02:24.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.515+0000 7f665b640700 1 Processor -- start 2023-11-28T12:02:24.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.515+0000 7f665b640700 1 -- start start 2023-11-28T12:02:24.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.516+0000 7f665b640700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f665406a270 0x7f66540b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.516+0000 7f665b640700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f665406a640 0x7f66540b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.516+0000 7f665b640700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f665406e450 0x7f66540b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.516+0000 7f665b640700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66540c7100 con 0x7f665406a640 2023-11-28T12:02:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.516+0000 7f665b640700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66540b5630 con 0x7f665406a270 2023-11-28T12:02:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.516+0000 7f665b640700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66540b5910 con 0x7f665406e450 2023-11-28T12:02:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.516+0000 7f6658bdb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f665406a640 0x7f66540b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.516+0000 7f6658bdb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f665406a640 0x7f66540b4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53538/0 (socket says 172.21.15.57:53538) 2023-11-28T12:02:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.516+0000 7f6658bdb700 1 -- 172.21.15.57:0/2000021022 learned_addr learned my addr 172.21.15.57:0/2000021022 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.516+0000 7f66593dc700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f665406a270 0x7f66540b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.516+0000 7f6659bdd700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f665406e450 0x7f66540b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.517+0000 7f6658bdb700 1 -- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f665406e450 msgr2=0x7f66540b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.517+0000 7f6658bdb700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f665406e450 0x7f66540b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.517+0000 7f6658bdb700 1 -- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f665406a270 msgr2=0x7f66540b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.517+0000 7f6658bdb700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f665406a270 0x7f66540b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.517+0000 7f6658bdb700 1 -- 172.21.15.57:0/2000021022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66500094a0 con 0x7f665406a640 2023-11-28T12:02:24.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.517+0000 7f6658bdb700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f665406a640 0x7f66540b4a90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f664800e810 tx=0x7f664800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:24.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.517+0000 7f6645ffb700 1 -- 172.21.15.57:0/2000021022 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664800c070 con 0x7f665406a640 2023-11-28T12:02:24.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.517+0000 7f6645ffb700 1 -- 172.21.15.57:0/2000021022 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6648013610 con 0x7f665406a640 2023-11-28T12:02:24.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.517+0000 7f665b640700 1 -- 172.21.15.57:0/2000021022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665415e950 con 0x7f665406a640 2023-11-28T12:02:24.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.517+0000 7f6645ffb700 1 -- 172.21.15.57:0/2000021022 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66480139c0 con 0x7f665406a640 2023-11-28T12:02:24.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.517+0000 7f665b640700 1 -- 172.21.15.57:0/2000021022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66540b21b0 con 0x7f665406a640 2023-11-28T12:02:24.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.518+0000 7f665b640700 1 -- 172.21.15.57:0/2000021022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665406aab0 con 0x7f665406a640 2023-11-28T12:02:24.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.519+0000 7f6645ffb700 1 -- 172.21.15.57:0/2000021022 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6648023070 con 0x7f665406a640 2023-11-28T12:02:24.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.519+0000 7f6645ffb700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f66380682f0 0x7f663806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.519+0000 7f6645ffb700 1 -- 172.21.15.57:0/2000021022 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6648085a30 con 0x7f665406a640 2023-11-28T12:02:24.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.519+0000 7f66593dc700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f66380682f0 0x7f663806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.520+0000 7f66593dc700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f66380682f0 0x7f663806a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f66500090b0 tx=0x7f665001c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:24.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.523+0000 7f6645ffb700 1 -- 172.21.15.57:0/2000021022 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6648052210 con 0x7f665406a640 2023-11-28T12:02:24.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.723+0000 7f665b640700 1 -- 172.21.15.57:0/2000021022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66540c7530 con 0x7f665406a640 2023-11-28T12:02:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.725+0000 7f6645ffb700 1 -- 172.21.15.57:0/2000021022 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7f664800f040 con 0x7f665406a640 2023-11-28T12:02:24.726 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:24.726 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.727+0000 7f665b640700 1 -- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f66380682f0 msgr2=0x7f663806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.727+0000 7f665b640700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f66380682f0 0x7f663806a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f66500090b0 tx=0x7f665001c040 comp rx=0 tx=0).stop 2023-11-28T12:02:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.727+0000 7f665b640700 1 -- 172.21.15.57:0/2000021022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f665406a640 msgr2=0x7f66540b4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.727+0000 7f665b640700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f665406a640 0x7f66540b4a90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f664800e810 tx=0x7f664800eb20 comp rx=0 tx=0).stop 2023-11-28T12:02:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.727+0000 7f665b640700 1 -- 172.21.15.57:0/2000021022 shutdown_connections 2023-11-28T12:02:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.727+0000 7f665b640700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f66380682f0 0x7f663806a7a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.727+0000 7f665b640700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f665406e450 0x7f66540b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.727+0000 7f665b640700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f665406a270 0x7f66540b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.727+0000 7f665b640700 1 --2- 172.21.15.57:0/2000021022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f665406a640 0x7f66540b4a90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.727+0000 7f665b640700 1 -- 172.21.15.57:0/2000021022 >> 172.21.15.57:0/2000021022 conn(0x7f665406c010 msgr2=0x7f66540cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.727+0000 7f665b640700 1 -- 172.21.15.57:0/2000021022 shutdown_connections 2023-11-28T12:02:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.727+0000 7f665b640700 1 -- 172.21.15.57:0/2000021022 wait complete. 2023-11-28T12:02:24.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:24.742 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:24.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.880+0000 7f277068b700 1 Processor -- start 2023-11-28T12:02:24.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.881+0000 7f277068b700 1 -- start start 2023-11-28T12:02:24.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f277068b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f27680cf710 0x7f27680ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f277068b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2768071890 0x7f2768071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f277068b700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f276815cc10 con 0x7f276806a270 2023-11-28T12:02:24.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f277068b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276806c9b0 con 0x7f2768071890 2023-11-28T12:02:24.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f277068b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276806cc90 con 0x7f27680cf710 2023-11-28T12:02:24.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f276dc26700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2768071890 0x7f2768071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f276ec28700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f276806a270 0x7f27680b1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40202/0 (socket says 172.21.15.57:40202) 2023-11-28T12:02:24.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f276dc26700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2768071890 0x7f2768071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53542/0 (socket says 172.21.15.57:53542) 2023-11-28T12:02:24.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f276ec28700 1 -- 172.21.15.57:0/2515946618 learned_addr learned my addr 172.21.15.57:0/2515946618 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:24.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f276e427700 1 --2- 172.21.15.57:0/2515946618 >> v2:172.21.15.181:3300/0 conn(0x7f27680cf710 0x7f27680ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f276dc26700 1 -- 172.21.15.57:0/2515946618 >> v2:172.21.15.181:3300/0 conn(0x7f27680cf710 msgr2=0x7f27680ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:24.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f276dc26700 1 --2- 172.21.15.57:0/2515946618 >> v2:172.21.15.181:3300/0 conn(0x7f27680cf710 0x7f27680ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f276dc26700 1 -- 172.21.15.57:0/2515946618 >> v1:172.21.15.57:6789/0 conn(0x7f276806a270 legacy=0x7f27680b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f276dc26700 1 -- 172.21.15.57:0/2515946618 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276806cf70 con 0x7f2768071890 2023-11-28T12:02:24.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f276d425700 1 -- 172.21.15.57:0/2515946618 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303545092 0 0) 0x7f276815cc10 con 0x7f276806a270 2023-11-28T12:02:24.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.882+0000 7f276dc26700 1 --2- 172.21.15.57:0/2515946618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2768071890 0x7f2768071220 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f275c009840 tx=0x7f275c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a53ff3df80b872 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:24.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.883+0000 7f276d425700 1 -- 172.21.15.57:0/2515946618 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c011830 con 0x7f2768071890 2023-11-28T12:02:24.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.883+0000 7f276d425700 1 -- 172.21.15.57:0/2515946618 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275c011990 con 0x7f2768071890 2023-11-28T12:02:24.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.883+0000 7f276d425700 1 -- 172.21.15.57:0/2515946618 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c011d20 con 0x7f2768071890 2023-11-28T12:02:24.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.883+0000 7f277068b700 1 -- 172.21.15.57:0/2515946618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2768071890 msgr2=0x7f2768071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.883+0000 7f277068b700 1 --2- 172.21.15.57:0/2515946618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2768071890 0x7f2768071220 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f275c009840 tx=0x7f275c009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:24.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.883+0000 7f277068b700 1 -- 172.21.15.57:0/2515946618 shutdown_connections 2023-11-28T12:02:24.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.883+0000 7f277068b700 1 --2- 172.21.15.57:0/2515946618 >> v2:172.21.15.181:3300/0 conn(0x7f27680cf710 0x7f27680ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.883+0000 7f277068b700 1 --2- 172.21.15.57:0/2515946618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2768071890 0x7f2768071220 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.883+0000 7f277068b700 1 -- 172.21.15.57:0/2515946618 >> 172.21.15.57:0/2515946618 conn(0x7f276806c010 msgr2=0x7f27680cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:24.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.883+0000 7f277068b700 1 -- 172.21.15.57:0/2515946618 shutdown_connections 2023-11-28T12:02:24.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.883+0000 7f277068b700 1 -- 172.21.15.57:0/2515946618 wait complete. 2023-11-28T12:02:24.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.885+0000 7f277068b700 1 Processor -- start 2023-11-28T12:02:24.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.885+0000 7f277068b700 1 -- start start 2023-11-28T12:02:24.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f277068b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f276806a270 0x7f2768155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f277068b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2768071890 0x7f27681f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f277068b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27680cf710 0x7f27680cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f277068b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27681f5d40 con 0x7f27680cf710 2023-11-28T12:02:24.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f277068b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276815ce00 con 0x7f276806a270 2023-11-28T12:02:24.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f277068b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27681f2ed0 con 0x7f2768071890 2023-11-28T12:02:24.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f276ec28700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27680cf710 0x7f27680cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f276ec28700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27680cf710 0x7f27680cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53548/0 (socket says 172.21.15.57:53548) 2023-11-28T12:02:24.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f276ec28700 1 -- 172.21.15.57:0/2927752424 learned_addr learned my addr 172.21.15.57:0/2927752424 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:24.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f276e427700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f276806a270 0x7f2768155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f276dc26700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2768071890 0x7f27681f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f276ec28700 1 -- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2768071890 msgr2=0x7f27681f2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f276ec28700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2768071890 0x7f27681f2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f276ec28700 1 -- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f276806a270 msgr2=0x7f2768155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:24.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f276ec28700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f276806a270 0x7f2768155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:24.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f276ec28700 1 -- 172.21.15.57:0/2927752424 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c0094a0 con 0x7f27680cf710 2023-11-28T12:02:24.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.886+0000 7f276e427700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f276806a270 0x7f2768155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:24.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.887+0000 7f276ec28700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27680cf710 0x7f27680cc290 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2760002230 tx=0x7f276000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:24.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.887+0000 7f275affd700 1 -- 172.21.15.57:0/2927752424 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276000ac20 con 0x7f27680cf710 2023-11-28T12:02:24.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.887+0000 7f275affd700 1 -- 172.21.15.57:0/2927752424 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276000ada0 con 0x7f27680cf710 2023-11-28T12:02:24.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.887+0000 7f277068b700 1 -- 172.21.15.57:0/2927752424 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276806d530 con 0x7f27680cf710 2023-11-28T12:02:24.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.887+0000 7f275affd700 1 -- 172.21.15.57:0/2927752424 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2760007310 con 0x7f27680cf710 2023-11-28T12:02:24.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.887+0000 7f277068b700 1 -- 172.21.15.57:0/2927752424 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276806d250 con 0x7f27680cf710 2023-11-28T12:02:24.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.888+0000 7f277068b700 1 -- 172.21.15.57:0/2927752424 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276806a6b0 con 0x7f27680cf710 2023-11-28T12:02:24.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.888+0000 7f275affd700 1 -- 172.21.15.57:0/2927752424 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2760010070 con 0x7f27680cf710 2023-11-28T12:02:24.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.889+0000 7f275affd700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f274c0682f0 0x7f274c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:24.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.889+0000 7f275affd700 1 -- 172.21.15.57:0/2927752424 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2760085680 con 0x7f27680cf710 2023-11-28T12:02:24.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.889+0000 7f276e427700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f274c0682f0 0x7f274c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:24.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.889+0000 7f276e427700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f274c0682f0 0x7f274c06a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2764009820 tx=0x7f2764008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:24.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:24.893+0000 7f275affd700 1 -- 172.21.15.57:0/2927752424 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2760051ee0 con 0x7f27680cf710 2023-11-28T12:02:25.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.094+0000 7f277068b700 1 -- 172.21.15.57:0/2927752424 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2768074420 con 0x7f27680cf710 2023-11-28T12:02:25.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.094+0000 7f275affd700 1 -- 172.21.15.57:0/2927752424 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2760018020 con 0x7f27680cf710 2023-11-28T12:02:25.096 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:25.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.097+0000 7f277068b700 1 -- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f274c0682f0 msgr2=0x7f274c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.097+0000 7f277068b700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f274c0682f0 0x7f274c06a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2764009820 tx=0x7f2764008040 comp rx=0 tx=0).stop 2023-11-28T12:02:25.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.097+0000 7f277068b700 1 -- 172.21.15.57:0/2927752424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27680cf710 msgr2=0x7f27680cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.097+0000 7f277068b700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27680cf710 0x7f27680cc290 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2760002230 tx=0x7f276000a060 comp rx=0 tx=0).stop 2023-11-28T12:02:25.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.097+0000 7f277068b700 1 -- 172.21.15.57:0/2927752424 shutdown_connections 2023-11-28T12:02:25.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.097+0000 7f277068b700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f274c0682f0 0x7f274c06a7a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.097+0000 7f277068b700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27680cf710 0x7f27680cc290 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.097+0000 7f277068b700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f276806a270 0x7f2768155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.097+0000 7f277068b700 1 --2- 172.21.15.57:0/2927752424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2768071890 0x7f27681f2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.097+0000 7f277068b700 1 -- 172.21.15.57:0/2927752424 >> 172.21.15.57:0/2927752424 conn(0x7f276806c010 msgr2=0x7f276815beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:25.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.097+0000 7f277068b700 1 -- 172.21.15.57:0/2927752424 shutdown_connections 2023-11-28T12:02:25.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.097+0000 7f277068b700 1 -- 172.21.15.57:0/2927752424 wait complete. 2023-11-28T12:02:25.111 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:25.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.248+0000 7ff3ddcc2700 1 Processor -- start 2023-11-28T12:02:25.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.248+0000 7ff3ddcc2700 1 -- start start 2023-11-28T12:02:25.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.249+0000 7ff3ddcc2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff3d80b1650 0x7ff3d80b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.249+0000 7ff3ddcc2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff3d81599b0 0x7ff3d815d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.249+0000 7ff3ddcc2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d8159ec0 con 0x7ff3d80af0a0 2023-11-28T12:02:25.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.249+0000 7ff3ddcc2700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d815aa60 con 0x7ff3d81599b0 2023-11-28T12:02:25.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.249+0000 7ff3ddcc2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d80b1f60 con 0x7ff3d80b1650 2023-11-28T12:02:25.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.249+0000 7ff3d6ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff3d81599b0 0x7ff3d815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.249+0000 7ff3d6ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff3d81599b0 0x7ff3d815d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53550/0 (socket says 172.21.15.57:53550) 2023-11-28T12:02:25.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.249+0000 7ff3d6ffd700 1 -- 172.21.15.57:0/1358922322 learned_addr learned my addr 172.21.15.57:0/1358922322 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:25.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.249+0000 7ff3d77fe700 1 --2- 172.21.15.57:0/1358922322 >> v2:172.21.15.181:3300/0 conn(0x7ff3d80b1650 0x7ff3d80b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.249+0000 7ff3d6ffd700 1 -- 172.21.15.57:0/1358922322 >> v2:172.21.15.181:3300/0 conn(0x7ff3d80b1650 msgr2=0x7ff3d80b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:25.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.249+0000 7ff3d6ffd700 1 --2- 172.21.15.57:0/1358922322 >> v2:172.21.15.181:3300/0 conn(0x7ff3d80b1650 0x7ff3d80b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.249+0000 7ff3d6ffd700 1 -- 172.21.15.57:0/1358922322 >> v1:172.21.15.181:6789/0 conn(0x7ff3d80af0a0 legacy=0x7ff3d815a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.250+0000 7ff3d6ffd700 1 -- 172.21.15.57:0/1358922322 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d815d8a0 con 0x7ff3d81599b0 2023-11-28T12:02:25.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.250+0000 7ff3d77fe700 1 --2- 172.21.15.57:0/1358922322 >> v2:172.21.15.181:3300/0 conn(0x7ff3d80b1650 0x7ff3d80b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:25.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.250+0000 7ff3d6ffd700 1 --2- 172.21.15.57:0/1358922322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3d81599b0 0x7ff3d815d050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8067310 tx=0x7ff3c000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2f24f7718f28ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:25.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.250+0000 7ff3d67fc700 1 -- 172.21.15.57:0/1358922322 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c000dd30 con 0x7ff3d81599b0 2023-11-28T12:02:25.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.250+0000 7ff3d67fc700 1 -- 172.21.15.57:0/1358922322 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c000de90 con 0x7ff3d81599b0 2023-11-28T12:02:25.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.250+0000 7ff3d67fc700 1 -- 172.21.15.57:0/1358922322 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0010690 con 0x7ff3d81599b0 2023-11-28T12:02:25.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.250+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1358922322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3d81599b0 msgr2=0x7ff3d815d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.250+0000 7ff3ddcc2700 1 --2- 172.21.15.57:0/1358922322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3d81599b0 0x7ff3d815d050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8067310 tx=0x7ff3c000be70 comp rx=0 tx=0).stop 2023-11-28T12:02:25.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.250+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1358922322 shutdown_connections 2023-11-28T12:02:25.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.250+0000 7ff3ddcc2700 1 --2- 172.21.15.57:0/1358922322 >> v2:172.21.15.181:3300/0 conn(0x7ff3d80b1650 0x7ff3d80b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.250+0000 7ff3ddcc2700 1 --2- 172.21.15.57:0/1358922322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3d81599b0 0x7ff3d815d050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.250+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1358922322 >> 172.21.15.57:0/1358922322 conn(0x7ff3d806c010 msgr2=0x7ff3d80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:25.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.251+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1358922322 shutdown_connections 2023-11-28T12:02:25.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.251+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1358922322 wait complete. 2023-11-28T12:02:25.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.252+0000 7ff3ddcc2700 1 Processor -- start 2023-11-28T12:02:25.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.253+0000 7ff3ddcc2700 1 -- start start 2023-11-28T12:02:25.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.253+0000 7ff3ddcc2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3d80af0a0 0x7ff3d815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.253+0000 7ff3ddcc2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3d80b1650 0x7ff3d814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.253+0000 7ff3ddcc2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3d814ea00 0x7ff3d80c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.253+0000 7ff3ddcc2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d81563b0 con 0x7ff3d814ea00 2023-11-28T12:02:25.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.253+0000 7ff3ddcc2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d80c4610 con 0x7ff3d80af0a0 2023-11-28T12:02:25.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.253+0000 7ff3ddcc2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d814ee80 con 0x7ff3d80b1650 2023-11-28T12:02:25.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3d7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3d814ea00 0x7ff3d80c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3d7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3d814ea00 0x7ff3d80c38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53560/0 (socket says 172.21.15.57:53560) 2023-11-28T12:02:25.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3d7fff700 1 -- 172.21.15.57:0/1425060349 learned_addr learned my addr 172.21.15.57:0/1425060349 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:25.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3d6ffd700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3d80b1650 0x7ff3d814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3d77fe700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3d80af0a0 0x7ff3d815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3d7fff700 1 -- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3d80b1650 msgr2=0x7ff3d814e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3d7fff700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3d80b1650 0x7ff3d814e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3d7fff700 1 -- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3d80af0a0 msgr2=0x7ff3d815bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3d7fff700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3d80af0a0 0x7ff3d815bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3d7fff700 1 -- 172.21.15.57:0/1425060349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c000b7f0 con 0x7ff3d814ea00 2023-11-28T12:02:25.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3d7fff700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3d814ea00 0x7ff3d80c38c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000cc50 tx=0x7ff3d000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:25.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3cffff700 1 -- 172.21.15.57:0/1425060349 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0003c00 con 0x7ff3d814ea00 2023-11-28T12:02:25.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3cffff700 1 -- 172.21.15.57:0/1425060349 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d0003d60 con 0x7ff3d814ea00 2023-11-28T12:02:25.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3cffff700 1 -- 172.21.15.57:0/1425060349 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000aab0 con 0x7ff3d814ea00 2023-11-28T12:02:25.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1425060349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d815de60 con 0x7ff3d814ea00 2023-11-28T12:02:25.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.254+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1425060349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d815db80 con 0x7ff3d814ea00 2023-11-28T12:02:25.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.255+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1425060349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d815d590 con 0x7ff3d814ea00 2023-11-28T12:02:25.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.256+0000 7ff3cffff700 1 -- 172.21.15.57:0/1425060349 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff3d0010050 con 0x7ff3d814ea00 2023-11-28T12:02:25.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.256+0000 7ff3cffff700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff3b80682f0 0x7ff3b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.256+0000 7ff3cffff700 1 -- 172.21.15.57:0/1425060349 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff3d0085ba0 con 0x7ff3d814ea00 2023-11-28T12:02:25.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.256+0000 7ff3d77fe700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff3b80682f0 0x7ff3b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.257+0000 7ff3d77fe700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff3b80682f0 0x7ff3b806a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80099d0 tx=0x7ff3c8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:25.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.261+0000 7ff3cffff700 1 -- 172.21.15.57:0/1425060349 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff3d0052380 con 0x7ff3d814ea00 2023-11-28T12:02:25.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.461+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1425060349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3d80743f0 con 0x7ff3d814ea00 2023-11-28T12:02:25.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.461+0000 7ff3cffff700 1 -- 172.21.15.57:0/1425060349 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3d00559a0 con 0x7ff3d814ea00 2023-11-28T12:02:25.462 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:25.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.463+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff3b80682f0 msgr2=0x7ff3b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.463+0000 7ff3ddcc2700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff3b80682f0 0x7ff3b806a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80099d0 tx=0x7ff3c8008040 comp rx=0 tx=0).stop 2023-11-28T12:02:25.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.463+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1425060349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3d814ea00 msgr2=0x7ff3d80c38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.463+0000 7ff3ddcc2700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3d814ea00 0x7ff3d80c38c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000cc50 tx=0x7ff3d000cf60 comp rx=0 tx=0).stop 2023-11-28T12:02:25.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.463+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1425060349 shutdown_connections 2023-11-28T12:02:25.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.463+0000 7ff3ddcc2700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff3b80682f0 0x7ff3b806a7a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.463+0000 7ff3ddcc2700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3d814ea00 0x7ff3d80c38c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.463+0000 7ff3ddcc2700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3d80af0a0 0x7ff3d815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.463+0000 7ff3ddcc2700 1 --2- 172.21.15.57:0/1425060349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3d80b1650 0x7ff3d814e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.463+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1425060349 >> 172.21.15.57:0/1425060349 conn(0x7ff3d806c010 msgr2=0x7ff3d80b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:25.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.464+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1425060349 shutdown_connections 2023-11-28T12:02:25.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.464+0000 7ff3ddcc2700 1 -- 172.21.15.57:0/1425060349 wait complete. 2023-11-28T12:02:25.478 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:02:25.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.616+0000 7f521385e700 1 Processor -- start 2023-11-28T12:02:25.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.616+0000 7f521385e700 1 -- start start 2023-11-28T12:02:25.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.617+0000 7f521385e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f520c15c8c0 0x7f520c160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.617+0000 7f521385e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520c0d2d30 con 0x7f520c06b220 2023-11-28T12:02:25.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.617+0000 7f521385e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520c0d0b40 con 0x7f520c0b5dc0 2023-11-28T12:02:25.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.617+0000 7f521385e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520c1611f0 con 0x7f520c15c8c0 2023-11-28T12:02:25.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.617+0000 7f5211dfb700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f520c06b220 0x7f520c0d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40218/0 (socket says 172.21.15.57:40218) 2023-11-28T12:02:25.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.617+0000 7f5211dfb700 1 -- 172.21.15.57:0/15138629 learned_addr learned my addr 172.21.15.57:0/15138629 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:25.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.617+0000 7f52115fa700 1 --2- 172.21.15.57:0/15138629 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f520c15c8c0 0x7f520c160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.618+0000 7f51fffff700 1 -- 172.21.15.57:0/15138629 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336426672 0 0) 0x7f5204001490 con 0x7f520c06b220 2023-11-28T12:02:25.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.618+0000 7f51fffff700 1 -- 172.21.15.57:0/15138629 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f80035b0 con 0x7f520c06b220 2023-11-28T12:02:25.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.618+0000 7f51fffff700 1 -- 172.21.15.57:0/15138629 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174897002 0 0) 0x7f520c0d0b40 con 0x7f520c0b5dc0 2023-11-28T12:02:25.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.618+0000 7f51fffff700 1 -- 172.21.15.57:0/15138629 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5204001490 con 0x7f520c0b5dc0 2023-11-28T12:02:25.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.618+0000 7f52115fa700 1 -- 172.21.15.57:0/15138629 >> v1:172.21.15.181:6789/0 conn(0x7f520c0b5dc0 legacy=0x7f520c06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.618+0000 7f52115fa700 1 -- 172.21.15.57:0/15138629 >> v1:172.21.15.57:6789/0 conn(0x7f520c06b220 legacy=0x7f520c0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.618+0000 7f52115fa700 1 -- 172.21.15.57:0/15138629 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520c0d2a50 con 0x7f520c15c8c0 2023-11-28T12:02:25.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.618+0000 7f52115fa700 1 --2- 172.21.15.57:0/15138629 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f520c15c8c0 0x7f520c160cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5208009870 tx=0x7f5208009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5a12149f49f44e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:25.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.618+0000 7f51fffff700 1 -- 172.21.15.57:0/15138629 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208011830 con 0x7f520c15c8c0 2023-11-28T12:02:25.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.618+0000 7f51fffff700 1 -- 172.21.15.57:0/15138629 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5208011990 con 0x7f520c15c8c0 2023-11-28T12:02:25.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.618+0000 7f51fffff700 1 -- 172.21.15.57:0/15138629 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208011d20 con 0x7f520c15c8c0 2023-11-28T12:02:25.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.619+0000 7f521385e700 1 -- 172.21.15.57:0/15138629 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f520c15c8c0 msgr2=0x7f520c160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.619+0000 7f521385e700 1 --2- 172.21.15.57:0/15138629 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f520c15c8c0 0x7f520c160cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5208009870 tx=0x7f5208009b80 comp rx=0 tx=0).stop 2023-11-28T12:02:25.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.619+0000 7f521385e700 1 -- 172.21.15.57:0/15138629 shutdown_connections 2023-11-28T12:02:25.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.619+0000 7f521385e700 1 --2- 172.21.15.57:0/15138629 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f520c15c8c0 0x7f520c160cb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.619+0000 7f521385e700 1 -- 172.21.15.57:0/15138629 >> 172.21.15.57:0/15138629 conn(0x7f520c06c520 msgr2=0x7f520c06c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:25.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.619+0000 7f521385e700 1 -- 172.21.15.57:0/15138629 shutdown_connections 2023-11-28T12:02:25.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.619+0000 7f521385e700 1 -- 172.21.15.57:0/15138629 wait complete. 2023-11-28T12:02:25.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.620+0000 7f521385e700 1 Processor -- start 2023-11-28T12:02:25.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.620+0000 7f521385e700 1 -- start start 2023-11-28T12:02:25.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f521385e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f520c06b220 0x7f520c0ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f521385e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f520c0b5dc0 0x7f520c0cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f521385e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f520c15c8c0 0x7f520c1f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f521385e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c1f32b0 con 0x7f520c0b5dc0 2023-11-28T12:02:25.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f521385e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c0ca620 con 0x7f520c06b220 2023-11-28T12:02:25.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f521385e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520c0ca900 con 0x7f520c15c8c0 2023-11-28T12:02:25.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f5210df9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f520c0b5dc0 0x7f520c0cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f5210df9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f520c0b5dc0 0x7f520c0cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53564/0 (socket says 172.21.15.57:53564) 2023-11-28T12:02:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f5210df9700 1 -- 172.21.15.57:0/1905957788 learned_addr learned my addr 172.21.15.57:0/1905957788 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f52115fa700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f520c06b220 0x7f520c0ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f5211dfb700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f520c15c8c0 0x7f520c1f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f5210df9700 1 -- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f520c15c8c0 msgr2=0x7f520c1f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f5210df9700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f520c15c8c0 0x7f520c1f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f5210df9700 1 -- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f520c06b220 msgr2=0x7f520c0ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f5210df9700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f520c06b220 0x7f520c0ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f5210df9700 1 -- 172.21.15.57:0/1905957788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52080094a0 con 0x7f520c0b5dc0 2023-11-28T12:02:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f52115fa700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f520c06b220 0x7f520c0ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.621+0000 7f5211dfb700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f520c15c8c0 0x7f520c1f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:25.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.622+0000 7f5210df9700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f520c0b5dc0 0x7f520c0cba00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f520000c520 tx=0x7f520000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:25.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.622+0000 7f51fdffb700 1 -- 172.21.15.57:0/1905957788 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52000052d0 con 0x7f520c0b5dc0 2023-11-28T12:02:25.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.622+0000 7f51fdffb700 1 -- 172.21.15.57:0/1905957788 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5200005430 con 0x7f520c0b5dc0 2023-11-28T12:02:25.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.622+0000 7f521385e700 1 -- 172.21.15.57:0/1905957788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520c0cda40 con 0x7f520c0b5dc0 2023-11-28T12:02:25.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.622+0000 7f51fdffb700 1 -- 172.21.15.57:0/1905957788 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52000038c0 con 0x7f520c0b5dc0 2023-11-28T12:02:25.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.622+0000 7f521385e700 1 -- 172.21.15.57:0/1905957788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520c0cd760 con 0x7f520c0b5dc0 2023-11-28T12:02:25.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.623+0000 7f521385e700 1 -- 172.21.15.57:0/1905957788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520c0d2700 con 0x7f520c0b5dc0 2023-11-28T12:02:25.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.623+0000 7f51fdffb700 1 -- 172.21.15.57:0/1905957788 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5200019030 con 0x7f520c0b5dc0 2023-11-28T12:02:25.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.624+0000 7f51fdffb700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f51f00682f0 0x7f51f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.624+0000 7f51fdffb700 1 -- 172.21.15.57:0/1905957788 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52000850b0 con 0x7f520c0b5dc0 2023-11-28T12:02:25.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.624+0000 7f52115fa700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f51f00682f0 0x7f51f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.625+0000 7f52115fa700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f51f00682f0 0x7f51f006a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f520c0d03f0 tx=0x7f520801c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:25.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.629+0000 7f51fdffb700 1 -- 172.21.15.57:0/1905957788 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5200051910 con 0x7f520c0b5dc0 2023-11-28T12:02:25.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.826+0000 7f521385e700 1 -- 172.21.15.57:0/1905957788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f520c067d70 con 0x7f520c0b5dc0 2023-11-28T12:02:25.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.827+0000 7f51fdffb700 1 -- 172.21.15.57:0/1905957788 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5200054f30 con 0x7f520c0b5dc0 2023-11-28T12:02:25.830 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:02:25.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.829+0000 7f521385e700 1 -- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f51f00682f0 msgr2=0x7f51f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.829+0000 7f521385e700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f51f00682f0 0x7f51f006a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f520c0d03f0 tx=0x7f520801c040 comp rx=0 tx=0).stop 2023-11-28T12:02:25.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.829+0000 7f521385e700 1 -- 172.21.15.57:0/1905957788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f520c0b5dc0 msgr2=0x7f520c0cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.829+0000 7f521385e700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f520c0b5dc0 0x7f520c0cba00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f520000c520 tx=0x7f520000c8e0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.829+0000 7f521385e700 1 -- 172.21.15.57:0/1905957788 shutdown_connections 2023-11-28T12:02:25.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.829+0000 7f521385e700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f51f00682f0 0x7f51f006a7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.829+0000 7f521385e700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f520c15c8c0 0x7f520c1f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.829+0000 7f521385e700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f520c06b220 0x7f520c0ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.829+0000 7f521385e700 1 --2- 172.21.15.57:0/1905957788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f520c0b5dc0 0x7f520c0cba00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.829+0000 7f521385e700 1 -- 172.21.15.57:0/1905957788 >> 172.21.15.57:0/1905957788 conn(0x7f520c06c520 msgr2=0x7f520c160750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:25.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.829+0000 7f521385e700 1 -- 172.21.15.57:0/1905957788 shutdown_connections 2023-11-28T12:02:25.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.829+0000 7f521385e700 1 -- 172.21.15.57:0/1905957788 wait complete. 2023-11-28T12:02:25.844 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:02:25.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.983+0000 7fdd72441700 1 Processor -- start 2023-11-28T12:02:25.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.983+0000 7fdd72441700 1 -- start start 2023-11-28T12:02:25.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.984+0000 7fdd72441700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd6c0cf710 0x7fdd6c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.984+0000 7fdd72441700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdd6c06a980 0x7fdd6c071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.984+0000 7fdd72441700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6c15cc10 con 0x7fdd6c071890 2023-11-28T12:02:25.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.984+0000 7fdd72441700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6c06c9b0 con 0x7fdd6c06a980 2023-11-28T12:02:25.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.984+0000 7fdd72441700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c06cc90 con 0x7fdd6c0cf710 2023-11-28T12:02:25.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.985+0000 7fdd6bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd6c0cf710 0x7fdd6c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.985+0000 7fdd709de700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdd6c06a980 0x7fdd6c071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.985+0000 7fdd6b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdd6c071890 0x7fdd6c06a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:55894/0 (socket says 172.21.15.57:55894) 2023-11-28T12:02:25.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.985+0000 7fdd6b7fe700 1 -- 172.21.15.57:0/2983101202 learned_addr learned my addr 172.21.15.57:0/2983101202 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:25.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.985+0000 7fdd6affd700 1 -- 172.21.15.57:0/2983101202 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477961563 0 0) 0x7fdd6c15cc10 con 0x7fdd6c071890 2023-11-28T12:02:25.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.985+0000 7fdd6bfff700 1 -- 172.21.15.57:0/2983101202 >> v2:172.21.15.181:3300/0 conn(0x7fdd6c06a980 msgr2=0x7fdd6c071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:25.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.985+0000 7fdd6bfff700 1 --2- 172.21.15.57:0/2983101202 >> v2:172.21.15.181:3300/0 conn(0x7fdd6c06a980 0x7fdd6c071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.985+0000 7fdd6bfff700 1 -- 172.21.15.57:0/2983101202 >> v1:172.21.15.181:6789/0 conn(0x7fdd6c071890 legacy=0x7fdd6c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.985+0000 7fdd6bfff700 1 -- 172.21.15.57:0/2983101202 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c06cf70 con 0x7fdd6c0cf710 2023-11-28T12:02:25.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.985+0000 7fdd6bfff700 1 --2- 172.21.15.57:0/2983101202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd6c0cf710 0x7fdd6c0ce740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009840 tx=0x7fdd5c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78aca7173b6b1752 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:25.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.985+0000 7fdd6affd700 1 -- 172.21.15.57:0/2983101202 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c011830 con 0x7fdd6c0cf710 2023-11-28T12:02:25.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.985+0000 7fdd6affd700 1 -- 172.21.15.57:0/2983101202 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c011990 con 0x7fdd6c0cf710 2023-11-28T12:02:25.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.985+0000 7fdd6affd700 1 -- 172.21.15.57:0/2983101202 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c011d20 con 0x7fdd6c0cf710 2023-11-28T12:02:25.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.986+0000 7fdd72441700 1 -- 172.21.15.57:0/2983101202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd6c0cf710 msgr2=0x7fdd6c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.986+0000 7fdd72441700 1 --2- 172.21.15.57:0/2983101202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd6c0cf710 0x7fdd6c0ce740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009840 tx=0x7fdd5c009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:25.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.986+0000 7fdd72441700 1 -- 172.21.15.57:0/2983101202 shutdown_connections 2023-11-28T12:02:25.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.986+0000 7fdd72441700 1 --2- 172.21.15.57:0/2983101202 >> v2:172.21.15.181:3300/0 conn(0x7fdd6c06a980 0x7fdd6c071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.986+0000 7fdd72441700 1 --2- 172.21.15.57:0/2983101202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd6c0cf710 0x7fdd6c0ce740 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.986+0000 7fdd72441700 1 -- 172.21.15.57:0/2983101202 >> 172.21.15.57:0/2983101202 conn(0x7fdd6c06c010 msgr2=0x7fdd6c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:25.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.986+0000 7fdd72441700 1 -- 172.21.15.57:0/2983101202 shutdown_connections 2023-11-28T12:02:25.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.986+0000 7fdd72441700 1 -- 172.21.15.57:0/2983101202 wait complete. 2023-11-28T12:02:25.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.987+0000 7fdd72441700 1 Processor -- start 2023-11-28T12:02:25.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.988+0000 7fdd72441700 1 -- start start 2023-11-28T12:02:25.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.988+0000 7fdd72441700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd6c06a980 0x7fdd6c157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.988+0000 7fdd72441700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd6c071890 0x7fdd6c156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.988+0000 7fdd72441700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd6c0cf710 0x7fdd6c157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.988+0000 7fdd72441700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c1f2820 con 0x7fdd6c06a980 2023-11-28T12:02:25.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.988+0000 7fdd72441700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c1583c0 con 0x7fdd6c0cf710 2023-11-28T12:02:25.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.988+0000 7fdd72441700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c157ba0 con 0x7fdd6c071890 2023-11-28T12:02:25.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.988+0000 7fdd6bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd6c06a980 0x7fdd6c157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.988+0000 7fdd6bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd6c06a980 0x7fdd6c157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53568/0 (socket says 172.21.15.57:53568) 2023-11-28T12:02:25.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.988+0000 7fdd6bfff700 1 -- 172.21.15.57:0/2934300742 learned_addr learned my addr 172.21.15.57:0/2934300742 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:25.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd6b7fe700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd6c071890 0x7fdd6c156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd709de700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd6c0cf710 0x7fdd6c157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd6bfff700 1 -- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd6c071890 msgr2=0x7fdd6c156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd6bfff700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd6c071890 0x7fdd6c156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd6bfff700 1 -- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd6c0cf710 msgr2=0x7fdd6c157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:25.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd6bfff700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd6c0cf710 0x7fdd6c157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:25.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd6bfff700 1 -- 172.21.15.57:0/2934300742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c0094a0 con 0x7fdd6c06a980 2023-11-28T12:02:25.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd709de700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd6c0cf710 0x7fdd6c157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:25.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd6bfff700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd6c06a980 0x7fdd6c157e80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009fd0 tx=0x7fdd5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:25.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd68ff9700 1 -- 172.21.15.57:0/2934300742 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c0036a0 con 0x7fdd6c06a980 2023-11-28T12:02:25.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd68ff9700 1 -- 172.21.15.57:0/2934300742 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c01f070 con 0x7fdd6c06a980 2023-11-28T12:02:25.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd68ff9700 1 -- 172.21.15.57:0/2934300742 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c003a00 con 0x7fdd6c06a980 2023-11-28T12:02:25.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd72441700 1 -- 172.21.15.57:0/2934300742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c06d530 con 0x7fdd6c06a980 2023-11-28T12:02:25.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.989+0000 7fdd72441700 1 -- 172.21.15.57:0/2934300742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c06d250 con 0x7fdd6c06a980 2023-11-28T12:02:25.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.990+0000 7fdd72441700 1 -- 172.21.15.57:0/2934300742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c160480 con 0x7fdd6c06a980 2023-11-28T12:02:25.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.991+0000 7fdd68ff9700 1 -- 172.21.15.57:0/2934300742 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdd5c00f050 con 0x7fdd6c06a980 2023-11-28T12:02:25.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.991+0000 7fdd68ff9700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd580682f0 0x7fdd5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:25.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.991+0000 7fdd68ff9700 1 -- 172.21.15.57:0/2934300742 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdd5c021070 con 0x7fdd6c06a980 2023-11-28T12:02:25.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.991+0000 7fdd6b7fe700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd580682f0 0x7fdd5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:25.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.992+0000 7fdd6b7fe700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd580682f0 0x7fdd5806a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c15c540 tx=0x7fdd60007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:25.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:25.996+0000 7fdd68ff9700 1 -- 172.21.15.57:0/2934300742 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdd5c053cf0 con 0x7fdd6c06a980 2023-11-28T12:02:26.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.194+0000 7fdd72441700 1 -- 172.21.15.57:0/2934300742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd6c068450 con 0x7fdd6c06a980 2023-11-28T12:02:26.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.194+0000 7fdd68ff9700 1 -- 172.21.15.57:0/2934300742 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd5c057310 con 0x7fdd6c06a980 2023-11-28T12:02:26.195 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:02:26.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.196+0000 7fdd72441700 1 -- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd580682f0 msgr2=0x7fdd5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:26.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.196+0000 7fdd72441700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd580682f0 0x7fdd5806a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c15c540 tx=0x7fdd60007de0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.196+0000 7fdd72441700 1 -- 172.21.15.57:0/2934300742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd6c06a980 msgr2=0x7fdd6c157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:26.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.196+0000 7fdd72441700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd6c06a980 0x7fdd6c157e80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009fd0 tx=0x7fdd5c004750 comp rx=0 tx=0).stop 2023-11-28T12:02:26.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.196+0000 7fdd72441700 1 -- 172.21.15.57:0/2934300742 shutdown_connections 2023-11-28T12:02:26.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.196+0000 7fdd72441700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd580682f0 0x7fdd5806a7a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.196+0000 7fdd72441700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd6c0cf710 0x7fdd6c157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.196+0000 7fdd72441700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd6c06a980 0x7fdd6c157e80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.196+0000 7fdd72441700 1 --2- 172.21.15.57:0/2934300742 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd6c071890 0x7fdd6c156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.196+0000 7fdd72441700 1 -- 172.21.15.57:0/2934300742 >> 172.21.15.57:0/2934300742 conn(0x7fdd6c06c010 msgr2=0x7fdd6c0b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:26.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.197+0000 7fdd72441700 1 -- 172.21.15.57:0/2934300742 shutdown_connections 2023-11-28T12:02:26.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.197+0000 7fdd72441700 1 -- 172.21.15.57:0/2934300742 wait complete. 2023-11-28T12:02:26.211 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T12:02:26.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.347+0000 7fe36c1bf700 1 Processor -- start 2023-11-28T12:02:26.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.347+0000 7fe36c1bf700 1 -- start start 2023-11-28T12:02:26.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe36c1bf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3640cf710 0x7fe3640ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:26.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe36c1bf700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe36406a980 0x7fe364071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:26.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe36c1bf700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36415cc10 con 0x7fe364071890 2023-11-28T12:02:26.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe36c1bf700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36406c9b0 con 0x7fe36406a980 2023-11-28T12:02:26.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe36c1bf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36406cc90 con 0x7fe3640cf710 2023-11-28T12:02:26.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe36a75c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe36406a980 0x7fe364071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:26.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe36a75c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe36406a980 0x7fe364071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53576/0 (socket says 172.21.15.57:53576) 2023-11-28T12:02:26.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe36a75c700 1 -- 172.21.15.57:0/731947303 learned_addr learned my addr 172.21.15.57:0/731947303 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:26.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe369f5b700 1 --2- 172.21.15.57:0/731947303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3640cf710 0x7fe3640ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:26.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe36a75c700 1 -- 172.21.15.57:0/731947303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3640cf710 msgr2=0x7fe3640ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:26.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe36a75c700 1 --2- 172.21.15.57:0/731947303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3640cf710 0x7fe3640ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe36a75c700 1 -- 172.21.15.57:0/731947303 >> v1:172.21.15.181:6789/0 conn(0x7fe364071890 legacy=0x7fe36406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:26.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe36a75c700 1 -- 172.21.15.57:0/731947303 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36406cf70 con 0x7fe36406a980 2023-11-28T12:02:26.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.348+0000 7fe36a75c700 1 --2- 172.21.15.57:0/731947303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe36406a980 0x7fe364071220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009e50 tx=0x7fe35c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df857651e844f5d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:26.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.349+0000 7fe368f59700 1 -- 172.21.15.57:0/731947303 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00e070 con 0x7fe36406a980 2023-11-28T12:02:26.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.349+0000 7fe368f59700 1 -- 172.21.15.57:0/731947303 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c00b9c0 con 0x7fe36406a980 2023-11-28T12:02:26.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.349+0000 7fe368f59700 1 -- 172.21.15.57:0/731947303 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00bd70 con 0x7fe36406a980 2023-11-28T12:02:26.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.349+0000 7fe36c1bf700 1 -- 172.21.15.57:0/731947303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe36406a980 msgr2=0x7fe364071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:26.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.349+0000 7fe36c1bf700 1 --2- 172.21.15.57:0/731947303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe36406a980 0x7fe364071220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009e50 tx=0x7fe35c00b330 comp rx=0 tx=0).stop 2023-11-28T12:02:26.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.349+0000 7fe36c1bf700 1 -- 172.21.15.57:0/731947303 shutdown_connections 2023-11-28T12:02:26.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.349+0000 7fe36c1bf700 1 --2- 172.21.15.57:0/731947303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe36406a980 0x7fe364071220 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.349+0000 7fe36c1bf700 1 --2- 172.21.15.57:0/731947303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3640cf710 0x7fe3640ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.349+0000 7fe36c1bf700 1 -- 172.21.15.57:0/731947303 >> 172.21.15.57:0/731947303 conn(0x7fe36406c010 msgr2=0x7fe3640cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:26.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.349+0000 7fe36c1bf700 1 -- 172.21.15.57:0/731947303 shutdown_connections 2023-11-28T12:02:26.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.349+0000 7fe36c1bf700 1 -- 172.21.15.57:0/731947303 wait complete. 2023-11-28T12:02:26.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.351+0000 7fe36c1bf700 1 Processor -- start 2023-11-28T12:02:26.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.351+0000 7fe36c1bf700 1 -- start start 2023-11-28T12:02:26.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.351+0000 7fe36c1bf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe36406a980 0x7fe3641f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:26.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.351+0000 7fe36c1bf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe364071890 0x7fe3640cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:26.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.351+0000 7fe36c1bf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3640cf710 0x7fe36415a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:26.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.351+0000 7fe36c1bf700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3640c86a0 con 0x7fe364071890 2023-11-28T12:02:26.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.351+0000 7fe36c1bf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3641f3340 con 0x7fe36406a980 2023-11-28T12:02:26.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.351+0000 7fe36c1bf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36415e3f0 con 0x7fe3640cf710 2023-11-28T12:02:26.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe36975a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe364071890 0x7fe3640cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:26.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe36975a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe364071890 0x7fe3640cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53592/0 (socket says 172.21.15.57:53592) 2023-11-28T12:02:26.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe36975a700 1 -- 172.21.15.57:0/2532889713 learned_addr learned my addr 172.21.15.57:0/2532889713 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:26.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe369f5b700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe36406a980 0x7fe3641f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:26.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe36a75c700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3640cf710 0x7fe36415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:26.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe36975a700 1 -- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3640cf710 msgr2=0x7fe36415a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:26.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe36975a700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3640cf710 0x7fe36415a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe36975a700 1 -- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe36406a980 msgr2=0x7fe3641f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:26.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe36975a700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe36406a980 0x7fe3641f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe36975a700 1 -- 172.21.15.57:0/2532889713 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c009b30 con 0x7fe364071890 2023-11-28T12:02:26.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe36975a700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe364071890 0x7fe3640cab60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe35400c4e0 tx=0x7fe35400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:26.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe35a7fc700 1 -- 172.21.15.57:0/2532889713 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3540052d0 con 0x7fe364071890 2023-11-28T12:02:26.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe35a7fc700 1 -- 172.21.15.57:0/2532889713 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe354005430 con 0x7fe364071890 2023-11-28T12:02:26.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe36c1bf700 1 -- 172.21.15.57:0/2532889713 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36406d530 con 0x7fe364071890 2023-11-28T12:02:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe35a7fc700 1 -- 172.21.15.57:0/2532889713 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3540038c0 con 0x7fe364071890 2023-11-28T12:02:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.352+0000 7fe36c1bf700 1 -- 172.21.15.57:0/2532889713 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36406d250 con 0x7fe364071890 2023-11-28T12:02:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.353+0000 7fe36c1bf700 1 -- 172.21.15.57:0/2532889713 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe364160480 con 0x7fe364071890 2023-11-28T12:02:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.354+0000 7fe35a7fc700 1 -- 172.21.15.57:0/2532889713 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe354019030 con 0x7fe364071890 2023-11-28T12:02:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.354+0000 7fe35a7fc700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe3480682f0 0x7fe34806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.354+0000 7fe35a7fc700 1 -- 172.21.15.57:0/2532889713 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe354085130 con 0x7fe364071890 2023-11-28T12:02:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.355+0000 7fe369f5b700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe3480682f0 0x7fe34806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.355+0000 7fe369f5b700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe3480682f0 0x7fe34806a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe36415f8e0 tx=0x7fe360008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:26.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.359+0000 7fe35a7fc700 1 -- 172.21.15.57:0/2532889713 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe3540518a0 con 0x7fe364071890 2023-11-28T12:02:26.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.575+0000 7fe36c1bf700 1 -- 172.21.15.57:0/2532889713 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe3640743f0 con 0x7fe364071890 2023-11-28T12:02:26.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.575+0000 7fe35a7fc700 1 -- 172.21.15.57:0/2532889713 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe354054ec0 con 0x7fe364071890 2023-11-28T12:02:26.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.578+0000 7fe36c1bf700 1 -- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe3480682f0 msgr2=0x7fe34806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:26.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.578+0000 7fe36c1bf700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe3480682f0 0x7fe34806a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe36415f8e0 tx=0x7fe360008040 comp rx=0 tx=0).stop 2023-11-28T12:02:26.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.578+0000 7fe36c1bf700 1 -- 172.21.15.57:0/2532889713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe364071890 msgr2=0x7fe3640cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:26.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.578+0000 7fe36c1bf700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe364071890 0x7fe3640cab60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe35400c4e0 tx=0x7fe35400c8a0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.578+0000 7fe36c1bf700 1 -- 172.21.15.57:0/2532889713 shutdown_connections 2023-11-28T12:02:26.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.578+0000 7fe36c1bf700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe3480682f0 0x7fe34806a7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.578+0000 7fe36c1bf700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3640cf710 0x7fe36415a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.578+0000 7fe36c1bf700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe36406a980 0x7fe3641f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.578+0000 7fe36c1bf700 1 --2- 172.21.15.57:0/2532889713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe364071890 0x7fe3640cab60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.578+0000 7fe36c1bf700 1 -- 172.21.15.57:0/2532889713 >> 172.21.15.57:0/2532889713 conn(0x7fe36406c010 msgr2=0x7fe3640ad780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:26.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.578+0000 7fe36c1bf700 1 -- 172.21.15.57:0/2532889713 shutdown_connections 2023-11-28T12:02:26.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.578+0000 7fe36c1bf700 1 -- 172.21.15.57:0/2532889713 wait complete. 2023-11-28T12:02:26.580 INFO:teuthology.orchestra.run.smithi057.stderr:listed 0 entries 2023-11-28T12:02:26.593 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2023-11-28T12:02:26.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:26.593 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-28T12:02:26.634 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:02:26.635 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:02:26.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:26.635 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T12:02:26.710 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:02:26.711 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:02:26.733 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:02:26.733 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:02:26.798 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:26.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.980+0000 7f1466e82700 1 Processor -- start 2023-11-28T12:02:26.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.980+0000 7f1466e82700 1 -- start start 2023-11-28T12:02:26.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.981+0000 7f1466e82700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f146006e400 0x7f146006e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:26.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.981+0000 7f1466e82700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14600d15f0 0x7f146015b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:26.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.981+0000 7f1466e82700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f146015e010 0x7f14600ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:26.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.981+0000 7f1466e82700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146015a9d0 con 0x7f146006e400 2023-11-28T12:02:26.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.981+0000 7f1466e82700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14601584e0 con 0x7f146015e010 2023-11-28T12:02:26.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.981+0000 7f1466e82700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146015af30 con 0x7f14600d15f0 2023-11-28T12:02:26.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.981+0000 7f1464c1e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f146006e400 0x7f146006e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:26.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.981+0000 7f1464c1e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f146006e400 0x7f146006e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53604/0 (socket says 172.21.15.57:53604) 2023-11-28T12:02:26.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.981+0000 7f1464c1e700 1 -- 172.21.15.57:0/650215687 learned_addr learned my addr 172.21.15.57:0/650215687 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:26.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.982+0000 7f146541f700 1 --2- 172.21.15.57:0/650215687 >> v2:172.21.15.181:3300/0 conn(0x7f146015e010 0x7f14600ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:26.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.982+0000 7f145ffff700 1 --2- 172.21.15.57:0/650215687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14600d15f0 0x7f146015b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:26.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.982+0000 7f1464c1e700 1 -- 172.21.15.57:0/650215687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14600d15f0 msgr2=0x7f146015b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:26.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.982+0000 7f1464c1e700 1 --2- 172.21.15.57:0/650215687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14600d15f0 0x7f146015b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.982+0000 7f1464c1e700 1 -- 172.21.15.57:0/650215687 >> v2:172.21.15.181:3300/0 conn(0x7f146015e010 msgr2=0x7f14600ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:26.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.982+0000 7f1464c1e700 1 --2- 172.21.15.57:0/650215687 >> v2:172.21.15.181:3300/0 conn(0x7f146015e010 0x7f14600ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.982+0000 7f1464c1e700 1 -- 172.21.15.57:0/650215687 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146015ef40 con 0x7f146006e400 2023-11-28T12:02:26.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.982+0000 7f145ffff700 1 --2- 172.21.15.57:0/650215687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14600d15f0 0x7f146015b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:26.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.982+0000 7f146541f700 1 --2- 172.21.15.57:0/650215687 >> v2:172.21.15.181:3300/0 conn(0x7f146015e010 0x7f14600ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:26.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.982+0000 7f1464c1e700 1 --2- 172.21.15.57:0/650215687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f146006e400 0x7f146006e7d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f14600cd8b0 tx=0x7f1454009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40db4764f5237c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:26.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.982+0000 7f145f7fe700 1 -- 172.21.15.57:0/650215687 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145400b870 con 0x7f146006e400 2023-11-28T12:02:26.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.982+0000 7f145f7fe700 1 -- 172.21.15.57:0/650215687 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145400b9d0 con 0x7f146006e400 2023-11-28T12:02:26.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.982+0000 7f145f7fe700 1 -- 172.21.15.57:0/650215687 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145400bd60 con 0x7f146006e400 2023-11-28T12:02:26.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.983+0000 7f1466e82700 1 -- 172.21.15.57:0/650215687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f146006e400 msgr2=0x7f146006e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:26.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.983+0000 7f1466e82700 1 --2- 172.21.15.57:0/650215687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f146006e400 0x7f146006e7d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f14600cd8b0 tx=0x7f1454009bd0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.983+0000 7f1466e82700 1 -- 172.21.15.57:0/650215687 shutdown_connections 2023-11-28T12:02:26.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.983+0000 7f1466e82700 1 --2- 172.21.15.57:0/650215687 >> v2:172.21.15.181:3300/0 conn(0x7f146015e010 0x7f14600ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.983+0000 7f1466e82700 1 --2- 172.21.15.57:0/650215687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f146006e400 0x7f146006e7d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.983+0000 7f1466e82700 1 --2- 172.21.15.57:0/650215687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14600d15f0 0x7f146015b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.983+0000 7f1466e82700 1 -- 172.21.15.57:0/650215687 >> 172.21.15.57:0/650215687 conn(0x7f146006a270 msgr2=0x7f146006a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:26.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.983+0000 7f1466e82700 1 -- 172.21.15.57:0/650215687 shutdown_connections 2023-11-28T12:02:26.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.983+0000 7f1466e82700 1 -- 172.21.15.57:0/650215687 wait complete. 2023-11-28T12:02:26.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.984+0000 7f1466e82700 1 Processor -- start 2023-11-28T12:02:26.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.984+0000 7f1466e82700 1 -- start start 2023-11-28T12:02:26.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1466e82700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f146006e400 0x7f146015c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:26.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1466e82700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14600d15f0 0x7f146006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:26.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1466e82700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f146015e010 0x7f14600b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:26.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1466e82700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146014c3f0 con 0x7f146006e400 2023-11-28T12:02:26.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1466e82700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146006bbf0 con 0x7f146015e010 2023-11-28T12:02:26.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1466e82700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146006bed0 con 0x7f14600d15f0 2023-11-28T12:02:26.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1464c1e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f146006e400 0x7f146015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:26.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1464c1e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f146006e400 0x7f146015c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53616/0 (socket says 172.21.15.57:53616) 2023-11-28T12:02:26.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1464c1e700 1 -- 172.21.15.57:0/345077511 learned_addr learned my addr 172.21.15.57:0/345077511 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:26.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1464c1e700 1 -- 172.21.15.57:0/345077511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14600d15f0 msgr2=0x7f146006cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:26.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1464c1e700 1 --2- 172.21.15.57:0/345077511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14600d15f0 0x7f146006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1464c1e700 1 -- 172.21.15.57:0/345077511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f146015e010 msgr2=0x7f14600b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:26.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1464c1e700 1 --2- 172.21.15.57:0/345077511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f146015e010 0x7f14600b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:26.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.985+0000 7f1464c1e700 1 -- 172.21.15.57:0/345077511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1454009550 con 0x7f146006e400 2023-11-28T12:02:26.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.986+0000 7f1464c1e700 1 --2- 172.21.15.57:0/345077511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f146006e400 0x7f146015c910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f14600b6eb0 tx=0x7f1454004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:26.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.986+0000 7f145d7fa700 1 -- 172.21.15.57:0/345077511 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454004310 con 0x7f146006e400 2023-11-28T12:02:26.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.986+0000 7f145d7fa700 1 -- 172.21.15.57:0/345077511 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145401f070 con 0x7f146006e400 2023-11-28T12:02:26.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.986+0000 7f1466e82700 1 -- 172.21.15.57:0/345077511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146015e980 con 0x7f146006e400 2023-11-28T12:02:26.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.986+0000 7f145d7fa700 1 -- 172.21.15.57:0/345077511 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540036a0 con 0x7f146006e400 2023-11-28T12:02:26.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.986+0000 7f1466e82700 1 -- 172.21.15.57:0/345077511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146006ed10 con 0x7f146006e400 2023-11-28T12:02:26.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.987+0000 7f1466e82700 1 -- 172.21.15.57:0/345077511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146015ec60 con 0x7f146006e400 2023-11-28T12:02:26.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.987+0000 7f145d7fa700 1 -- 172.21.15.57:0/345077511 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f145402a020 con 0x7f146006e400 2023-11-28T12:02:26.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.988+0000 7f145d7fa700 1 --2- 172.21.15.57:0/345077511 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f14440682f0 0x7f144406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:26.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.988+0000 7f145d7fa700 1 -- 172.21.15.57:0/345077511 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1454087320 con 0x7f146006e400 2023-11-28T12:02:26.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.988+0000 7f145ffff700 1 --2- 172.21.15.57:0/345077511 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f14440682f0 0x7f144406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:26.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.989+0000 7f145ffff700 1 --2- 172.21.15.57:0/345077511 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f14440682f0 0x7f144406a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f144c006fd0 tx=0x7f144c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:26.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:26.992+0000 7f145d7fa700 1 -- 172.21.15.57:0/345077511 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1454053b00 con 0x7f146006e400 2023-11-28T12:02:27.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.192+0000 7f1466e82700 1 -- 172.21.15.57:0/345077511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14600743f0 con 0x7f146006e400 2023-11-28T12:02:27.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.194+0000 7f145d7fa700 1 -- 172.21.15.57:0/345077511 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7f1454057120 con 0x7f146006e400 2023-11-28T12:02:27.195 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:27.195 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.196+0000 7f1466e82700 1 -- 172.21.15.57:0/345077511 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f14440682f0 msgr2=0x7f144406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.196+0000 7f1466e82700 1 --2- 172.21.15.57:0/345077511 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f14440682f0 0x7f144406a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f144c006fd0 tx=0x7f144c008040 comp rx=0 tx=0).stop 2023-11-28T12:02:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.196+0000 7f1466e82700 1 -- 172.21.15.57:0/345077511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f146006e400 msgr2=0x7f146015c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:27.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.196+0000 7f1466e82700 1 --2- 172.21.15.57:0/345077511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f146006e400 0x7f146015c910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f14600b6eb0 tx=0x7f1454004750 comp rx=0 tx=0).stop 2023-11-28T12:02:27.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.196+0000 7f1466e82700 1 -- 172.21.15.57:0/345077511 shutdown_connections 2023-11-28T12:02:27.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.196+0000 7f1466e82700 1 --2- 172.21.15.57:0/345077511 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f14440682f0 0x7f144406a7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.196+0000 7f1466e82700 1 --2- 172.21.15.57:0/345077511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f146015e010 0x7f14600b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.196+0000 7f1466e82700 1 --2- 172.21.15.57:0/345077511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f146006e400 0x7f146015c910 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.196+0000 7f1466e82700 1 --2- 172.21.15.57:0/345077511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14600d15f0 0x7f146006cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.196+0000 7f1466e82700 1 -- 172.21.15.57:0/345077511 >> 172.21.15.57:0/345077511 conn(0x7f146006a270 msgr2=0x7f14601568a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:27.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.196+0000 7f1466e82700 1 -- 172.21.15.57:0/345077511 shutdown_connections 2023-11-28T12:02:27.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.196+0000 7f1466e82700 1 -- 172.21.15.57:0/345077511 wait complete. 2023-11-28T12:02:27.211 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:27.212 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:27.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.350+0000 7f8f87eb7700 1 Processor -- start 2023-11-28T12:02:27.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.350+0000 7f8f87eb7700 1 -- start start 2023-11-28T12:02:27.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.351+0000 7f8f87eb7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8f8015a5f0 0x7f8f800d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:27.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.351+0000 7f8f87eb7700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8f800cc200 0x7f8f800b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:27.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.351+0000 7f8f87eb7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f8015a9c0 con 0x7f8f800b2d10 2023-11-28T12:02:27.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.351+0000 7f8f87eb7700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f800b3190 con 0x7f8f800cc200 2023-11-28T12:02:27.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.351+0000 7f8f87eb7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f800c7b80 con 0x7f8f8015a5f0 2023-11-28T12:02:27.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.351+0000 7f8f85452700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8f800cc200 0x7f8f800b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:27.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.351+0000 7f8f85452700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8f800cc200 0x7f8f800b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53632/0 (socket says 172.21.15.57:53632) 2023-11-28T12:02:27.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.351+0000 7f8f85452700 1 -- 172.21.15.57:0/655076067 learned_addr learned my addr 172.21.15.57:0/655076067 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:27.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.351+0000 7f8f85c53700 1 --2- 172.21.15.57:0/655076067 >> v2:172.21.15.181:3300/0 conn(0x7f8f8015a5f0 0x7f8f800d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:27.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.352+0000 7f8f85452700 1 -- 172.21.15.57:0/655076067 >> v2:172.21.15.181:3300/0 conn(0x7f8f8015a5f0 msgr2=0x7f8f800d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:27.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.352+0000 7f8f85452700 1 --2- 172.21.15.57:0/655076067 >> v2:172.21.15.181:3300/0 conn(0x7f8f8015a5f0 0x7f8f800d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.352+0000 7f8f85452700 1 -- 172.21.15.57:0/655076067 >> v1:172.21.15.181:6789/0 conn(0x7f8f800b2d10 legacy=0x7f8f8015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:27.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.352+0000 7f8f85452700 1 -- 172.21.15.57:0/655076067 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800cc990 con 0x7f8f800cc200 2023-11-28T12:02:27.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.352+0000 7f8f85452700 1 --2- 172.21.15.57:0/655076067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f800cc200 0x7f8f800b27d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f800d1d40 tx=0x7f8f70009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0efceb31c4e0a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:27.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.352+0000 7f8f84c51700 1 -- 172.21.15.57:0/655076067 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7000b870 con 0x7f8f800cc200 2023-11-28T12:02:27.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.352+0000 7f8f84c51700 1 -- 172.21.15.57:0/655076067 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7000b9d0 con 0x7f8f800cc200 2023-11-28T12:02:27.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.352+0000 7f8f84c51700 1 -- 172.21.15.57:0/655076067 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7000bd60 con 0x7f8f800cc200 2023-11-28T12:02:27.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.352+0000 7f8f87eb7700 1 -- 172.21.15.57:0/655076067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f800cc200 msgr2=0x7f8f800b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:27.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.352+0000 7f8f87eb7700 1 --2- 172.21.15.57:0/655076067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f800cc200 0x7f8f800b27d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f800d1d40 tx=0x7f8f70009bd0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.353+0000 7f8f87eb7700 1 -- 172.21.15.57:0/655076067 shutdown_connections 2023-11-28T12:02:27.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.353+0000 7f8f87eb7700 1 --2- 172.21.15.57:0/655076067 >> v2:172.21.15.181:3300/0 conn(0x7f8f8015a5f0 0x7f8f800d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.353+0000 7f8f87eb7700 1 --2- 172.21.15.57:0/655076067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f800cc200 0x7f8f800b27d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.353+0000 7f8f87eb7700 1 -- 172.21.15.57:0/655076067 >> 172.21.15.57:0/655076067 conn(0x7f8f8006ea80 msgr2=0x7f8f8006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:27.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.353+0000 7f8f87eb7700 1 -- 172.21.15.57:0/655076067 shutdown_connections 2023-11-28T12:02:27.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.353+0000 7f8f87eb7700 1 -- 172.21.15.57:0/655076067 wait complete. 2023-11-28T12:02:27.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.354+0000 7f8f87eb7700 1 Processor -- start 2023-11-28T12:02:27.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.354+0000 7f8f87eb7700 1 -- start start 2023-11-28T12:02:27.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f87eb7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f800b2d10 0x7f8f80153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:27.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f87eb7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f800cc200 0x7f8f800ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:27.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f87eb7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f8015a5f0 0x7f8f800c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:27.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f87eb7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f80154470 con 0x7f8f8015a5f0 2023-11-28T12:02:27.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f87eb7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f801e51a0 con 0x7f8f800cc200 2023-11-28T12:02:27.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f87eb7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f801e5480 con 0x7f8f800b2d10 2023-11-28T12:02:27.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f86454700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f8015a5f0 0x7f8f800c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:27.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f86454700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f8015a5f0 0x7f8f800c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53638/0 (socket says 172.21.15.57:53638) 2023-11-28T12:02:27.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f86454700 1 -- 172.21.15.57:0/239953281 learned_addr learned my addr 172.21.15.57:0/239953281 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:27.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f85452700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f800cc200 0x7f8f800ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:27.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f85c53700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f800b2d10 0x7f8f80153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:27.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f86454700 1 -- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f800b2d10 msgr2=0x7f8f80153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:27.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f86454700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f800b2d10 0x7f8f80153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f86454700 1 -- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f800cc200 msgr2=0x7f8f800ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:27.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f86454700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f800cc200 0x7f8f800ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f86454700 1 -- 172.21.15.57:0/239953281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f70009550 con 0x7f8f8015a5f0 2023-11-28T12:02:27.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.355+0000 7f8f86454700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f8015a5f0 0x7f8f800c8e70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800cb50 tx=0x7f8f7800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:27.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.356+0000 7f8f767fc700 1 -- 172.21.15.57:0/239953281 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78003c00 con 0x7f8f8015a5f0 2023-11-28T12:02:27.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.356+0000 7f8f767fc700 1 -- 172.21.15.57:0/239953281 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f78003d60 con 0x7f8f8015a5f0 2023-11-28T12:02:27.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.356+0000 7f8f87eb7700 1 -- 172.21.15.57:0/239953281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f800ccf50 con 0x7f8f8015a5f0 2023-11-28T12:02:27.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.356+0000 7f8f767fc700 1 -- 172.21.15.57:0/239953281 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7800aa20 con 0x7f8f8015a5f0 2023-11-28T12:02:27.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.356+0000 7f8f87eb7700 1 -- 172.21.15.57:0/239953281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f800ccc70 con 0x7f8f8015a5f0 2023-11-28T12:02:27.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.357+0000 7f8f87eb7700 1 -- 172.21.15.57:0/239953281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f800cd230 con 0x7f8f8015a5f0 2023-11-28T12:02:27.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.357+0000 7f8f767fc700 1 -- 172.21.15.57:0/239953281 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8f78010050 con 0x7f8f8015a5f0 2023-11-28T12:02:27.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.358+0000 7f8f767fc700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8f640682f0 0x7f8f6406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:27.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.358+0000 7f8f767fc700 1 -- 172.21.15.57:0/239953281 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8f78085b30 con 0x7f8f8015a5f0 2023-11-28T12:02:27.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.358+0000 7f8f85c53700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8f640682f0 0x7f8f6406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:27.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.358+0000 7f8f85c53700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8f640682f0 0x7f8f6406a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8f800c7800 tx=0x7f8f7c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:27.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.362+0000 7f8f767fc700 1 -- 172.21.15.57:0/239953281 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8f78052390 con 0x7f8f8015a5f0 2023-11-28T12:02:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.560+0000 7f8f87eb7700 1 -- 172.21.15.57:0/239953281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8f800743f0 con 0x7f8f8015a5f0 2023-11-28T12:02:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.560+0000 7f8f767fc700 1 -- 172.21.15.57:0/239953281 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8f780559b0 con 0x7f8f8015a5f0 2023-11-28T12:02:27.561 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:27.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.562+0000 7f8f87eb7700 1 -- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8f640682f0 msgr2=0x7f8f6406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.562+0000 7f8f87eb7700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8f640682f0 0x7f8f6406a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8f800c7800 tx=0x7f8f7c008040 comp rx=0 tx=0).stop 2023-11-28T12:02:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.562+0000 7f8f87eb7700 1 -- 172.21.15.57:0/239953281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f8015a5f0 msgr2=0x7f8f800c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.562+0000 7f8f87eb7700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f8015a5f0 0x7f8f800c8e70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800cb50 tx=0x7f8f7800ce60 comp rx=0 tx=0).stop 2023-11-28T12:02:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.562+0000 7f8f87eb7700 1 -- 172.21.15.57:0/239953281 shutdown_connections 2023-11-28T12:02:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.562+0000 7f8f87eb7700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8f640682f0 0x7f8f6406a7a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.562+0000 7f8f87eb7700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f8015a5f0 0x7f8f800c8e70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.562+0000 7f8f87eb7700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f800b2d10 0x7f8f80153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.562+0000 7f8f87eb7700 1 --2- 172.21.15.57:0/239953281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f800cc200 0x7f8f800ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.562+0000 7f8f87eb7700 1 -- 172.21.15.57:0/239953281 >> 172.21.15.57:0/239953281 conn(0x7f8f8006ea80 msgr2=0x7f8f800b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.562+0000 7f8f87eb7700 1 -- 172.21.15.57:0/239953281 shutdown_connections 2023-11-28T12:02:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.562+0000 7f8f87eb7700 1 -- 172.21.15.57:0/239953281 wait complete. 2023-11-28T12:02:27.577 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:27.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.714+0000 7fd485341700 1 Processor -- start 2023-11-28T12:02:27.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.714+0000 7fd485341700 1 -- start start 2023-11-28T12:02:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.715+0000 7fd485341700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd48015c8c0 0x7fd480160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.715+0000 7fd485341700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd4800b5dc0 0x7fd48006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.715+0000 7fd485341700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd48006b050 0x7fd4800d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.715+0000 7fd485341700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48006b4d0 con 0x7fd48015c8c0 2023-11-28T12:02:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.715+0000 7fd485341700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4800d2d30 con 0x7fd4800b5dc0 2023-11-28T12:02:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.715+0000 7fd485341700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800d0b40 con 0x7fd48006b050 2023-11-28T12:02:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.715+0000 7fd47effd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd48015c8c0 0x7fd480160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.715+0000 7fd47effd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd48015c8c0 0x7fd480160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53640/0 (socket says 172.21.15.57:53640) 2023-11-28T12:02:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.715+0000 7fd47effd700 1 -- 172.21.15.57:0/1309653660 learned_addr learned my addr 172.21.15.57:0/1309653660 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.715+0000 7fd47f7fe700 1 --2- 172.21.15.57:0/1309653660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd48006b050 0x7fd4800d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.716+0000 7fd47e7fc700 1 --2- 172.21.15.57:0/1309653660 >> v2:172.21.15.181:3300/0 conn(0x7fd4800b5dc0 0x7fd48006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:27.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.716+0000 7fd47effd700 1 -- 172.21.15.57:0/1309653660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd48006b050 msgr2=0x7fd4800d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:27.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.716+0000 7fd47effd700 1 --2- 172.21.15.57:0/1309653660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd48006b050 0x7fd4800d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.716+0000 7fd47effd700 1 -- 172.21.15.57:0/1309653660 >> v2:172.21.15.181:3300/0 conn(0x7fd4800b5dc0 msgr2=0x7fd48006ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:27.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.716+0000 7fd47effd700 1 --2- 172.21.15.57:0/1309653660 >> v2:172.21.15.181:3300/0 conn(0x7fd4800b5dc0 0x7fd48006ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.716+0000 7fd47effd700 1 -- 172.21.15.57:0/1309653660 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4801611f0 con 0x7fd48015c8c0 2023-11-28T12:02:27.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.716+0000 7fd47effd700 1 --2- 172.21.15.57:0/1309653660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48015c8c0 0x7fd480160cb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd4800d24c0 tx=0x7fd468009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e438a38df414ad48 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:27.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.716+0000 7fd47dffb700 1 -- 172.21.15.57:0/1309653660 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46800b870 con 0x7fd48015c8c0 2023-11-28T12:02:27.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.716+0000 7fd47dffb700 1 -- 172.21.15.57:0/1309653660 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46800b9d0 con 0x7fd48015c8c0 2023-11-28T12:02:27.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.716+0000 7fd47dffb700 1 -- 172.21.15.57:0/1309653660 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46800bd60 con 0x7fd48015c8c0 2023-11-28T12:02:27.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.717+0000 7fd485341700 1 -- 172.21.15.57:0/1309653660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48015c8c0 msgr2=0x7fd480160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:27.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.717+0000 7fd485341700 1 --2- 172.21.15.57:0/1309653660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48015c8c0 0x7fd480160cb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd4800d24c0 tx=0x7fd468009bd0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.717+0000 7fd485341700 1 -- 172.21.15.57:0/1309653660 shutdown_connections 2023-11-28T12:02:27.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.717+0000 7fd485341700 1 --2- 172.21.15.57:0/1309653660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd48006b050 0x7fd4800d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.717+0000 7fd485341700 1 --2- 172.21.15.57:0/1309653660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48015c8c0 0x7fd480160cb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.717+0000 7fd485341700 1 --2- 172.21.15.57:0/1309653660 >> v2:172.21.15.181:3300/0 conn(0x7fd4800b5dc0 0x7fd48006ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.717+0000 7fd485341700 1 -- 172.21.15.57:0/1309653660 >> 172.21.15.57:0/1309653660 conn(0x7fd48006c520 msgr2=0x7fd48006c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:27.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.717+0000 7fd485341700 1 -- 172.21.15.57:0/1309653660 shutdown_connections 2023-11-28T12:02:27.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.717+0000 7fd485341700 1 -- 172.21.15.57:0/1309653660 wait complete. 2023-11-28T12:02:27.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.718+0000 7fd485341700 1 Processor -- start 2023-11-28T12:02:27.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.718+0000 7fd485341700 1 -- start start 2023-11-28T12:02:27.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd485341700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd48006b050 0x7fd480155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:27.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd485341700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4800b5dc0 0x7fd48015f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:27.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd485341700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48015c8c0 0x7fd480157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:27.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd485341700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48015fc80 con 0x7fd48015c8c0 2023-11-28T12:02:27.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd485341700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48015d3b0 con 0x7fd48006b050 2023-11-28T12:02:27.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd485341700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800cb610 con 0x7fd4800b5dc0 2023-11-28T12:02:27.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd47f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48015c8c0 0x7fd480157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:27.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd47f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48015c8c0 0x7fd480157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53644/0 (socket says 172.21.15.57:53644) 2023-11-28T12:02:27.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd47f7fe700 1 -- 172.21.15.57:0/2993859928 learned_addr learned my addr 172.21.15.57:0/2993859928 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:27.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd47e7fc700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4800b5dc0 0x7fd48015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:27.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd47effd700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd48006b050 0x7fd480155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:27.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd47f7fe700 1 -- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4800b5dc0 msgr2=0x7fd48015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:27.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd47f7fe700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4800b5dc0 0x7fd48015f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd47f7fe700 1 -- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd48006b050 msgr2=0x7fd480155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:27.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd47f7fe700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd48006b050 0x7fd480155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.719+0000 7fd47f7fe700 1 -- 172.21.15.57:0/2993859928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd468009550 con 0x7fd48015c8c0 2023-11-28T12:02:27.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.720+0000 7fd47f7fe700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48015c8c0 0x7fd480157070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd478007e60 tx=0x7fd47800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:27.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.720+0000 7fd4777fe700 1 -- 172.21.15.57:0/2993859928 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47800ed30 con 0x7fd48015c8c0 2023-11-28T12:02:27.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.720+0000 7fd4777fe700 1 -- 172.21.15.57:0/2993859928 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd478014420 con 0x7fd48015c8c0 2023-11-28T12:02:27.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.720+0000 7fd485341700 1 -- 172.21.15.57:0/2993859928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4800cd760 con 0x7fd48015c8c0 2023-11-28T12:02:27.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.720+0000 7fd4777fe700 1 -- 172.21.15.57:0/2993859928 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780147d0 con 0x7fd48015c8c0 2023-11-28T12:02:27.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.720+0000 7fd485341700 1 -- 172.21.15.57:0/2993859928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4800d2a50 con 0x7fd48015c8c0 2023-11-28T12:02:27.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.721+0000 7fd485341700 1 -- 172.21.15.57:0/2993859928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4800cda40 con 0x7fd48015c8c0 2023-11-28T12:02:27.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.721+0000 7fd4777fe700 1 -- 172.21.15.57:0/2993859928 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd478012050 con 0x7fd48015c8c0 2023-11-28T12:02:27.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.722+0000 7fd4777fe700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd4600682f0 0x7fd46006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:27.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.722+0000 7fd4777fe700 1 -- 172.21.15.57:0/2993859928 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd478086110 con 0x7fd48015c8c0 2023-11-28T12:02:27.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.722+0000 7fd47effd700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd4600682f0 0x7fd46006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:27.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.723+0000 7fd47effd700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd4600682f0 0x7fd46006a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd468014040 tx=0x7fd468008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:27.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.727+0000 7fd4777fe700 1 -- 172.21.15.57:0/2993859928 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd478052970 con 0x7fd48015c8c0 2023-11-28T12:02:27.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.929+0000 7fd485341700 1 -- 172.21.15.57:0/2993859928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd480067d70 con 0x7fd48015c8c0 2023-11-28T12:02:27.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.929+0000 7fd4777fe700 1 -- 172.21.15.57:0/2993859928 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd478055f90 con 0x7fd48015c8c0 2023-11-28T12:02:27.931 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:27.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.932+0000 7fd485341700 1 -- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd4600682f0 msgr2=0x7fd46006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:27.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.932+0000 7fd485341700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd4600682f0 0x7fd46006a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd468014040 tx=0x7fd468008030 comp rx=0 tx=0).stop 2023-11-28T12:02:27.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.932+0000 7fd485341700 1 -- 172.21.15.57:0/2993859928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48015c8c0 msgr2=0x7fd480157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:27.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.932+0000 7fd485341700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48015c8c0 0x7fd480157070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd478007e60 tx=0x7fd47800e3c0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.932+0000 7fd485341700 1 -- 172.21.15.57:0/2993859928 shutdown_connections 2023-11-28T12:02:27.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.932+0000 7fd485341700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd4600682f0 0x7fd46006a7a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.932+0000 7fd485341700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48015c8c0 0x7fd480157070 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.932+0000 7fd485341700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd48006b050 0x7fd480155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.932+0000 7fd485341700 1 --2- 172.21.15.57:0/2993859928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4800b5dc0 0x7fd48015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:27.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.932+0000 7fd485341700 1 -- 172.21.15.57:0/2993859928 >> 172.21.15.57:0/2993859928 conn(0x7fd48006c520 msgr2=0x7fd4800b8930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:27.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.932+0000 7fd485341700 1 -- 172.21.15.57:0/2993859928 shutdown_connections 2023-11-28T12:02:27.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:27.932+0000 7fd485341700 1 -- 172.21.15.57:0/2993859928 wait complete. 2023-11-28T12:02:27.946 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:02:28.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.084+0000 7fb7c711a700 1 Processor -- start 2023-11-28T12:02:28.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.084+0000 7fb7c711a700 1 -- start start 2023-11-28T12:02:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.085+0000 7fb7c711a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb7c00cf710 0x7fb7c00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.085+0000 7fb7c711a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c006a980 0x7fb7c0071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.085+0000 7fb7c711a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c015cc10 con 0x7fb7c0071890 2023-11-28T12:02:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.085+0000 7fb7c711a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c006c9b0 con 0x7fb7c00cf710 2023-11-28T12:02:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.085+0000 7fb7c711a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c006cc90 con 0x7fb7c006a980 2023-11-28T12:02:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.085+0000 7fb7c4eb6700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb7c00cf710 0x7fb7c00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.085+0000 7fb7bffff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fb7c0071890 0x7fb7c006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40230/0 (socket says 172.21.15.57:40230) 2023-11-28T12:02:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.085+0000 7fb7bffff700 1 -- 172.21.15.57:0/782255372 learned_addr learned my addr 172.21.15.57:0/782255372 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:28.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.086+0000 7fb7c56b7700 1 --2- 172.21.15.57:0/782255372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c006a980 0x7fb7c0071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.086+0000 7fb7c4eb6700 1 -- 172.21.15.57:0/782255372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c006a980 msgr2=0x7fb7c0071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:28.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.086+0000 7fb7c4eb6700 1 --2- 172.21.15.57:0/782255372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c006a980 0x7fb7c0071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.086+0000 7fb7bf7fe700 1 -- 172.21.15.57:0/782255372 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679847599 0 0) 0x7fb7c015cc10 con 0x7fb7c0071890 2023-11-28T12:02:28.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.086+0000 7fb7c4eb6700 1 -- 172.21.15.57:0/782255372 >> v1:172.21.15.57:6789/0 conn(0x7fb7c0071890 legacy=0x7fb7c006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.086+0000 7fb7c4eb6700 1 -- 172.21.15.57:0/782255372 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c006cf70 con 0x7fb7c00cf710 2023-11-28T12:02:28.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.086+0000 7fb7c4eb6700 1 --2- 172.21.15.57:0/782255372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c00cf710 0x7fb7c00ce740 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009870 tx=0x7fb7b0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58c822341383e1e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:28.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.086+0000 7fb7bf7fe700 1 -- 172.21.15.57:0/782255372 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0011830 con 0x7fb7c00cf710 2023-11-28T12:02:28.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.086+0000 7fb7bf7fe700 1 -- 172.21.15.57:0/782255372 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b0011990 con 0x7fb7c00cf710 2023-11-28T12:02:28.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.086+0000 7fb7bf7fe700 1 -- 172.21.15.57:0/782255372 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0011d20 con 0x7fb7c00cf710 2023-11-28T12:02:28.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.087+0000 7fb7c711a700 1 -- 172.21.15.57:0/782255372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c00cf710 msgr2=0x7fb7c00ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.087+0000 7fb7c711a700 1 --2- 172.21.15.57:0/782255372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c00cf710 0x7fb7c00ce740 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009870 tx=0x7fb7b0009b80 comp rx=0 tx=0).stop 2023-11-28T12:02:28.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.087+0000 7fb7c711a700 1 -- 172.21.15.57:0/782255372 shutdown_connections 2023-11-28T12:02:28.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.087+0000 7fb7c711a700 1 --2- 172.21.15.57:0/782255372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c00cf710 0x7fb7c00ce740 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.087+0000 7fb7c711a700 1 --2- 172.21.15.57:0/782255372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c006a980 0x7fb7c0071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.087+0000 7fb7c711a700 1 -- 172.21.15.57:0/782255372 >> 172.21.15.57:0/782255372 conn(0x7fb7c006c010 msgr2=0x7fb7c00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:28.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.087+0000 7fb7c711a700 1 -- 172.21.15.57:0/782255372 shutdown_connections 2023-11-28T12:02:28.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.087+0000 7fb7c711a700 1 -- 172.21.15.57:0/782255372 wait complete. 2023-11-28T12:02:28.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.088+0000 7fb7c711a700 1 Processor -- start 2023-11-28T12:02:28.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.088+0000 7fb7c711a700 1 -- start start 2023-11-28T12:02:28.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.089+0000 7fb7c711a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c006a980 0x7fb7c00b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.089+0000 7fb7c711a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c0071890 0x7fb7c00c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.089+0000 7fb7c711a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c00cf710 0x7fb7c014b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.089+0000 7fb7c711a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c00b08b0 con 0x7fb7c00cf710 2023-11-28T12:02:28.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.089+0000 7fb7c711a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c00b0b90 con 0x7fb7c006a980 2023-11-28T12:02:28.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.089+0000 7fb7c711a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c00b0e70 con 0x7fb7c0071890 2023-11-28T12:02:28.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.089+0000 7fb7c56b7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c00cf710 0x7fb7c014b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.089+0000 7fb7c56b7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c00cf710 0x7fb7c014b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53668/0 (socket says 172.21.15.57:53668) 2023-11-28T12:02:28.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.089+0000 7fb7c56b7700 1 -- 172.21.15.57:0/4205849976 learned_addr learned my addr 172.21.15.57:0/4205849976 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:28.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.089+0000 7fb7bffff700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c0071890 0x7fb7c00c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.089+0000 7fb7c4eb6700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c006a980 0x7fb7c00b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.090+0000 7fb7c56b7700 1 -- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c0071890 msgr2=0x7fb7c00c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.090+0000 7fb7c56b7700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c0071890 0x7fb7c00c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.090+0000 7fb7c56b7700 1 -- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c006a980 msgr2=0x7fb7c00b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.090+0000 7fb7c56b7700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c006a980 0x7fb7c00b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.090+0000 7fb7c56b7700 1 -- 172.21.15.57:0/4205849976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b00094a0 con 0x7fb7c00cf710 2023-11-28T12:02:28.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.090+0000 7fb7c56b7700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c00cf710 0x7fb7c014b950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8007de0 tx=0x7fb7b800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:28.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.090+0000 7fb7bd7fa700 1 -- 172.21.15.57:0/4205849976 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b800ed30 con 0x7fb7c00cf710 2023-11-28T12:02:28.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.090+0000 7fb7bd7fa700 1 -- 172.21.15.57:0/4205849976 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b8014420 con 0x7fb7c00cf710 2023-11-28T12:02:28.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.090+0000 7fb7c711a700 1 -- 172.21.15.57:0/4205849976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c006d530 con 0x7fb7c00cf710 2023-11-28T12:02:28.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.090+0000 7fb7bd7fa700 1 -- 172.21.15.57:0/4205849976 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80147d0 con 0x7fb7c00cf710 2023-11-28T12:02:28.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.090+0000 7fb7c711a700 1 -- 172.21.15.57:0/4205849976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c006d250 con 0x7fb7c00cf710 2023-11-28T12:02:28.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.091+0000 7fb7bd7fa700 1 -- 172.21.15.57:0/4205849976 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb7b8012050 con 0x7fb7c00cf710 2023-11-28T12:02:28.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.092+0000 7fb7c711a700 1 -- 172.21.15.57:0/4205849976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c0160480 con 0x7fb7c00cf710 2023-11-28T12:02:28.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.092+0000 7fb7bd7fa700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7a00682f0 0x7fb7a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.092+0000 7fb7bd7fa700 1 -- 172.21.15.57:0/4205849976 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb7b8086310 con 0x7fb7c00cf710 2023-11-28T12:02:28.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.092+0000 7fb7c4eb6700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7a00682f0 0x7fb7a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.093+0000 7fb7c4eb6700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7a00682f0 0x7fb7a006a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00090b0 tx=0x7fb7b001c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:28.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.096+0000 7fb7bd7fa700 1 -- 172.21.15.57:0/4205849976 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb7b8052b10 con 0x7fb7c00cf710 2023-11-28T12:02:28.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.295+0000 7fb7c711a700 1 -- 172.21.15.57:0/4205849976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7c00743f0 con 0x7fb7c00cf710 2023-11-28T12:02:28.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.296+0000 7fb7bd7fa700 1 -- 172.21.15.57:0/4205849976 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7b8056130 con 0x7fb7c00cf710 2023-11-28T12:02:28.297 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:02:28.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.298+0000 7fb7c711a700 1 -- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7a00682f0 msgr2=0x7fb7a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.298+0000 7fb7c711a700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7a00682f0 0x7fb7a006a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00090b0 tx=0x7fb7b001c040 comp rx=0 tx=0).stop 2023-11-28T12:02:28.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.298+0000 7fb7c711a700 1 -- 172.21.15.57:0/4205849976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c00cf710 msgr2=0x7fb7c014b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.298+0000 7fb7c711a700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c00cf710 0x7fb7c014b950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8007de0 tx=0x7fb7b800e3c0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.298+0000 7fb7c711a700 1 -- 172.21.15.57:0/4205849976 shutdown_connections 2023-11-28T12:02:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.298+0000 7fb7c711a700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7a00682f0 0x7fb7a006a7a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.298+0000 7fb7c711a700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7c00cf710 0x7fb7c014b950 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.298+0000 7fb7c711a700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7c006a980 0x7fb7c00b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.298+0000 7fb7c711a700 1 --2- 172.21.15.57:0/4205849976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7c0071890 0x7fb7c00c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.298+0000 7fb7c711a700 1 -- 172.21.15.57:0/4205849976 >> 172.21.15.57:0/4205849976 conn(0x7fb7c006c010 msgr2=0x7fb7c015ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.298+0000 7fb7c711a700 1 -- 172.21.15.57:0/4205849976 shutdown_connections 2023-11-28T12:02:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.298+0000 7fb7c711a700 1 -- 172.21.15.57:0/4205849976 wait complete. 2023-11-28T12:02:28.313 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:02:28.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.451+0000 7f9bf5260700 1 Processor -- start 2023-11-28T12:02:28.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.451+0000 7f9bf5260700 1 -- start start 2023-11-28T12:02:28.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.452+0000 7f9bf5260700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9bf00d0ad0 0x7f9bf00cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.452+0000 7f9bf5260700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf00cc740 con 0x7f9bf015a5f0 2023-11-28T12:02:28.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.452+0000 7f9bf5260700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf015a9c0 con 0x7f9bf00b2ee0 2023-11-28T12:02:28.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.452+0000 7f9bf5260700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf00c7b80 con 0x7f9bf00d0ad0 2023-11-28T12:02:28.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.452+0000 7f9beed9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f9bf015a5f0 0x7f9bf00b27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40246/0 (socket says 172.21.15.57:40246) 2023-11-28T12:02:28.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.452+0000 7f9beed9d700 1 -- 172.21.15.57:0/3821529413 learned_addr learned my addr 172.21.15.57:0/3821529413 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:28.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.452+0000 7f9bef59e700 1 --2- 172.21.15.57:0/3821529413 >> v2:172.21.15.181:3300/0 conn(0x7f9bf00d0ad0 0x7f9bf00cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.452+0000 7f9bedd9b700 1 -- 172.21.15.57:0/3821529413 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250824956 0 0) 0x7f9bf00cc740 con 0x7f9bf015a5f0 2023-11-28T12:02:28.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.452+0000 7f9bedd9b700 1 -- 172.21.15.57:0/3821529413 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bdc0035b0 con 0x7f9bf015a5f0 2023-11-28T12:02:28.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.452+0000 7f9bedd9b700 1 -- 172.21.15.57:0/3821529413 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16054833 0 0) 0x7f9bf015a9c0 con 0x7f9bf00b2ee0 2023-11-28T12:02:28.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.453+0000 7f9bedd9b700 1 -- 172.21.15.57:0/3821529413 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bf00cc740 con 0x7f9bf00b2ee0 2023-11-28T12:02:28.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.453+0000 7f9bedd9b700 1 -- 172.21.15.57:0/3821529413 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 150348643 0 0) 0x7f9bdc0035b0 con 0x7f9bf015a5f0 2023-11-28T12:02:28.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.453+0000 7f9bef59e700 1 -- 172.21.15.57:0/3821529413 >> v1:172.21.15.181:6789/0 conn(0x7f9bf00b2ee0 legacy=0x7f9bf00ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.453+0000 7f9bef59e700 1 -- 172.21.15.57:0/3821529413 >> v1:172.21.15.57:6789/0 conn(0x7f9bf015a5f0 legacy=0x7f9bf00b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.453+0000 7f9bef59e700 1 -- 172.21.15.57:0/3821529413 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf00cc990 con 0x7f9bf00d0ad0 2023-11-28T12:02:28.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.453+0000 7f9bef59e700 1 --2- 172.21.15.57:0/3821529413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00d0ad0 0x7f9bf00cc200 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009d90 tx=0x7f9be800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb08b6e8262281c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:28.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.453+0000 7f9bedd9b700 1 -- 172.21.15.57:0/3821529413 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be800e070 con 0x7f9bf00d0ad0 2023-11-28T12:02:28.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.453+0000 7f9bedd9b700 1 -- 172.21.15.57:0/3821529413 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8003a20 con 0x7f9bf00d0ad0 2023-11-28T12:02:28.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.453+0000 7f9bedd9b700 1 -- 172.21.15.57:0/3821529413 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0001670 con 0x7f9bf00d0ad0 2023-11-28T12:02:28.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.454+0000 7f9bf5260700 1 -- 172.21.15.57:0/3821529413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00d0ad0 msgr2=0x7f9bf00cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.454+0000 7f9bf5260700 1 --2- 172.21.15.57:0/3821529413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00d0ad0 0x7f9bf00cc200 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009d90 tx=0x7f9be800b330 comp rx=0 tx=0).stop 2023-11-28T12:02:28.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.454+0000 7f9bf5260700 1 -- 172.21.15.57:0/3821529413 shutdown_connections 2023-11-28T12:02:28.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.454+0000 7f9bf5260700 1 --2- 172.21.15.57:0/3821529413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00d0ad0 0x7f9bf00cc200 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.454+0000 7f9bf5260700 1 -- 172.21.15.57:0/3821529413 >> 172.21.15.57:0/3821529413 conn(0x7f9bf006ea80 msgr2=0x7f9bf006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:28.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.454+0000 7f9bf5260700 1 -- 172.21.15.57:0/3821529413 shutdown_connections 2023-11-28T12:02:28.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.454+0000 7f9bf5260700 1 -- 172.21.15.57:0/3821529413 wait complete. 2023-11-28T12:02:28.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.455+0000 7f9bf5260700 1 Processor -- start 2023-11-28T12:02:28.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.455+0000 7f9bf5260700 1 -- start start 2023-11-28T12:02:28.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.456+0000 7f9bf5260700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00b2ee0 0x7f9bf0140830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.456+0000 7f9bf5260700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bf00d0ad0 0x7f9bf00c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.456+0000 7f9bf5260700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bf015a5f0 0x7f9bf00b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.456+0000 7f9bf5260700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0143c90 con 0x7f9bf015a5f0 2023-11-28T12:02:28.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.456+0000 7f9bf5260700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf00b6620 con 0x7f9bf00b2ee0 2023-11-28T12:02:28.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.456+0000 7f9bf5260700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0140d70 con 0x7f9bf00d0ad0 2023-11-28T12:02:28.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.456+0000 7f9bef59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bf015a5f0 0x7f9bf00b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.456+0000 7f9bef59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bf015a5f0 0x7f9bf00b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53678/0 (socket says 172.21.15.57:53678) 2023-11-28T12:02:28.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.456+0000 7f9bef59e700 1 -- 172.21.15.57:0/695758981 learned_addr learned my addr 172.21.15.57:0/695758981 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:28.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.456+0000 7f9beed9d700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00b2ee0 0x7f9bf0140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.456+0000 7f9bee59c700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bf00d0ad0 0x7f9bf00c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.457+0000 7f9bef59e700 1 -- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bf00d0ad0 msgr2=0x7f9bf00c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.457+0000 7f9bef59e700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bf00d0ad0 0x7f9bf00c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.457+0000 7f9bef59e700 1 -- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00b2ee0 msgr2=0x7f9bf0140830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.457+0000 7f9bef59e700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00b2ee0 0x7f9bf0140830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.457+0000 7f9bef59e700 1 -- 172.21.15.57:0/695758981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be8009a70 con 0x7f9bf015a5f0 2023-11-28T12:02:28.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.457+0000 7f9bef59e700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bf015a5f0 0x7f9bf00b1800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9be800b370 tx=0x7f9be8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:28.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.457+0000 7f9be77fe700 1 -- 172.21.15.57:0/695758981 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80045c0 con 0x7f9bf015a5f0 2023-11-28T12:02:28.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.457+0000 7f9be77fe700 1 -- 172.21.15.57:0/695758981 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8004720 con 0x7f9bf015a5f0 2023-11-28T12:02:28.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.457+0000 7f9bf5260700 1 -- 172.21.15.57:0/695758981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf00ccf50 con 0x7f9bf015a5f0 2023-11-28T12:02:28.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.457+0000 7f9be77fe700 1 -- 172.21.15.57:0/695758981 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be801d470 con 0x7f9bf015a5f0 2023-11-28T12:02:28.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.457+0000 7f9bf5260700 1 -- 172.21.15.57:0/695758981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf00ccc70 con 0x7f9bf015a5f0 2023-11-28T12:02:28.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.458+0000 7f9bf5260700 1 -- 172.21.15.57:0/695758981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf00cd230 con 0x7f9bf015a5f0 2023-11-28T12:02:28.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.459+0000 7f9be77fe700 1 -- 172.21.15.57:0/695758981 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9be8018070 con 0x7f9bf015a5f0 2023-11-28T12:02:28.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.459+0000 7f9be77fe700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9bd00682f0 0x7f9bd006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.459+0000 7f9be77fe700 1 -- 172.21.15.57:0/695758981 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9be80873a0 con 0x7f9bf015a5f0 2023-11-28T12:02:28.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.459+0000 7f9beed9d700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9bd00682f0 0x7f9bd006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.460+0000 7f9beed9d700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9bd00682f0 0x7f9bd006a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9be0000fd0 tx=0x7f9be0001c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:28.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.463+0000 7f9be77fe700 1 -- 172.21.15.57:0/695758981 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9be8053c00 con 0x7f9bf015a5f0 2023-11-28T12:02:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.663+0000 7f9bf5260700 1 -- 172.21.15.57:0/695758981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9bf00b5a70 con 0x7f9bf015a5f0 2023-11-28T12:02:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.663+0000 7f9be77fe700 1 -- 172.21.15.57:0/695758981 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9be8057220 con 0x7f9bf015a5f0 2023-11-28T12:02:28.665 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:02:28.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.666+0000 7f9bf5260700 1 -- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9bd00682f0 msgr2=0x7f9bd006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.666+0000 7f9bf5260700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9bd00682f0 0x7f9bd006a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9be0000fd0 tx=0x7f9be0001c00 comp rx=0 tx=0).stop 2023-11-28T12:02:28.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.666+0000 7f9bf5260700 1 -- 172.21.15.57:0/695758981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bf015a5f0 msgr2=0x7f9bf00b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.666+0000 7f9bf5260700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bf015a5f0 0x7f9bf00b1800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9be800b370 tx=0x7f9be8004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.666+0000 7f9bf5260700 1 -- 172.21.15.57:0/695758981 shutdown_connections 2023-11-28T12:02:28.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.666+0000 7f9bf5260700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9bd00682f0 0x7f9bd006a7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.666+0000 7f9bf5260700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bf015a5f0 0x7f9bf00b1800 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.666+0000 7f9bf5260700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00b2ee0 0x7f9bf0140830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.666+0000 7f9bf5260700 1 --2- 172.21.15.57:0/695758981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bf00d0ad0 0x7f9bf00c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.666+0000 7f9bf5260700 1 -- 172.21.15.57:0/695758981 >> 172.21.15.57:0/695758981 conn(0x7f9bf006ea80 msgr2=0x7f9bf0154f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:28.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.666+0000 7f9bf5260700 1 -- 172.21.15.57:0/695758981 shutdown_connections 2023-11-28T12:02:28.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.666+0000 7f9bf5260700 1 -- 172.21.15.57:0/695758981 wait complete. 2023-11-28T12:02:28.681 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T12:02:28.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.815+0000 7f0611e38700 1 Processor -- start 2023-11-28T12:02:28.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.815+0000 7f0611e38700 1 -- start start 2023-11-28T12:02:28.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.816+0000 7f0611e38700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f060c159710 0x7f060c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.816+0000 7f0611e38700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f060c0cee40 0x7f060c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.816+0000 7f0611e38700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060c06a310 con 0x7f060c0c9a50 2023-11-28T12:02:28.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.816+0000 7f0611e38700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060c0cf2b0 con 0x7f060c0cee40 2023-11-28T12:02:28.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.816+0000 7f0611e38700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0ce530 con 0x7f060c159710 2023-11-28T12:02:28.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.816+0000 7f060affd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f060c0cee40 0x7f060c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.816+0000 7f060bfff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f060c0c9a50 0x7f060c06adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40252/0 (socket says 172.21.15.57:40252) 2023-11-28T12:02:28.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.816+0000 7f060bfff700 1 -- 172.21.15.57:0/3025667663 learned_addr learned my addr 172.21.15.57:0/3025667663 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:28.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.816+0000 7f060affd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f060c0cee40 0x7f060c0cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53692/0 (socket says 172.21.15.57:53692) 2023-11-28T12:02:28.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.816+0000 7f060b7fe700 1 --2- 172.21.15.57:0/3025667663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f060c159710 0x7f060c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.817+0000 7f060affd700 1 -- 172.21.15.57:0/3025667663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f060c159710 msgr2=0x7f060c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:28.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.817+0000 7f060affd700 1 --2- 172.21.15.57:0/3025667663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f060c159710 0x7f060c070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.817+0000 7f060affd700 1 -- 172.21.15.57:0/3025667663 >> v1:172.21.15.57:6789/0 conn(0x7f060c0c9a50 legacy=0x7f060c06adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.817+0000 7f060affd700 1 -- 172.21.15.57:0/3025667663 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c0ce7e0 con 0x7f060c0cee40 2023-11-28T12:02:28.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.817+0000 7f060affd700 1 --2- 172.21.15.57:0/3025667663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f060c0cee40 0x7f060c0cdff0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009840 tx=0x7f05f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e72e710199948f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:28.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.817+0000 7f060a7fc700 1 -- 172.21.15.57:0/3025667663 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4011830 con 0x7f060c0cee40 2023-11-28T12:02:28.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.817+0000 7f060a7fc700 1 -- 172.21.15.57:0/3025667663 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f4011990 con 0x7f060c0cee40 2023-11-28T12:02:28.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.817+0000 7f060a7fc700 1 -- 172.21.15.57:0/3025667663 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4011d20 con 0x7f060c0cee40 2023-11-28T12:02:28.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.818+0000 7f0611e38700 1 -- 172.21.15.57:0/3025667663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f060c0cee40 msgr2=0x7f060c0cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.818+0000 7f0611e38700 1 --2- 172.21.15.57:0/3025667663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f060c0cee40 0x7f060c0cdff0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009840 tx=0x7f05f4009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:28.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.818+0000 7f0611e38700 1 -- 172.21.15.57:0/3025667663 shutdown_connections 2023-11-28T12:02:28.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.818+0000 7f0611e38700 1 --2- 172.21.15.57:0/3025667663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f060c159710 0x7f060c070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.818+0000 7f0611e38700 1 --2- 172.21.15.57:0/3025667663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f060c0cee40 0x7f060c0cdff0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.818+0000 7f0611e38700 1 -- 172.21.15.57:0/3025667663 >> 172.21.15.57:0/3025667663 conn(0x7f060c0cf400 msgr2=0x7f060c0cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:28.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.818+0000 7f0611e38700 1 -- 172.21.15.57:0/3025667663 shutdown_connections 2023-11-28T12:02:28.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.818+0000 7f0611e38700 1 -- 172.21.15.57:0/3025667663 wait complete. 2023-11-28T12:02:28.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.819+0000 7f0611e38700 1 Processor -- start 2023-11-28T12:02:28.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.819+0000 7f0611e38700 1 -- start start 2023-11-28T12:02:28.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f0611e38700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f060c0c9a50 0x7f060c1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f0611e38700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f060c0cee40 0x7f060c0b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f0611e38700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f060c159710 0x7f060c0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f0611e38700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0b3480 con 0x7f060c0cee40 2023-11-28T12:02:28.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f0611e38700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0ad050 con 0x7f060c159710 2023-11-28T12:02:28.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f0611e38700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0ad330 con 0x7f060c0c9a50 2023-11-28T12:02:28.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f060affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f060c0cee40 0x7f060c0b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f060affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f060c0cee40 0x7f060c0b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53706/0 (socket says 172.21.15.57:53706) 2023-11-28T12:02:28.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f060affd700 1 -- 172.21.15.57:0/1814618475 learned_addr learned my addr 172.21.15.57:0/1814618475 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:28.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f060bfff700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f060c159710 0x7f060c0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f060b7fe700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f060c0c9a50 0x7f060c1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f060affd700 1 -- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f060c0c9a50 msgr2=0x7f060c1588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f060affd700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f060c0c9a50 0x7f060c1588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f060affd700 1 -- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f060c159710 msgr2=0x7f060c0cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f060affd700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f060c159710 0x7f060c0cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.820+0000 7f060affd700 1 -- 172.21.15.57:0/1814618475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f40094a0 con 0x7f060c0cee40 2023-11-28T12:02:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.821+0000 7f060bfff700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f060c159710 0x7f060c0cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.821+0000 7f060b7fe700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f060c0c9a50 0x7f060c1588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.821+0000 7f060affd700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f060c0cee40 0x7f060c0b36d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009fd0 tx=0x7f05f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.821+0000 7f0603fff700 1 -- 172.21.15.57:0/1814618475 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40036a0 con 0x7f060c0cee40 2023-11-28T12:02:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.821+0000 7f0603fff700 1 -- 172.21.15.57:0/1814618475 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f401e070 con 0x7f060c0cee40 2023-11-28T12:02:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.821+0000 7f0611e38700 1 -- 172.21.15.57:0/1814618475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060c06a890 con 0x7f060c0cee40 2023-11-28T12:02:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.821+0000 7f0603fff700 1 -- 172.21.15.57:0/1814618475 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4003a60 con 0x7f060c0cee40 2023-11-28T12:02:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.821+0000 7f0611e38700 1 -- 172.21.15.57:0/1814618475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060c06a5b0 con 0x7f060c0cee40 2023-11-28T12:02:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.822+0000 7f0611e38700 1 -- 172.21.15.57:0/1814618475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060c0ceac0 con 0x7f060c0cee40 2023-11-28T12:02:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.823+0000 7f0603fff700 1 -- 172.21.15.57:0/1814618475 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f05f400f050 con 0x7f060c0cee40 2023-11-28T12:02:28.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.823+0000 7f0603fff700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f05ec0682f0 0x7f05ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:28.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.824+0000 7f0603fff700 1 -- 172.21.15.57:0/1814618475 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f05f4087090 con 0x7f060c0cee40 2023-11-28T12:02:28.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.824+0000 7f060b7fe700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f05ec0682f0 0x7f05ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:28.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.824+0000 7f060b7fe700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f05ec0682f0 0x7f05ec06a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f060c14e630 tx=0x7f05fc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:28.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:28.828+0000 7f0603fff700 1 -- 172.21.15.57:0/1814618475 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f05f4053990 con 0x7f060c0cee40 2023-11-28T12:02:29.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.043+0000 7f0611e38700 1 -- 172.21.15.57:0/1814618475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f060c0743f0 con 0x7f060c0cee40 2023-11-28T12:02:29.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.043+0000 7f0603fff700 1 -- 172.21.15.57:0/1814618475 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f05f4020080 con 0x7f060c0cee40 2023-11-28T12:02:29.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.045+0000 7f0611e38700 1 -- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f05ec0682f0 msgr2=0x7f05ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.045+0000 7f0611e38700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f05ec0682f0 0x7f05ec06a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f060c14e630 tx=0x7f05fc008040 comp rx=0 tx=0).stop 2023-11-28T12:02:29.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.045+0000 7f0611e38700 1 -- 172.21.15.57:0/1814618475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f060c0cee40 msgr2=0x7f060c0b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.045+0000 7f0611e38700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f060c0cee40 0x7f060c0b36d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009fd0 tx=0x7f05f4004750 comp rx=0 tx=0).stop 2023-11-28T12:02:29.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.045+0000 7f0611e38700 1 -- 172.21.15.57:0/1814618475 shutdown_connections 2023-11-28T12:02:29.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.045+0000 7f0611e38700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f05ec0682f0 0x7f05ec06a7a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.045+0000 7f0611e38700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f060c159710 0x7f060c0cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.045+0000 7f0611e38700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f060c0c9a50 0x7f060c1588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.045+0000 7f0611e38700 1 --2- 172.21.15.57:0/1814618475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f060c0cee40 0x7f060c0b36d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.045+0000 7f0611e38700 1 -- 172.21.15.57:0/1814618475 >> 172.21.15.57:0/1814618475 conn(0x7f060c0cf400 msgr2=0x7f060c0aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:29.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.045+0000 7f0611e38700 1 -- 172.21.15.57:0/1814618475 shutdown_connections 2023-11-28T12:02:29.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.045+0000 7f0611e38700 1 -- 172.21.15.57:0/1814618475 wait complete. 2023-11-28T12:02:29.048 INFO:teuthology.orchestra.run.smithi057.stderr:listed 0 entries 2023-11-28T12:02:29.060 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:02:29.060 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:02:29.081 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:29.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.259+0000 7f2b4ff88700 1 Processor -- start 2023-11-28T12:02:29.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.259+0000 7f2b4ff88700 1 -- start start 2023-11-28T12:02:29.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.260+0000 7f2b4ff88700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2b480b0880 0x7f2b4806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.260+0000 7f2b4ff88700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2b480ad1f0 0x7f2b480ad610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.260+0000 7f2b4ff88700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b480adb50 con 0x7f2b48072180 2023-11-28T12:02:29.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.260+0000 7f2b4ff88700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b48072620 con 0x7f2b480b0880 2023-11-28T12:02:29.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.260+0000 7f2b4ff88700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b480b43f0 con 0x7f2b480ad1f0 2023-11-28T12:02:29.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.260+0000 7f2b4dd24700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2b480b0880 0x7f2b4806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.260+0000 7f2b4dd24700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2b480b0880 0x7f2b4806b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53720/0 (socket says 172.21.15.57:53720) 2023-11-28T12:02:29.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.260+0000 7f2b4dd24700 1 -- 172.21.15.57:0/540622380 learned_addr learned my addr 172.21.15.57:0/540622380 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:29.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.260+0000 7f2b4e525700 1 --2- 172.21.15.57:0/540622380 >> v2:172.21.15.181:3300/0 conn(0x7f2b480ad1f0 0x7f2b480ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.260+0000 7f2b4dd24700 1 -- 172.21.15.57:0/540622380 >> v2:172.21.15.181:3300/0 conn(0x7f2b480ad1f0 msgr2=0x7f2b480ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:29.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.260+0000 7f2b4dd24700 1 --2- 172.21.15.57:0/540622380 >> v2:172.21.15.181:3300/0 conn(0x7f2b480ad1f0 0x7f2b480ad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.260+0000 7f2b4dd24700 1 -- 172.21.15.57:0/540622380 >> v1:172.21.15.181:6789/0 conn(0x7f2b48072180 legacy=0x7f2b480aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.260+0000 7f2b4dd24700 1 -- 172.21.15.57:0/540622380 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b480b46d0 con 0x7f2b480b0880 2023-11-28T12:02:29.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.261+0000 7f2b4dd24700 1 --2- 172.21.15.57:0/540622380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b480b0880 0x7f2b4806b7b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2b48073cf0 tx=0x7f2b44009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781b150d939bc1d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:29.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.261+0000 7f2b4cd22700 1 -- 172.21.15.57:0/540622380 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44011830 con 0x7f2b480b0880 2023-11-28T12:02:29.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.261+0000 7f2b4cd22700 1 -- 172.21.15.57:0/540622380 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b44011990 con 0x7f2b480b0880 2023-11-28T12:02:29.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.261+0000 7f2b4cd22700 1 -- 172.21.15.57:0/540622380 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44011d20 con 0x7f2b480b0880 2023-11-28T12:02:29.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.261+0000 7f2b4ff88700 1 -- 172.21.15.57:0/540622380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b480b0880 msgr2=0x7f2b4806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.261+0000 7f2b4ff88700 1 --2- 172.21.15.57:0/540622380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b480b0880 0x7f2b4806b7b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2b48073cf0 tx=0x7f2b44009bd0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.261+0000 7f2b4ff88700 1 -- 172.21.15.57:0/540622380 shutdown_connections 2023-11-28T12:02:29.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.261+0000 7f2b4ff88700 1 --2- 172.21.15.57:0/540622380 >> v2:172.21.15.181:3300/0 conn(0x7f2b480ad1f0 0x7f2b480ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.261+0000 7f2b4ff88700 1 --2- 172.21.15.57:0/540622380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b480b0880 0x7f2b4806b7b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.261+0000 7f2b4ff88700 1 -- 172.21.15.57:0/540622380 >> 172.21.15.57:0/540622380 conn(0x7f2b480b2fc0 msgr2=0x7f2b48071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:29.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.262+0000 7f2b4ff88700 1 -- 172.21.15.57:0/540622380 shutdown_connections 2023-11-28T12:02:29.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.262+0000 7f2b4ff88700 1 -- 172.21.15.57:0/540622380 wait complete. 2023-11-28T12:02:29.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.263+0000 7f2b4ff88700 1 Processor -- start 2023-11-28T12:02:29.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.263+0000 7f2b4ff88700 1 -- start start 2023-11-28T12:02:29.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4ff88700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b48072180 0x7f2b4815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4ff88700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b480ad1f0 0x7f2b480c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4ff88700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b480b0880 0x7f2b480c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4ff88700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4815a8f0 con 0x7f2b48072180 2023-11-28T12:02:29.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4ff88700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4815f250 con 0x7f2b480b0880 2023-11-28T12:02:29.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4ff88700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4815ec40 con 0x7f2b480ad1f0 2023-11-28T12:02:29.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4dd24700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b48072180 0x7f2b4815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4dd24700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b48072180 0x7f2b4815a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53730/0 (socket says 172.21.15.57:53730) 2023-11-28T12:02:29.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4dd24700 1 -- 172.21.15.57:0/197812795 learned_addr learned my addr 172.21.15.57:0/197812795 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:29.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4e525700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b480b0880 0x7f2b480c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4d523700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b480ad1f0 0x7f2b480c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4dd24700 1 -- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b480ad1f0 msgr2=0x7f2b480c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4dd24700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b480ad1f0 0x7f2b480c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4dd24700 1 -- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b480b0880 msgr2=0x7f2b480c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4dd24700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b480b0880 0x7f2b480c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4dd24700 1 -- 172.21.15.57:0/197812795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b44009550 con 0x7f2b48072180 2023-11-28T12:02:29.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b4dd24700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b48072180 0x7f2b4815a3b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2b44000c00 tx=0x7f2b44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:29.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.264+0000 7f2b3e7fc700 1 -- 172.21.15.57:0/197812795 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44004380 con 0x7f2b48072180 2023-11-28T12:02:29.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.265+0000 7f2b3e7fc700 1 -- 172.21.15.57:0/197812795 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4401f070 con 0x7f2b48072180 2023-11-28T12:02:29.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.265+0000 7f2b3e7fc700 1 -- 172.21.15.57:0/197812795 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b440036a0 con 0x7f2b48072180 2023-11-28T12:02:29.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.265+0000 7f2b4ff88700 1 -- 172.21.15.57:0/197812795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b480b4fe0 con 0x7f2b48072180 2023-11-28T12:02:29.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.265+0000 7f2b4ff88700 1 -- 172.21.15.57:0/197812795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b480b4d00 con 0x7f2b48072180 2023-11-28T12:02:29.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.266+0000 7f2b4ff88700 1 -- 172.21.15.57:0/197812795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b480b49b0 con 0x7f2b48072180 2023-11-28T12:02:29.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.266+0000 7f2b3e7fc700 1 -- 172.21.15.57:0/197812795 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2b4402a020 con 0x7f2b48072180 2023-11-28T12:02:29.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.266+0000 7f2b3e7fc700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2b2c0682f0 0x7f2b2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.266+0000 7f2b3e7fc700 1 -- 172.21.15.57:0/197812795 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b44087200 con 0x7f2b48072180 2023-11-28T12:02:29.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.267+0000 7f2b4d523700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2b2c0682f0 0x7f2b2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.267+0000 7f2b4d523700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2b2c0682f0 0x7f2b2c06a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2b48099c90 tx=0x7f2b38009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:29.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.271+0000 7f2b3e7fc700 1 -- 172.21.15.57:0/197812795 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2b440539e0 con 0x7f2b48072180 2023-11-28T12:02:29.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.470+0000 7f2b4ff88700 1 -- 172.21.15.57:0/197812795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b480743f0 con 0x7f2b48072180 2023-11-28T12:02:29.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.472+0000 7f2b3e7fc700 1 -- 172.21.15.57:0/197812795 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17849 (secure 0 0 0) 0x7f2b44057000 con 0x7f2b48072180 2023-11-28T12:02:29.473 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:29.473 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:01:44.921300+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:29.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.474+0000 7f2b4ff88700 1 -- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2b2c0682f0 msgr2=0x7f2b2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.474+0000 7f2b4ff88700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2b2c0682f0 0x7f2b2c06a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2b48099c90 tx=0x7f2b38009040 comp rx=0 tx=0).stop 2023-11-28T12:02:29.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.474+0000 7f2b4ff88700 1 -- 172.21.15.57:0/197812795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b48072180 msgr2=0x7f2b4815a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.474+0000 7f2b4ff88700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b48072180 0x7f2b4815a3b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2b44000c00 tx=0x7f2b44004750 comp rx=0 tx=0).stop 2023-11-28T12:02:29.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.474+0000 7f2b4ff88700 1 -- 172.21.15.57:0/197812795 shutdown_connections 2023-11-28T12:02:29.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.474+0000 7f2b4ff88700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2b2c0682f0 0x7f2b2c06a7a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.474+0000 7f2b4ff88700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b480b0880 0x7f2b480c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.474+0000 7f2b4ff88700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b48072180 0x7f2b4815a3b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.474+0000 7f2b4ff88700 1 --2- 172.21.15.57:0/197812795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b480ad1f0 0x7f2b480c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.474+0000 7f2b4ff88700 1 -- 172.21.15.57:0/197812795 >> 172.21.15.57:0/197812795 conn(0x7f2b480b2fc0 msgr2=0x7f2b4806ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:29.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.474+0000 7f2b4ff88700 1 -- 172.21.15.57:0/197812795 shutdown_connections 2023-11-28T12:02:29.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.474+0000 7f2b4ff88700 1 -- 172.21.15.57:0/197812795 wait complete. 2023-11-28T12:02:29.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:29.490 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:29.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.627+0000 7fdd0dae8700 1 Processor -- start 2023-11-28T12:02:29.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.627+0000 7fdd0dae8700 1 -- start start 2023-11-28T12:02:29.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.628+0000 7fdd0dae8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdd0815a5f0 0x7fdd080d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.628+0000 7fdd0dae8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd080cc200 0x7fdd080b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.628+0000 7fdd0dae8700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0815a9c0 con 0x7fdd080b2d10 2023-11-28T12:02:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.628+0000 7fdd0dae8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd080b3190 con 0x7fdd0815a5f0 2023-11-28T12:02:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.628+0000 7fdd0dae8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080c7b80 con 0x7fdd080cc200 2023-11-28T12:02:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.628+0000 7fdd07fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fdd080b2d10 0x7fdd0815e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40262/0 (socket says 172.21.15.57:40262) 2023-11-28T12:02:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.628+0000 7fdd07fff700 1 -- 172.21.15.57:0/1256141331 learned_addr learned my addr 172.21.15.57:0/1256141331 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd077fe700 1 --2- 172.21.15.57:0/1256141331 >> v2:172.21.15.181:3300/0 conn(0x7fdd0815a5f0 0x7fdd080d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd06ffd700 1 --2- 172.21.15.57:0/1256141331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd080cc200 0x7fdd080b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922420953 0 0) 0x7fdd0815a9c0 con 0x7fdd080b2d10 2023-11-28T12:02:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcf40035b0 con 0x7fdd080b2d10 2023-11-28T12:02:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3766775227 0 0) 0x7fdcf40035b0 con 0x7fdd080b2d10 2023-11-28T12:02:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd080cc200 msgr2=0x7fdd080b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:29.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 --2- 172.21.15.57:0/1256141331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd080cc200 0x7fdd080b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 >> v2:172.21.15.181:3300/0 conn(0x7fdd0815a5f0 msgr2=0x7fdd080d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:29.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 --2- 172.21.15.57:0/1256141331 >> v2:172.21.15.181:3300/0 conn(0x7fdd0815a5f0 0x7fdd080d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd080cc990 con 0x7fdd080b2d10 2023-11-28T12:02:29.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fdd000019f0 con 0x7fdd080b2d10 2023-11-28T12:02:29.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 >> v1:172.21.15.57:6789/0 conn(0x7fdd080b2d10 legacy=0x7fdd0815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 --2- 172.21.15.57:0/1256141331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcf4003a00 0x7fdcf4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 --2- 172.21.15.57:0/1256141331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcf4004430 0x7fdcf40048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 --2- 172.21.15.57:0/1256141331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcf4004f10 0x7fdcf400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080b3190 con 0x7fdcf4004f10 2023-11-28T12:02:29.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080c7b80 con 0x7fdcf4003a00 2023-11-28T12:02:29.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf400bf50 con 0x7fdcf4004430 2023-11-28T12:02:29.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd06ffd700 1 --2- 172.21.15.57:0/1256141331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd080cc200 0x7fdd080b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:29.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd07fff700 1 --2- 172.21.15.57:0/1256141331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcf4004f10 0x7fdcf400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd077fe700 1 --2- 172.21.15.57:0/1256141331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcf4003a00 0x7fdcf4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.629+0000 7fdd06ffd700 1 --2- 172.21.15.57:0/1256141331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcf4004430 0x7fdcf40048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.630+0000 7fdd07fff700 1 -- 172.21.15.57:0/1256141331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcf4004430 msgr2=0x7fdcf40048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:29.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.630+0000 7fdd07fff700 1 --2- 172.21.15.57:0/1256141331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcf4004430 0x7fdcf40048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.630+0000 7fdd07fff700 1 -- 172.21.15.57:0/1256141331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcf4003a00 msgr2=0x7fdcf4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:29.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.630+0000 7fdd07fff700 1 --2- 172.21.15.57:0/1256141331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcf4003a00 0x7fdcf4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.630+0000 7fdd07fff700 1 -- 172.21.15.57:0/1256141331 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf40035b0 con 0x7fdcf4004f10 2023-11-28T12:02:29.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.630+0000 7fdd07fff700 1 --2- 172.21.15.57:0/1256141331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcf4004f10 0x7fdcf400ba10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000a860 tx=0x7fdcf4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932a0d570efdc2a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:29.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.630+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00012600 con 0x7fdcf4004f10 2023-11-28T12:02:29.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.630+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd00012780 con 0x7fdcf4004f10 2023-11-28T12:02:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.630+0000 7fdd067fc700 1 -- 172.21.15.57:0/1256141331 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00012b30 con 0x7fdcf4004f10 2023-11-28T12:02:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.630+0000 7fdd0dae8700 1 -- 172.21.15.57:0/1256141331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcf4004f10 msgr2=0x7fdcf400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.630+0000 7fdd0dae8700 1 --2- 172.21.15.57:0/1256141331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcf4004f10 0x7fdcf400ba10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000a860 tx=0x7fdcf4005880 comp rx=0 tx=0).stop 2023-11-28T12:02:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.630+0000 7fdd07fff700 1 -- 172.21.15.57:0/1256141331 reap_dead start 2023-11-28T12:02:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.631+0000 7fdd0dae8700 1 -- 172.21.15.57:0/1256141331 shutdown_connections 2023-11-28T12:02:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.631+0000 7fdd0dae8700 1 -- 172.21.15.57:0/1256141331 >> 172.21.15.57:0/1256141331 conn(0x7fdd0806ea80 msgr2=0x7fdd0806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.631+0000 7fdd0dae8700 1 -- 172.21.15.57:0/1256141331 shutdown_connections 2023-11-28T12:02:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.631+0000 7fdd0dae8700 1 -- 172.21.15.57:0/1256141331 wait complete. 2023-11-28T12:02:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.632+0000 7fdd0dae8700 1 Processor -- start 2023-11-28T12:02:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.632+0000 7fdd0dae8700 1 -- start start 2023-11-28T12:02:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd0dae8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd0814bb90 0x7fdd081585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd0dae8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd08158ae0 0x7fdd080c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd0dae8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd0814bf60 0x7fdd080c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd0dae8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd08159540 con 0x7fdd0814bb90 2023-11-28T12:02:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd0dae8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd08158f80 con 0x7fdd0814bf60 2023-11-28T12:02:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd0dae8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd08159260 con 0x7fdd08158ae0 2023-11-28T12:02:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd077fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd0814bb90 0x7fdd081585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd077fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd0814bb90 0x7fdd081585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53760/0 (socket says 172.21.15.57:53760) 2023-11-28T12:02:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd077fe700 1 -- 172.21.15.57:0/713619521 learned_addr learned my addr 172.21.15.57:0/713619521 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd06ffd700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd08158ae0 0x7fdd080c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd07fff700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd0814bf60 0x7fdd080c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd077fe700 1 -- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd08158ae0 msgr2=0x7fdd080c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd077fe700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd08158ae0 0x7fdd080c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd077fe700 1 -- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd0814bf60 msgr2=0x7fdd080c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd077fe700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd0814bf60 0x7fdd080c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.633+0000 7fdd077fe700 1 -- 172.21.15.57:0/713619521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd080ccf50 con 0x7fdd0814bb90 2023-11-28T12:02:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.634+0000 7fdd077fe700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd0814bb90 0x7fdd081585a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8011aa0 tx=0x7fdcf8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.634+0000 7fdcfffff700 1 -- 172.21.15.57:0/713619521 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8011ee0 con 0x7fdd0814bb90 2023-11-28T12:02:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.634+0000 7fdcfffff700 1 -- 172.21.15.57:0/713619521 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf800f940 con 0x7fdd0814bb90 2023-11-28T12:02:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.634+0000 7fdd0dae8700 1 -- 172.21.15.57:0/713619521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd080ccc70 con 0x7fdd0814bb90 2023-11-28T12:02:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.634+0000 7fdcfffff700 1 -- 172.21.15.57:0/713619521 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf800fd10 con 0x7fdd0814bb90 2023-11-28T12:02:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.634+0000 7fdd0dae8700 1 -- 172.21.15.57:0/713619521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd080cd230 con 0x7fdd0814bb90 2023-11-28T12:02:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.635+0000 7fdd0dae8700 1 -- 172.21.15.57:0/713619521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd080743f0 con 0x7fdd0814bb90 2023-11-28T12:02:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.635+0000 7fdcfffff700 1 -- 172.21.15.57:0/713619521 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdcf8014440 con 0x7fdd0814bb90 2023-11-28T12:02:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.636+0000 7fdcfffff700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdce80682f0 0x7fdce806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.636+0000 7fdcfffff700 1 -- 172.21.15.57:0/713619521 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdcf8086400 con 0x7fdd0814bb90 2023-11-28T12:02:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.636+0000 7fdd06ffd700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdce80682f0 0x7fdce806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.636+0000 7fdd06ffd700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdce80682f0 0x7fdce806a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdd08158eb0 tx=0x7fdcf0009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.640+0000 7fdcfffff700 1 -- 172.21.15.57:0/713619521 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdcf8052d70 con 0x7fdd0814bb90 2023-11-28T12:02:29.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.841+0000 7fdd0dae8700 1 -- 172.21.15.57:0/713619521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdd0806ab80 con 0x7fdd0814bb90 2023-11-28T12:02:29.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.841+0000 7fdcfffff700 1 -- 172.21.15.57:0/713619521 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdcf8056390 con 0x7fdd0814bb90 2023-11-28T12:02:29.843 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:29.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.844+0000 7fdd0dae8700 1 -- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdce80682f0 msgr2=0x7fdce806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.844+0000 7fdd0dae8700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdce80682f0 0x7fdce806a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdd08158eb0 tx=0x7fdcf0009210 comp rx=0 tx=0).stop 2023-11-28T12:02:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.844+0000 7fdd0dae8700 1 -- 172.21.15.57:0/713619521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd0814bb90 msgr2=0x7fdd081585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.844+0000 7fdd0dae8700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd0814bb90 0x7fdd081585a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8011aa0 tx=0x7fdcf8011db0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.844+0000 7fdd0dae8700 1 -- 172.21.15.57:0/713619521 shutdown_connections 2023-11-28T12:02:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.844+0000 7fdd0dae8700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdce80682f0 0x7fdce806a7a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.844+0000 7fdd0dae8700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd0814bf60 0x7fdd080c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.844+0000 7fdd0dae8700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd0814bb90 0x7fdd081585a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.844+0000 7fdd0dae8700 1 --2- 172.21.15.57:0/713619521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd08158ae0 0x7fdd080c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:29.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.844+0000 7fdd0dae8700 1 -- 172.21.15.57:0/713619521 >> 172.21.15.57:0/713619521 conn(0x7fdd0806ea80 msgr2=0x7fdd080cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:29.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.844+0000 7fdd0dae8700 1 -- 172.21.15.57:0/713619521 shutdown_connections 2023-11-28T12:02:29.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.844+0000 7fdd0dae8700 1 -- 172.21.15.57:0/713619521 wait complete. 2023-11-28T12:02:29.859 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:29.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.996+0000 7f8c538b2700 1 Processor -- start 2023-11-28T12:02:29.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.996+0000 7f8c538b2700 1 -- start start 2023-11-28T12:02:30.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.997+0000 7f8c538b2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c4c160a50 0x7f8c4c070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.997+0000 7f8c538b2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c4c0ae660 0x7f8c4c06c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.997+0000 7f8c538b2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4c0ce740 con 0x7f8c4c160e20 2023-11-28T12:02:30.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.997+0000 7f8c538b2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4c071870 con 0x7f8c4c0ae660 2023-11-28T12:02:30.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.997+0000 7f8c538b2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0af450 con 0x7f8c4c160a50 2023-11-28T12:02:30.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c51e4f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c4c0ae660 0x7f8c4c06c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c5164e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c4c160a50 0x7f8c4c070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c50e4d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8c4c160e20 0x7f8c4c0cf170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:55938/0 (socket says 172.21.15.57:55938) 2023-11-28T12:02:30.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c51e4f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c4c0ae660 0x7f8c4c06c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:44146/0 (socket says 172.21.15.57:44146) 2023-11-28T12:02:30.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c5164e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c4c160a50 0x7f8c4c070bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:40836/0 (socket says 172.21.15.57:40836) 2023-11-28T12:02:30.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c50e4d700 1 -- 172.21.15.57:0/1929681723 learned_addr learned my addr 172.21.15.57:0/1929681723 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:30.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c3ffff700 1 -- 172.21.15.57:0/1929681723 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771730865 0 0) 0x7f8c4c0ce740 con 0x7f8c4c160e20 2023-11-28T12:02:30.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c3ffff700 1 -- 172.21.15.57:0/1929681723 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c380035b0 con 0x7f8c4c160e20 2023-11-28T12:02:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c51e4f700 1 -- 172.21.15.57:0/1929681723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c4c160a50 msgr2=0x7f8c4c070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c51e4f700 1 --2- 172.21.15.57:0/1929681723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c4c160a50 0x7f8c4c070bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c51e4f700 1 -- 172.21.15.57:0/1929681723 >> v1:172.21.15.181:6789/0 conn(0x7f8c4c160e20 legacy=0x7f8c4c0cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c51e4f700 1 -- 172.21.15.57:0/1929681723 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c0712b0 con 0x7f8c4c0ae660 2023-11-28T12:02:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c5164e700 1 --2- 172.21.15.57:0/1929681723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c4c160a50 0x7f8c4c070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.998+0000 7f8c51e4f700 1 --2- 172.21.15.57:0/1929681723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4c0ae660 0x7f8c4c06c880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009d90 tx=0x7f8c4400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0cb3cc6fadbd700 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:30.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.999+0000 7f8c3ffff700 1 -- 172.21.15.57:0/1929681723 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4400e070 con 0x7f8c4c0ae660 2023-11-28T12:02:30.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.999+0000 7f8c3ffff700 1 -- 172.21.15.57:0/1929681723 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4400bb20 con 0x7f8c4c0ae660 2023-11-28T12:02:30.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.999+0000 7f8c3ffff700 1 -- 172.21.15.57:0/1929681723 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44012430 con 0x7f8c4c0ae660 2023-11-28T12:02:30.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.999+0000 7f8c538b2700 1 -- 172.21.15.57:0/1929681723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4c0ae660 msgr2=0x7f8c4c06c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.999+0000 7f8c538b2700 1 --2- 172.21.15.57:0/1929681723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4c0ae660 0x7f8c4c06c880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009d90 tx=0x7f8c4400b330 comp rx=0 tx=0).stop 2023-11-28T12:02:30.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.999+0000 7f8c538b2700 1 -- 172.21.15.57:0/1929681723 shutdown_connections 2023-11-28T12:02:30.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.999+0000 7f8c538b2700 1 --2- 172.21.15.57:0/1929681723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4c0ae660 0x7f8c4c06c880 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.999+0000 7f8c538b2700 1 --2- 172.21.15.57:0/1929681723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c4c160a50 0x7f8c4c070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.999+0000 7f8c538b2700 1 -- 172.21.15.57:0/1929681723 >> 172.21.15.57:0/1929681723 conn(0x7f8c4c06c010 msgr2=0x7f8c4c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:30.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.999+0000 7f8c538b2700 1 -- 172.21.15.57:0/1929681723 shutdown_connections 2023-11-28T12:02:30.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:29.999+0000 7f8c538b2700 1 -- 172.21.15.57:0/1929681723 wait complete. 2023-11-28T12:02:30.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.000+0000 7f8c538b2700 1 Processor -- start 2023-11-28T12:02:30.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.001+0000 7f8c538b2700 1 -- start start 2023-11-28T12:02:30.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.001+0000 7f8c538b2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c4c0ae660 0x7f8c4c0cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.001+0000 7f8c538b2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c160a50 0x7f8c4c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.001+0000 7f8c538b2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4c160e20 0x7f8c4c159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.001+0000 7f8c538b2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c163e40 con 0x7f8c4c160a50 2023-11-28T12:02:30.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.001+0000 7f8c538b2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0cd090 con 0x7f8c4c160e20 2023-11-28T12:02:30.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.001+0000 7f8c538b2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0cb360 con 0x7f8c4c0ae660 2023-11-28T12:02:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.001+0000 7f8c50e4d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c160a50 0x7f8c4c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.001+0000 7f8c50e4d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c160a50 0x7f8c4c0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53770/0 (socket says 172.21.15.57:53770) 2023-11-28T12:02:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.001+0000 7f8c50e4d700 1 -- 172.21.15.57:0/1615003501 learned_addr learned my addr 172.21.15.57:0/1615003501 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.001+0000 7f8c5164e700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c4c0ae660 0x7f8c4c0cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.001+0000 7f8c51e4f700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4c160e20 0x7f8c4c159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.002+0000 7f8c50e4d700 1 -- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c4c0ae660 msgr2=0x7f8c4c0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.002+0000 7f8c50e4d700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c4c0ae660 0x7f8c4c0cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.002+0000 7f8c50e4d700 1 -- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4c160e20 msgr2=0x7f8c4c159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.002+0000 7f8c50e4d700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4c160e20 0x7f8c4c159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.002+0000 7f8c50e4d700 1 -- 172.21.15.57:0/1615003501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c480094a0 con 0x7f8c4c160a50 2023-11-28T12:02:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.002+0000 7f8c50e4d700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c160a50 0x7f8c4c0c5980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8c40001eb0 tx=0x7f8c4000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.002+0000 7f8c3dffb700 1 -- 172.21.15.57:0/1615003501 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40005d70 con 0x7f8c4c160a50 2023-11-28T12:02:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.002+0000 7f8c3dffb700 1 -- 172.21.15.57:0/1615003501 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c400070d0 con 0x7f8c4c160a50 2023-11-28T12:02:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.002+0000 7f8c538b2700 1 -- 172.21.15.57:0/1615003501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c44009a70 con 0x7f8c4c160a50 2023-11-28T12:02:30.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.002+0000 7f8c3dffb700 1 -- 172.21.15.57:0/1615003501 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40007460 con 0x7f8c4c160a50 2023-11-28T12:02:30.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.002+0000 7f8c538b2700 1 -- 172.21.15.57:0/1615003501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c0ce970 con 0x7f8c4c160a50 2023-11-28T12:02:30.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.004+0000 7f8c538b2700 1 -- 172.21.15.57:0/1615003501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4c071590 con 0x7f8c4c160a50 2023-11-28T12:02:30.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.004+0000 7f8c3dffb700 1 -- 172.21.15.57:0/1615003501 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8c40019030 con 0x7f8c4c160a50 2023-11-28T12:02:30.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.005+0000 7f8c3dffb700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c300682f0 0x7f8c3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.005+0000 7f8c3dffb700 1 -- 172.21.15.57:0/1615003501 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8c40084f20 con 0x7f8c4c160a50 2023-11-28T12:02:30.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.005+0000 7f8c5164e700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c300682f0 0x7f8c3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.005+0000 7f8c5164e700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c300682f0 0x7f8c3006a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009d70 tx=0x7f8c48009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:30.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.009+0000 7f8c3dffb700 1 -- 172.21.15.57:0/1615003501 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8c40051780 con 0x7f8c4c160a50 2023-11-28T12:02:30.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.209+0000 7f8c538b2700 1 -- 172.21.15.57:0/1615003501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c4c0cec50 con 0x7f8c4c160a50 2023-11-28T12:02:30.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.209+0000 7f8c3dffb700 1 -- 172.21.15.57:0/1615003501 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c400170a0 con 0x7f8c4c160a50 2023-11-28T12:02:30.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:30.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.212+0000 7f8c538b2700 1 -- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c300682f0 msgr2=0x7f8c3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.212+0000 7f8c538b2700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c300682f0 0x7f8c3006a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009d70 tx=0x7f8c48009210 comp rx=0 tx=0).stop 2023-11-28T12:02:30.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.212+0000 7f8c538b2700 1 -- 172.21.15.57:0/1615003501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c160a50 msgr2=0x7f8c4c0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.212+0000 7f8c538b2700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c160a50 0x7f8c4c0c5980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8c40001eb0 tx=0x7f8c4000c7b0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.212+0000 7f8c538b2700 1 -- 172.21.15.57:0/1615003501 shutdown_connections 2023-11-28T12:02:30.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.212+0000 7f8c538b2700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c300682f0 0x7f8c3006a7a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.212+0000 7f8c538b2700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c4c160e20 0x7f8c4c159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.212+0000 7f8c538b2700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c4c0ae660 0x7f8c4c0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.212+0000 7f8c538b2700 1 --2- 172.21.15.57:0/1615003501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c160a50 0x7f8c4c0c5980 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.212+0000 7f8c538b2700 1 -- 172.21.15.57:0/1615003501 >> 172.21.15.57:0/1615003501 conn(0x7f8c4c06c010 msgr2=0x7f8c4c15bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:30.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.212+0000 7f8c538b2700 1 -- 172.21.15.57:0/1615003501 shutdown_connections 2023-11-28T12:02:30.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.212+0000 7f8c538b2700 1 -- 172.21.15.57:0/1615003501 wait complete. 2023-11-28T12:02:30.226 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:02:30.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.364+0000 7fc9a5ed5700 1 Processor -- start 2023-11-28T12:02:30.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.364+0000 7fc9a5ed5700 1 -- start start 2023-11-28T12:02:30.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.365+0000 7fc9a5ed5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc9a00d0ad0 0x7fc9a00cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.365+0000 7fc9a5ed5700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a00cc740 con 0x7fc9a015a5f0 2023-11-28T12:02:30.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.365+0000 7fc9a5ed5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a015a9c0 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.365+0000 7fc9a5ed5700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a00c7b80 con 0x7fc9a00d0ad0 2023-11-28T12:02:30.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.365+0000 7fc99ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc9a00d0ad0 0x7fc9a00cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.365+0000 7fc99f7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc9a015a5f0 0x7fc9a00b27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40276/0 (socket says 172.21.15.57:40276) 2023-11-28T12:02:30.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.365+0000 7fc99f7fe700 1 -- 172.21.15.57:0/1190497800 learned_addr learned my addr 172.21.15.57:0/1190497800 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:30.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.365+0000 7fc99ffff700 1 -- 172.21.15.57:0/1190497800 >> v1:172.21.15.181:6789/0 conn(0x7fc9a00b2ee0 legacy=0x7fc9a00ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:30.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.365+0000 7fc99ffff700 1 -- 172.21.15.57:0/1190497800 >> v1:172.21.15.57:6789/0 conn(0x7fc9a015a5f0 legacy=0x7fc9a00b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.365+0000 7fc99ffff700 1 -- 172.21.15.57:0/1190497800 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a00cc990 con 0x7fc9a00d0ad0 2023-11-28T12:02:30.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.365+0000 7fc99e7fc700 1 -- 172.21.15.57:0/1190497800 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179189587 0 0) 0x7fc9a00cc740 con 0x7fc9a015a5f0 2023-11-28T12:02:30.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.365+0000 7fc99ffff700 1 --2- 172.21.15.57:0/1190497800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9a00d0ad0 0x7fc9a00cc200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc998009d90 tx=0x7fc99800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd80021ac81842f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:30.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.365+0000 7fc99e7fc700 1 -- 172.21.15.57:0/1190497800 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99800e070 con 0x7fc9a00d0ad0 2023-11-28T12:02:30.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.366+0000 7fc99e7fc700 1 -- 172.21.15.57:0/1190497800 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99800bb90 con 0x7fc9a00d0ad0 2023-11-28T12:02:30.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.366+0000 7fc99e7fc700 1 -- 172.21.15.57:0/1190497800 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc998003a20 con 0x7fc9a00d0ad0 2023-11-28T12:02:30.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.366+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1190497800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9a00d0ad0 msgr2=0x7fc9a00cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.366+0000 7fc9a5ed5700 1 --2- 172.21.15.57:0/1190497800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9a00d0ad0 0x7fc9a00cc200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc998009d90 tx=0x7fc99800b330 comp rx=0 tx=0).stop 2023-11-28T12:02:30.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.366+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1190497800 shutdown_connections 2023-11-28T12:02:30.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.366+0000 7fc9a5ed5700 1 --2- 172.21.15.57:0/1190497800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9a00d0ad0 0x7fc9a00cc200 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.366+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1190497800 >> 172.21.15.57:0/1190497800 conn(0x7fc9a006ea80 msgr2=0x7fc9a006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:30.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.366+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1190497800 shutdown_connections 2023-11-28T12:02:30.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.366+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1190497800 wait complete. 2023-11-28T12:02:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.367+0000 7fc9a5ed5700 1 Processor -- start 2023-11-28T12:02:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.368+0000 7fc9a5ed5700 1 -- start start 2023-11-28T12:02:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.368+0000 7fc9a5ed5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9a00b2ee0 0x7fc9a014b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.368+0000 7fc9a5ed5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a00d0ad0 0x7fc9a00c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.368+0000 7fc9a5ed5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9a015a5f0 0x7fc9a00b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.368+0000 7fc9a5ed5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a00c4810 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.368+0000 7fc9a5ed5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a00b5640 con 0x7fc9a00d0ad0 2023-11-28T12:02:30.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.368+0000 7fc9a5ed5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a00b5c00 con 0x7fc9a015a5f0 2023-11-28T12:02:30.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc99f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9a00b2ee0 0x7fc9a014b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc99f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9a00b2ee0 0x7fc9a014b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53800/0 (socket says 172.21.15.57:53800) 2023-11-28T12:02:30.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc99f7fe700 1 -- 172.21.15.57:0/1272435239 learned_addr learned my addr 172.21.15.57:0/1272435239 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:30.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc99ffff700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9a015a5f0 0x7fc9a00b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc99effd700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a00d0ad0 0x7fc9a00c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc99f7fe700 1 -- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9a015a5f0 msgr2=0x7fc9a00b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc99f7fe700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9a015a5f0 0x7fc9a00b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc99f7fe700 1 -- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a00d0ad0 msgr2=0x7fc9a00c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc99f7fe700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a00d0ad0 0x7fc9a00c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc99f7fe700 1 -- 172.21.15.57:0/1272435239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc998009a70 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc99ffff700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9a015a5f0 0x7fc9a00b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:30.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc99effd700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a00d0ad0 0x7fc9a00c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:30.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc99f7fe700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9a00b2ee0 0x7fc9a014b390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc98800c750 tx=0x7fc98800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:30.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc997fff700 1 -- 172.21.15.57:0/1272435239 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988004c80 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc997fff700 1 -- 172.21.15.57:0/1272435239 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc988004e00 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc997fff700 1 -- 172.21.15.57:0/1272435239 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988007310 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1272435239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a00ccf50 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.369+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1272435239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a00ccc70 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.370+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1272435239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a00cd230 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.371+0000 7fc997fff700 1 -- 172.21.15.57:0/1272435239 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc98801a020 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.371+0000 7fc997fff700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc9800682f0 0x7fc98006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.372+0000 7fc997fff700 1 -- 172.21.15.57:0/1272435239 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc988085960 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.372+0000 7fc99effd700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc9800682f0 0x7fc98006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.372+0000 7fc99effd700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc9800682f0 0x7fc98006a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc990006fd0 tx=0x7fc990008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.376+0000 7fc997fff700 1 -- 172.21.15.57:0/1272435239 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc988052270 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.574+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1272435239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc9a00743f0 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.574+0000 7fc997fff700 1 -- 172.21.15.57:0/1272435239 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc988055890 con 0x7fc9a00b2ee0 2023-11-28T12:02:30.577 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:02:30.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.577+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc9800682f0 msgr2=0x7fc98006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.577+0000 7fc9a5ed5700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc9800682f0 0x7fc98006a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc990006fd0 tx=0x7fc990008040 comp rx=0 tx=0).stop 2023-11-28T12:02:30.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.577+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1272435239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9a00b2ee0 msgr2=0x7fc9a014b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.577+0000 7fc9a5ed5700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9a00b2ee0 0x7fc9a014b390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc98800c750 tx=0x7fc98800cb10 comp rx=0 tx=0).stop 2023-11-28T12:02:30.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.577+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1272435239 shutdown_connections 2023-11-28T12:02:30.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.577+0000 7fc9a5ed5700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc9800682f0 0x7fc98006a7a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.577+0000 7fc9a5ed5700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9a015a5f0 0x7fc9a00b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.577+0000 7fc9a5ed5700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9a00b2ee0 0x7fc9a014b390 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.577+0000 7fc9a5ed5700 1 --2- 172.21.15.57:0/1272435239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a00d0ad0 0x7fc9a00c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.577+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1272435239 >> 172.21.15.57:0/1272435239 conn(0x7fc9a006ea80 msgr2=0x7fc9a0158f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:30.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.577+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1272435239 shutdown_connections 2023-11-28T12:02:30.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.577+0000 7fc9a5ed5700 1 -- 172.21.15.57:0/1272435239 wait complete. 2023-11-28T12:02:30.591 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:02:30.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.729+0000 7f3c415a9700 1 Processor -- start 2023-11-28T12:02:30.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.729+0000 7f3c415a9700 1 -- start start 2023-11-28T12:02:30.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.730+0000 7f3c415a9700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3c3c0b1650 0x7f3c3c0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.730+0000 7f3c415a9700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c3c159ec0 con 0x7f3c3c1599b0 2023-11-28T12:02:30.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.730+0000 7f3c415a9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c3c15aa60 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.730+0000 7f3c415a9700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3c0b1f60 con 0x7f3c3c0b1650 2023-11-28T12:02:30.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.730+0000 7f3c3affd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3c3c0b1650 0x7f3c3c0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.730+0000 7f3c3a7fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f3c3c1599b0 0x7f3c3c15d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40284/0 (socket says 172.21.15.57:40284) 2023-11-28T12:02:30.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.730+0000 7f3c3a7fc700 1 -- 172.21.15.57:0/1781858896 learned_addr learned my addr 172.21.15.57:0/1781858896 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:30.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.730+0000 7f3c3affd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3c3c0b1650 0x7f3c3c0b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53810/0 (socket says 172.21.15.57:53810) 2023-11-28T12:02:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.731+0000 7f3c3affd700 1 -- 172.21.15.57:0/1781858896 >> v1:172.21.15.181:6789/0 conn(0x7f3c3c0af0a0 legacy=0x7f3c3c15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.731+0000 7f3c3affd700 1 -- 172.21.15.57:0/1781858896 >> v1:172.21.15.57:6789/0 conn(0x7f3c3c1599b0 legacy=0x7f3c3c15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.731+0000 7f3c3affd700 1 -- 172.21.15.57:0/1781858896 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c15d8a0 con 0x7f3c3c0b1650 2023-11-28T12:02:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.731+0000 7f3c3affd700 1 --2- 172.21.15.57:0/1781858896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c0b1650 0x7f3c3c0b1a20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009840 tx=0x7f3c24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189422b0e892f507 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.731+0000 7f3c39ffb700 1 -- 172.21.15.57:0/1781858896 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24011830 con 0x7f3c3c0b1650 2023-11-28T12:02:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.731+0000 7f3c39ffb700 1 -- 172.21.15.57:0/1781858896 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c24011990 con 0x7f3c3c0b1650 2023-11-28T12:02:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.731+0000 7f3c39ffb700 1 -- 172.21.15.57:0/1781858896 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24011d20 con 0x7f3c3c0b1650 2023-11-28T12:02:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.732+0000 7f3c415a9700 1 -- 172.21.15.57:0/1781858896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c0b1650 msgr2=0x7f3c3c0b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.732+0000 7f3c415a9700 1 --2- 172.21.15.57:0/1781858896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c0b1650 0x7f3c3c0b1a20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009840 tx=0x7f3c24009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.732+0000 7f3c415a9700 1 -- 172.21.15.57:0/1781858896 shutdown_connections 2023-11-28T12:02:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.732+0000 7f3c415a9700 1 --2- 172.21.15.57:0/1781858896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c0b1650 0x7f3c3c0b1a20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.732+0000 7f3c415a9700 1 -- 172.21.15.57:0/1781858896 >> 172.21.15.57:0/1781858896 conn(0x7f3c3c06c010 msgr2=0x7f3c3c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.732+0000 7f3c415a9700 1 -- 172.21.15.57:0/1781858896 shutdown_connections 2023-11-28T12:02:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.732+0000 7f3c415a9700 1 -- 172.21.15.57:0/1781858896 wait complete. 2023-11-28T12:02:30.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.733+0000 7f3c415a9700 1 Processor -- start 2023-11-28T12:02:30.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.733+0000 7f3c415a9700 1 -- start start 2023-11-28T12:02:30.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c415a9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c0af0a0 0x7f3c3c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c415a9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c3c0b1650 0x7f3c3c0c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c415a9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c3c0b5130 0x7f3c3c0c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c415a9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c0c5970 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c415a9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c0c5d90 con 0x7f3c3c0b1650 2023-11-28T12:02:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c415a9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c159420 con 0x7f3c3c0b5130 2023-11-28T12:02:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c3affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c0af0a0 0x7f3c3c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c3affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c0af0a0 0x7f3c3c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53820/0 (socket says 172.21.15.57:53820) 2023-11-28T12:02:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c3affd700 1 -- 172.21.15.57:0/1518175504 learned_addr learned my addr 172.21.15.57:0/1518175504 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c3a7fc700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c3c0b1650 0x7f3c3c0c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c3b7fe700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c3c0b5130 0x7f3c3c0c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c3affd700 1 -- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c3c0b5130 msgr2=0x7f3c3c0c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c3affd700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c3c0b5130 0x7f3c3c0c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c3affd700 1 -- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c3c0b1650 msgr2=0x7f3c3c0c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c3affd700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c3c0b1650 0x7f3c3c0c4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c3affd700 1 -- 172.21.15.57:0/1518175504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c240094a0 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c3b7fe700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c3c0b5130 0x7f3c3c0c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:30.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.734+0000 7f3c3a7fc700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c3c0b1650 0x7f3c3c0c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:30.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.735+0000 7f3c3affd700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c0af0a0 0x7f3c3c0b4bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0708b0 tx=0x7f3c24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:30.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.735+0000 7f3c337fe700 1 -- 172.21.15.57:0/1518175504 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c240045f0 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.735+0000 7f3c337fe700 1 -- 172.21.15.57:0/1518175504 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2401f070 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.735+0000 7f3c415a9700 1 -- 172.21.15.57:0/1518175504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3c15de60 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.735+0000 7f3c337fe700 1 -- 172.21.15.57:0/1518175504 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24003b00 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.735+0000 7f3c415a9700 1 -- 172.21.15.57:0/1518175504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c15db80 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.736+0000 7f3c415a9700 1 -- 172.21.15.57:0/1518175504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3c0743f0 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.736+0000 7f3c337fe700 1 -- 172.21.15.57:0/1518175504 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3c2400f050 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.737+0000 7f3c337fe700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3c1c0682f0 0x7f3c1c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:30.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.737+0000 7f3c337fe700 1 -- 172.21.15.57:0/1518175504 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3c24021070 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.737+0000 7f3c3a7fc700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3c1c0682f0 0x7f3c1c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:30.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.738+0000 7f3c3a7fc700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3c1c0682f0 0x7f3c1c06a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0b2240 tx=0x7f3c2c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:30.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.742+0000 7f3c337fe700 1 -- 172.21.15.57:0/1518175504 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3c24053fc0 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.938+0000 7f3c415a9700 1 -- 172.21.15.57:0/1518175504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3c3c06cd10 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.939+0000 7f3c337fe700 1 -- 172.21.15.57:0/1518175504 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3c240575e0 con 0x7f3c3c0af0a0 2023-11-28T12:02:30.940 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:02:30.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.941+0000 7f3c415a9700 1 -- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3c1c0682f0 msgr2=0x7f3c1c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.941+0000 7f3c415a9700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3c1c0682f0 0x7f3c1c06a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0b2240 tx=0x7f3c2c009040 comp rx=0 tx=0).stop 2023-11-28T12:02:30.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.941+0000 7f3c415a9700 1 -- 172.21.15.57:0/1518175504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c0af0a0 msgr2=0x7f3c3c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:30.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.941+0000 7f3c415a9700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c0af0a0 0x7f3c3c0b4bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0708b0 tx=0x7f3c24004750 comp rx=0 tx=0).stop 2023-11-28T12:02:30.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.941+0000 7f3c415a9700 1 -- 172.21.15.57:0/1518175504 shutdown_connections 2023-11-28T12:02:30.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.941+0000 7f3c415a9700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3c1c0682f0 0x7f3c1c06a7a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.941+0000 7f3c415a9700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c3c0b5130 0x7f3c3c0c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.941+0000 7f3c415a9700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c0af0a0 0x7f3c3c0b4bf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.941+0000 7f3c415a9700 1 --2- 172.21.15.57:0/1518175504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c3c0b1650 0x7f3c3c0c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:30.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.941+0000 7f3c415a9700 1 -- 172.21.15.57:0/1518175504 >> 172.21.15.57:0/1518175504 conn(0x7f3c3c06c010 msgr2=0x7f3c3c070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:30.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.941+0000 7f3c415a9700 1 -- 172.21.15.57:0/1518175504 shutdown_connections 2023-11-28T12:02:30.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:30.941+0000 7f3c415a9700 1 -- 172.21.15.57:0/1518175504 wait complete. 2023-11-28T12:02:30.956 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T12:02:31.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.091+0000 7f71993f3700 1 Processor -- start 2023-11-28T12:02:31.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.091+0000 7f71993f3700 1 -- start start 2023-11-28T12:02:31.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.092+0000 7f71993f3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f719415c8c0 0x7f7194160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:31.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.092+0000 7f71993f3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71940b5dc0 0x7f719406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:31.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.092+0000 7f71993f3700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f719406b050 0x7f71940d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:31.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.092+0000 7f71993f3700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f719406b4d0 con 0x7f719406b050 2023-11-28T12:02:31.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.092+0000 7f71993f3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71940d2d30 con 0x7f719415c8c0 2023-11-28T12:02:31.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.092+0000 7f71993f3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71940d0b40 con 0x7f71940b5dc0 2023-11-28T12:02:31.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.092+0000 7f71937fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f719406b050 0x7f71940d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:31.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.092+0000 7f71937fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f719406b050 0x7f71940d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53830/0 (socket says 172.21.15.57:53830) 2023-11-28T12:02:31.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.092+0000 7f71937fe700 1 -- 172.21.15.57:0/3216275995 learned_addr learned my addr 172.21.15.57:0/3216275995 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:31.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.092+0000 7f71927fc700 1 --2- 172.21.15.57:0/3216275995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71940b5dc0 0x7f719406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:31.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.092+0000 7f7192ffd700 1 --2- 172.21.15.57:0/3216275995 >> v2:172.21.15.181:3300/0 conn(0x7f719415c8c0 0x7f7194160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:31.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.093+0000 7f71937fe700 1 -- 172.21.15.57:0/3216275995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71940b5dc0 msgr2=0x7f719406ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:31.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.093+0000 7f71937fe700 1 --2- 172.21.15.57:0/3216275995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71940b5dc0 0x7f719406ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.093+0000 7f71937fe700 1 -- 172.21.15.57:0/3216275995 >> v2:172.21.15.181:3300/0 conn(0x7f719415c8c0 msgr2=0x7f7194160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.093+0000 7f71937fe700 1 --2- 172.21.15.57:0/3216275995 >> v2:172.21.15.181:3300/0 conn(0x7f719415c8c0 0x7f7194160cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.093+0000 7f71937fe700 1 -- 172.21.15.57:0/3216275995 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71941611f0 con 0x7f719406b050 2023-11-28T12:02:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.093+0000 7f71937fe700 1 --2- 172.21.15.57:0/3216275995 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f719406b050 0x7f71940d0fd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f71940d2ed0 tx=0x7f718c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55fdb85fb0ed490 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.093+0000 7f7191ffb700 1 -- 172.21.15.57:0/3216275995 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c00e070 con 0x7f719406b050 2023-11-28T12:02:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.093+0000 7f7191ffb700 1 -- 172.21.15.57:0/3216275995 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718c00b910 con 0x7f719406b050 2023-11-28T12:02:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.093+0000 7f7191ffb700 1 -- 172.21.15.57:0/3216275995 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c00bcc0 con 0x7f719406b050 2023-11-28T12:02:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.094+0000 7f71993f3700 1 -- 172.21.15.57:0/3216275995 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f719406b050 msgr2=0x7f71940d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:31.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.094+0000 7f71993f3700 1 --2- 172.21.15.57:0/3216275995 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f719406b050 0x7f71940d0fd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f71940d2ed0 tx=0x7f718c00b330 comp rx=0 tx=0).stop 2023-11-28T12:02:31.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.094+0000 7f71993f3700 1 -- 172.21.15.57:0/3216275995 shutdown_connections 2023-11-28T12:02:31.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.094+0000 7f71993f3700 1 --2- 172.21.15.57:0/3216275995 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f719406b050 0x7f71940d0fd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.094+0000 7f71993f3700 1 --2- 172.21.15.57:0/3216275995 >> v2:172.21.15.181:3300/0 conn(0x7f719415c8c0 0x7f7194160cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.094+0000 7f71993f3700 1 --2- 172.21.15.57:0/3216275995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71940b5dc0 0x7f719406ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.094+0000 7f71993f3700 1 -- 172.21.15.57:0/3216275995 >> 172.21.15.57:0/3216275995 conn(0x7f719406c520 msgr2=0x7f719406c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:31.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.094+0000 7f71993f3700 1 -- 172.21.15.57:0/3216275995 shutdown_connections 2023-11-28T12:02:31.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.094+0000 7f71993f3700 1 -- 172.21.15.57:0/3216275995 wait complete. 2023-11-28T12:02:31.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.095+0000 7f71993f3700 1 Processor -- start 2023-11-28T12:02:31.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.095+0000 7f71993f3700 1 -- start start 2023-11-28T12:02:31.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.096+0000 7f71993f3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f719406b050 0x7f71941f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:31.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.096+0000 7f71993f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71940b5dc0 0x7f71940cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:31.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.096+0000 7f71993f3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f719415c8c0 0x7f71940cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:31.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.096+0000 7f71993f3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71940ca770 con 0x7f719406b050 2023-11-28T12:02:31.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.096+0000 7f71993f3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71940b6d50 con 0x7f71940b5dc0 2023-11-28T12:02:31.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.096+0000 7f71993f3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71940ccea0 con 0x7f719415c8c0 2023-11-28T12:02:31.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.096+0000 7f7192ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f719406b050 0x7f71941f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:31.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.096+0000 7f7192ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f719406b050 0x7f71941f3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53842/0 (socket says 172.21.15.57:53842) 2023-11-28T12:02:31.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.096+0000 7f7192ffd700 1 -- 172.21.15.57:0/4158589510 learned_addr learned my addr 172.21.15.57:0/4158589510 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:31.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.096+0000 7f71937fe700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f719415c8c0 0x7f71940cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:31.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.096+0000 7f71927fc700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71940b5dc0 0x7f71940cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:31.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.096+0000 7f7192ffd700 1 -- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f719415c8c0 msgr2=0x7f71940cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:31.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.097+0000 7f7192ffd700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f719415c8c0 0x7f71940cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.097+0000 7f7192ffd700 1 -- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71940b5dc0 msgr2=0x7f71940cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:31.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.097+0000 7f7192ffd700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71940b5dc0 0x7f71940cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.097+0000 7f7192ffd700 1 -- 172.21.15.57:0/4158589510 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c009b30 con 0x7f719406b050 2023-11-28T12:02:31.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.097+0000 7f71937fe700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f719415c8c0 0x7f71940cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.097+0000 7f71927fc700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71940b5dc0 0x7f71940cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.097+0000 7f7192ffd700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f719406b050 0x7f71941f3210 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f717c00e870 tx=0x7f717c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.097+0000 7f718b7fe700 1 -- 172.21.15.57:0/4158589510 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c009590 con 0x7f719406b050 2023-11-28T12:02:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.097+0000 7f718b7fe700 1 -- 172.21.15.57:0/4158589510 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c00ee70 con 0x7f719406b050 2023-11-28T12:02:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.097+0000 7f71993f3700 1 -- 172.21.15.57:0/4158589510 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71940cd760 con 0x7f719406b050 2023-11-28T12:02:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.097+0000 7f718b7fe700 1 -- 172.21.15.57:0/4158589510 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c00ca50 con 0x7f719406b050 2023-11-28T12:02:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.097+0000 7f71993f3700 1 -- 172.21.15.57:0/4158589510 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71940d2a50 con 0x7f719406b050 2023-11-28T12:02:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.098+0000 7f71993f3700 1 -- 172.21.15.57:0/4158589510 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71940cda40 con 0x7f719406b050 2023-11-28T12:02:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.099+0000 7f718b7fe700 1 -- 172.21.15.57:0/4158589510 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f717c019030 con 0x7f719406b050 2023-11-28T12:02:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.099+0000 7f718b7fe700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f71740682f0 0x7f717406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.099+0000 7f718b7fe700 1 -- 172.21.15.57:0/4158589510 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f717c085ad0 con 0x7f719406b050 2023-11-28T12:02:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.099+0000 7f71927fc700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f71740682f0 0x7f717406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.100+0000 7f71927fc700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f71740682f0 0x7f717406a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f71940cd240 tx=0x7f7184008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.103+0000 7f718b7fe700 1 -- 172.21.15.57:0/4158589510 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f717c052330 con 0x7f719406b050 2023-11-28T12:02:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.321+0000 7f71993f3700 1 -- 172.21.15.57:0/4158589510 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7194067d70 con 0x7f719406b050 2023-11-28T12:02:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.321+0000 7f718b7fe700 1 -- 172.21.15.57:0/4158589510 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f717c021020 con 0x7f719406b050 2023-11-28T12:02:31.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.324+0000 7f71993f3700 1 -- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f71740682f0 msgr2=0x7f717406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:31.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.324+0000 7f71993f3700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f71740682f0 0x7f717406a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f71940cd240 tx=0x7f7184008040 comp rx=0 tx=0).stop 2023-11-28T12:02:31.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.324+0000 7f71993f3700 1 -- 172.21.15.57:0/4158589510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f719406b050 msgr2=0x7f71941f3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:31.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.324+0000 7f71993f3700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f719406b050 0x7f71941f3210 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f717c00e870 tx=0x7f717c00ec30 comp rx=0 tx=0).stop 2023-11-28T12:02:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.324+0000 7f71993f3700 1 -- 172.21.15.57:0/4158589510 shutdown_connections 2023-11-28T12:02:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.324+0000 7f71993f3700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f71740682f0 0x7f717406a7a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.324+0000 7f71993f3700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f719415c8c0 0x7f71940cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.324+0000 7f71993f3700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f719406b050 0x7f71941f3210 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.324+0000 7f71993f3700 1 --2- 172.21.15.57:0/4158589510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71940b5dc0 0x7f71940cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.324+0000 7f71993f3700 1 -- 172.21.15.57:0/4158589510 >> 172.21.15.57:0/4158589510 conn(0x7f719406c520 msgr2=0x7f7194156e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.324+0000 7f71993f3700 1 -- 172.21.15.57:0/4158589510 shutdown_connections 2023-11-28T12:02:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.324+0000 7f71993f3700 1 -- 172.21.15.57:0/4158589510 wait complete. 2023-11-28T12:02:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:listed 0 entries 2023-11-28T12:02:31.339 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2023-11-28T12:02:31.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:31.339 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-28T12:02:31.377 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:02:31.377 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:02:31.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:31.378 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T12:02:31.440 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:02:31.440 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:02:31.463 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:02:31.464 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:02:31.526 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:31.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.669+0000 7fa333c33700 1 Processor -- start 2023-11-28T12:02:31.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.669+0000 7fa333c33700 1 -- start start 2023-11-28T12:02:31.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.670+0000 7fa333c33700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa32c0ce040 0x7fa32c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:31.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.670+0000 7fa333c33700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa32c0b1770 0x7fa32c0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:31.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.670+0000 7fa333c33700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32c13fc90 con 0x7fa32c142db0 2023-11-28T12:02:31.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.670+0000 7fa333c33700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32c0b2130 con 0x7fa32c0b1770 2023-11-28T12:02:31.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.670+0000 7fa333c33700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c13f480 con 0x7fa32c0ce040 2023-11-28T12:02:31.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.670+0000 7fa3321d0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa32c0b1770 0x7fa32c0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:31.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.670+0000 7fa3311ce700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa32c142db0 0x7fa32c13ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40292/0 (socket says 172.21.15.57:40292) 2023-11-28T12:02:31.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.670+0000 7fa3311ce700 1 -- 172.21.15.57:0/1461690593 learned_addr learned my addr 172.21.15.57:0/1461690593 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:31.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.671+0000 7fa3319cf700 1 --2- 172.21.15.57:0/1461690593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa32c0ce040 0x7fa32c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:31.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.671+0000 7fa3321d0700 1 -- 172.21.15.57:0/1461690593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa32c0ce040 msgr2=0x7fa32c15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:31.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.671+0000 7fa3321d0700 1 --2- 172.21.15.57:0/1461690593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa32c0ce040 0x7fa32c15d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.671+0000 7fa3321d0700 1 -- 172.21.15.57:0/1461690593 >> v1:172.21.15.57:6789/0 conn(0x7fa32c142db0 legacy=0x7fa32c13ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:31.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.671+0000 7fa3321d0700 1 -- 172.21.15.57:0/1461690593 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c13f6d0 con 0x7fa32c0b1770 2023-11-28T12:02:31.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.671+0000 7fa3321d0700 1 --2- 172.21.15.57:0/1461690593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa32c0b1770 0x7fa32c0b1bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa32400bfa0 tx=0x7fa32400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2cc70dc4d1255b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:31.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.671+0000 7fa3309cd700 1 -- 172.21.15.57:0/1461690593 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324010070 con 0x7fa32c0b1770 2023-11-28T12:02:31.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.671+0000 7fa3309cd700 1 -- 172.21.15.57:0/1461690593 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32400b2e0 con 0x7fa32c0b1770 2023-11-28T12:02:31.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.671+0000 7fa3309cd700 1 -- 172.21.15.57:0/1461690593 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32400dc40 con 0x7fa32c0b1770 2023-11-28T12:02:31.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.672+0000 7fa333c33700 1 -- 172.21.15.57:0/1461690593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa32c0b1770 msgr2=0x7fa32c0b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:31.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.672+0000 7fa333c33700 1 --2- 172.21.15.57:0/1461690593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa32c0b1770 0x7fa32c0b1bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa32400bfa0 tx=0x7fa32400d330 comp rx=0 tx=0).stop 2023-11-28T12:02:31.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.672+0000 7fa333c33700 1 -- 172.21.15.57:0/1461690593 shutdown_connections 2023-11-28T12:02:31.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.672+0000 7fa333c33700 1 --2- 172.21.15.57:0/1461690593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa32c0b1770 0x7fa32c0b1bf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.672+0000 7fa333c33700 1 --2- 172.21.15.57:0/1461690593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa32c0ce040 0x7fa32c15d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.672+0000 7fa333c33700 1 -- 172.21.15.57:0/1461690593 >> 172.21.15.57:0/1461690593 conn(0x7fa32c06c010 msgr2=0x7fa32c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:31.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.672+0000 7fa333c33700 1 -- 172.21.15.57:0/1461690593 shutdown_connections 2023-11-28T12:02:31.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.672+0000 7fa333c33700 1 -- 172.21.15.57:0/1461690593 wait complete. 2023-11-28T12:02:31.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.673+0000 7fa333c33700 1 Processor -- start 2023-11-28T12:02:31.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.673+0000 7fa333c33700 1 -- start start 2023-11-28T12:02:31.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.674+0000 7fa333c33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa32c0b1770 0x7fa32c0c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:31.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.674+0000 7fa333c33700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa32c0ce040 0x7fa32c1f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:31.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.674+0000 7fa333c33700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa32c142db0 0x7fa32c0c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:31.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.674+0000 7fa333c33700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0c60f0 con 0x7fa32c0ce040 2023-11-28T12:02:31.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.674+0000 7fa333c33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0c54e0 con 0x7fa32c0b1770 2023-11-28T12:02:31.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.674+0000 7fa333c33700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0c57c0 con 0x7fa32c142db0 2023-11-28T12:02:31.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.674+0000 7fa3311ce700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa32c0ce040 0x7fa32c1f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:31.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.674+0000 7fa3311ce700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa32c0ce040 0x7fa32c1f1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53864/0 (socket says 172.21.15.57:53864) 2023-11-28T12:02:31.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.674+0000 7fa3311ce700 1 -- 172.21.15.57:0/1838875300 learned_addr learned my addr 172.21.15.57:0/1838875300 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:31.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.674+0000 7fa3321d0700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa32c142db0 0x7fa32c0c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:31.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.674+0000 7fa3319cf700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa32c0b1770 0x7fa32c0c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:31.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.674+0000 7fa3311ce700 1 -- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa32c142db0 msgr2=0x7fa32c0c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:31.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.675+0000 7fa3311ce700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa32c142db0 0x7fa32c0c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.675+0000 7fa3311ce700 1 -- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa32c0b1770 msgr2=0x7fa32c0c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:31.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.675+0000 7fa3311ce700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa32c0b1770 0x7fa32c0c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.675+0000 7fa3311ce700 1 -- 172.21.15.57:0/1838875300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32400bc80 con 0x7fa32c0ce040 2023-11-28T12:02:31.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.675+0000 7fa3311ce700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa32c0ce040 0x7fa32c1f1da0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa32000c520 tx=0x7fa32000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:31.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.675+0000 7fa31e7fc700 1 -- 172.21.15.57:0/1838875300 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320005d70 con 0x7fa32c0ce040 2023-11-28T12:02:31.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.675+0000 7fa31e7fc700 1 -- 172.21.15.57:0/1838875300 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3200070d0 con 0x7fa32c0ce040 2023-11-28T12:02:31.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.675+0000 7fa333c33700 1 -- 172.21.15.57:0/1838875300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c15d100 con 0x7fa32c0ce040 2023-11-28T12:02:31.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.675+0000 7fa31e7fc700 1 -- 172.21.15.57:0/1838875300 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320007440 con 0x7fa32c0ce040 2023-11-28T12:02:31.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.675+0000 7fa333c33700 1 -- 172.21.15.57:0/1838875300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c13f9b0 con 0x7fa32c0ce040 2023-11-28T12:02:31.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.676+0000 7fa333c33700 1 -- 172.21.15.57:0/1838875300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32c15d3e0 con 0x7fa32c0ce040 2023-11-28T12:02:31.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.676+0000 7fa31e7fc700 1 -- 172.21.15.57:0/1838875300 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa320019030 con 0x7fa32c0ce040 2023-11-28T12:02:31.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.677+0000 7fa31e7fc700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa3100682f0 0x7fa31006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:31.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.677+0000 7fa31e7fc700 1 -- 172.21.15.57:0/1838875300 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa320084ef0 con 0x7fa32c0ce040 2023-11-28T12:02:31.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.677+0000 7fa3319cf700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa3100682f0 0x7fa31006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:31.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.678+0000 7fa3319cf700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa3100682f0 0x7fa31006a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa328009be0 tx=0x7fa328009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:31.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.682+0000 7fa31e7fc700 1 -- 172.21.15.57:0/1838875300 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa32004b5e0 con 0x7fa32c0ce040 2023-11-28T12:02:31.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.890+0000 7fa333c33700 1 -- 172.21.15.57:0/1838875300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa32c0743f0 con 0x7fa32c0ce040 2023-11-28T12:02:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.891+0000 7fa31e7fc700 1 -- 172.21.15.57:0/1838875300 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17945 (secure 0 0 0) 0x7fa320017090 con 0x7fa32c0ce040 2023-11-28T12:02:31.893 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:31.893 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":22,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:02:31.840283+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:31.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.893+0000 7fa333c33700 1 -- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa3100682f0 msgr2=0x7fa31006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:31.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.893+0000 7fa333c33700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa3100682f0 0x7fa31006a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa328009be0 tx=0x7fa328009210 comp rx=0 tx=0).stop 2023-11-28T12:02:31.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.893+0000 7fa333c33700 1 -- 172.21.15.57:0/1838875300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa32c0ce040 msgr2=0x7fa32c1f1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:31.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.893+0000 7fa333c33700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa32c0ce040 0x7fa32c1f1da0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa32000c520 tx=0x7fa32000c830 comp rx=0 tx=0).stop 2023-11-28T12:02:31.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.894+0000 7fa333c33700 1 -- 172.21.15.57:0/1838875300 shutdown_connections 2023-11-28T12:02:31.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.894+0000 7fa333c33700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa3100682f0 0x7fa31006a7a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.894+0000 7fa333c33700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa32c142db0 0x7fa32c0c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.894+0000 7fa333c33700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa32c0b1770 0x7fa32c0c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.894+0000 7fa333c33700 1 --2- 172.21.15.57:0/1838875300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa32c0ce040 0x7fa32c1f1da0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:31.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.894+0000 7fa333c33700 1 -- 172.21.15.57:0/1838875300 >> 172.21.15.57:0/1838875300 conn(0x7fa32c06c010 msgr2=0x7fa32c0cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:31.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.894+0000 7fa333c33700 1 -- 172.21.15.57:0/1838875300 shutdown_connections 2023-11-28T12:02:31.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:31.894+0000 7fa333c33700 1 -- 172.21.15.57:0/1838875300 wait complete. 2023-11-28T12:02:31.909 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:31.910 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:32.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.049+0000 7f8df3ebc700 1 Processor -- start 2023-11-28T12:02:32.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.049+0000 7f8df3ebc700 1 -- start start 2023-11-28T12:02:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.050+0000 7f8df3ebc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8dec0b1650 0x7f8dec0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.050+0000 7f8df3ebc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8dec1599b0 0x7f8dec15d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.050+0000 7f8df3ebc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dec0af0a0 0x7f8dec15a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.050+0000 7f8df3ebc700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dec159ec0 con 0x7f8dec0b1650 2023-11-28T12:02:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.050+0000 7f8df3ebc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dec0b1f60 con 0x7f8dec1599b0 2023-11-28T12:02:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.050+0000 7f8df3ebc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec15d8a0 con 0x7f8dec0af0a0 2023-11-28T12:02:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.050+0000 7f8df1c58700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8dec0b1650 0x7f8dec0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.050+0000 7f8df1c58700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8dec0b1650 0x7f8dec0b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53868/0 (socket says 172.21.15.57:53868) 2023-11-28T12:02:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.050+0000 7f8df1c58700 1 -- 172.21.15.57:0/27836560 learned_addr learned my addr 172.21.15.57:0/27836560 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:32.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.050+0000 7f8df1457700 1 --2- 172.21.15.57:0/27836560 >> v2:172.21.15.181:3300/0 conn(0x7f8dec1599b0 0x7f8dec15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.050+0000 7f8df2459700 1 --2- 172.21.15.57:0/27836560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dec0af0a0 0x7f8dec15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.050+0000 7f8df1c58700 1 -- 172.21.15.57:0/27836560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dec0af0a0 msgr2=0x7f8dec15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:32.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.050+0000 7f8df1c58700 1 --2- 172.21.15.57:0/27836560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dec0af0a0 0x7f8dec15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.051+0000 7f8df1c58700 1 -- 172.21.15.57:0/27836560 >> v2:172.21.15.181:3300/0 conn(0x7f8dec1599b0 msgr2=0x7f8dec15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:32.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.051+0000 7f8df1c58700 1 --2- 172.21.15.57:0/27836560 >> v2:172.21.15.181:3300/0 conn(0x7f8dec1599b0 0x7f8dec15d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.051+0000 7f8df1c58700 1 -- 172.21.15.57:0/27836560 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec15db80 con 0x7f8dec0b1650 2023-11-28T12:02:32.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.051+0000 7f8df1c58700 1 --2- 172.21.15.57:0/27836560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dec0b1650 0x7f8dec0b1a20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8dec06c730 tx=0x7f8de800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e784dcef697a2b59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:32.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.051+0000 7f8df0c56700 1 -- 172.21.15.57:0/27836560 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de800dd30 con 0x7f8dec0b1650 2023-11-28T12:02:32.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.051+0000 7f8df0c56700 1 -- 172.21.15.57:0/27836560 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de800de90 con 0x7f8dec0b1650 2023-11-28T12:02:32.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.051+0000 7f8df0c56700 1 -- 172.21.15.57:0/27836560 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de8010690 con 0x7f8dec0b1650 2023-11-28T12:02:32.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.051+0000 7f8df3ebc700 1 -- 172.21.15.57:0/27836560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dec0b1650 msgr2=0x7f8dec0b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.051+0000 7f8df3ebc700 1 --2- 172.21.15.57:0/27836560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dec0b1650 0x7f8dec0b1a20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8dec06c730 tx=0x7f8de800be70 comp rx=0 tx=0).stop 2023-11-28T12:02:32.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.052+0000 7f8df3ebc700 1 -- 172.21.15.57:0/27836560 shutdown_connections 2023-11-28T12:02:32.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.052+0000 7f8df3ebc700 1 --2- 172.21.15.57:0/27836560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dec0af0a0 0x7f8dec15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.052+0000 7f8df3ebc700 1 --2- 172.21.15.57:0/27836560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dec0b1650 0x7f8dec0b1a20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.052+0000 7f8df3ebc700 1 --2- 172.21.15.57:0/27836560 >> v2:172.21.15.181:3300/0 conn(0x7f8dec1599b0 0x7f8dec15d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.052+0000 7f8df3ebc700 1 -- 172.21.15.57:0/27836560 >> 172.21.15.57:0/27836560 conn(0x7f8dec06c010 msgr2=0x7f8dec0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:32.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.052+0000 7f8df3ebc700 1 -- 172.21.15.57:0/27836560 shutdown_connections 2023-11-28T12:02:32.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.052+0000 7f8df3ebc700 1 -- 172.21.15.57:0/27836560 wait complete. 2023-11-28T12:02:32.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.053+0000 7f8df3ebc700 1 Processor -- start 2023-11-28T12:02:32.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.053+0000 7f8df3ebc700 1 -- start start 2023-11-28T12:02:32.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df3ebc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dec0af0a0 0x7f8dec14e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df3ebc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dec0b1650 0x7f8dec0b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df3ebc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dec14f130 0x7f8dec0c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df3ebc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec14ef20 con 0x7f8dec0b1650 2023-11-28T12:02:32.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df3ebc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec0c6720 con 0x7f8dec14f130 2023-11-28T12:02:32.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df3ebc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec0b3d20 con 0x7f8dec0af0a0 2023-11-28T12:02:32.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df1457700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dec0b1650 0x7f8dec0b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df1457700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dec0b1650 0x7f8dec0b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53876/0 (socket says 172.21.15.57:53876) 2023-11-28T12:02:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df1457700 1 -- 172.21.15.57:0/2907627244 learned_addr learned my addr 172.21.15.57:0/2907627244 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df2459700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dec14f130 0x7f8dec0c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df1c58700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dec0af0a0 0x7f8dec14e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df1457700 1 -- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dec0af0a0 msgr2=0x7f8dec14e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df1457700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dec0af0a0 0x7f8dec14e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df1457700 1 -- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dec14f130 msgr2=0x7f8dec0c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df1457700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dec14f130 0x7f8dec0c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df1457700 1 -- 172.21.15.57:0/2907627244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de800b7f0 con 0x7f8dec0b1650 2023-11-28T12:02:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.054+0000 7f8df1457700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dec0b1650 0x7f8dec0b4a60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8de000d830 tx=0x7f8de000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.055+0000 7f8dde7fc700 1 -- 172.21.15.57:0/2907627244 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de00125f0 con 0x7f8dec0b1650 2023-11-28T12:02:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.055+0000 7f8dde7fc700 1 -- 172.21.15.57:0/2907627244 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de0012750 con 0x7f8dec0b1650 2023-11-28T12:02:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.055+0000 7f8df3ebc700 1 -- 172.21.15.57:0/2907627244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dec15d590 con 0x7f8dec0b1650 2023-11-28T12:02:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.055+0000 7f8dde7fc700 1 -- 172.21.15.57:0/2907627244 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0012b00 con 0x7f8dec0b1650 2023-11-28T12:02:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.055+0000 7f8df3ebc700 1 -- 172.21.15.57:0/2907627244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dec15de60 con 0x7f8dec0b1650 2023-11-28T12:02:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.056+0000 7f8df3ebc700 1 -- 172.21.15.57:0/2907627244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dec15a890 con 0x7f8dec0b1650 2023-11-28T12:02:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.056+0000 7f8dde7fc700 1 -- 172.21.15.57:0/2907627244 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8de0012c60 con 0x7f8dec0b1650 2023-11-28T12:02:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.057+0000 7f8dde7fc700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8dd00682f0 0x7f8dd006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.057+0000 7f8dde7fc700 1 -- 172.21.15.57:0/2907627244 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8de0085ee0 con 0x7f8dec0b1650 2023-11-28T12:02:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.057+0000 7f8df1c58700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8dd00682f0 0x7f8dd006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.057+0000 7f8df1c58700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8dd00682f0 0x7f8dd006a7a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8de800beb0 tx=0x7f8de8012000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:32.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.061+0000 7f8dde7fc700 1 -- 172.21.15.57:0/2907627244 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8de00525b0 con 0x7f8dec0b1650 2023-11-28T12:02:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.260+0000 7f8df3ebc700 1 -- 172.21.15.57:0/2907627244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8dec0743f0 con 0x7f8dec0b1650 2023-11-28T12:02:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.261+0000 7f8dde7fc700 1 -- 172.21.15.57:0/2907627244 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8de0055bd0 con 0x7f8dec0b1650 2023-11-28T12:02:32.262 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:32.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.263+0000 7f8df3ebc700 1 -- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8dd00682f0 msgr2=0x7f8dd006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.263+0000 7f8df3ebc700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8dd00682f0 0x7f8dd006a7a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8de800beb0 tx=0x7f8de8012000 comp rx=0 tx=0).stop 2023-11-28T12:02:32.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.263+0000 7f8df3ebc700 1 -- 172.21.15.57:0/2907627244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dec0b1650 msgr2=0x7f8dec0b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.263+0000 7f8df3ebc700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dec0b1650 0x7f8dec0b4a60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8de000d830 tx=0x7f8de000db40 comp rx=0 tx=0).stop 2023-11-28T12:02:32.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.263+0000 7f8df3ebc700 1 -- 172.21.15.57:0/2907627244 shutdown_connections 2023-11-28T12:02:32.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.263+0000 7f8df3ebc700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8dd00682f0 0x7f8dd006a7a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.263+0000 7f8df3ebc700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dec14f130 0x7f8dec0c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.263+0000 7f8df3ebc700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dec0af0a0 0x7f8dec14e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.263+0000 7f8df3ebc700 1 --2- 172.21.15.57:0/2907627244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8dec0b1650 0x7f8dec0b4a60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.263+0000 7f8df3ebc700 1 -- 172.21.15.57:0/2907627244 >> 172.21.15.57:0/2907627244 conn(0x7f8dec06c010 msgr2=0x7f8dec06dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:32.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.263+0000 7f8df3ebc700 1 -- 172.21.15.57:0/2907627244 shutdown_connections 2023-11-28T12:02:32.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.263+0000 7f8df3ebc700 1 -- 172.21.15.57:0/2907627244 wait complete. 2023-11-28T12:02:32.278 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:32.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.415+0000 7f3d4b320700 1 Processor -- start 2023-11-28T12:02:32.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.415+0000 7f3d4b320700 1 -- start start 2023-11-28T12:02:32.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.416+0000 7f3d4b320700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d4406a980 0x7f3d440ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.416+0000 7f3d4b320700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4415cc10 con 0x7f3d44071890 2023-11-28T12:02:32.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.416+0000 7f3d4b320700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4406c9b0 con 0x7f3d440cf710 2023-11-28T12:02:32.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.416+0000 7f3d4b320700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d4406cc90 con 0x7f3d4406a980 2023-11-28T12:02:32.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.417+0000 7f3d498bd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f3d44071890 0x7f3d44160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40294/0 (socket says 172.21.15.57:40294) 2023-11-28T12:02:32.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.417+0000 7f3d498bd700 1 -- 172.21.15.57:0/3891509783 learned_addr learned my addr 172.21.15.57:0/3891509783 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:32.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.417+0000 7f3d488bb700 1 --2- 172.21.15.57:0/3891509783 >> v2:172.21.15.181:3300/0 conn(0x7f3d4406a980 0x7f3d440ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.417+0000 7f3d3bfff700 1 -- 172.21.15.57:0/3891509783 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887779870 0 0) 0x7f3d4415cc10 con 0x7f3d44071890 2023-11-28T12:02:32.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.417+0000 7f3d3bfff700 1 -- 172.21.15.57:0/3891509783 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d300035b0 con 0x7f3d44071890 2023-11-28T12:02:32.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.417+0000 7f3d3bfff700 1 -- 172.21.15.57:0/3891509783 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239748065 0 0) 0x7f3d4406c9b0 con 0x7f3d440cf710 2023-11-28T12:02:32.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.417+0000 7f3d3bfff700 1 -- 172.21.15.57:0/3891509783 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d4415cc10 con 0x7f3d440cf710 2023-11-28T12:02:32.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.417+0000 7f3d488bb700 1 -- 172.21.15.57:0/3891509783 >> v1:172.21.15.181:6789/0 conn(0x7f3d440cf710 legacy=0x7f3d4406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.417+0000 7f3d488bb700 1 -- 172.21.15.57:0/3891509783 >> v1:172.21.15.57:6789/0 conn(0x7f3d44071890 legacy=0x7f3d44160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.417+0000 7f3d3bfff700 1 -- 172.21.15.57:0/3891509783 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1152801313 0 0) 0x7f3d300035b0 con 0x7f3d44071890 2023-11-28T12:02:32.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.417+0000 7f3d488bb700 1 -- 172.21.15.57:0/3891509783 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4406cf70 con 0x7f3d4406a980 2023-11-28T12:02:32.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.417+0000 7f3d488bb700 1 --2- 172.21.15.57:0/3891509783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d4406a980 0x7f3d440ce740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009840 tx=0x7f3d40009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d980067a6255fe80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:32.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.418+0000 7f3d3bfff700 1 -- 172.21.15.57:0/3891509783 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0019f0 con 0x7f3d4406a980 2023-11-28T12:02:32.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.418+0000 7f3d3bfff700 1 -- 172.21.15.57:0/3891509783 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d40004030 con 0x7f3d4406a980 2023-11-28T12:02:32.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.418+0000 7f3d3bfff700 1 -- 172.21.15.57:0/3891509783 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d400043c0 con 0x7f3d4406a980 2023-11-28T12:02:32.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.418+0000 7f3d4b320700 1 -- 172.21.15.57:0/3891509783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d4406a980 msgr2=0x7f3d440ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.418+0000 7f3d4b320700 1 --2- 172.21.15.57:0/3891509783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d4406a980 0x7f3d440ce740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009840 tx=0x7f3d40009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:32.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.418+0000 7f3d4b320700 1 -- 172.21.15.57:0/3891509783 shutdown_connections 2023-11-28T12:02:32.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.418+0000 7f3d4b320700 1 --2- 172.21.15.57:0/3891509783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d4406a980 0x7f3d440ce740 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.418+0000 7f3d4b320700 1 -- 172.21.15.57:0/3891509783 >> 172.21.15.57:0/3891509783 conn(0x7f3d4406c010 msgr2=0x7f3d440cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:32.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.418+0000 7f3d4b320700 1 -- 172.21.15.57:0/3891509783 shutdown_connections 2023-11-28T12:02:32.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.418+0000 7f3d4b320700 1 -- 172.21.15.57:0/3891509783 wait complete. 2023-11-28T12:02:32.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.420+0000 7f3d4b320700 1 Processor -- start 2023-11-28T12:02:32.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.420+0000 7f3d4b320700 1 -- start start 2023-11-28T12:02:32.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.421+0000 7f3d4b320700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d4406a980 0x7f3d44151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.421+0000 7f3d4b320700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d44071890 0x7f3d440b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.421+0000 7f3d4b320700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d440cf710 0x7f3d441f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.421+0000 7f3d4b320700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d440c5a20 con 0x7f3d4406a980 2023-11-28T12:02:32.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.421+0000 7f3d4b320700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d440c5740 con 0x7f3d44071890 2023-11-28T12:02:32.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.421+0000 7f3d4b320700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d440af810 con 0x7f3d440cf710 2023-11-28T12:02:32.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.421+0000 7f3d490bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d4406a980 0x7f3d44151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.421+0000 7f3d490bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d4406a980 0x7f3d44151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53892/0 (socket says 172.21.15.57:53892) 2023-11-28T12:02:32.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.421+0000 7f3d490bc700 1 -- 172.21.15.57:0/2929159666 learned_addr learned my addr 172.21.15.57:0/2929159666 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:32.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.421+0000 7f3d488bb700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d44071890 0x7f3d440b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.422+0000 7f3d498bd700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d440cf710 0x7f3d441f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.422+0000 7f3d490bc700 1 -- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d440cf710 msgr2=0x7f3d441f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.422+0000 7f3d490bc700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d440cf710 0x7f3d441f22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.422+0000 7f3d490bc700 1 -- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d44071890 msgr2=0x7f3d440b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.422+0000 7f3d490bc700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d44071890 0x7f3d440b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.422+0000 7f3d490bc700 1 -- 172.21.15.57:0/2929159666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d400094a0 con 0x7f3d4406a980 2023-11-28T12:02:32.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.422+0000 7f3d488bb700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d44071890 0x7f3d440b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:32.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.422+0000 7f3d490bc700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d4406a980 0x7f3d44151430 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c500 tx=0x7f3d3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:32.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.422+0000 7f3d39ffb700 1 -- 172.21.15.57:0/2929159666 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d340052d0 con 0x7f3d4406a980 2023-11-28T12:02:32.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.422+0000 7f3d39ffb700 1 -- 172.21.15.57:0/2929159666 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d34005450 con 0x7f3d4406a980 2023-11-28T12:02:32.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.422+0000 7f3d4b320700 1 -- 172.21.15.57:0/2929159666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4406d530 con 0x7f3d4406a980 2023-11-28T12:02:32.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.422+0000 7f3d39ffb700 1 -- 172.21.15.57:0/2929159666 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d340038c0 con 0x7f3d4406a980 2023-11-28T12:02:32.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.422+0000 7f3d4b320700 1 -- 172.21.15.57:0/2929159666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4406d250 con 0x7f3d4406a980 2023-11-28T12:02:32.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.423+0000 7f3d4b320700 1 -- 172.21.15.57:0/2929159666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d440743f0 con 0x7f3d4406a980 2023-11-28T12:02:32.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.424+0000 7f3d39ffb700 1 -- 172.21.15.57:0/2929159666 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3d34019030 con 0x7f3d4406a980 2023-11-28T12:02:32.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.424+0000 7f3d39ffb700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d280682f0 0x7f3d2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.424+0000 7f3d39ffb700 1 -- 172.21.15.57:0/2929159666 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3d34085080 con 0x7f3d4406a980 2023-11-28T12:02:32.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.424+0000 7f3d488bb700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d280682f0 0x7f3d2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.425+0000 7f3d488bb700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d280682f0 0x7f3d2806a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3d400090b0 tx=0x7f3d40008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:32.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.428+0000 7f3d39ffb700 1 -- 172.21.15.57:0/2929159666 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3d340518a0 con 0x7f3d4406a980 2023-11-28T12:02:32.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.624+0000 7f3d4b320700 1 -- 172.21.15.57:0/2929159666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d440711a0 con 0x7f3d4406a980 2023-11-28T12:02:32.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.625+0000 7f3d39ffb700 1 -- 172.21.15.57:0/2929159666 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d34054ec0 con 0x7f3d4406a980 2023-11-28T12:02:32.627 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:32.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.627+0000 7f3d4b320700 1 -- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d280682f0 msgr2=0x7f3d2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.627+0000 7f3d4b320700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d280682f0 0x7f3d2806a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3d400090b0 tx=0x7f3d40008030 comp rx=0 tx=0).stop 2023-11-28T12:02:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.627+0000 7f3d4b320700 1 -- 172.21.15.57:0/2929159666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d4406a980 msgr2=0x7f3d44151430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.627+0000 7f3d4b320700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d4406a980 0x7f3d44151430 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c500 tx=0x7f3d3400c8c0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.627+0000 7f3d4b320700 1 -- 172.21.15.57:0/2929159666 shutdown_connections 2023-11-28T12:02:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.627+0000 7f3d4b320700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d280682f0 0x7f3d2806a7a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.627+0000 7f3d4b320700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d440cf710 0x7f3d441f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.627+0000 7f3d4b320700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d4406a980 0x7f3d44151430 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.627+0000 7f3d4b320700 1 --2- 172.21.15.57:0/2929159666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d44071890 0x7f3d440b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.627+0000 7f3d4b320700 1 -- 172.21.15.57:0/2929159666 >> 172.21.15.57:0/2929159666 conn(0x7f3d4406c010 msgr2=0x7f3d440b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.627+0000 7f3d4b320700 1 -- 172.21.15.57:0/2929159666 shutdown_connections 2023-11-28T12:02:32.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.627+0000 7f3d4b320700 1 -- 172.21.15.57:0/2929159666 wait complete. 2023-11-28T12:02:32.642 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:02:32.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.783+0000 7f0122835700 1 Processor -- start 2023-11-28T12:02:32.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.783+0000 7f0122835700 1 -- start start 2023-11-28T12:02:32.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.784+0000 7f0122835700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f011c15fe50 0x7f011c06d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.784+0000 7f0122835700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f011c0b3b80 0x7f011c160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.784+0000 7f0122835700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011c161370 con 0x7f011c0b27d0 2023-11-28T12:02:32.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.784+0000 7f0122835700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011c160220 con 0x7f011c15fe50 2023-11-28T12:02:32.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.784+0000 7f0122835700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011c0cf700 con 0x7f011c0b3b80 2023-11-28T12:02:32.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.784+0000 7f011bfff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f011c15fe50 0x7f011c06d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.784+0000 7f011bfff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f011c15fe50 0x7f011c06d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53906/0 (socket says 172.21.15.57:53906) 2023-11-28T12:02:32.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.784+0000 7f011bfff700 1 -- 172.21.15.57:0/2395404270 learned_addr learned my addr 172.21.15.57:0/2395404270 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:32.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.785+0000 7f011b7fe700 1 --2- 172.21.15.57:0/2395404270 >> v2:172.21.15.181:3300/0 conn(0x7f011c0b3b80 0x7f011c160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.785+0000 7f011bfff700 1 -- 172.21.15.57:0/2395404270 >> v2:172.21.15.181:3300/0 conn(0x7f011c0b3b80 msgr2=0x7f011c160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:32.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.785+0000 7f011bfff700 1 --2- 172.21.15.57:0/2395404270 >> v2:172.21.15.181:3300/0 conn(0x7f011c0b3b80 0x7f011c160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.785+0000 7f011bfff700 1 -- 172.21.15.57:0/2395404270 >> v1:172.21.15.181:6789/0 conn(0x7f011c0b27d0 legacy=0x7f011c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:32.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.785+0000 7f011bfff700 1 -- 172.21.15.57:0/2395404270 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c0b3fd0 con 0x7f011c15fe50 2023-11-28T12:02:32.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.785+0000 7f011bfff700 1 --2- 172.21.15.57:0/2395404270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f011c15fe50 0x7f011c06d170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f011c06f350 tx=0x7f010c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75c4503f8006c8d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:32.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.785+0000 7f011affd700 1 -- 172.21.15.57:0/2395404270 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c004230 con 0x7f011c15fe50 2023-11-28T12:02:32.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.785+0000 7f011affd700 1 -- 172.21.15.57:0/2395404270 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010c004390 con 0x7f011c15fe50 2023-11-28T12:02:32.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.785+0000 7f011affd700 1 -- 172.21.15.57:0/2395404270 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c0036a0 con 0x7f011c15fe50 2023-11-28T12:02:32.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.786+0000 7f0122835700 1 -- 172.21.15.57:0/2395404270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f011c15fe50 msgr2=0x7f011c06d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.786+0000 7f0122835700 1 --2- 172.21.15.57:0/2395404270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f011c15fe50 0x7f011c06d170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f011c06f350 tx=0x7f010c009bd0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.786+0000 7f0122835700 1 -- 172.21.15.57:0/2395404270 shutdown_connections 2023-11-28T12:02:32.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.786+0000 7f0122835700 1 --2- 172.21.15.57:0/2395404270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f011c15fe50 0x7f011c06d170 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.786+0000 7f0122835700 1 --2- 172.21.15.57:0/2395404270 >> v2:172.21.15.181:3300/0 conn(0x7f011c0b3b80 0x7f011c160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.786+0000 7f0122835700 1 -- 172.21.15.57:0/2395404270 >> 172.21.15.57:0/2395404270 conn(0x7f011c06ea80 msgr2=0x7f011c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:32.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.786+0000 7f0122835700 1 -- 172.21.15.57:0/2395404270 shutdown_connections 2023-11-28T12:02:32.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.786+0000 7f0122835700 1 -- 172.21.15.57:0/2395404270 wait complete. 2023-11-28T12:02:32.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.787+0000 7f0122835700 1 Processor -- start 2023-11-28T12:02:32.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.787+0000 7f0122835700 1 -- start start 2023-11-28T12:02:32.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.788+0000 7f0122835700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f011c0b27d0 0x7f011c15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.788+0000 7f0122835700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f011c0b3b80 0x7f011c0c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.788+0000 7f0122835700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f011c15fe50 0x7f011c15a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.788+0000 7f0122835700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c0ca390 con 0x7f011c0b3b80 2023-11-28T12:02:32.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.788+0000 7f0122835700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c0c9dd0 con 0x7f011c0b27d0 2023-11-28T12:02:32.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.788+0000 7f0122835700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011c0ca0b0 con 0x7f011c15fe50 2023-11-28T12:02:32.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.788+0000 7f011b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f011c0b3b80 0x7f011c0c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.788+0000 7f011b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f011c0b3b80 0x7f011c0c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53922/0 (socket says 172.21.15.57:53922) 2023-11-28T12:02:32.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.788+0000 7f011b7fe700 1 -- 172.21.15.57:0/4099653275 learned_addr learned my addr 172.21.15.57:0/4099653275 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:32.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.788+0000 7f011bfff700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f011c0b27d0 0x7f011c15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.788+0000 7f0120dd2700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f011c15fe50 0x7f011c15a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.789+0000 7f011b7fe700 1 -- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f011c15fe50 msgr2=0x7f011c15a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.789+0000 7f011b7fe700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f011c15fe50 0x7f011c15a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.789+0000 7f011b7fe700 1 -- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f011c0b27d0 msgr2=0x7f011c15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:32.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.789+0000 7f011b7fe700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f011c0b27d0 0x7f011c15b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:32.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.789+0000 7f011b7fe700 1 -- 172.21.15.57:0/4099653275 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c009550 con 0x7f011c0b3b80 2023-11-28T12:02:32.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.789+0000 7f0120dd2700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f011c15fe50 0x7f011c15a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:32.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.789+0000 7f011b7fe700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f011c0b3b80 0x7f011c0c9420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f011000c7f0 tx=0x7f011000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:32.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.789+0000 7f0118ff9700 1 -- 172.21.15.57:0/4099653275 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110011800 con 0x7f011c0b3b80 2023-11-28T12:02:32.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.789+0000 7f0118ff9700 1 -- 172.21.15.57:0/4099653275 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0110011960 con 0x7f011c0b3b80 2023-11-28T12:02:32.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.789+0000 7f0118ff9700 1 -- 172.21.15.57:0/4099653275 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110011cd0 con 0x7f011c0b3b80 2023-11-28T12:02:32.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.789+0000 7f0122835700 1 -- 172.21.15.57:0/4099653275 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011c0cf0b0 con 0x7f011c0b3b80 2023-11-28T12:02:32.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.789+0000 7f0122835700 1 -- 172.21.15.57:0/4099653275 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011c0cedd0 con 0x7f011c0b3b80 2023-11-28T12:02:32.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.790+0000 7f0122835700 1 -- 172.21.15.57:0/4099653275 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011c0cf390 con 0x7f011c0b3b80 2023-11-28T12:02:32.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.791+0000 7f0118ff9700 1 -- 172.21.15.57:0/4099653275 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0110017030 con 0x7f011c0b3b80 2023-11-28T12:02:32.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.791+0000 7f0118ff9700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f01080682f0 0x7f010806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:32.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.791+0000 7f0118ff9700 1 -- 172.21.15.57:0/4099653275 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0110085700 con 0x7f011c0b3b80 2023-11-28T12:02:32.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.791+0000 7f011bfff700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f01080682f0 0x7f010806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:32.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.792+0000 7f011bfff700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f01080682f0 0x7f010806a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f010c009130 tx=0x7f010c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:32.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.796+0000 7f0118ff9700 1 -- 172.21.15.57:0/4099653275 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0110051ea0 con 0x7f011c0b3b80 2023-11-28T12:02:33.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.999+0000 7f0122835700 1 -- 172.21.15.57:0/4099653275 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f011c160410 con 0x7f011c0b3b80 2023-11-28T12:02:33.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:32.999+0000 7f0118ff9700 1 -- 172.21.15.57:0/4099653275 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f01100554c0 con 0x7f011c0b3b80 2023-11-28T12:02:33.001 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:02:33.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.001+0000 7f0122835700 1 -- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f01080682f0 msgr2=0x7f010806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.001+0000 7f0122835700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f01080682f0 0x7f010806a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f010c009130 tx=0x7f010c008030 comp rx=0 tx=0).stop 2023-11-28T12:02:33.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.001+0000 7f0122835700 1 -- 172.21.15.57:0/4099653275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f011c0b3b80 msgr2=0x7f011c0c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.001+0000 7f0122835700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f011c0b3b80 0x7f011c0c9420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f011000c7f0 tx=0x7f011000cb00 comp rx=0 tx=0).stop 2023-11-28T12:02:33.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.001+0000 7f0122835700 1 -- 172.21.15.57:0/4099653275 shutdown_connections 2023-11-28T12:02:33.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.001+0000 7f0122835700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f01080682f0 0x7f010806a7a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.001+0000 7f0122835700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f011c15fe50 0x7f011c15a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.001+0000 7f0122835700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f011c0b27d0 0x7f011c15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.001+0000 7f0122835700 1 --2- 172.21.15.57:0/4099653275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f011c0b3b80 0x7f011c0c9420 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.001+0000 7f0122835700 1 -- 172.21.15.57:0/4099653275 >> 172.21.15.57:0/4099653275 conn(0x7f011c06ea80 msgr2=0x7f011c0b67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:33.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.002+0000 7f0122835700 1 -- 172.21.15.57:0/4099653275 shutdown_connections 2023-11-28T12:02:33.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.002+0000 7f0122835700 1 -- 172.21.15.57:0/4099653275 wait complete. 2023-11-28T12:02:33.016 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:02:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.154+0000 7f4e0c5b0700 1 Processor -- start 2023-11-28T12:02:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.154+0000 7f4e0c5b0700 1 -- start start 2023-11-28T12:02:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.155+0000 7f4e0c5b0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e0406a980 0x7f4e040ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.155+0000 7f4e0c5b0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4e04071890 0x7f4e04071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.155+0000 7f4e0c5b0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0415cc10 con 0x7f4e040cf710 2023-11-28T12:02:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.155+0000 7f4e0c5b0700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e0406c9b0 con 0x7f4e04071890 2023-11-28T12:02:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.155+0000 7f4e0c5b0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0406cc90 con 0x7f4e0406a980 2023-11-28T12:02:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.155+0000 7f4e0ab4d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4e04071890 0x7f4e04071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.155+0000 7f4e0ab4d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4e04071890 0x7f4e04071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53924/0 (socket says 172.21.15.57:53924) 2023-11-28T12:02:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.155+0000 7f4e0ab4d700 1 -- 172.21.15.57:0/1811777779 learned_addr learned my addr 172.21.15.57:0/1811777779 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.155+0000 7f4e09b4b700 1 --2- 172.21.15.57:0/1811777779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e0406a980 0x7f4e040ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.155+0000 7f4e0ab4d700 1 -- 172.21.15.57:0/1811777779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e0406a980 msgr2=0x7f4e040ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.155+0000 7f4e0ab4d700 1 --2- 172.21.15.57:0/1811777779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e0406a980 0x7f4e040ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.155+0000 7f4e0ab4d700 1 -- 172.21.15.57:0/1811777779 >> v1:172.21.15.181:6789/0 conn(0x7f4e040cf710 legacy=0x7f4e0406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.155+0000 7f4e0ab4d700 1 -- 172.21.15.57:0/1811777779 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0406cf70 con 0x7f4e04071890 2023-11-28T12:02:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.156+0000 7f4e0ab4d700 1 --2- 172.21.15.57:0/1811777779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e04071890 0x7f4e04071220 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009e50 tx=0x7f4dfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a26f5159ead8740 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.156+0000 7f4e0934a700 1 -- 172.21.15.57:0/1811777779 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc00e070 con 0x7f4e04071890 2023-11-28T12:02:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.156+0000 7f4e0934a700 1 -- 172.21.15.57:0/1811777779 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dfc00b9c0 con 0x7f4e04071890 2023-11-28T12:02:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.156+0000 7f4e0934a700 1 -- 172.21.15.57:0/1811777779 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc00bd70 con 0x7f4e04071890 2023-11-28T12:02:33.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.156+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/1811777779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e04071890 msgr2=0x7f4e04071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.156+0000 7f4e0c5b0700 1 --2- 172.21.15.57:0/1811777779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e04071890 0x7f4e04071220 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009e50 tx=0x7f4dfc00b330 comp rx=0 tx=0).stop 2023-11-28T12:02:33.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.156+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/1811777779 shutdown_connections 2023-11-28T12:02:33.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.156+0000 7f4e0c5b0700 1 --2- 172.21.15.57:0/1811777779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e04071890 0x7f4e04071220 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.156+0000 7f4e0c5b0700 1 --2- 172.21.15.57:0/1811777779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e0406a980 0x7f4e040ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.156+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/1811777779 >> 172.21.15.57:0/1811777779 conn(0x7f4e0406c010 msgr2=0x7f4e040cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:33.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.157+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/1811777779 shutdown_connections 2023-11-28T12:02:33.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.157+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/1811777779 wait complete. 2023-11-28T12:02:33.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.158+0000 7f4e0c5b0700 1 Processor -- start 2023-11-28T12:02:33.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.158+0000 7f4e0c5b0700 1 -- start start 2023-11-28T12:02:33.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.159+0000 7f4e0c5b0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e0406a980 0x7f4e041f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.159+0000 7f4e0c5b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e04071890 0x7f4e040b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.159+0000 7f4e0c5b0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e040cf710 0x7f4e0415e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.159+0000 7f4e0c5b0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e041f2750 con 0x7f4e0406a980 2023-11-28T12:02:33.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.159+0000 7f4e0c5b0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e040b11a0 con 0x7f4e04071890 2023-11-28T12:02:33.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.159+0000 7f4e0c5b0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e041f3200 con 0x7f4e040cf710 2023-11-28T12:02:33.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.159+0000 7f4e0a34c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e0406a980 0x7f4e041f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.159+0000 7f4e0a34c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e0406a980 0x7f4e041f2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53940/0 (socket says 172.21.15.57:53940) 2023-11-28T12:02:33.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.159+0000 7f4e0a34c700 1 -- 172.21.15.57:0/3584300743 learned_addr learned my addr 172.21.15.57:0/3584300743 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:33.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4e09b4b700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e04071890 0x7f4e040b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4e0ab4d700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e040cf710 0x7f4e0415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4e0a34c700 1 -- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e040cf710 msgr2=0x7f4e0415e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4e0a34c700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e040cf710 0x7f4e0415e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4e0a34c700 1 -- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e04071890 msgr2=0x7f4e040b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4e0a34c700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e04071890 0x7f4e040b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4e0a34c700 1 -- 172.21.15.57:0/3584300743 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc009b30 con 0x7f4e0406a980 2023-11-28T12:02:33.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4e09b4b700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e04071890 0x7f4e040b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:33.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4e0a34c700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e0406a980 0x7f4e041f2210 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000c450 tx=0x7f4e0000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:33.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4dfaffd700 1 -- 172.21.15.57:0/3584300743 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e000052d0 con 0x7f4e0406a980 2023-11-28T12:02:33.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4dfaffd700 1 -- 172.21.15.57:0/3584300743 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e00005430 con 0x7f4e0406a980 2023-11-28T12:02:33.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/3584300743 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0406d530 con 0x7f4e0406a980 2023-11-28T12:02:33.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4dfaffd700 1 -- 172.21.15.57:0/3584300743 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e000038c0 con 0x7f4e0406a980 2023-11-28T12:02:33.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.160+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/3584300743 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0406d250 con 0x7f4e0406a980 2023-11-28T12:02:33.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.161+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/3584300743 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e04160480 con 0x7f4e0406a980 2023-11-28T12:02:33.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.162+0000 7f4dfaffd700 1 -- 172.21.15.57:0/3584300743 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e00019030 con 0x7f4e0406a980 2023-11-28T12:02:33.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.162+0000 7f4dfaffd700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4de80682f0 0x7f4de806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.162+0000 7f4dfaffd700 1 -- 172.21.15.57:0/3584300743 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4e00084ec0 con 0x7f4e0406a980 2023-11-28T12:02:33.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.162+0000 7f4e09b4b700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4de80682f0 0x7f4de806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.163+0000 7f4e09b4b700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4de80682f0 0x7f4de806a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4e041f68c0 tx=0x7f4df4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:33.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.167+0000 7f4dfaffd700 1 -- 172.21.15.57:0/3584300743 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e000516e0 con 0x7f4e0406a980 2023-11-28T12:02:33.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.366+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/3584300743 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e0415e790 con 0x7f4e0406a980 2023-11-28T12:02:33.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.367+0000 7f4dfaffd700 1 -- 172.21.15.57:0/3584300743 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e00054d00 con 0x7f4e0406a980 2023-11-28T12:02:33.368 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:02:33.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.369+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4de80682f0 msgr2=0x7f4de806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.369+0000 7f4e0c5b0700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4de80682f0 0x7f4de806a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4e041f68c0 tx=0x7f4df4008040 comp rx=0 tx=0).stop 2023-11-28T12:02:33.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.369+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/3584300743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e0406a980 msgr2=0x7f4e041f2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.369+0000 7f4e0c5b0700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e0406a980 0x7f4e041f2210 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000c450 tx=0x7f4e0000c810 comp rx=0 tx=0).stop 2023-11-28T12:02:33.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.369+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/3584300743 shutdown_connections 2023-11-28T12:02:33.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.369+0000 7f4e0c5b0700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4de80682f0 0x7f4de806a7a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.369+0000 7f4e0c5b0700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e040cf710 0x7f4e0415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.369+0000 7f4e0c5b0700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e0406a980 0x7f4e041f2210 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.369+0000 7f4e0c5b0700 1 --2- 172.21.15.57:0/3584300743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e04071890 0x7f4e040b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.369+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/3584300743 >> 172.21.15.57:0/3584300743 conn(0x7f4e0406c010 msgr2=0x7f4e0406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:33.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.369+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/3584300743 shutdown_connections 2023-11-28T12:02:33.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.369+0000 7f4e0c5b0700 1 -- 172.21.15.57:0/3584300743 wait complete. 2023-11-28T12:02:33.384 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T12:02:33.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.519+0000 7fdd9992f700 1 Processor -- start 2023-11-28T12:02:33.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.519+0000 7fdd9992f700 1 -- start start 2023-11-28T12:02:33.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.520+0000 7fdd9992f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd941599b0 0x7fdd9415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.520+0000 7fdd9992f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdd940af0a0 0x7fdd9415a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.520+0000 7fdd9992f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd94159ec0 con 0x7fdd940b1650 2023-11-28T12:02:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.520+0000 7fdd9992f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd9415d8a0 con 0x7fdd940af0a0 2023-11-28T12:02:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.520+0000 7fdd9992f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9415db80 con 0x7fdd941599b0 2023-11-28T12:02:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.520+0000 7fdd92ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fdd940b1650 0x7fdd940b1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40298/0 (socket says 172.21.15.57:40298) 2023-11-28T12:02:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.520+0000 7fdd92ffd700 1 -- 172.21.15.57:0/346433808 learned_addr learned my addr 172.21.15.57:0/346433808 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.521+0000 7fdd937fe700 1 --2- 172.21.15.57:0/346433808 >> v2:172.21.15.181:3300/0 conn(0x7fdd940af0a0 0x7fdd9415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.521+0000 7fdd927fc700 1 --2- 172.21.15.57:0/346433808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd941599b0 0x7fdd9415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.521+0000 7fdd91ffb700 1 -- 172.21.15.57:0/346433808 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050700953 0 0) 0x7fdd94159ec0 con 0x7fdd940b1650 2023-11-28T12:02:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.521+0000 7fdd91ffb700 1 -- 172.21.15.57:0/346433808 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd800035b0 con 0x7fdd940b1650 2023-11-28T12:02:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.521+0000 7fdd937fe700 1 -- 172.21.15.57:0/346433808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd941599b0 msgr2=0x7fdd9415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.521+0000 7fdd937fe700 1 --2- 172.21.15.57:0/346433808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd941599b0 0x7fdd9415d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.521+0000 7fdd937fe700 1 -- 172.21.15.57:0/346433808 >> v1:172.21.15.57:6789/0 conn(0x7fdd940b1650 legacy=0x7fdd940b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.521+0000 7fdd91ffb700 1 -- 172.21.15.57:0/346433808 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4068719825 0 0) 0x7fdd800035b0 con 0x7fdd940b1650 2023-11-28T12:02:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.521+0000 7fdd937fe700 1 -- 172.21.15.57:0/346433808 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9415de60 con 0x7fdd940af0a0 2023-11-28T12:02:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.521+0000 7fdd937fe700 1 --2- 172.21.15.57:0/346433808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd940af0a0 0x7fdd9415a350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009d90 tx=0x7fdd8c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f7e395511d5704d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.521+0000 7fdd91ffb700 1 -- 172.21.15.57:0/346433808 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c001670 con 0x7fdd940af0a0 2023-11-28T12:02:33.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.521+0000 7fdd91ffb700 1 -- 172.21.15.57:0/346433808 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8c003a20 con 0x7fdd940af0a0 2023-11-28T12:02:33.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.522+0000 7fdd91ffb700 1 -- 172.21.15.57:0/346433808 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c003dd0 con 0x7fdd940af0a0 2023-11-28T12:02:33.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.522+0000 7fdd9992f700 1 -- 172.21.15.57:0/346433808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd940af0a0 msgr2=0x7fdd9415a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.522+0000 7fdd9992f700 1 --2- 172.21.15.57:0/346433808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd940af0a0 0x7fdd9415a350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009d90 tx=0x7fdd8c00b330 comp rx=0 tx=0).stop 2023-11-28T12:02:33.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.522+0000 7fdd9992f700 1 -- 172.21.15.57:0/346433808 shutdown_connections 2023-11-28T12:02:33.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.522+0000 7fdd9992f700 1 --2- 172.21.15.57:0/346433808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd940af0a0 0x7fdd9415a350 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.522+0000 7fdd9992f700 1 --2- 172.21.15.57:0/346433808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd941599b0 0x7fdd9415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.522+0000 7fdd9992f700 1 -- 172.21.15.57:0/346433808 >> 172.21.15.57:0/346433808 conn(0x7fdd9406c010 msgr2=0x7fdd940cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:33.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.522+0000 7fdd9992f700 1 -- 172.21.15.57:0/346433808 shutdown_connections 2023-11-28T12:02:33.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.522+0000 7fdd9992f700 1 -- 172.21.15.57:0/346433808 wait complete. 2023-11-28T12:02:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.523+0000 7fdd9992f700 1 Processor -- start 2023-11-28T12:02:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.524+0000 7fdd9992f700 1 -- start start 2023-11-28T12:02:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.524+0000 7fdd9992f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd940af0a0 0x7fdd940b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.524+0000 7fdd9992f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd940b1650 0x7fdd941517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.524+0000 7fdd9992f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd94152080 0x7fdd940c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.524+0000 7fdd9992f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940b5710 con 0x7fdd94152080 2023-11-28T12:02:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.524+0000 7fdd9992f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940b2b80 con 0x7fdd940af0a0 2023-11-28T12:02:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.524+0000 7fdd9992f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940b2e60 con 0x7fdd940b1650 2023-11-28T12:02:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd937fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd94152080 0x7fdd940c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd937fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd94152080 0x7fdd940c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53942/0 (socket says 172.21.15.57:53942) 2023-11-28T12:02:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd937fe700 1 -- 172.21.15.57:0/1244381624 learned_addr learned my addr 172.21.15.57:0/1244381624 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd927fc700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd940b1650 0x7fdd941517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd92ffd700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd940af0a0 0x7fdd940b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd937fe700 1 -- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd940b1650 msgr2=0x7fdd941517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd937fe700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd940b1650 0x7fdd941517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd937fe700 1 -- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd940af0a0 msgr2=0x7fdd940b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd937fe700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd940af0a0 0x7fdd940b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd937fe700 1 -- 172.21.15.57:0/1244381624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c009a70 con 0x7fdd94152080 2023-11-28T12:02:33.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd937fe700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd94152080 0x7fdd940c4e30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdd940708b0 tx=0x7fdd8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:33.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd8bfff700 1 -- 172.21.15.57:0/1244381624 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c009410 con 0x7fdd94152080 2023-11-28T12:02:33.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd8bfff700 1 -- 172.21.15.57:0/1244381624 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8c004460 con 0x7fdd94152080 2023-11-28T12:02:33.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd8bfff700 1 -- 172.21.15.57:0/1244381624 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c004810 con 0x7fdd94152080 2023-11-28T12:02:33.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd9992f700 1 -- 172.21.15.57:0/1244381624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9415a890 con 0x7fdd94152080 2023-11-28T12:02:33.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.525+0000 7fdd9992f700 1 -- 172.21.15.57:0/1244381624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9415d590 con 0x7fdd94152080 2023-11-28T12:02:33.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.526+0000 7fdd9992f700 1 -- 172.21.15.57:0/1244381624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd9415aa20 con 0x7fdd94152080 2023-11-28T12:02:33.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.527+0000 7fdd8bfff700 1 -- 172.21.15.57:0/1244381624 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdd8c02b030 con 0x7fdd94152080 2023-11-28T12:02:33.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.527+0000 7fdd8bfff700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd740682f0 0x7fdd7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.527+0000 7fdd8bfff700 1 -- 172.21.15.57:0/1244381624 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdd8c0896f0 con 0x7fdd94152080 2023-11-28T12:02:33.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.528+0000 7fdd92ffd700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd740682f0 0x7fdd7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.528+0000 7fdd92ffd700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd740682f0 0x7fdd7406a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c000f60 tx=0x7fdd7c001230 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:33.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.532+0000 7fdd8bfff700 1 -- 172.21.15.57:0/1244381624 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdd8c055f10 con 0x7fdd94152080 2023-11-28T12:02:33.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.748+0000 7fdd9992f700 1 -- 172.21.15.57:0/1244381624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdd940b3320 con 0x7fdd94152080 2023-11-28T12:02:33.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.749+0000 7fdd8bfff700 1 -- 172.21.15.57:0/1244381624 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdd8c059530 con 0x7fdd94152080 2023-11-28T12:02:33.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.751+0000 7fdd9992f700 1 -- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd740682f0 msgr2=0x7fdd7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.751+0000 7fdd9992f700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd740682f0 0x7fdd7406a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c000f60 tx=0x7fdd7c001230 comp rx=0 tx=0).stop 2023-11-28T12:02:33.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.751+0000 7fdd9992f700 1 -- 172.21.15.57:0/1244381624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd94152080 msgr2=0x7fdd940c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.751+0000 7fdd9992f700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd94152080 0x7fdd940c4e30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdd940708b0 tx=0x7fdd8c004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.751+0000 7fdd9992f700 1 -- 172.21.15.57:0/1244381624 shutdown_connections 2023-11-28T12:02:33.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.751+0000 7fdd9992f700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdd740682f0 0x7fdd7406a7a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.751+0000 7fdd9992f700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd94152080 0x7fdd940c4e30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.751+0000 7fdd9992f700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd940af0a0 0x7fdd940b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.751+0000 7fdd9992f700 1 --2- 172.21.15.57:0/1244381624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd940b1650 0x7fdd941517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.751+0000 7fdd9992f700 1 -- 172.21.15.57:0/1244381624 >> 172.21.15.57:0/1244381624 conn(0x7fdd9406c010 msgr2=0x7fdd940cfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:33.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.751+0000 7fdd9992f700 1 -- 172.21.15.57:0/1244381624 shutdown_connections 2023-11-28T12:02:33.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.751+0000 7fdd9992f700 1 -- 172.21.15.57:0/1244381624 wait complete. 2023-11-28T12:02:33.754 INFO:teuthology.orchestra.run.smithi057.stderr:listed 0 entries 2023-11-28T12:02:33.766 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:02:33.767 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:02:33.790 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:33.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.926+0000 7ffa70e51700 1 Processor -- start 2023-11-28T12:02:33.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.926+0000 7ffa70e51700 1 -- start start 2023-11-28T12:02:33.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa70e51700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa6c06a270 0x7ffa6c06a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa70e51700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ffa6c15cc80 0x7ffa6c06de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa70e51700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6c06ab80 con 0x7ffa6c06e3a0 2023-11-28T12:02:33.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa70e51700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6c0b2490 con 0x7ffa6c15cc80 2023-11-28T12:02:33.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa70e51700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c0b1810 con 0x7ffa6c06a270 2023-11-28T12:02:33.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa69d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ffa6c15cc80 0x7ffa6c06de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa6ad9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ffa6c06e3a0 0x7ffa6c06ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40306/0 (socket says 172.21.15.57:40306) 2023-11-28T12:02:33.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa6ad9d700 1 -- 172.21.15.57:0/2413211516 learned_addr learned my addr 172.21.15.57:0/2413211516 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:33.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa6a59c700 1 --2- 172.21.15.57:0/2413211516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa6c06a270 0x7ffa6c06a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa69d9b700 1 -- 172.21.15.57:0/2413211516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa6c06a270 msgr2=0x7ffa6c06a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:33.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa69d9b700 1 --2- 172.21.15.57:0/2413211516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa6c06a270 0x7ffa6c06a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa69d9b700 1 -- 172.21.15.57:0/2413211516 >> v1:172.21.15.57:6789/0 conn(0x7ffa6c06e3a0 legacy=0x7ffa6c06ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa69d9b700 1 -- 172.21.15.57:0/2413211516 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c0b1af0 con 0x7ffa6c15cc80 2023-11-28T12:02:33.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.927+0000 7ffa6959a700 1 -- 172.21.15.57:0/2413211516 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28418734 0 0) 0x7ffa6c06ab80 con 0x7ffa6c06e3a0 2023-11-28T12:02:33.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.928+0000 7ffa69d9b700 1 --2- 172.21.15.57:0/2413211516 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa6c15cc80 0x7ffa6c06de60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009840 tx=0x7ffa60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e856f491aba7fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:33.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.928+0000 7ffa6959a700 1 -- 172.21.15.57:0/2413211516 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60011830 con 0x7ffa6c15cc80 2023-11-28T12:02:33.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.928+0000 7ffa6959a700 1 -- 172.21.15.57:0/2413211516 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60011990 con 0x7ffa6c15cc80 2023-11-28T12:02:33.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.928+0000 7ffa6959a700 1 -- 172.21.15.57:0/2413211516 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60011d20 con 0x7ffa6c15cc80 2023-11-28T12:02:33.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.928+0000 7ffa70e51700 1 -- 172.21.15.57:0/2413211516 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa6c15cc80 msgr2=0x7ffa6c06de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.928+0000 7ffa70e51700 1 --2- 172.21.15.57:0/2413211516 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa6c15cc80 0x7ffa6c06de60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009840 tx=0x7ffa60009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:33.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.928+0000 7ffa70e51700 1 -- 172.21.15.57:0/2413211516 shutdown_connections 2023-11-28T12:02:33.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.928+0000 7ffa70e51700 1 --2- 172.21.15.57:0/2413211516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa6c06a270 0x7ffa6c06a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.928+0000 7ffa70e51700 1 --2- 172.21.15.57:0/2413211516 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa6c15cc80 0x7ffa6c06de60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.928+0000 7ffa70e51700 1 -- 172.21.15.57:0/2413211516 >> 172.21.15.57:0/2413211516 conn(0x7ffa6c06c010 msgr2=0x7ffa6c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:33.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.928+0000 7ffa70e51700 1 -- 172.21.15.57:0/2413211516 shutdown_connections 2023-11-28T12:02:33.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.928+0000 7ffa70e51700 1 -- 172.21.15.57:0/2413211516 wait complete. 2023-11-28T12:02:33.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.930+0000 7ffa70e51700 1 Processor -- start 2023-11-28T12:02:33.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.930+0000 7ffa70e51700 1 -- start start 2023-11-28T12:02:33.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.931+0000 7ffa70e51700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa6c06a270 0x7ffa6c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.931+0000 7ffa70e51700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa6c06e3a0 0x7ffa6c1523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.931+0000 7ffa70e51700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa6c15cc80 0x7ffa6c152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.931+0000 7ffa70e51700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c0c7300 con 0x7ffa6c06e3a0 2023-11-28T12:02:33.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.931+0000 7ffa70e51700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c1528f0 con 0x7ffa6c15cc80 2023-11-28T12:02:33.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.931+0000 7ffa70e51700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c0b59f0 con 0x7ffa6c06a270 2023-11-28T12:02:33.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa69d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa6c06e3a0 0x7ffa6c1523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa69d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa6c06e3a0 0x7ffa6c1523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53950/0 (socket says 172.21.15.57:53950) 2023-11-28T12:02:33.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa69d9b700 1 -- 172.21.15.57:0/1107868269 learned_addr learned my addr 172.21.15.57:0/1107868269 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:33.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa6a59c700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa6c06a270 0x7ffa6c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa6ad9d700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa6c15cc80 0x7ffa6c152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa69d9b700 1 -- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa6c06a270 msgr2=0x7ffa6c0c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa69d9b700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa6c06a270 0x7ffa6c0c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa69d9b700 1 -- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa6c15cc80 msgr2=0x7ffa6c152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:33.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa69d9b700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa6c15cc80 0x7ffa6c152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:33.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa69d9b700 1 -- 172.21.15.57:0/1107868269 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa600094a0 con 0x7ffa6c06e3a0 2023-11-28T12:02:33.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa6ad9d700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa6c15cc80 0x7ffa6c152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:33.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa69d9b700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa6c06e3a0 0x7ffa6c1523b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009fd0 tx=0x7ffa60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:33.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa5affd700 1 -- 172.21.15.57:0/1107868269 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa600036a0 con 0x7ffa6c06e3a0 2023-11-28T12:02:33.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa5affd700 1 -- 172.21.15.57:0/1107868269 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6001f070 con 0x7ffa6c06e3a0 2023-11-28T12:02:33.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa70e51700 1 -- 172.21.15.57:0/1107868269 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6c0b20b0 con 0x7ffa6c06e3a0 2023-11-28T12:02:33.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa5affd700 1 -- 172.21.15.57:0/1107868269 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60003a20 con 0x7ffa6c06e3a0 2023-11-28T12:02:33.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.932+0000 7ffa70e51700 1 -- 172.21.15.57:0/1107868269 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6c0b1dd0 con 0x7ffa6c06e3a0 2023-11-28T12:02:33.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.933+0000 7ffa70e51700 1 -- 172.21.15.57:0/1107868269 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6c06e7e0 con 0x7ffa6c06e3a0 2023-11-28T12:02:33.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.934+0000 7ffa5affd700 1 -- 172.21.15.57:0/1107868269 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffa6000f050 con 0x7ffa6c06e3a0 2023-11-28T12:02:33.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.934+0000 7ffa5affd700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ffa4c0682f0 0x7ffa4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:33.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.934+0000 7ffa5affd700 1 -- 172.21.15.57:0/1107868269 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ffa60021070 con 0x7ffa6c06e3a0 2023-11-28T12:02:33.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.934+0000 7ffa6a59c700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ffa4c0682f0 0x7ffa4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:33.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.935+0000 7ffa6a59c700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ffa4c0682f0 0x7ffa4c06a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009b40 tx=0x7ffa5c009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:33.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:33.939+0000 7ffa5affd700 1 -- 172.21.15.57:0/1107868269 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffa60053c30 con 0x7ffa6c06e3a0 2023-11-28T12:02:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.139+0000 7ffa70e51700 1 -- 172.21.15.57:0/1107868269 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa6c0743f0 con 0x7ffa6c06e3a0 2023-11-28T12:02:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.140+0000 7ffa5affd700 1 -- 172.21.15.57:0/1107868269 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17945 (secure 0 0 0) 0x7ffa60057250 con 0x7ffa6c06e3a0 2023-11-28T12:02:34.142 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:34.142 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:02:32.851731+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:34.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.143+0000 7ffa70e51700 1 -- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ffa4c0682f0 msgr2=0x7ffa4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.143+0000 7ffa70e51700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ffa4c0682f0 0x7ffa4c06a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009b40 tx=0x7ffa5c009210 comp rx=0 tx=0).stop 2023-11-28T12:02:34.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.143+0000 7ffa70e51700 1 -- 172.21.15.57:0/1107868269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa6c06e3a0 msgr2=0x7ffa6c1523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.143+0000 7ffa70e51700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa6c06e3a0 0x7ffa6c1523b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009fd0 tx=0x7ffa60004750 comp rx=0 tx=0).stop 2023-11-28T12:02:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.143+0000 7ffa70e51700 1 -- 172.21.15.57:0/1107868269 shutdown_connections 2023-11-28T12:02:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.143+0000 7ffa70e51700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ffa4c0682f0 0x7ffa4c06a7a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.143+0000 7ffa70e51700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa6c15cc80 0x7ffa6c152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.143+0000 7ffa70e51700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa6c06a270 0x7ffa6c0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.143+0000 7ffa70e51700 1 --2- 172.21.15.57:0/1107868269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa6c06e3a0 0x7ffa6c1523b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.143+0000 7ffa70e51700 1 -- 172.21.15.57:0/1107868269 >> 172.21.15.57:0/1107868269 conn(0x7ffa6c06c010 msgr2=0x7ffa6c0b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.143+0000 7ffa70e51700 1 -- 172.21.15.57:0/1107868269 shutdown_connections 2023-11-28T12:02:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.143+0000 7ffa70e51700 1 -- 172.21.15.57:0/1107868269 wait complete. 2023-11-28T12:02:34.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:34.158 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:34.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.297+0000 7fec67465700 1 Processor -- start 2023-11-28T12:02:34.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.297+0000 7fec67465700 1 -- start start 2023-11-28T12:02:34.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.298+0000 7fec67465700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fec6006a980 0x7fec600cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:34.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.298+0000 7fec67465700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec600cdd50 0x7fec600cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:34.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.298+0000 7fec67465700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6015e0d0 con 0x7fec6015a4d0 2023-11-28T12:02:34.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.298+0000 7fec67465700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec600ce560 con 0x7fec6006a980 2023-11-28T12:02:34.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.298+0000 7fec67465700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6015e320 con 0x7fec600cdd50 2023-11-28T12:02:34.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.299+0000 7fec65201700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fec6015a4d0 0x7fec6006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40308/0 (socket says 172.21.15.57:40308) 2023-11-28T12:02:34.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.299+0000 7fec65201700 1 -- 172.21.15.57:0/2281136313 learned_addr learned my addr 172.21.15.57:0/2281136313 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:34.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.299+0000 7fec64a00700 1 --2- 172.21.15.57:0/2281136313 >> v2:172.21.15.181:3300/0 conn(0x7fec6006a980 0x7fec600cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:34.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.299+0000 7fec65a02700 1 --2- 172.21.15.57:0/2281136313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec600cdd50 0x7fec600cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:34.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.299+0000 7fec57fff700 1 -- 172.21.15.57:0/2281136313 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205444575 0 0) 0x7fec6015e0d0 con 0x7fec6015a4d0 2023-11-28T12:02:34.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.299+0000 7fec57fff700 1 -- 172.21.15.57:0/2281136313 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec480035b0 con 0x7fec6015a4d0 2023-11-28T12:02:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.299+0000 7fec64a00700 1 -- 172.21.15.57:0/2281136313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec600cdd50 msgr2=0x7fec600cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.299+0000 7fec64a00700 1 --2- 172.21.15.57:0/2281136313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec600cdd50 0x7fec600cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.299+0000 7fec64a00700 1 -- 172.21.15.57:0/2281136313 >> v1:172.21.15.57:6789/0 conn(0x7fec6015a4d0 legacy=0x7fec6006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.299+0000 7fec64a00700 1 -- 172.21.15.57:0/2281136313 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6015e600 con 0x7fec6006a980 2023-11-28T12:02:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.299+0000 7fec64a00700 1 --2- 172.21.15.57:0/2281136313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec6006a980 0x7fec600cd810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009840 tx=0x7fec5c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71ee8454fcab7cae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.300+0000 7fec57fff700 1 -- 172.21.15.57:0/2281136313 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c011830 con 0x7fec6006a980 2023-11-28T12:02:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.300+0000 7fec57fff700 1 -- 172.21.15.57:0/2281136313 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c011990 con 0x7fec6006a980 2023-11-28T12:02:34.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.300+0000 7fec57fff700 1 -- 172.21.15.57:0/2281136313 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c011d20 con 0x7fec6006a980 2023-11-28T12:02:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.300+0000 7fec67465700 1 -- 172.21.15.57:0/2281136313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec6006a980 msgr2=0x7fec600cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.300+0000 7fec67465700 1 --2- 172.21.15.57:0/2281136313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec6006a980 0x7fec600cd810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009840 tx=0x7fec5c009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.300+0000 7fec67465700 1 -- 172.21.15.57:0/2281136313 shutdown_connections 2023-11-28T12:02:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.300+0000 7fec67465700 1 --2- 172.21.15.57:0/2281136313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec600cdd50 0x7fec600cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.300+0000 7fec67465700 1 --2- 172.21.15.57:0/2281136313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec6006a980 0x7fec600cd810 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.300+0000 7fec67465700 1 -- 172.21.15.57:0/2281136313 >> 172.21.15.57:0/2281136313 conn(0x7fec6006c010 msgr2=0x7fec600cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.301+0000 7fec67465700 1 -- 172.21.15.57:0/2281136313 shutdown_connections 2023-11-28T12:02:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.301+0000 7fec67465700 1 -- 172.21.15.57:0/2281136313 wait complete. 2023-11-28T12:02:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.302+0000 7fec67465700 1 Processor -- start 2023-11-28T12:02:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.302+0000 7fec67465700 1 -- start start 2023-11-28T12:02:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec67465700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec6006a980 0x7fec600c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec67465700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec600cdd50 0x7fec600aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec67465700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec6015a4d0 0x7fec60148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec67465700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec600af0c0 con 0x7fec600cdd50 2023-11-28T12:02:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec67465700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec601544c0 con 0x7fec6006a980 2023-11-28T12:02:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec67465700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec601494d0 con 0x7fec6015a4d0 2023-11-28T12:02:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec64a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec600cdd50 0x7fec600aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec64a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec600cdd50 0x7fec600aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53958/0 (socket says 172.21.15.57:53958) 2023-11-28T12:02:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec64a00700 1 -- 172.21.15.57:0/1429085961 learned_addr learned my addr 172.21.15.57:0/1429085961 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:34.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec65a02700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec6015a4d0 0x7fec60148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec65201700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec6006a980 0x7fec600c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec64a00700 1 -- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec6015a4d0 msgr2=0x7fec60148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec64a00700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec6015a4d0 0x7fec60148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec64a00700 1 -- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec6006a980 msgr2=0x7fec600c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec64a00700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec6006a980 0x7fec600c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec64a00700 1 -- 172.21.15.57:0/1429085961 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c0094a0 con 0x7fec600cdd50 2023-11-28T12:02:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec64a00700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec600cdd50 0x7fec600aeb80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fec5c000c00 tx=0x7fec5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec55ffb700 1 -- 172.21.15.57:0/1429085961 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c004590 con 0x7fec600cdd50 2023-11-28T12:02:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.303+0000 7fec55ffb700 1 -- 172.21.15.57:0/1429085961 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c01f070 con 0x7fec600cdd50 2023-11-28T12:02:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.304+0000 7fec67465700 1 -- 172.21.15.57:0/1429085961 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec600ce1b0 con 0x7fec600cdd50 2023-11-28T12:02:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.304+0000 7fec55ffb700 1 -- 172.21.15.57:0/1429085961 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0038e0 con 0x7fec600cdd50 2023-11-28T12:02:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.304+0000 7fec67465700 1 -- 172.21.15.57:0/1429085961 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6015e8e0 con 0x7fec600cdd50 2023-11-28T12:02:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.305+0000 7fec67465700 1 -- 172.21.15.57:0/1429085961 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6015a8a0 con 0x7fec600cdd50 2023-11-28T12:02:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.305+0000 7fec55ffb700 1 -- 172.21.15.57:0/1429085961 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fec5c00f050 con 0x7fec600cdd50 2023-11-28T12:02:34.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.305+0000 7fec55ffb700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec4c0682f0 0x7fec4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:34.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.306+0000 7fec55ffb700 1 -- 172.21.15.57:0/1429085961 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fec5c021070 con 0x7fec600cdd50 2023-11-28T12:02:34.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.306+0000 7fec65201700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec4c0682f0 0x7fec4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:34.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.306+0000 7fec65201700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec4c0682f0 0x7fec4c06a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fec50007fd0 tx=0x7fec50007dc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.310+0000 7fec55ffb700 1 -- 172.21.15.57:0/1429085961 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fec5c053c20 con 0x7fec600cdd50 2023-11-28T12:02:34.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.508+0000 7fec67465700 1 -- 172.21.15.57:0/1429085961 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fec600743f0 con 0x7fec600cdd50 2023-11-28T12:02:34.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.509+0000 7fec55ffb700 1 -- 172.21.15.57:0/1429085961 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fec5c057240 con 0x7fec600cdd50 2023-11-28T12:02:34.510 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:34.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.511+0000 7fec67465700 1 -- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec4c0682f0 msgr2=0x7fec4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.511+0000 7fec67465700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec4c0682f0 0x7fec4c06a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fec50007fd0 tx=0x7fec50007dc0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.511+0000 7fec67465700 1 -- 172.21.15.57:0/1429085961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec600cdd50 msgr2=0x7fec600aeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.511+0000 7fec67465700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec600cdd50 0x7fec600aeb80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fec5c000c00 tx=0x7fec5c004750 comp rx=0 tx=0).stop 2023-11-28T12:02:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.511+0000 7fec67465700 1 -- 172.21.15.57:0/1429085961 shutdown_connections 2023-11-28T12:02:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.511+0000 7fec67465700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec4c0682f0 0x7fec4c06a7a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.511+0000 7fec67465700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec6015a4d0 0x7fec60148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.511+0000 7fec67465700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec6006a980 0x7fec600c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.511+0000 7fec67465700 1 --2- 172.21.15.57:0/1429085961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec600cdd50 0x7fec600aeb80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.511+0000 7fec67465700 1 -- 172.21.15.57:0/1429085961 >> 172.21.15.57:0/1429085961 conn(0x7fec6006c010 msgr2=0x7fec600caed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.511+0000 7fec67465700 1 -- 172.21.15.57:0/1429085961 shutdown_connections 2023-11-28T12:02:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.511+0000 7fec67465700 1 -- 172.21.15.57:0/1429085961 wait complete. 2023-11-28T12:02:34.526 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:34.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.663+0000 7f20d391f700 1 Processor -- start 2023-11-28T12:02:34.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.663+0000 7f20d391f700 1 -- start start 2023-11-28T12:02:34.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.664+0000 7f20d391f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20cc0b1650 0x7f20cc0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:34.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.664+0000 7f20d391f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f20cc0af0a0 0x7f20cc15a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:34.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.664+0000 7f20d391f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20cc159ec0 con 0x7f20cc1599b0 2023-11-28T12:02:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.664+0000 7f20d391f700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20cc0b1f60 con 0x7f20cc0af0a0 2023-11-28T12:02:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.664+0000 7f20d391f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc15d8a0 con 0x7f20cc0b1650 2023-11-28T12:02:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.664+0000 7f20d0eba700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f20cc1599b0 0x7f20cc15d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37362/0 (socket says 172.21.15.57:37362) 2023-11-28T12:02:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.664+0000 7f20d1ebc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f20cc0af0a0 0x7f20cc15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.664+0000 7f20d0eba700 1 -- 172.21.15.57:0/1322713691 learned_addr learned my addr 172.21.15.57:0/1322713691 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.664+0000 7f20d1ebc700 1 -- 172.21.15.57:0/1322713691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20cc0b1650 msgr2=0x7f20cc0b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.664+0000 7f20d1ebc700 1 --2- 172.21.15.57:0/1322713691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20cc0b1650 0x7f20cc0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.664+0000 7f20d1ebc700 1 -- 172.21.15.57:0/1322713691 >> v1:172.21.15.57:6789/0 conn(0x7f20cc1599b0 legacy=0x7f20cc15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.664+0000 7f20d1ebc700 1 -- 172.21.15.57:0/1322713691 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc15db80 con 0x7f20cc0af0a0 2023-11-28T12:02:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.664+0000 7f20c3fff700 1 -- 172.21.15.57:0/1322713691 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160354338 0 0) 0x7f20cc159ec0 con 0x7f20cc1599b0 2023-11-28T12:02:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.665+0000 7f20d1ebc700 1 --2- 172.21.15.57:0/1322713691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20cc0af0a0 0x7f20cc15a350 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009d90 tx=0x7f20c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411107a505d6ee09 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:34.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.665+0000 7f20c3fff700 1 -- 172.21.15.57:0/1322713691 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c400e070 con 0x7f20cc0af0a0 2023-11-28T12:02:34.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.665+0000 7f20c3fff700 1 -- 172.21.15.57:0/1322713691 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c400bbd0 con 0x7f20cc0af0a0 2023-11-28T12:02:34.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.665+0000 7f20c3fff700 1 -- 172.21.15.57:0/1322713691 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4012430 con 0x7f20cc0af0a0 2023-11-28T12:02:34.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.665+0000 7f20d391f700 1 -- 172.21.15.57:0/1322713691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20cc0af0a0 msgr2=0x7f20cc15a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.665+0000 7f20d391f700 1 --2- 172.21.15.57:0/1322713691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20cc0af0a0 0x7f20cc15a350 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009d90 tx=0x7f20c400b330 comp rx=0 tx=0).stop 2023-11-28T12:02:34.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.665+0000 7f20d391f700 1 -- 172.21.15.57:0/1322713691 shutdown_connections 2023-11-28T12:02:34.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.665+0000 7f20d391f700 1 --2- 172.21.15.57:0/1322713691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20cc0af0a0 0x7f20cc15a350 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.665+0000 7f20d391f700 1 --2- 172.21.15.57:0/1322713691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20cc0b1650 0x7f20cc0b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.665+0000 7f20d391f700 1 -- 172.21.15.57:0/1322713691 >> 172.21.15.57:0/1322713691 conn(0x7f20cc06c010 msgr2=0x7f20cc0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:34.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.666+0000 7f20d391f700 1 -- 172.21.15.57:0/1322713691 shutdown_connections 2023-11-28T12:02:34.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.666+0000 7f20d391f700 1 -- 172.21.15.57:0/1322713691 wait complete. 2023-11-28T12:02:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.667+0000 7f20d391f700 1 Processor -- start 2023-11-28T12:02:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.667+0000 7f20d391f700 1 -- start start 2023-11-28T12:02:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d391f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20cc0af0a0 0x7f20cc15c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d391f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20cc0b1650 0x7f20cc157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d391f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20cc157d80 0x7f20cc0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d391f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc0a8b40 con 0x7f20cc0af0a0 2023-11-28T12:02:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d391f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc15ac10 con 0x7f20cc0b1650 2023-11-28T12:02:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d391f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc0a8860 con 0x7f20cc157d80 2023-11-28T12:02:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d16bb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20cc0af0a0 0x7f20cc15c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d16bb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20cc0af0a0 0x7f20cc15c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50684/0 (socket says 172.21.15.57:50684) 2023-11-28T12:02:34.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d16bb700 1 -- 172.21.15.57:0/319171862 learned_addr learned my addr 172.21.15.57:0/319171862 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:34.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d0eba700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20cc0b1650 0x7f20cc157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:34.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d1ebc700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20cc157d80 0x7f20cc0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:34.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d16bb700 1 -- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20cc157d80 msgr2=0x7f20cc0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d16bb700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20cc157d80 0x7f20cc0713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d16bb700 1 -- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20cc0b1650 msgr2=0x7f20cc157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d16bb700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20cc0b1650 0x7f20cc157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d16bb700 1 -- 172.21.15.57:0/319171862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c4009a70 con 0x7f20cc0af0a0 2023-11-28T12:02:34.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d0eba700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20cc0b1650 0x7f20cc157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:34.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d16bb700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20cc0af0a0 0x7f20cc15c850 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f20c800b4a0 tx=0x7f20c800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:34.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20c1ffb700 1 -- 172.21.15.57:0/319171862 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8011830 con 0x7f20cc0af0a0 2023-11-28T12:02:34.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20c1ffb700 1 -- 172.21.15.57:0/319171862 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c8011990 con 0x7f20cc0af0a0 2023-11-28T12:02:34.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20d391f700 1 -- 172.21.15.57:0/319171862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20cc15a890 con 0x7f20cc0af0a0 2023-11-28T12:02:34.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.668+0000 7f20c1ffb700 1 -- 172.21.15.57:0/319171862 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8011d40 con 0x7f20cc0af0a0 2023-11-28T12:02:34.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.669+0000 7f20d391f700 1 -- 172.21.15.57:0/319171862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20cc15de60 con 0x7f20cc0af0a0 2023-11-28T12:02:34.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.670+0000 7f20d391f700 1 -- 172.21.15.57:0/319171862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20cc15aa20 con 0x7f20cc0af0a0 2023-11-28T12:02:34.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.670+0000 7f20c1ffb700 1 -- 172.21.15.57:0/319171862 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f20c8021070 con 0x7f20cc0af0a0 2023-11-28T12:02:34.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.670+0000 7f20c1ffb700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20b00682f0 0x7f20b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:34.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.670+0000 7f20c1ffb700 1 -- 172.21.15.57:0/319171862 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f20c8085c40 con 0x7f20cc0af0a0 2023-11-28T12:02:34.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.671+0000 7f20d0eba700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20b00682f0 0x7f20b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:34.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.671+0000 7f20d0eba700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20b00682f0 0x7f20b006a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0b5b20 tx=0x7f20bc007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:34.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.675+0000 7f20c1ffb700 1 -- 172.21.15.57:0/319171862 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f20c80523e0 con 0x7f20cc0af0a0 2023-11-28T12:02:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.872+0000 7f20d391f700 1 -- 172.21.15.57:0/319171862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f20cc0743f0 con 0x7f20cc0af0a0 2023-11-28T12:02:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.872+0000 7f20c1ffb700 1 -- 172.21.15.57:0/319171862 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20c80150a0 con 0x7f20cc0af0a0 2023-11-28T12:02:34.874 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.875+0000 7f20d391f700 1 -- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20b00682f0 msgr2=0x7f20b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.875+0000 7f20d391f700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20b00682f0 0x7f20b006a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0b5b20 tx=0x7f20bc007d30 comp rx=0 tx=0).stop 2023-11-28T12:02:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.875+0000 7f20d391f700 1 -- 172.21.15.57:0/319171862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20cc0af0a0 msgr2=0x7f20cc15c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.875+0000 7f20d391f700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20cc0af0a0 0x7f20cc15c850 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f20c800b4a0 tx=0x7f20c800b860 comp rx=0 tx=0).stop 2023-11-28T12:02:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.875+0000 7f20d391f700 1 -- 172.21.15.57:0/319171862 shutdown_connections 2023-11-28T12:02:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.875+0000 7f20d391f700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20b00682f0 0x7f20b006a7a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.875+0000 7f20d391f700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20cc157d80 0x7f20cc0713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.875+0000 7f20d391f700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20cc0af0a0 0x7f20cc15c850 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.875+0000 7f20d391f700 1 --2- 172.21.15.57:0/319171862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20cc0b1650 0x7f20cc157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:34.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.875+0000 7f20d391f700 1 -- 172.21.15.57:0/319171862 >> 172.21.15.57:0/319171862 conn(0x7f20cc06c010 msgr2=0x7f20cc159060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:34.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.875+0000 7f20d391f700 1 -- 172.21.15.57:0/319171862 shutdown_connections 2023-11-28T12:02:34.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:34.875+0000 7f20d391f700 1 -- 172.21.15.57:0/319171862 wait complete. 2023-11-28T12:02:34.889 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:02:35.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.026+0000 7ff5b0b3d700 1 Processor -- start 2023-11-28T12:02:35.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.026+0000 7ff5b0b3d700 1 -- start start 2023-11-28T12:02:35.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.027+0000 7ff5b0b3d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff5ac15fe50 0x7ff5ac06d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.027+0000 7ff5b0b3d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac0b27d0 0x7ff5ac0b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.027+0000 7ff5b0b3d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac161370 con 0x7ff5ac0b3b80 2023-11-28T12:02:35.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.027+0000 7ff5b0b3d700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ac0b3190 con 0x7ff5ac15fe50 2023-11-28T12:02:35.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.027+0000 7ff5b0b3d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac160220 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.027+0000 7ff5aa59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff5ac15fe50 0x7ff5ac06d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.027+0000 7ff5a9d9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff5ac0b3b80 0x7ff5ac160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37374/0 (socket says 172.21.15.57:37374) 2023-11-28T12:02:35.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.027+0000 7ff5a9d9b700 1 -- 172.21.15.57:0/1951035150 learned_addr learned my addr 172.21.15.57:0/1951035150 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:35.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.027+0000 7ff5aad9d700 1 --2- 172.21.15.57:0/1951035150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac0b27d0 0x7ff5ac0b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.028+0000 7ff5aa59c700 1 -- 172.21.15.57:0/1951035150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac0b27d0 msgr2=0x7ff5ac0b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:35.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.028+0000 7ff5aa59c700 1 --2- 172.21.15.57:0/1951035150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac0b27d0 0x7ff5ac0b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.028+0000 7ff5aa59c700 1 -- 172.21.15.57:0/1951035150 >> v1:172.21.15.57:6789/0 conn(0x7ff5ac0b3b80 legacy=0x7ff5ac160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.028+0000 7ff5aa59c700 1 -- 172.21.15.57:0/1951035150 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac0cf700 con 0x7ff5ac15fe50 2023-11-28T12:02:35.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.028+0000 7ff5a959a700 1 -- 172.21.15.57:0/1951035150 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149056698 0 0) 0x7ff5ac161370 con 0x7ff5ac0b3b80 2023-11-28T12:02:35.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.028+0000 7ff5aa59c700 1 --2- 172.21.15.57:0/1951035150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5ac15fe50 0x7ff5ac06d170 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009840 tx=0x7ff59c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399376f39a7ffce4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:35.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.028+0000 7ff5a959a700 1 -- 172.21.15.57:0/1951035150 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c011830 con 0x7ff5ac15fe50 2023-11-28T12:02:35.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.028+0000 7ff5a959a700 1 -- 172.21.15.57:0/1951035150 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c011990 con 0x7ff5ac15fe50 2023-11-28T12:02:35.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.028+0000 7ff5a959a700 1 -- 172.21.15.57:0/1951035150 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c011d20 con 0x7ff5ac15fe50 2023-11-28T12:02:35.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.029+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1951035150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5ac15fe50 msgr2=0x7ff5ac06d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.029+0000 7ff5b0b3d700 1 --2- 172.21.15.57:0/1951035150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5ac15fe50 0x7ff5ac06d170 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009840 tx=0x7ff59c009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:35.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.029+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1951035150 shutdown_connections 2023-11-28T12:02:35.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.029+0000 7ff5b0b3d700 1 --2- 172.21.15.57:0/1951035150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5ac15fe50 0x7ff5ac06d170 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.029+0000 7ff5b0b3d700 1 --2- 172.21.15.57:0/1951035150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac0b27d0 0x7ff5ac0b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.029+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1951035150 >> 172.21.15.57:0/1951035150 conn(0x7ff5ac06ea80 msgr2=0x7ff5ac06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:35.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.029+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1951035150 shutdown_connections 2023-11-28T12:02:35.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.029+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1951035150 wait complete. 2023-11-28T12:02:35.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.030+0000 7ff5b0b3d700 1 Processor -- start 2023-11-28T12:02:35.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.030+0000 7ff5b0b3d700 1 -- start start 2023-11-28T12:02:35.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.031+0000 7ff5b0b3d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5ac0b27d0 0x7ff5ac0adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.031+0000 7ff5b0b3d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac0b3b80 0x7ff5ac15e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.031+0000 7ff5b0b3d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac15fe50 0x7ff5ac0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.031+0000 7ff5b0b3d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac0cc160 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.031+0000 7ff5b0b3d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac0cbea0 con 0x7ff5ac0b3b80 2023-11-28T12:02:35.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.031+0000 7ff5b0b3d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac1e7ef0 con 0x7ff5ac15fe50 2023-11-28T12:02:35.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.031+0000 7ff5aa59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5ac0b27d0 0x7ff5ac0adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.031+0000 7ff5aa59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5ac0b27d0 0x7ff5ac0adfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50696/0 (socket says 172.21.15.57:50696) 2023-11-28T12:02:35.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.031+0000 7ff5aa59c700 1 -- 172.21.15.57:0/1124014549 learned_addr learned my addr 172.21.15.57:0/1124014549 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:35.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.031+0000 7ff5aad9d700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac15fe50 0x7ff5ac0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.031+0000 7ff5a9d9b700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac0b3b80 0x7ff5ac15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.031+0000 7ff5aa59c700 1 -- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac15fe50 msgr2=0x7ff5ac0c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.031+0000 7ff5aa59c700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac15fe50 0x7ff5ac0c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.032+0000 7ff5aa59c700 1 -- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac0b3b80 msgr2=0x7ff5ac15e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.032+0000 7ff5aa59c700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac0b3b80 0x7ff5ac15e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.032+0000 7ff5aa59c700 1 -- 172.21.15.57:0/1124014549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c0094a0 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.032+0000 7ff5aa59c700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5ac0b27d0 0x7ff5ac0adfd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff59c014040 tx=0x7ff59c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:35.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.032+0000 7ff5a2ffd700 1 -- 172.21.15.57:0/1124014549 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0038a0 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.032+0000 7ff5a2ffd700 1 -- 172.21.15.57:0/1124014549 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c01e070 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.032+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1124014549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ac0cedd0 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.032+0000 7ff5a2ffd700 1 -- 172.21.15.57:0/1124014549 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c003c20 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.032+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1124014549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac0b3fd0 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.033+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1124014549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ac0cf0b0 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.033+0000 7ff5a2ffd700 1 -- 172.21.15.57:0/1124014549 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff59c00f050 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.034+0000 7ff5a2ffd700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff58c0682f0 0x7ff58c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.034+0000 7ff5a2ffd700 1 -- 172.21.15.57:0/1124014549 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff59c086d40 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.034+0000 7ff5a9d9b700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff58c0682f0 0x7ff58c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.035+0000 7ff5a9d9b700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff58c0682f0 0x7ff58c06a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0af9f0 tx=0x7ff594007c90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:35.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.038+0000 7ff5a2ffd700 1 -- 172.21.15.57:0/1124014549 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff59c053560 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.238+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1124014549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5ac0743f0 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.239+0000 7ff5a2ffd700 1 -- 172.21.15.57:0/1124014549 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff59c015020 con 0x7ff5ac0b27d0 2023-11-28T12:02:35.240 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:02:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.241+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff58c0682f0 msgr2=0x7ff58c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.241+0000 7ff5b0b3d700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff58c0682f0 0x7ff58c06a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0af9f0 tx=0x7ff594007c90 comp rx=0 tx=0).stop 2023-11-28T12:02:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.241+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1124014549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5ac0b27d0 msgr2=0x7ff5ac0adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.241+0000 7ff5b0b3d700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5ac0b27d0 0x7ff5ac0adfd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff59c014040 tx=0x7ff59c004750 comp rx=0 tx=0).stop 2023-11-28T12:02:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.241+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1124014549 shutdown_connections 2023-11-28T12:02:35.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.241+0000 7ff5b0b3d700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff58c0682f0 0x7ff58c06a7a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.241+0000 7ff5b0b3d700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac15fe50 0x7ff5ac0c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.241+0000 7ff5b0b3d700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5ac0b27d0 0x7ff5ac0adfd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.241+0000 7ff5b0b3d700 1 --2- 172.21.15.57:0/1124014549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac0b3b80 0x7ff5ac15e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.241+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1124014549 >> 172.21.15.57:0/1124014549 conn(0x7ff5ac06ea80 msgr2=0x7ff5ac0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:35.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.241+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1124014549 shutdown_connections 2023-11-28T12:02:35.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.241+0000 7ff5b0b3d700 1 -- 172.21.15.57:0/1124014549 wait complete. 2023-11-28T12:02:35.256 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:02:35.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.392+0000 7ff630562700 1 Processor -- start 2023-11-28T12:02:35.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.392+0000 7ff630562700 1 -- start start 2023-11-28T12:02:35.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.393+0000 7ff630562700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff62815a4d0 0x7ff62806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.393+0000 7ff630562700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff62806a7b0 0x7ff6280cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.393+0000 7ff630562700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6280cdd50 0x7ff6280b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.393+0000 7ff630562700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62806ac30 con 0x7ff62815a4d0 2023-11-28T12:02:35.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.393+0000 7ff630562700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62815e0d0 con 0x7ff62806a7b0 2023-11-28T12:02:35.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.393+0000 7ff630562700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280ce560 con 0x7ff6280cdd50 2023-11-28T12:02:35.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.393+0000 7ff62e2fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff62815a4d0 0x7ff62806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.393+0000 7ff62e2fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff62815a4d0 0x7ff62806a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50708/0 (socket says 172.21.15.57:50708) 2023-11-28T12:02:35.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.393+0000 7ff62e2fe700 1 -- 172.21.15.57:0/1112532635 learned_addr learned my addr 172.21.15.57:0/1112532635 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:35.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.394+0000 7ff62eaff700 1 --2- 172.21.15.57:0/1112532635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6280cdd50 0x7ff6280b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.394+0000 7ff62dafd700 1 --2- 172.21.15.57:0/1112532635 >> v2:172.21.15.181:3300/0 conn(0x7ff62806a7b0 0x7ff6280cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.394+0000 7ff62e2fe700 1 -- 172.21.15.57:0/1112532635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6280cdd50 msgr2=0x7ff6280b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:35.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.394+0000 7ff62e2fe700 1 --2- 172.21.15.57:0/1112532635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6280cdd50 0x7ff6280b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.394+0000 7ff62e2fe700 1 -- 172.21.15.57:0/1112532635 >> v2:172.21.15.181:3300/0 conn(0x7ff62806a7b0 msgr2=0x7ff6280cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:35.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.394+0000 7ff62e2fe700 1 --2- 172.21.15.57:0/1112532635 >> v2:172.21.15.181:3300/0 conn(0x7ff62806a7b0 0x7ff6280cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.394+0000 7ff62e2fe700 1 -- 172.21.15.57:0/1112532635 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62815e320 con 0x7ff62815a4d0 2023-11-28T12:02:35.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.394+0000 7ff62e2fe700 1 --2- 172.21.15.57:0/1112532635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff62815a4d0 0x7ff62806a270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff628070e20 tx=0x7ff624009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d40266ee837a165 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:35.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.394+0000 7ff62d2fc700 1 -- 172.21.15.57:0/1112532635 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62400b870 con 0x7ff62815a4d0 2023-11-28T12:02:35.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.394+0000 7ff62d2fc700 1 -- 172.21.15.57:0/1112532635 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62400b9d0 con 0x7ff62815a4d0 2023-11-28T12:02:35.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.394+0000 7ff62d2fc700 1 -- 172.21.15.57:0/1112532635 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62400bd60 con 0x7ff62815a4d0 2023-11-28T12:02:35.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.395+0000 7ff630562700 1 -- 172.21.15.57:0/1112532635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff62815a4d0 msgr2=0x7ff62806a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.395+0000 7ff630562700 1 --2- 172.21.15.57:0/1112532635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff62815a4d0 0x7ff62806a270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff628070e20 tx=0x7ff624009bd0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.395+0000 7ff630562700 1 -- 172.21.15.57:0/1112532635 shutdown_connections 2023-11-28T12:02:35.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.395+0000 7ff630562700 1 --2- 172.21.15.57:0/1112532635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6280cdd50 0x7ff6280b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.395+0000 7ff630562700 1 --2- 172.21.15.57:0/1112532635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff62815a4d0 0x7ff62806a270 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.395+0000 7ff630562700 1 --2- 172.21.15.57:0/1112532635 >> v2:172.21.15.181:3300/0 conn(0x7ff62806a7b0 0x7ff6280cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.395+0000 7ff630562700 1 -- 172.21.15.57:0/1112532635 >> 172.21.15.57:0/1112532635 conn(0x7ff62806c010 msgr2=0x7ff6280cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:35.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.395+0000 7ff630562700 1 -- 172.21.15.57:0/1112532635 shutdown_connections 2023-11-28T12:02:35.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.395+0000 7ff630562700 1 -- 172.21.15.57:0/1112532635 wait complete. 2023-11-28T12:02:35.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.397+0000 7ff630562700 1 Processor -- start 2023-11-28T12:02:35.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.397+0000 7ff630562700 1 -- start start 2023-11-28T12:02:35.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff630562700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff62806a7b0 0x7ff6280c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff630562700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6280cdd50 0x7ff62814b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff630562700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff62815a4d0 0x7ff6281f0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff630562700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62814c2f0 con 0x7ff62806a7b0 2023-11-28T12:02:35.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff630562700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280c6b20 con 0x7ff62815a4d0 2023-11-28T12:02:35.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff630562700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62814b810 con 0x7ff6280cdd50 2023-11-28T12:02:35.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff62e2fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff62806a7b0 0x7ff6280c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff62e2fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff62806a7b0 0x7ff6280c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50710/0 (socket says 172.21.15.57:50710) 2023-11-28T12:02:35.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff62e2fe700 1 -- 172.21.15.57:0/2033804391 learned_addr learned my addr 172.21.15.57:0/2033804391 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:35.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff62dafd700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6280cdd50 0x7ff62814b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff62eaff700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff62815a4d0 0x7ff6281f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff62e2fe700 1 -- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6280cdd50 msgr2=0x7ff62814b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff62e2fe700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6280cdd50 0x7ff62814b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff62e2fe700 1 -- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff62815a4d0 msgr2=0x7ff6281f0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff62e2fe700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff62815a4d0 0x7ff6281f0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff62e2fe700 1 -- 172.21.15.57:0/2033804391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff624009550 con 0x7ff62806a7b0 2023-11-28T12:02:35.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff62e2fe700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff62806a7b0 0x7ff6280c65e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff624014040 tx=0x7ff624004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:35.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff61affd700 1 -- 172.21.15.57:0/2033804391 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624004250 con 0x7ff62806a7b0 2023-11-28T12:02:35.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff61affd700 1 -- 172.21.15.57:0/2033804391 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62401e070 con 0x7ff62806a7b0 2023-11-28T12:02:35.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff61affd700 1 -- 172.21.15.57:0/2033804391 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240045d0 con 0x7ff62806a7b0 2023-11-28T12:02:35.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.398+0000 7ff630562700 1 -- 172.21.15.57:0/2033804391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62815e8e0 con 0x7ff62806a7b0 2023-11-28T12:02:35.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.399+0000 7ff630562700 1 -- 172.21.15.57:0/2033804391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62815e600 con 0x7ff62806a7b0 2023-11-28T12:02:35.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.399+0000 7ff630562700 1 -- 172.21.15.57:0/2033804391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62815a8a0 con 0x7ff62806a7b0 2023-11-28T12:02:35.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.400+0000 7ff61affd700 1 -- 172.21.15.57:0/2033804391 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff624029020 con 0x7ff62806a7b0 2023-11-28T12:02:35.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.400+0000 7ff61affd700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff60c0682f0 0x7ff60c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.401+0000 7ff61affd700 1 -- 172.21.15.57:0/2033804391 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff624086ee0 con 0x7ff62806a7b0 2023-11-28T12:02:35.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.401+0000 7ff62dafd700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff60c0682f0 0x7ff60c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.401+0000 7ff62dafd700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff60c0682f0 0x7ff60c06a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff6280ca210 tx=0x7ff61c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:35.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.405+0000 7ff61affd700 1 -- 172.21.15.57:0/2033804391 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff624053610 con 0x7ff62806a7b0 2023-11-28T12:02:35.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.603+0000 7ff630562700 1 -- 172.21.15.57:0/2033804391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff62814c070 con 0x7ff62806a7b0 2023-11-28T12:02:35.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.604+0000 7ff61affd700 1 -- 172.21.15.57:0/2033804391 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff624056c30 con 0x7ff62806a7b0 2023-11-28T12:02:35.605 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:02:35.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.606+0000 7ff630562700 1 -- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff60c0682f0 msgr2=0x7ff60c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.606+0000 7ff630562700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff60c0682f0 0x7ff60c06a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff6280ca210 tx=0x7ff61c008040 comp rx=0 tx=0).stop 2023-11-28T12:02:35.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.606+0000 7ff630562700 1 -- 172.21.15.57:0/2033804391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff62806a7b0 msgr2=0x7ff6280c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.606+0000 7ff630562700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff62806a7b0 0x7ff6280c65e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff624014040 tx=0x7ff624004750 comp rx=0 tx=0).stop 2023-11-28T12:02:35.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.606+0000 7ff630562700 1 -- 172.21.15.57:0/2033804391 shutdown_connections 2023-11-28T12:02:35.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.606+0000 7ff630562700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff60c0682f0 0x7ff60c06a7a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.606+0000 7ff630562700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff62815a4d0 0x7ff6281f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.606+0000 7ff630562700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff62806a7b0 0x7ff6280c65e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.606+0000 7ff630562700 1 --2- 172.21.15.57:0/2033804391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6280cdd50 0x7ff62814b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.606+0000 7ff630562700 1 -- 172.21.15.57:0/2033804391 >> 172.21.15.57:0/2033804391 conn(0x7ff62806c010 msgr2=0x7ff6280b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:35.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.606+0000 7ff630562700 1 -- 172.21.15.57:0/2033804391 shutdown_connections 2023-11-28T12:02:35.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.606+0000 7ff630562700 1 -- 172.21.15.57:0/2033804391 wait complete. 2023-11-28T12:02:35.621 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T12:02:35.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.757+0000 7ff5477f1700 1 Processor -- start 2023-11-28T12:02:35.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.757+0000 7ff5477f1700 1 -- start start 2023-11-28T12:02:35.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff5477f1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff5401599b0 0x7ff54015d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff5477f1700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff540159ec0 con 0x7ff5400b1650 2023-11-28T12:02:35.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff5477f1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54015aa60 con 0x7ff5400af0a0 2023-11-28T12:02:35.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff5477f1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54015d8a0 con 0x7ff5401599b0 2023-11-28T12:02:35.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff54558d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff5400b1650 0x7ff5400b1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37386/0 (socket says 172.21.15.57:37386) 2023-11-28T12:02:35.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff54558d700 1 -- 172.21.15.57:0/1012200664 learned_addr learned my addr 172.21.15.57:0/1012200664 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:35.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff544d8c700 1 --2- 172.21.15.57:0/1012200664 >> v2:172.21.15.181:3300/0 conn(0x7ff5401599b0 0x7ff54015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544945090 0 0) 0x7ff540159ec0 con 0x7ff5400b1650 2023-11-28T12:02:35.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff52c0057c0 con 0x7ff5400b1650 2023-11-28T12:02:35.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3818709235 0 0) 0x7ff52c0057c0 con 0x7ff5400b1650 2023-11-28T12:02:35.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 >> v2:172.21.15.181:3300/0 conn(0x7ff5401599b0 msgr2=0x7ff54015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:35.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff537fff700 1 --2- 172.21.15.57:0/1012200664 >> v2:172.21.15.181:3300/0 conn(0x7ff5401599b0 0x7ff54015d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 >> v1:172.21.15.181:6789/0 conn(0x7ff5400af0a0 legacy=0x7ff54015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54015db80 con 0x7ff5400b1650 2023-11-28T12:02:35.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7ff53c001670 con 0x7ff5400b1650 2023-11-28T12:02:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 >> v1:172.21.15.57:6789/0 conn(0x7ff5400b1650 legacy=0x7ff5400b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff537fff700 1 --2- 172.21.15.57:0/1012200664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff52c005c10 0x7ff52c006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.758+0000 7ff537fff700 1 --2- 172.21.15.57:0/1012200664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff52c006a60 0x7ff52c006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff537fff700 1 --2- 172.21.15.57:0/1012200664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff52c007410 0x7ff52c00dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54015d8a0 con 0x7ff52c005c10 2023-11-28T12:02:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c00e290 con 0x7ff52c006a60 2023-11-28T12:02:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c00e450 con 0x7ff52c007410 2023-11-28T12:02:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff54558d700 1 --2- 172.21.15.57:0/1012200664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff52c005c10 0x7ff52c006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff544d8c700 1 --2- 172.21.15.57:0/1012200664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff52c006a60 0x7ff52c006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff545d8e700 1 --2- 172.21.15.57:0/1012200664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff52c007410 0x7ff52c00dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff54558d700 1 -- 172.21.15.57:0/1012200664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff52c007410 msgr2=0x7ff52c00dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff54558d700 1 --2- 172.21.15.57:0/1012200664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff52c007410 0x7ff52c00dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff54558d700 1 -- 172.21.15.57:0/1012200664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff52c006a60 msgr2=0x7ff52c006ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff54558d700 1 --2- 172.21.15.57:0/1012200664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff52c006a60 0x7ff52c006ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff54558d700 1 -- 172.21.15.57:0/1012200664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c0057c0 con 0x7ff52c005c10 2023-11-28T12:02:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff545d8e700 1 -- 172.21.15.57:0/1012200664 reap_dead start 2023-11-28T12:02:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff54558d700 1 --2- 172.21.15.57:0/1012200664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff52c005c10 0x7ff52c006020 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff53c001050 tx=0x7ff52c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e01c2d073355efa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00cd80 con 0x7ff52c005c10 2023-11-28T12:02:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c005960 con 0x7ff52c005c10 2023-11-28T12:02:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.759+0000 7ff537fff700 1 -- 172.21.15.57:0/1012200664 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c005d10 con 0x7ff52c005c10 2023-11-28T12:02:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.760+0000 7ff5477f1700 1 -- 172.21.15.57:0/1012200664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff52c005c10 msgr2=0x7ff52c006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.760+0000 7ff5477f1700 1 --2- 172.21.15.57:0/1012200664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff52c005c10 0x7ff52c006020 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff53c001050 tx=0x7ff52c007a90 comp rx=0 tx=0).stop 2023-11-28T12:02:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.760+0000 7ff5477f1700 1 -- 172.21.15.57:0/1012200664 shutdown_connections 2023-11-28T12:02:35.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.760+0000 7ff5477f1700 1 --2- 172.21.15.57:0/1012200664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff52c005c10 0x7ff52c006020 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.760+0000 7ff5477f1700 1 -- 172.21.15.57:0/1012200664 >> 172.21.15.57:0/1012200664 conn(0x7ff54006c010 msgr2=0x7ff5400cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:35.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.760+0000 7ff5477f1700 1 -- 172.21.15.57:0/1012200664 shutdown_connections 2023-11-28T12:02:35.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.760+0000 7ff5477f1700 1 -- 172.21.15.57:0/1012200664 wait complete. 2023-11-28T12:02:35.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.761+0000 7ff5477f1700 1 Processor -- start 2023-11-28T12:02:35.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.761+0000 7ff5477f1700 1 -- start start 2023-11-28T12:02:35.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.762+0000 7ff5477f1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff52c005c10 0x7ff5400b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.762+0000 7ff5477f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff540144f30 0x7ff5400af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.762+0000 7ff5477f1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5401453a0 0x7ff5400afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.762+0000 7ff5477f1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400b3da0 con 0x7ff5401453a0 2023-11-28T12:02:35.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.762+0000 7ff5477f1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5401583d0 con 0x7ff540144f30 2023-11-28T12:02:35.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.762+0000 7ff5477f1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400b2aa0 con 0x7ff52c005c10 2023-11-28T12:02:35.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.762+0000 7ff545d8e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5401453a0 0x7ff5400afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.762+0000 7ff545d8e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5401453a0 0x7ff5400afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50728/0 (socket says 172.21.15.57:50728) 2023-11-28T12:02:35.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.762+0000 7ff54558d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff52c005c10 0x7ff5400b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.762+0000 7ff545d8e700 1 -- 172.21.15.57:0/1441258294 learned_addr learned my addr 172.21.15.57:0/1441258294 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:35.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.762+0000 7ff544d8c700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff540144f30 0x7ff5400af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.763+0000 7ff545d8e700 1 -- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff52c005c10 msgr2=0x7ff5400b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.763+0000 7ff545d8e700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff52c005c10 0x7ff5400b3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.763+0000 7ff545d8e700 1 -- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff540144f30 msgr2=0x7ff5400af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.763+0000 7ff545d8e700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff540144f30 0x7ff5400af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.763+0000 7ff545d8e700 1 -- 172.21.15.57:0/1441258294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54015a350 con 0x7ff5401453a0 2023-11-28T12:02:35.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.763+0000 7ff54558d700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff52c005c10 0x7ff5400b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:35.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.763+0000 7ff545d8e700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5401453a0 0x7ff5400afdb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff5400b2620 tx=0x7ff53800d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:35.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.763+0000 7ff535ffb700 1 -- 172.21.15.57:0/1441258294 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff538003c10 con 0x7ff5401453a0 2023-11-28T12:02:35.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.763+0000 7ff535ffb700 1 -- 172.21.15.57:0/1441258294 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff538003d90 con 0x7ff5401453a0 2023-11-28T12:02:35.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.763+0000 7ff5477f1700 1 -- 172.21.15.57:0/1441258294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54015de60 con 0x7ff5401453a0 2023-11-28T12:02:35.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.763+0000 7ff535ffb700 1 -- 172.21.15.57:0/1441258294 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53800d960 con 0x7ff5401453a0 2023-11-28T12:02:35.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.763+0000 7ff5477f1700 1 -- 172.21.15.57:0/1441258294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54015a4e0 con 0x7ff5401453a0 2023-11-28T12:02:35.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.764+0000 7ff5477f1700 1 -- 172.21.15.57:0/1441258294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5400743f0 con 0x7ff5401453a0 2023-11-28T12:02:35.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.765+0000 7ff535ffb700 1 -- 172.21.15.57:0/1441258294 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff53800dac0 con 0x7ff5401453a0 2023-11-28T12:02:35.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.765+0000 7ff535ffb700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff5240682f0 0x7ff52406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:35.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.765+0000 7ff535ffb700 1 -- 172.21.15.57:0/1441258294 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff538085fc0 con 0x7ff5401453a0 2023-11-28T12:02:35.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.766+0000 7ff54558d700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff5240682f0 0x7ff52406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:35.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.766+0000 7ff54558d700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff5240682f0 0x7ff52406a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff53c000c00 tx=0x7ff53c00a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:35.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.770+0000 7ff535ffb700 1 -- 172.21.15.57:0/1441258294 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff538052750 con 0x7ff5401453a0 2023-11-28T12:02:35.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.984+0000 7ff5477f1700 1 -- 172.21.15.57:0/1441258294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff54006e620 con 0x7ff5401453a0 2023-11-28T12:02:35.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.985+0000 7ff535ffb700 1 -- 172.21.15.57:0/1441258294 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff538055d70 con 0x7ff5401453a0 2023-11-28T12:02:35.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.987+0000 7ff5477f1700 1 -- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff5240682f0 msgr2=0x7ff52406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.987+0000 7ff5477f1700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff5240682f0 0x7ff52406a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff53c000c00 tx=0x7ff53c00a000 comp rx=0 tx=0).stop 2023-11-28T12:02:35.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.987+0000 7ff5477f1700 1 -- 172.21.15.57:0/1441258294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5401453a0 msgr2=0x7ff5400afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:35.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.987+0000 7ff5477f1700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5401453a0 0x7ff5400afdb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff5400b2620 tx=0x7ff53800d150 comp rx=0 tx=0).stop 2023-11-28T12:02:35.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.987+0000 7ff5477f1700 1 -- 172.21.15.57:0/1441258294 shutdown_connections 2023-11-28T12:02:35.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.987+0000 7ff5477f1700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff5240682f0 0x7ff52406a7a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.987+0000 7ff5477f1700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5401453a0 0x7ff5400afdb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.987+0000 7ff5477f1700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff52c005c10 0x7ff5400b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.987+0000 7ff5477f1700 1 --2- 172.21.15.57:0/1441258294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff540144f30 0x7ff5400af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:35.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.987+0000 7ff5477f1700 1 -- 172.21.15.57:0/1441258294 >> 172.21.15.57:0/1441258294 conn(0x7ff54006c010 msgr2=0x7ff5400b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:35.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.987+0000 7ff5477f1700 1 -- 172.21.15.57:0/1441258294 shutdown_connections 2023-11-28T12:02:35.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:35.987+0000 7ff5477f1700 1 -- 172.21.15.57:0/1441258294 wait complete. 2023-11-28T12:02:35.990 INFO:teuthology.orchestra.run.smithi057.stderr:listed 0 entries 2023-11-28T12:02:36.002 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2023-11-28T12:02:36.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:36.002 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-28T12:02:36.044 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:02:36.044 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:02:36.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:02:36.045 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T12:02:36.106 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T12:02:36.107 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:02:36.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.245+0000 7f4d96652700 1 Processor -- start 2023-11-28T12:02:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.245+0000 7f4d96652700 1 -- start start 2023-11-28T12:02:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.246+0000 7f4d96652700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d90142db0 0x7f4d9013ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.246+0000 7f4d96652700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d900b1770 0x7f4d900b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.246+0000 7f4d96652700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d900b2110 con 0x7f4d900ce040 2023-11-28T12:02:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.246+0000 7f4d96652700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d9015e010 con 0x7f4d90142db0 2023-11-28T12:02:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.246+0000 7f4d96652700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9013fc90 con 0x7f4d900b1770 2023-11-28T12:02:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.246+0000 7f4d8ffff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4d900ce040 0x7f4d9015d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37402/0 (socket says 172.21.15.57:37402) 2023-11-28T12:02:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.246+0000 7f4d8ffff700 1 -- 172.21.15.57:0/2885913240 learned_addr learned my addr 172.21.15.57:0/2885913240 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808829378 0 0) 0x7f4d900b2110 con 0x7f4d900ce040 2023-11-28T12:02:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d780035b0 con 0x7f4d900ce040 2023-11-28T12:02:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d94bef700 1 --2- 172.21.15.57:0/2885913240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d900b1770 0x7f4d900b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8f7fe700 1 --2- 172.21.15.57:0/2885913240 >> v2:172.21.15.181:3300/0 conn(0x7f4d90142db0 0x7f4d9013ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1730717542 0 0) 0x7f4d780035b0 con 0x7f4d900ce040 2023-11-28T12:02:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d900b1770 msgr2=0x7f4d900b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 --2- 172.21.15.57:0/2885913240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d900b1770 0x7f4d900b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 >> v2:172.21.15.181:3300/0 conn(0x7f4d90142db0 msgr2=0x7f4d9013ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 --2- 172.21.15.57:0/2885913240 >> v2:172.21.15.181:3300/0 conn(0x7f4d90142db0 0x7f4d9013ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9013f6d0 con 0x7f4d900ce040 2023-11-28T12:02:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f4d80001670 con 0x7f4d900ce040 2023-11-28T12:02:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8f7fe700 1 --2- 172.21.15.57:0/2885913240 >> v2:172.21.15.181:3300/0 conn(0x7f4d90142db0 0x7f4d9013ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d94bef700 1 --2- 172.21.15.57:0/2885913240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d900b1770 0x7f4d900b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 >> v1:172.21.15.57:6789/0 conn(0x7f4d900ce040 legacy=0x7f4d9015d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 --2- 172.21.15.57:0/2885913240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d78003a00 0x7f4d78003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 --2- 172.21.15.57:0/2885913240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78004850 0x7f4d78004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 --2- 172.21.15.57:0/2885913240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d900b1770 0x7f4d78005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9015e010 con 0x7f4d78004850 2023-11-28T12:02:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9013fc90 con 0x7f4d78003a00 2023-11-28T12:02:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7800ba60 con 0x7f4d900b1770 2023-11-28T12:02:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8f7fe700 1 --2- 172.21.15.57:0/2885913240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78004850 0x7f4d78004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d94bef700 1 --2- 172.21.15.57:0/2885913240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d900b1770 0x7f4d78005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8ffff700 1 --2- 172.21.15.57:0/2885913240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d78003a00 0x7f4d78003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.247+0000 7f4d8f7fe700 1 -- 172.21.15.57:0/2885913240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d900b1770 msgr2=0x7f4d78005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.248+0000 7f4d8f7fe700 1 --2- 172.21.15.57:0/2885913240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d900b1770 0x7f4d78005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.248+0000 7f4d8f7fe700 1 -- 172.21.15.57:0/2885913240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d78003a00 msgr2=0x7f4d78003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.248+0000 7f4d8f7fe700 1 --2- 172.21.15.57:0/2885913240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d78003a00 0x7f4d78003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.248+0000 7f4d8f7fe700 1 -- 172.21.15.57:0/2885913240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d780035b0 con 0x7f4d78004850 2023-11-28T12:02:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.248+0000 7f4d8f7fe700 1 --2- 172.21.15.57:0/2885913240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78004850 0x7f4d78004cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4d84009980 tx=0x7f4d78005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7af7c506559ea60 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.248+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8400c070 con 0x7f4d78004850 2023-11-28T12:02:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.248+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d84013610 con 0x7f4d78004850 2023-11-28T12:02:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.248+0000 7f4d8effd700 1 -- 172.21.15.57:0/2885913240 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d84013980 con 0x7f4d78004850 2023-11-28T12:02:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.248+0000 7f4d96652700 1 -- 172.21.15.57:0/2885913240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78004850 msgr2=0x7f4d78004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.248+0000 7f4d96652700 1 --2- 172.21.15.57:0/2885913240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78004850 0x7f4d78004cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4d84009980 tx=0x7f4d78005880 comp rx=0 tx=0).stop 2023-11-28T12:02:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.248+0000 7f4d94bef700 1 -- 172.21.15.57:0/2885913240 reap_dead start 2023-11-28T12:02:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.249+0000 7f4d96652700 1 -- 172.21.15.57:0/2885913240 shutdown_connections 2023-11-28T12:02:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.249+0000 7f4d96652700 1 -- 172.21.15.57:0/2885913240 >> 172.21.15.57:0/2885913240 conn(0x7f4d9006c010 msgr2=0x7f4d900cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.249+0000 7f4d96652700 1 -- 172.21.15.57:0/2885913240 shutdown_connections 2023-11-28T12:02:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.249+0000 7f4d96652700 1 -- 172.21.15.57:0/2885913240 wait complete. 2023-11-28T12:02:36.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.250+0000 7f4d96652700 1 Processor -- start 2023-11-28T12:02:36.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.250+0000 7f4d96652700 1 -- start start 2023-11-28T12:02:36.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d96652700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78004850 0x7f4d9006a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d96652700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d90155060 0x7f4d900b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d96652700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d900c5b80 0x7f4d9006ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d96652700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9006a850 con 0x7f4d78004850 2023-11-28T12:02:36.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d96652700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9006ab30 con 0x7f4d900c5b80 2023-11-28T12:02:36.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d96652700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d900b4ed0 con 0x7f4d90155060 2023-11-28T12:02:36.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d8ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78004850 0x7f4d9006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d8ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78004850 0x7f4d9006a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50758/0 (socket says 172.21.15.57:50758) 2023-11-28T12:02:36.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d8ffff700 1 -- 172.21.15.57:0/580456865 learned_addr learned my addr 172.21.15.57:0/580456865 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:36.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d8f7fe700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d90155060 0x7f4d900b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d94bef700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d900c5b80 0x7f4d9006ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d8ffff700 1 -- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d90155060 msgr2=0x7f4d900b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:36.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d8ffff700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d90155060 0x7f4d900b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d8ffff700 1 -- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d900c5b80 msgr2=0x7f4d9006ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:36.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d8ffff700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d900c5b80 0x7f4d9006ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d8ffff700 1 -- 172.21.15.57:0/580456865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9015d100 con 0x7f4d78004850 2023-11-28T12:02:36.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d8f7fe700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d90155060 0x7f4d900b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:36.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.251+0000 7f4d94bef700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d900c5b80 0x7f4d9006ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:36.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.252+0000 7f4d8ffff700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78004850 0x7f4d9006a270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4d80007340 tx=0x7f4d80007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:36.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.252+0000 7f4d8cff9700 1 -- 172.21.15.57:0/580456865 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8000f6c0 con 0x7f4d78004850 2023-11-28T12:02:36.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.252+0000 7f4d8cff9700 1 -- 172.21.15.57:0/580456865 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8000f840 con 0x7f4d78004850 2023-11-28T12:02:36.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.252+0000 7f4d8cff9700 1 -- 172.21.15.57:0/580456865 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8000fbf0 con 0x7f4d78004850 2023-11-28T12:02:36.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.252+0000 7f4d96652700 1 -- 172.21.15.57:0/580456865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9013f9b0 con 0x7f4d78004850 2023-11-28T12:02:36.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.252+0000 7f4d96652700 1 -- 172.21.15.57:0/580456865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9015d3e0 con 0x7f4d78004850 2023-11-28T12:02:36.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.253+0000 7f4d96652700 1 -- 172.21.15.57:0/580456865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d900b4d30 con 0x7f4d78004850 2023-11-28T12:02:36.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.253+0000 7f4d8cff9700 1 -- 172.21.15.57:0/580456865 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d8000fd50 con 0x7f4d78004850 2023-11-28T12:02:36.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.254+0000 7f4d8cff9700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4d7c0682f0 0x7f4d7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.254+0000 7f4d8cff9700 1 -- 172.21.15.57:0/580456865 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4d80085ed0 con 0x7f4d78004850 2023-11-28T12:02:36.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.254+0000 7f4d8f7fe700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4d7c0682f0 0x7f4d7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.254+0000 7f4d8f7fe700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4d7c0682f0 0x7f4d7c06a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4d90155470 tx=0x7f4d8400c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:36.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.258+0000 7f4d8cff9700 1 -- 172.21.15.57:0/580456865 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d900b4d30 con 0x7f4d78004850 2023-11-28T12:02:36.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.506+0000 7f4d96652700 1 -- 172.21.15.57:0/580456865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d900b4d30 con 0x7f4d78004850 2023-11-28T12:02:36.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.507+0000 7f4d8cff9700 1 -- 172.21.15.57:0/580456865 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4642 (secure 0 0 0) 0x7f4d900b4d30 con 0x7f4d78004850 2023-11-28T12:02:36.508 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:36.508 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/890913444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":890913444},{"type":"v1","addr":"172.21.15.181:6836","nonce":890913444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/847658150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":847658150},{"type":"v1","addr":"172.21.15.181:6837","nonce":847658150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1379013929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1379013929},{"type":"v1","addr":"172.21.15.57:6835","nonce":1379013929}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:01:43.919415+0000","modified":"2023-11-28T12:01:46.937696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6833/3776085595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3776085595},{"type":"v1","addr":"172.21.15.57:6833","nonce":3776085595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T12:02:36.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.509+0000 7f4d96652700 1 -- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4d7c0682f0 msgr2=0x7f4d7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:36.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.509+0000 7f4d96652700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4d7c0682f0 0x7f4d7c06a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4d90155470 tx=0x7f4d8400c000 comp rx=0 tx=0).stop 2023-11-28T12:02:36.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.509+0000 7f4d96652700 1 -- 172.21.15.57:0/580456865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78004850 msgr2=0x7f4d9006a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:36.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.509+0000 7f4d96652700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78004850 0x7f4d9006a270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4d80007340 tx=0x7f4d80007650 comp rx=0 tx=0).stop 2023-11-28T12:02:36.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.509+0000 7f4d96652700 1 -- 172.21.15.57:0/580456865 shutdown_connections 2023-11-28T12:02:36.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.509+0000 7f4d96652700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4d7c0682f0 0x7f4d7c06a7a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.509+0000 7f4d96652700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d900c5b80 0x7f4d9006ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.509+0000 7f4d96652700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78004850 0x7f4d9006a270 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.509+0000 7f4d96652700 1 --2- 172.21.15.57:0/580456865 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d90155060 0x7f4d900b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.509+0000 7f4d96652700 1 -- 172.21.15.57:0/580456865 >> 172.21.15.57:0/580456865 conn(0x7f4d9006c010 msgr2=0x7f4d9006e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:36.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.509+0000 7f4d96652700 1 -- 172.21.15.57:0/580456865 shutdown_connections 2023-11-28T12:02:36.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.509+0000 7f4d96652700 1 -- 172.21.15.57:0/580456865 wait complete. 2023-11-28T12:02:36.512 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 8 2023-11-28T12:02:36.524 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:36.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.658+0000 7f67049fe700 1 Processor -- start 2023-11-28T12:02:36.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.658+0000 7f67049fe700 1 -- start start 2023-11-28T12:02:36.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.659+0000 7f67049fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006a270 0x7f670006a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.659+0000 7f67049fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f670015cc80 0x7f670006de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.659+0000 7f67049fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f670006e3a0 0x7f670006ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.659+0000 7f67049fe700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67000b2490 con 0x7f670015cc80 2023-11-28T12:02:36.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.659+0000 7f67049fe700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670006ab80 con 0x7f670006e3a0 2023-11-28T12:02:36.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.659+0000 7f67049fe700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67000b1810 con 0x7f670006a270 2023-11-28T12:02:36.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.659+0000 7f66fdd9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f670015cc80 0x7f670006de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.659+0000 7f66fdd9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f670015cc80 0x7f670006de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50768/0 (socket says 172.21.15.57:50768) 2023-11-28T12:02:36.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.659+0000 7f66fdd9b700 1 -- 172.21.15.57:0/2690477130 learned_addr learned my addr 172.21.15.57:0/2690477130 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:36.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.660+0000 7f66fed9d700 1 --2- 172.21.15.57:0/2690477130 >> v2:172.21.15.181:3300/0 conn(0x7f670006e3a0 0x7f670006ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.660+0000 7f66fdd9b700 1 -- 172.21.15.57:0/2690477130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006a270 msgr2=0x7f670006a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:36.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.660+0000 7f66fdd9b700 1 --2- 172.21.15.57:0/2690477130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006a270 0x7f670006a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.660+0000 7f66fdd9b700 1 -- 172.21.15.57:0/2690477130 >> v2:172.21.15.181:3300/0 conn(0x7f670006e3a0 msgr2=0x7f670006ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:36.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.660+0000 7f66fdd9b700 1 --2- 172.21.15.57:0/2690477130 >> v2:172.21.15.181:3300/0 conn(0x7f670006e3a0 0x7f670006ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.660+0000 7f66fdd9b700 1 -- 172.21.15.57:0/2690477130 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67000b1af0 con 0x7f670015cc80 2023-11-28T12:02:36.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.660+0000 7f66fed9d700 1 --2- 172.21.15.57:0/2690477130 >> v2:172.21.15.181:3300/0 conn(0x7f670006e3a0 0x7f670006ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:36.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.660+0000 7f66fdd9b700 1 --2- 172.21.15.57:0/2690477130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f670015cc80 0x7f670006de60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f67000af700 tx=0x7f66e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c6dd78aed15861 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:36.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.660+0000 7f66fd59a700 1 -- 172.21.15.57:0/2690477130 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e800b870 con 0x7f670015cc80 2023-11-28T12:02:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.660+0000 7f66fd59a700 1 -- 172.21.15.57:0/2690477130 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e800b9d0 con 0x7f670015cc80 2023-11-28T12:02:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.660+0000 7f66fd59a700 1 -- 172.21.15.57:0/2690477130 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e800bd60 con 0x7f670015cc80 2023-11-28T12:02:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.661+0000 7f67049fe700 1 -- 172.21.15.57:0/2690477130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f670015cc80 msgr2=0x7f670006de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.661+0000 7f67049fe700 1 --2- 172.21.15.57:0/2690477130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f670015cc80 0x7f670006de60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f67000af700 tx=0x7f66e8009bd0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.661+0000 7f67049fe700 1 -- 172.21.15.57:0/2690477130 shutdown_connections 2023-11-28T12:02:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.661+0000 7f67049fe700 1 --2- 172.21.15.57:0/2690477130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006a270 0x7f670006a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.661+0000 7f67049fe700 1 --2- 172.21.15.57:0/2690477130 >> v2:172.21.15.181:3300/0 conn(0x7f670006e3a0 0x7f670006ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.661+0000 7f67049fe700 1 --2- 172.21.15.57:0/2690477130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f670015cc80 0x7f670006de60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.661+0000 7f67049fe700 1 -- 172.21.15.57:0/2690477130 >> 172.21.15.57:0/2690477130 conn(0x7f670006c010 msgr2=0x7f67000cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.661+0000 7f67049fe700 1 -- 172.21.15.57:0/2690477130 shutdown_connections 2023-11-28T12:02:36.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.661+0000 7f67049fe700 1 -- 172.21.15.57:0/2690477130 wait complete. 2023-11-28T12:02:36.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.662+0000 7f67049fe700 1 Processor -- start 2023-11-28T12:02:36.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.662+0000 7f67049fe700 1 -- start start 2023-11-28T12:02:36.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f67049fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006a270 0x7f67000ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f67049fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f670006e3a0 0x7f67000c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f67049fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f670015cc80 0x7f67000b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f67049fe700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000b2b20 con 0x7f670006e3a0 2023-11-28T12:02:36.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f67049fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6700153090 con 0x7f670015cc80 2023-11-28T12:02:36.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f67049fe700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67000c98c0 con 0x7f670006a270 2023-11-28T12:02:36.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f66fdd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f670006e3a0 0x7f67000c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f66fdd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f670006e3a0 0x7f67000c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50780/0 (socket says 172.21.15.57:50780) 2023-11-28T12:02:36.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f66fdd9b700 1 -- 172.21.15.57:0/2969499689 learned_addr learned my addr 172.21.15.57:0/2969499689 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:36.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f66fed9d700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f670015cc80 0x7f67000b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f66fe59c700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006a270 0x7f67000ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f66fdd9b700 1 -- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006a270 msgr2=0x7f67000ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:36.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f66fdd9b700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006a270 0x7f67000ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f66fdd9b700 1 -- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f670015cc80 msgr2=0x7f67000b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:36.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.663+0000 7f66fdd9b700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f670015cc80 0x7f67000b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.664+0000 7f66fdd9b700 1 -- 172.21.15.57:0/2969499689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e8009550 con 0x7f670006e3a0 2023-11-28T12:02:36.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.664+0000 7f66fed9d700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f670015cc80 0x7f67000b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:36.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.664+0000 7f66fe59c700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006a270 0x7f67000ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:36.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.664+0000 7f66fdd9b700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f670006e3a0 0x7f67000c9090 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f66e8014040 tx=0x7f66e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:36.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.664+0000 7f66f6ffd700 1 -- 172.21.15.57:0/2969499689 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8004310 con 0x7f670006e3a0 2023-11-28T12:02:36.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.664+0000 7f66f6ffd700 1 -- 172.21.15.57:0/2969499689 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e801f070 con 0x7f670006e3a0 2023-11-28T12:02:36.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.664+0000 7f67049fe700 1 -- 172.21.15.57:0/2969499689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67000b20b0 con 0x7f670006e3a0 2023-11-28T12:02:36.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.664+0000 7f66f6ffd700 1 -- 172.21.15.57:0/2969499689 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e80036a0 con 0x7f670006e3a0 2023-11-28T12:02:36.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.664+0000 7f67049fe700 1 -- 172.21.15.57:0/2969499689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67000b1dd0 con 0x7f670006e3a0 2023-11-28T12:02:36.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.665+0000 7f67049fe700 1 -- 172.21.15.57:0/2969499689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670006e7e0 con 0x7f670006e3a0 2023-11-28T12:02:36.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.665+0000 7f66f6ffd700 1 -- 172.21.15.57:0/2969499689 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f66e800f050 con 0x7f670006e3a0 2023-11-28T12:02:36.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.666+0000 7f66f6ffd700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f66e0068320 0x7f66e006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:36.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.666+0000 7f66f6ffd700 1 -- 172.21.15.57:0/2969499689 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f66e8087180 con 0x7f670006e3a0 2023-11-28T12:02:36.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.666+0000 7f66fe59c700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f66e0068320 0x7f66e006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:36.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.667+0000 7f66fe59c700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f66e0068320 0x7f66e006a7d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f66f000a8f0 tx=0x7f66f0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:36.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.671+0000 7f66f6ffd700 1 -- 172.21.15.57:0/2969499689 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f66e80539a0 con 0x7f670006e3a0 2023-11-28T12:02:36.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.870+0000 7f67049fe700 1 -- 172.21.15.57:0/2969499689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67000743f0 con 0x7f670006e3a0 2023-11-28T12:02:36.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.874+0000 7f66f6ffd700 1 -- 172.21.15.57:0/2969499689 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17945 (secure 0 0 0) 0x7f66e8056fc0 con 0x7f670006e3a0 2023-11-28T12:02:36.875 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:36.875 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:02:32.851731+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:36.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.876+0000 7f67049fe700 1 -- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f66e0068320 msgr2=0x7f66e006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.876+0000 7f67049fe700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f66e0068320 0x7f66e006a7d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f66f000a8f0 tx=0x7f66f0008040 comp rx=0 tx=0).stop 2023-11-28T12:02:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.876+0000 7f67049fe700 1 -- 172.21.15.57:0/2969499689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f670006e3a0 msgr2=0x7f67000c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.876+0000 7f67049fe700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f670006e3a0 0x7f67000c9090 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f66e8014040 tx=0x7f66e8004750 comp rx=0 tx=0).stop 2023-11-28T12:02:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.876+0000 7f67049fe700 1 -- 172.21.15.57:0/2969499689 shutdown_connections 2023-11-28T12:02:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.876+0000 7f67049fe700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f66e0068320 0x7f66e006a7d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.876+0000 7f67049fe700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f670015cc80 0x7f67000b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.876+0000 7f67049fe700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006a270 0x7f67000ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.876+0000 7f67049fe700 1 --2- 172.21.15.57:0/2969499689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f670006e3a0 0x7f67000c9090 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.876+0000 7f67049fe700 1 -- 172.21.15.57:0/2969499689 >> 172.21.15.57:0/2969499689 conn(0x7f670006c010 msgr2=0x7f67000cc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.876+0000 7f67049fe700 1 -- 172.21.15.57:0/2969499689 shutdown_connections 2023-11-28T12:02:36.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:36.876+0000 7f67049fe700 1 -- 172.21.15.57:0/2969499689 wait complete. 2023-11-28T12:02:36.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:01:41.687926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:01:42.895826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:36.892 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:37.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.029+0000 7f3bd1dc1700 1 Processor -- start 2023-11-28T12:02:37.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.029+0000 7f3bd1dc1700 1 -- start start 2023-11-28T12:02:37.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.030+0000 7f3bd1dc1700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3bcc0cf710 0x7f3bcc0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.030+0000 7f3bd1dc1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3bcc06a980 0x7f3bcc071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.030+0000 7f3bd1dc1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bcc15cc10 con 0x7f3bcc071890 2023-11-28T12:02:37.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.030+0000 7f3bd1dc1700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc06c9b0 con 0x7f3bcc0cf710 2023-11-28T12:02:37.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.030+0000 7f3bd1dc1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc06cc90 con 0x7f3bcc06a980 2023-11-28T12:02:37.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.030+0000 7f3bcb7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3bcc0cf710 0x7f3bcc0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.030+0000 7f3bcb7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3bcc0cf710 0x7f3bcc0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50784/0 (socket says 172.21.15.57:50784) 2023-11-28T12:02:37.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.030+0000 7f3bcb7fe700 1 -- 172.21.15.57:0/3928782849 learned_addr learned my addr 172.21.15.57:0/3928782849 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:37.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.030+0000 7f3bcbfff700 1 --2- 172.21.15.57:0/3928782849 >> v2:172.21.15.181:3300/0 conn(0x7f3bcc06a980 0x7f3bcc071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.031+0000 7f3bcb7fe700 1 -- 172.21.15.57:0/3928782849 >> v2:172.21.15.181:3300/0 conn(0x7f3bcc06a980 msgr2=0x7f3bcc071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:37.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.031+0000 7f3bcb7fe700 1 --2- 172.21.15.57:0/3928782849 >> v2:172.21.15.181:3300/0 conn(0x7f3bcc06a980 0x7f3bcc071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.031+0000 7f3bcb7fe700 1 -- 172.21.15.57:0/3928782849 >> v1:172.21.15.181:6789/0 conn(0x7f3bcc071890 legacy=0x7f3bcc06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.031+0000 7f3bcb7fe700 1 -- 172.21.15.57:0/3928782849 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc06cf70 con 0x7f3bcc0cf710 2023-11-28T12:02:37.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.031+0000 7f3bcb7fe700 1 --2- 172.21.15.57:0/3928782849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc0cf710 0x7f3bcc0ce740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0098f0 tx=0x7f3bbc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ff53e519bfc9c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:37.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.031+0000 7f3bca7fc700 1 -- 172.21.15.57:0/3928782849 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc011830 con 0x7f3bcc0cf710 2023-11-28T12:02:37.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.031+0000 7f3bca7fc700 1 -- 172.21.15.57:0/3928782849 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc011990 con 0x7f3bcc0cf710 2023-11-28T12:02:37.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.031+0000 7f3bca7fc700 1 -- 172.21.15.57:0/3928782849 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc011d20 con 0x7f3bcc0cf710 2023-11-28T12:02:37.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.032+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/3928782849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc0cf710 msgr2=0x7f3bcc0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.032+0000 7f3bd1dc1700 1 --2- 172.21.15.57:0/3928782849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc0cf710 0x7f3bcc0ce740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0098f0 tx=0x7f3bbc009c00 comp rx=0 tx=0).stop 2023-11-28T12:02:37.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.032+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/3928782849 shutdown_connections 2023-11-28T12:02:37.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.032+0000 7f3bd1dc1700 1 --2- 172.21.15.57:0/3928782849 >> v2:172.21.15.181:3300/0 conn(0x7f3bcc06a980 0x7f3bcc071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.032+0000 7f3bd1dc1700 1 --2- 172.21.15.57:0/3928782849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc0cf710 0x7f3bcc0ce740 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.032+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/3928782849 >> 172.21.15.57:0/3928782849 conn(0x7f3bcc06c010 msgr2=0x7f3bcc0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:37.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.032+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/3928782849 shutdown_connections 2023-11-28T12:02:37.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.032+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/3928782849 wait complete. 2023-11-28T12:02:37.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.033+0000 7f3bd1dc1700 1 Processor -- start 2023-11-28T12:02:37.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.033+0000 7f3bd1dc1700 1 -- start start 2023-11-28T12:02:37.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bd1dc1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc06a980 0x7f3bcc1f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bd1dc1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bcc071890 0x7f3bcc1f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bd1dc1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc0cf710 0x7f3bcc0ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bd1dc1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0cb200 con 0x7f3bcc06a980 2023-11-28T12:02:37.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bd1dc1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc15ee40 con 0x7f3bcc071890 2023-11-28T12:02:37.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bd1dc1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0cbd60 con 0x7f3bcc0cf710 2023-11-28T12:02:37.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bcb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc06a980 0x7f3bcc1f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bcb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc06a980 0x7f3bcc1f5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50794/0 (socket says 172.21.15.57:50794) 2023-11-28T12:02:37.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bcb7fe700 1 -- 172.21.15.57:0/828288124 learned_addr learned my addr 172.21.15.57:0/828288124 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:37.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bcb7fe700 1 -- 172.21.15.57:0/828288124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc0cf710 msgr2=0x7f3bcc0ad0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bcb7fe700 1 --2- 172.21.15.57:0/828288124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc0cf710 0x7f3bcc0ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bcb7fe700 1 -- 172.21.15.57:0/828288124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bcc071890 msgr2=0x7f3bcc1f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bcb7fe700 1 --2- 172.21.15.57:0/828288124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bcc071890 0x7f3bcc1f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.034+0000 7f3bcb7fe700 1 -- 172.21.15.57:0/828288124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc009550 con 0x7f3bcc06a980 2023-11-28T12:02:37.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.035+0000 7f3bcb7fe700 1 --2- 172.21.15.57:0/828288124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc06a980 0x7f3bcc1f5590 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009130 tx=0x7f3bbc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:37.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.035+0000 7f3bb3fff700 1 -- 172.21.15.57:0/828288124 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0036a0 con 0x7f3bcc06a980 2023-11-28T12:02:37.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.035+0000 7f3bb3fff700 1 -- 172.21.15.57:0/828288124 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc01e070 con 0x7f3bcc06a980 2023-11-28T12:02:37.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.035+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/828288124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc06d530 con 0x7f3bcc06a980 2023-11-28T12:02:37.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.035+0000 7f3bb3fff700 1 -- 172.21.15.57:0/828288124 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc003a20 con 0x7f3bcc06a980 2023-11-28T12:02:37.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.035+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/828288124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc06d250 con 0x7f3bcc06a980 2023-11-28T12:02:37.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.036+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/828288124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc160480 con 0x7f3bcc06a980 2023-11-28T12:02:37.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.036+0000 7f3bb3fff700 1 -- 172.21.15.57:0/828288124 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3bbc00f050 con 0x7f3bcc06a980 2023-11-28T12:02:37.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.037+0000 7f3bb3fff700 1 --2- 172.21.15.57:0/828288124 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3bb80682f0 0x7f3bb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.037+0000 7f3bb3fff700 1 -- 172.21.15.57:0/828288124 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3bbc086c90 con 0x7f3bcc06a980 2023-11-28T12:02:37.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.037+0000 7f3bcaffd700 1 --2- 172.21.15.57:0/828288124 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3bb80682f0 0x7f3bb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.037+0000 7f3bcaffd700 1 --2- 172.21.15.57:0/828288124 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3bb80682f0 0x7f3bb806a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00012e0 tx=0x7f3bc0001780 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:37.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.041+0000 7f3bb3fff700 1 -- 172.21.15.57:0/828288124 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3bbc0534b0 con 0x7f3bcc06a980 2023-11-28T12:02:37.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.238+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/828288124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3bcc1f5ad0 con 0x7f3bcc06a980 2023-11-28T12:02:37.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.239+0000 7f3bb3fff700 1 -- 172.21.15.57:0/828288124 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3bbc0150a0 con 0x7f3bcc06a980 2023-11-28T12:02:37.240 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:37.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.241+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/828288124 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3bb80682f0 msgr2=0x7f3bb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.241+0000 7f3bd1dc1700 1 --2- 172.21.15.57:0/828288124 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3bb80682f0 0x7f3bb806a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00012e0 tx=0x7f3bc0001780 comp rx=0 tx=0).stop 2023-11-28T12:02:37.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.241+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/828288124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc06a980 msgr2=0x7f3bcc1f5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.241+0000 7f3bd1dc1700 1 --2- 172.21.15.57:0/828288124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc06a980 0x7f3bcc1f5590 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009130 tx=0x7f3bbc004750 comp rx=0 tx=0).stop 2023-11-28T12:02:37.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.241+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/828288124 shutdown_connections 2023-11-28T12:02:37.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.241+0000 7f3bd1dc1700 1 --2- 172.21.15.57:0/828288124 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3bb80682f0 0x7f3bb806a7a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.241+0000 7f3bd1dc1700 1 --2- 172.21.15.57:0/828288124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc0cf710 0x7f3bcc0ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.241+0000 7f3bd1dc1700 1 --2- 172.21.15.57:0/828288124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc06a980 0x7f3bcc1f5590 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.241+0000 7f3bd1dc1700 1 --2- 172.21.15.57:0/828288124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bcc071890 0x7f3bcc1f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.241+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/828288124 >> 172.21.15.57:0/828288124 conn(0x7f3bcc06c010 msgr2=0x7f3bcc0b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:37.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.241+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/828288124 shutdown_connections 2023-11-28T12:02:37.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.242+0000 7f3bd1dc1700 1 -- 172.21.15.57:0/828288124 wait complete. 2023-11-28T12:02:37.256 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:37.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.393+0000 7fbd07db6700 1 Processor -- start 2023-11-28T12:02:37.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.393+0000 7fbd07db6700 1 -- start start 2023-11-28T12:02:37.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.394+0000 7fbd07db6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbd000b1650 0x7fbd000b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.394+0000 7fbd07db6700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd00159ec0 con 0x7fbd000af0a0 2023-11-28T12:02:37.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.394+0000 7fbd07db6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd0015aa60 con 0x7fbd001599b0 2023-11-28T12:02:37.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.394+0000 7fbd07db6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd000b1f60 con 0x7fbd000b1650 2023-11-28T12:02:37.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.394+0000 7fbd06353700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fbd000af0a0 0x7fbd0015a350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37406/0 (socket says 172.21.15.57:37406) 2023-11-28T12:02:37.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.394+0000 7fbd06353700 1 -- 172.21.15.57:0/248457898 learned_addr learned my addr 172.21.15.57:0/248457898 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:37.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.394+0000 7fbd05b52700 1 --2- 172.21.15.57:0/248457898 >> v2:172.21.15.181:3300/0 conn(0x7fbd000b1650 0x7fbd000b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.395+0000 7fbd04b50700 1 -- 172.21.15.57:0/248457898 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018864398 0 0) 0x7fbd00159ec0 con 0x7fbd000af0a0 2023-11-28T12:02:37.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.395+0000 7fbd04b50700 1 -- 172.21.15.57:0/248457898 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcec0035b0 con 0x7fbd000af0a0 2023-11-28T12:02:37.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.395+0000 7fbd05b52700 1 -- 172.21.15.57:0/248457898 >> v1:172.21.15.181:6789/0 conn(0x7fbd001599b0 legacy=0x7fbd0015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.395+0000 7fbd05b52700 1 -- 172.21.15.57:0/248457898 >> v1:172.21.15.57:6789/0 conn(0x7fbd000af0a0 legacy=0x7fbd0015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.395+0000 7fbd05b52700 1 -- 172.21.15.57:0/248457898 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0015d8a0 con 0x7fbd000b1650 2023-11-28T12:02:37.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.395+0000 7fbd05b52700 1 --2- 172.21.15.57:0/248457898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd000b1650 0x7fbd000b1a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009840 tx=0x7fbcfc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f64d6aebd01b7b00 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:37.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.395+0000 7fbd04b50700 1 -- 172.21.15.57:0/248457898 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc011830 con 0x7fbd000b1650 2023-11-28T12:02:37.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.395+0000 7fbd04b50700 1 -- 172.21.15.57:0/248457898 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc011990 con 0x7fbd000b1650 2023-11-28T12:02:37.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.395+0000 7fbd04b50700 1 -- 172.21.15.57:0/248457898 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc011d20 con 0x7fbd000b1650 2023-11-28T12:02:37.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.396+0000 7fbd07db6700 1 -- 172.21.15.57:0/248457898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd000b1650 msgr2=0x7fbd000b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.396+0000 7fbd07db6700 1 --2- 172.21.15.57:0/248457898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd000b1650 0x7fbd000b1a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009840 tx=0x7fbcfc009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:37.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.396+0000 7fbd07db6700 1 -- 172.21.15.57:0/248457898 shutdown_connections 2023-11-28T12:02:37.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.396+0000 7fbd07db6700 1 --2- 172.21.15.57:0/248457898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd000b1650 0x7fbd000b1a20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.396+0000 7fbd07db6700 1 -- 172.21.15.57:0/248457898 >> 172.21.15.57:0/248457898 conn(0x7fbd0006c010 msgr2=0x7fbd000cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:37.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.396+0000 7fbd07db6700 1 -- 172.21.15.57:0/248457898 shutdown_connections 2023-11-28T12:02:37.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.396+0000 7fbd07db6700 1 -- 172.21.15.57:0/248457898 wait complete. 2023-11-28T12:02:37.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.397+0000 7fbd07db6700 1 Processor -- start 2023-11-28T12:02:37.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.397+0000 7fbd07db6700 1 -- start start 2023-11-28T12:02:37.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.398+0000 7fbd07db6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd000af0a0 0x7fbd00147050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.398+0000 7fbd07db6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd000b1650 0x7fbd000b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.398+0000 7fbd07db6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd000b5150 0x7fbd001f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.398+0000 7fbd07db6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd00156a80 con 0x7fbd000b1650 2023-11-28T12:02:37.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.398+0000 7fbd07db6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd001f4bc0 con 0x7fbd000af0a0 2023-11-28T12:02:37.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.398+0000 7fbd07db6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd000b55f0 con 0x7fbd000b5150 2023-11-28T12:02:37.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.398+0000 7fbd05351700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd000b1650 0x7fbd000b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.398+0000 7fbd05351700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd000b1650 0x7fbd000b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50804/0 (socket says 172.21.15.57:50804) 2023-11-28T12:02:37.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.398+0000 7fbd05351700 1 -- 172.21.15.57:0/3299734834 learned_addr learned my addr 172.21.15.57:0/3299734834 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:37.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.398+0000 7fbd05b52700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd000af0a0 0x7fbd00147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.399+0000 7fbd06353700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd000b5150 0x7fbd001f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.399+0000 7fbd05351700 1 -- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd000b5150 msgr2=0x7fbd001f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.399+0000 7fbd05351700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd000b5150 0x7fbd001f3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.399+0000 7fbd05351700 1 -- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd000af0a0 msgr2=0x7fbd00147050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.399+0000 7fbd05351700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd000af0a0 0x7fbd00147050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.399+0000 7fbd05351700 1 -- 172.21.15.57:0/3299734834 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc0094a0 con 0x7fbd000b1650 2023-11-28T12:02:37.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.399+0000 7fbd06353700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd000b5150 0x7fbd001f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:37.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.399+0000 7fbd05351700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd000b1650 0x7fbd000b4c10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400c460 tx=0x7fbcf400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:37.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.399+0000 7fbcf27fc700 1 -- 172.21.15.57:0/3299734834 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40052d0 con 0x7fbd000b1650 2023-11-28T12:02:37.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.399+0000 7fbcf27fc700 1 -- 172.21.15.57:0/3299734834 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf4005430 con 0x7fbd000b1650 2023-11-28T12:02:37.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.399+0000 7fbd07db6700 1 -- 172.21.15.57:0/3299734834 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0015de60 con 0x7fbd000b1650 2023-11-28T12:02:37.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.399+0000 7fbcf27fc700 1 -- 172.21.15.57:0/3299734834 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40038c0 con 0x7fbd000b1650 2023-11-28T12:02:37.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.399+0000 7fbd07db6700 1 -- 172.21.15.57:0/3299734834 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0015db80 con 0x7fbd000b1650 2023-11-28T12:02:37.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.400+0000 7fbd07db6700 1 -- 172.21.15.57:0/3299734834 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd000743f0 con 0x7fbd000b1650 2023-11-28T12:02:37.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.401+0000 7fbcf27fc700 1 -- 172.21.15.57:0/3299734834 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbcf4019030 con 0x7fbd000b1650 2023-11-28T12:02:37.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.401+0000 7fbcf27fc700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbce40682f0 0x7fbce406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.401+0000 7fbcf27fc700 1 -- 172.21.15.57:0/3299734834 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbcf4084f10 con 0x7fbd000b1650 2023-11-28T12:02:37.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.401+0000 7fbd05b52700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbce40682f0 0x7fbce406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.402+0000 7fbd05b52700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbce40682f0 0x7fbce406a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0090b0 tx=0x7fbcfc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:37.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.405+0000 7fbcf27fc700 1 -- 172.21.15.57:0/3299734834 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbcf4051730 con 0x7fbd000b1650 2023-11-28T12:02:37.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.603+0000 7fbd07db6700 1 -- 172.21.15.57:0/3299734834 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd0006e620 con 0x7fbd000b1650 2023-11-28T12:02:37.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.603+0000 7fbcf27fc700 1 -- 172.21.15.57:0/3299734834 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbcf4054d50 con 0x7fbd000b1650 2023-11-28T12:02:37.605 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:37.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.606+0000 7fbd07db6700 1 -- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbce40682f0 msgr2=0x7fbce406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.606+0000 7fbd07db6700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbce40682f0 0x7fbce406a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0090b0 tx=0x7fbcfc008030 comp rx=0 tx=0).stop 2023-11-28T12:02:37.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.606+0000 7fbd07db6700 1 -- 172.21.15.57:0/3299734834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd000b1650 msgr2=0x7fbd000b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.606+0000 7fbd07db6700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd000b1650 0x7fbd000b4c10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400c460 tx=0x7fbcf400c820 comp rx=0 tx=0).stop 2023-11-28T12:02:37.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.606+0000 7fbd07db6700 1 -- 172.21.15.57:0/3299734834 shutdown_connections 2023-11-28T12:02:37.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.606+0000 7fbd07db6700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbce40682f0 0x7fbce406a7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.606+0000 7fbd07db6700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd000b5150 0x7fbd001f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.606+0000 7fbd07db6700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd000af0a0 0x7fbd00147050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.606+0000 7fbd07db6700 1 --2- 172.21.15.57:0/3299734834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd000b1650 0x7fbd000b4c10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.606+0000 7fbd07db6700 1 -- 172.21.15.57:0/3299734834 >> 172.21.15.57:0/3299734834 conn(0x7fbd0006c010 msgr2=0x7fbd000b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:37.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.606+0000 7fbd07db6700 1 -- 172.21.15.57:0/3299734834 shutdown_connections 2023-11-28T12:02:37.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.606+0000 7fbd07db6700 1 -- 172.21.15.57:0/3299734834 wait complete. 2023-11-28T12:02:37.620 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:02:37.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.758+0000 7f3beb1d3700 1 Processor -- start 2023-11-28T12:02:37.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.758+0000 7f3beb1d3700 1 -- start start 2023-11-28T12:02:37.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.759+0000 7f3beb1d3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3be406a980 0x7f3be40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.759+0000 7f3beb1d3700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be415cc10 con 0x7f3be40cf710 2023-11-28T12:02:37.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.759+0000 7f3beb1d3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be406c9b0 con 0x7f3be4071890 2023-11-28T12:02:37.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.759+0000 7f3beb1d3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be406cc90 con 0x7f3be406a980 2023-11-28T12:02:37.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.759+0000 7f3be8f6f700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f3be40cf710 0x7f3be406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37418/0 (socket says 172.21.15.57:37418) 2023-11-28T12:02:37.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.759+0000 7f3be8f6f700 1 -- 172.21.15.57:0/3641908119 learned_addr learned my addr 172.21.15.57:0/3641908119 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:37.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.759+0000 7f3be3fff700 1 --2- 172.21.15.57:0/3641908119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3be406a980 0x7f3be40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.760+0000 7f3be37fe700 1 -- 172.21.15.57:0/3641908119 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039649067 0 0) 0x7f3be415cc10 con 0x7f3be40cf710 2023-11-28T12:02:37.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.760+0000 7f3be37fe700 1 -- 172.21.15.57:0/3641908119 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bd00035b0 con 0x7f3be40cf710 2023-11-28T12:02:37.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.760+0000 7f3be37fe700 1 -- 172.21.15.57:0/3641908119 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111681390 0 0) 0x7f3be406c9b0 con 0x7f3be4071890 2023-11-28T12:02:37.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.760+0000 7f3be37fe700 1 -- 172.21.15.57:0/3641908119 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3be415cc10 con 0x7f3be4071890 2023-11-28T12:02:37.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.760+0000 7f3be3fff700 1 -- 172.21.15.57:0/3641908119 >> v1:172.21.15.181:6789/0 conn(0x7f3be4071890 legacy=0x7f3be4160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.760+0000 7f3be3fff700 1 -- 172.21.15.57:0/3641908119 >> v1:172.21.15.57:6789/0 conn(0x7f3be40cf710 legacy=0x7f3be406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.760+0000 7f3be3fff700 1 -- 172.21.15.57:0/3641908119 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be406cf70 con 0x7f3be406a980 2023-11-28T12:02:37.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.760+0000 7f3be3fff700 1 --2- 172.21.15.57:0/3641908119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3be406a980 0x7f3be40ce740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009870 tx=0x7f3bcc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=525e9876b75e527d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:37.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.760+0000 7f3be37fe700 1 -- 172.21.15.57:0/3641908119 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc011830 con 0x7f3be406a980 2023-11-28T12:02:37.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.760+0000 7f3be37fe700 1 -- 172.21.15.57:0/3641908119 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc011990 con 0x7f3be406a980 2023-11-28T12:02:37.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.760+0000 7f3be37fe700 1 -- 172.21.15.57:0/3641908119 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc011d00 con 0x7f3be406a980 2023-11-28T12:02:37.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.761+0000 7f3beb1d3700 1 -- 172.21.15.57:0/3641908119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3be406a980 msgr2=0x7f3be40ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.761+0000 7f3beb1d3700 1 --2- 172.21.15.57:0/3641908119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3be406a980 0x7f3be40ce740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009870 tx=0x7f3bcc009b80 comp rx=0 tx=0).stop 2023-11-28T12:02:37.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.761+0000 7f3beb1d3700 1 -- 172.21.15.57:0/3641908119 shutdown_connections 2023-11-28T12:02:37.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.761+0000 7f3beb1d3700 1 --2- 172.21.15.57:0/3641908119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3be406a980 0x7f3be40ce740 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.761+0000 7f3beb1d3700 1 -- 172.21.15.57:0/3641908119 >> 172.21.15.57:0/3641908119 conn(0x7f3be406c010 msgr2=0x7f3be40cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:37.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.761+0000 7f3beb1d3700 1 -- 172.21.15.57:0/3641908119 shutdown_connections 2023-11-28T12:02:37.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.761+0000 7f3beb1d3700 1 -- 172.21.15.57:0/3641908119 wait complete. 2023-11-28T12:02:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.762+0000 7f3beb1d3700 1 Processor -- start 2023-11-28T12:02:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.762+0000 7f3beb1d3700 1 -- start start 2023-11-28T12:02:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.763+0000 7f3beb1d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3be406a980 0x7f3be414abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.763+0000 7f3beb1d3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3be4071890 0x7f3be41f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.763+0000 7f3beb1d3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3be40cf710 0x7f3be40ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.763+0000 7f3beb1d3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be41f65a0 con 0x7f3be40cf710 2023-11-28T12:02:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.763+0000 7f3beb1d3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be41516b0 con 0x7f3be406a980 2023-11-28T12:02:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.763+0000 7f3beb1d3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be414d840 con 0x7f3be4071890 2023-11-28T12:02:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.763+0000 7f3be9770700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3be40cf710 0x7f3be40ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.763+0000 7f3be9770700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3be40cf710 0x7f3be40ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50816/0 (socket says 172.21.15.57:50816) 2023-11-28T12:02:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.763+0000 7f3be9770700 1 -- 172.21.15.57:0/2851047788 learned_addr learned my addr 172.21.15.57:0/2851047788 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.763+0000 7f3be3fff700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3be4071890 0x7f3be41f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.763+0000 7f3be8f6f700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3be406a980 0x7f3be414abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.764+0000 7f3be9770700 1 -- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3be4071890 msgr2=0x7f3be41f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.764+0000 7f3be9770700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3be4071890 0x7f3be41f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.764+0000 7f3be9770700 1 -- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3be406a980 msgr2=0x7f3be414abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.764+0000 7f3be9770700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3be406a980 0x7f3be414abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.764+0000 7f3be9770700 1 -- 172.21.15.57:0/2851047788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc0094a0 con 0x7f3be40cf710 2023-11-28T12:02:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.764+0000 7f3be8f6f700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3be406a980 0x7f3be414abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.764+0000 7f3be9770700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3be40cf710 0x7f3be40ad900 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00cc20 tx=0x7f3bdc00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.764+0000 7f3be17fa700 1 -- 172.21.15.57:0/2851047788 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc003c00 con 0x7f3be40cf710 2023-11-28T12:02:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.764+0000 7f3be17fa700 1 -- 172.21.15.57:0/2851047788 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc003d60 con 0x7f3be40cf710 2023-11-28T12:02:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.764+0000 7f3beb1d3700 1 -- 172.21.15.57:0/2851047788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be406d530 con 0x7f3be40cf710 2023-11-28T12:02:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.764+0000 7f3be17fa700 1 -- 172.21.15.57:0/2851047788 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc00ab90 con 0x7f3be40cf710 2023-11-28T12:02:37.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.764+0000 7f3beb1d3700 1 -- 172.21.15.57:0/2851047788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be406d250 con 0x7f3be40cf710 2023-11-28T12:02:37.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.765+0000 7f3beb1d3700 1 -- 172.21.15.57:0/2851047788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be40cd2f0 con 0x7f3be40cf710 2023-11-28T12:02:37.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.766+0000 7f3be17fa700 1 -- 172.21.15.57:0/2851047788 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3bdc010050 con 0x7f3be40cf710 2023-11-28T12:02:37.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.766+0000 7f3be17fa700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3bc40682f0 0x7f3bc406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:37.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.766+0000 7f3be17fa700 1 -- 172.21.15.57:0/2851047788 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3bdc085c60 con 0x7f3be40cf710 2023-11-28T12:02:37.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.766+0000 7f3be8f6f700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3bc40682f0 0x7f3bc406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:37.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.767+0000 7f3be8f6f700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3bc40682f0 0x7f3bc406a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40011e0 tx=0x7f3bd4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:37.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.771+0000 7f3be17fa700 1 -- 172.21.15.57:0/2851047788 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3bdc052480 con 0x7f3be40cf710 2023-11-28T12:02:37.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.969+0000 7f3beb1d3700 1 -- 172.21.15.57:0/2851047788 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3be40743f0 con 0x7f3be40cf710 2023-11-28T12:02:37.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.969+0000 7f3be17fa700 1 -- 172.21.15.57:0/2851047788 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3bdc055aa0 con 0x7f3be40cf710 2023-11-28T12:02:37.970 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:02:37.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.971+0000 7f3beb1d3700 1 -- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3bc40682f0 msgr2=0x7f3bc406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.971+0000 7f3beb1d3700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3bc40682f0 0x7f3bc406a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40011e0 tx=0x7f3bd4009040 comp rx=0 tx=0).stop 2023-11-28T12:02:37.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.971+0000 7f3beb1d3700 1 -- 172.21.15.57:0/2851047788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3be40cf710 msgr2=0x7f3be40ad900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:37.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.971+0000 7f3beb1d3700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3be40cf710 0x7f3be40ad900 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00cc20 tx=0x7f3bdc00cf30 comp rx=0 tx=0).stop 2023-11-28T12:02:37.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.971+0000 7f3beb1d3700 1 -- 172.21.15.57:0/2851047788 shutdown_connections 2023-11-28T12:02:37.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.971+0000 7f3beb1d3700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3bc40682f0 0x7f3bc406a7a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.971+0000 7f3beb1d3700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3be40cf710 0x7f3be40ad900 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.971+0000 7f3beb1d3700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3be406a980 0x7f3be414abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.971+0000 7f3beb1d3700 1 --2- 172.21.15.57:0/2851047788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3be4071890 0x7f3be41f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:37.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.971+0000 7f3beb1d3700 1 -- 172.21.15.57:0/2851047788 >> 172.21.15.57:0/2851047788 conn(0x7f3be406c010 msgr2=0x7f3be40b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:37.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.971+0000 7f3beb1d3700 1 -- 172.21.15.57:0/2851047788 shutdown_connections 2023-11-28T12:02:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:37.971+0000 7f3beb1d3700 1 -- 172.21.15.57:0/2851047788 wait complete. 2023-11-28T12:02:37.986 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:02:38.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.124+0000 7fa9a40dd700 1 Processor -- start 2023-11-28T12:02:38.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.124+0000 7fa9a40dd700 1 -- start start 2023-11-28T12:02:38.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.125+0000 7fa9a40dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa99c160a50 0x7fa99c070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.125+0000 7fa9a40dd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa99c0ae660 0x7fa99c06c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.125+0000 7fa9a40dd700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99c0ce740 con 0x7fa99c160e20 2023-11-28T12:02:38.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.125+0000 7fa9a40dd700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa99c071870 con 0x7fa99c0ae660 2023-11-28T12:02:38.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.125+0000 7fa9a40dd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c0af450 con 0x7fa99c160a50 2023-11-28T12:02:38.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.125+0000 7fa9a267a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa99c0ae660 0x7fa99c06c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.125+0000 7fa9a1678700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa99c160e20 0x7fa99c0cf170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37426/0 (socket says 172.21.15.57:37426) 2023-11-28T12:02:38.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.125+0000 7fa9a1678700 1 -- 172.21.15.57:0/1359462639 learned_addr learned my addr 172.21.15.57:0/1359462639 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:38.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.126+0000 7fa9a1e79700 1 --2- 172.21.15.57:0/1359462639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa99c160a50 0x7fa99c070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.126+0000 7fa9a267a700 1 -- 172.21.15.57:0/1359462639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa99c160a50 msgr2=0x7fa99c070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:38.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.126+0000 7fa9a267a700 1 --2- 172.21.15.57:0/1359462639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa99c160a50 0x7fa99c070bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.126+0000 7fa9a267a700 1 -- 172.21.15.57:0/1359462639 >> v1:172.21.15.57:6789/0 conn(0x7fa99c160e20 legacy=0x7fa99c0cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.126+0000 7fa9a267a700 1 -- 172.21.15.57:0/1359462639 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c0712b0 con 0x7fa99c0ae660 2023-11-28T12:02:38.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.126+0000 7fa9a267a700 1 --2- 172.21.15.57:0/1359462639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa99c0ae660 0x7fa99c06c880 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa994009d90 tx=0x7fa99400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e358a6e3950a6399 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:38.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.126+0000 7fa9a0e77700 1 -- 172.21.15.57:0/1359462639 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99400e070 con 0x7fa99c0ae660 2023-11-28T12:02:38.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.126+0000 7fa9a0e77700 1 -- 172.21.15.57:0/1359462639 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99400bbd0 con 0x7fa99c0ae660 2023-11-28T12:02:38.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.126+0000 7fa9a0e77700 1 -- 172.21.15.57:0/1359462639 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa994012430 con 0x7fa99c0ae660 2023-11-28T12:02:38.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.127+0000 7fa9a40dd700 1 -- 172.21.15.57:0/1359462639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa99c0ae660 msgr2=0x7fa99c06c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.127+0000 7fa9a40dd700 1 --2- 172.21.15.57:0/1359462639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa99c0ae660 0x7fa99c06c880 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa994009d90 tx=0x7fa99400b330 comp rx=0 tx=0).stop 2023-11-28T12:02:38.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.127+0000 7fa9a40dd700 1 -- 172.21.15.57:0/1359462639 shutdown_connections 2023-11-28T12:02:38.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.127+0000 7fa9a40dd700 1 --2- 172.21.15.57:0/1359462639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa99c0ae660 0x7fa99c06c880 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.127+0000 7fa9a40dd700 1 --2- 172.21.15.57:0/1359462639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa99c160a50 0x7fa99c070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.127+0000 7fa9a40dd700 1 -- 172.21.15.57:0/1359462639 >> 172.21.15.57:0/1359462639 conn(0x7fa99c06c010 msgr2=0x7fa99c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:38.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.127+0000 7fa9a40dd700 1 -- 172.21.15.57:0/1359462639 shutdown_connections 2023-11-28T12:02:38.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.127+0000 7fa9a40dd700 1 -- 172.21.15.57:0/1359462639 wait complete. 2023-11-28T12:02:38.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.128+0000 7fa9a40dd700 1 Processor -- start 2023-11-28T12:02:38.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.128+0000 7fa9a40dd700 1 -- start start 2023-11-28T12:02:38.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a40dd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa99c0ae660 0x7fa99c1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a40dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa99c160a50 0x7fa99c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a40dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa99c160e20 0x7fa99c0c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a40dd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c0c4d00 con 0x7fa99c0ae660 2023-11-28T12:02:38.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a40dd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c159b20 con 0x7fa99c160e20 2023-11-28T12:02:38.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a40dd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c0b5990 con 0x7fa99c160a50 2023-11-28T12:02:38.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a1e79700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa99c0ae660 0x7fa99c1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a1e79700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa99c0ae660 0x7fa99c1595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50834/0 (socket says 172.21.15.57:50834) 2023-11-28T12:02:38.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a1e79700 1 -- 172.21.15.57:0/42145036 learned_addr learned my addr 172.21.15.57:0/42145036 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:38.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a1678700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa99c160a50 0x7fa99c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a267a700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa99c160e20 0x7fa99c0c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a1e79700 1 -- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa99c160a50 msgr2=0x7fa99c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a1e79700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa99c160a50 0x7fa99c0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a1e79700 1 -- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa99c160e20 msgr2=0x7fa99c0c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a1e79700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa99c160e20 0x7fa99c0c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a1e79700 1 -- 172.21.15.57:0/42145036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa994009a70 con 0x7fa99c0ae660 2023-11-28T12:02:38.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a267a700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa99c160e20 0x7fa99c0c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:38.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.129+0000 7fa9a1e79700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa99c0ae660 0x7fa99c1595e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa99800e840 tx=0x7fa99800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:38.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.130+0000 7fa98e7fc700 1 -- 172.21.15.57:0/42145036 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998013610 con 0x7fa99c0ae660 2023-11-28T12:02:38.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.130+0000 7fa98e7fc700 1 -- 172.21.15.57:0/42145036 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa998013770 con 0x7fa99c0ae660 2023-11-28T12:02:38.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.130+0000 7fa9a40dd700 1 -- 172.21.15.57:0/42145036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99c0ce970 con 0x7fa99c0ae660 2023-11-28T12:02:38.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.130+0000 7fa98e7fc700 1 -- 172.21.15.57:0/42145036 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998013b20 con 0x7fa99c0ae660 2023-11-28T12:02:38.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.130+0000 7fa9a40dd700 1 -- 172.21.15.57:0/42145036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99c071590 con 0x7fa99c0ae660 2023-11-28T12:02:38.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.131+0000 7fa9a40dd700 1 -- 172.21.15.57:0/42145036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99c0cec50 con 0x7fa99c0ae660 2023-11-28T12:02:38.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.132+0000 7fa98e7fc700 1 -- 172.21.15.57:0/42145036 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa998023070 con 0x7fa99c0ae660 2023-11-28T12:02:38.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.132+0000 7fa98e7fc700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa9800682f0 0x7fa98006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.132+0000 7fa98e7fc700 1 -- 172.21.15.57:0/42145036 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa998085d90 con 0x7fa99c0ae660 2023-11-28T12:02:38.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.133+0000 7fa9a1678700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa9800682f0 0x7fa98006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.133+0000 7fa9a1678700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa9800682f0 0x7fa98006a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0c2da0 tx=0x7fa990009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:38.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.136+0000 7fa98e7fc700 1 -- 172.21.15.57:0/42145036 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa9980524c0 con 0x7fa99c0ae660 2023-11-28T12:02:38.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.337+0000 7fa9a40dd700 1 -- 172.21.15.57:0/42145036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa99c06e630 con 0x7fa99c0ae660 2023-11-28T12:02:38.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.337+0000 7fa98e7fc700 1 -- 172.21.15.57:0/42145036 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa9980210a0 con 0x7fa99c0ae660 2023-11-28T12:02:38.338 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:02:38.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.340+0000 7fa9a40dd700 1 -- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa9800682f0 msgr2=0x7fa98006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.340+0000 7fa9a40dd700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa9800682f0 0x7fa98006a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0c2da0 tx=0x7fa990009040 comp rx=0 tx=0).stop 2023-11-28T12:02:38.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.340+0000 7fa9a40dd700 1 -- 172.21.15.57:0/42145036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa99c0ae660 msgr2=0x7fa99c1595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.340+0000 7fa9a40dd700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa99c0ae660 0x7fa99c1595e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa99800e840 tx=0x7fa99800eb50 comp rx=0 tx=0).stop 2023-11-28T12:02:38.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.340+0000 7fa9a40dd700 1 -- 172.21.15.57:0/42145036 shutdown_connections 2023-11-28T12:02:38.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.340+0000 7fa9a40dd700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa9800682f0 0x7fa98006a7a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.340+0000 7fa9a40dd700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa99c160e20 0x7fa99c0c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.340+0000 7fa9a40dd700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa99c0ae660 0x7fa99c1595e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.340+0000 7fa9a40dd700 1 --2- 172.21.15.57:0/42145036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa99c160a50 0x7fa99c0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.340+0000 7fa9a40dd700 1 -- 172.21.15.57:0/42145036 >> 172.21.15.57:0/42145036 conn(0x7fa99c06c010 msgr2=0x7fa99c15cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:38.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.340+0000 7fa9a40dd700 1 -- 172.21.15.57:0/42145036 shutdown_connections 2023-11-28T12:02:38.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.340+0000 7fa9a40dd700 1 -- 172.21.15.57:0/42145036 wait complete. 2023-11-28T12:02:38.354 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:02:38.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.490+0000 7f991b1d8700 1 Processor -- start 2023-11-28T12:02:38.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.491+0000 7f991b1d8700 1 -- start start 2023-11-28T12:02:38.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.491+0000 7f991b1d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991406a270 0x7f991406a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.491+0000 7f991b1d8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f991415cc80 0x7f991406de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.491+0000 7f991b1d8700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991406ab80 con 0x7f991406e3a0 2023-11-28T12:02:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.491+0000 7f991b1d8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99140b2490 con 0x7f991415cc80 2023-11-28T12:02:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.491+0000 7f991b1d8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99140b1810 con 0x7f991406a270 2023-11-28T12:02:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.492+0000 7f9919775700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f991406e3a0 0x7f991406ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37430/0 (socket says 172.21.15.57:37430) 2023-11-28T12:02:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.492+0000 7f9919775700 1 -- 172.21.15.57:0/2090992645 learned_addr learned my addr 172.21.15.57:0/2090992645 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.492+0000 7f9918f74700 1 --2- 172.21.15.57:0/2090992645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991406a270 0x7f991406a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.492+0000 7f9913fff700 1 --2- 172.21.15.57:0/2090992645 >> v2:172.21.15.181:3300/0 conn(0x7f991415cc80 0x7f991406de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.492+0000 7f99137fe700 1 -- 172.21.15.57:0/2090992645 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101666508 0 0) 0x7f991406ab80 con 0x7f991406e3a0 2023-11-28T12:02:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.492+0000 7f99137fe700 1 -- 172.21.15.57:0/2090992645 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99000035b0 con 0x7f991406e3a0 2023-11-28T12:02:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.492+0000 7f9918f74700 1 -- 172.21.15.57:0/2090992645 >> v2:172.21.15.181:3300/0 conn(0x7f991415cc80 msgr2=0x7f991406de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:38.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.492+0000 7f9918f74700 1 --2- 172.21.15.57:0/2090992645 >> v2:172.21.15.181:3300/0 conn(0x7f991415cc80 0x7f991406de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.492+0000 7f9918f74700 1 -- 172.21.15.57:0/2090992645 >> v1:172.21.15.57:6789/0 conn(0x7f991406e3a0 legacy=0x7f991406ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.492+0000 7f9918f74700 1 -- 172.21.15.57:0/2090992645 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99140b1af0 con 0x7f991406a270 2023-11-28T12:02:38.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.492+0000 7f9918f74700 1 --2- 172.21.15.57:0/2090992645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991406a270 0x7f991406a640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9904009840 tx=0x7f9904009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff5054efe8b899 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:38.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.493+0000 7f99137fe700 1 -- 172.21.15.57:0/2090992645 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904011830 con 0x7f991406a270 2023-11-28T12:02:38.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.493+0000 7f99137fe700 1 -- 172.21.15.57:0/2090992645 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9904011990 con 0x7f991406a270 2023-11-28T12:02:38.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.493+0000 7f99137fe700 1 -- 172.21.15.57:0/2090992645 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904011d00 con 0x7f991406a270 2023-11-28T12:02:38.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.493+0000 7f991b1d8700 1 -- 172.21.15.57:0/2090992645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991406a270 msgr2=0x7f991406a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.493+0000 7f991b1d8700 1 --2- 172.21.15.57:0/2090992645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991406a270 0x7f991406a640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9904009840 tx=0x7f9904009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:38.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.493+0000 7f991b1d8700 1 -- 172.21.15.57:0/2090992645 shutdown_connections 2023-11-28T12:02:38.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.493+0000 7f991b1d8700 1 --2- 172.21.15.57:0/2090992645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991406a270 0x7f991406a640 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.493+0000 7f991b1d8700 1 --2- 172.21.15.57:0/2090992645 >> v2:172.21.15.181:3300/0 conn(0x7f991415cc80 0x7f991406de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.493+0000 7f991b1d8700 1 -- 172.21.15.57:0/2090992645 >> 172.21.15.57:0/2090992645 conn(0x7f991406c010 msgr2=0x7f99140cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:38.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.493+0000 7f991b1d8700 1 -- 172.21.15.57:0/2090992645 shutdown_connections 2023-11-28T12:02:38.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.493+0000 7f991b1d8700 1 -- 172.21.15.57:0/2090992645 wait complete. 2023-11-28T12:02:38.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.494+0000 7f991b1d8700 1 Processor -- start 2023-11-28T12:02:38.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.495+0000 7f991b1d8700 1 -- start start 2023-11-28T12:02:38.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.495+0000 7f991b1d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991406a270 0x7f9914152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.495+0000 7f991b1d8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991406e3a0 0x7f9914152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.495+0000 7f991b1d8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f991415cc80 0x7f99140b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.495+0000 7f991b1d8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99140c77d0 con 0x7f991415cc80 2023-11-28T12:02:38.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f991b1d8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9914152860 con 0x7f991406e3a0 2023-11-28T12:02:38.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f991b1d8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991415bed0 con 0x7f991406a270 2023-11-28T12:02:38.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f9919775700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f991415cc80 0x7f99140b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f9919775700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f991415cc80 0x7f99140b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50840/0 (socket says 172.21.15.57:50840) 2023-11-28T12:02:38.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f9919775700 1 -- 172.21.15.57:0/2074155180 learned_addr learned my addr 172.21.15.57:0/2074155180 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:38.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f9913fff700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991406e3a0 0x7f9914152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f9918f74700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991406a270 0x7f9914152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f9919775700 1 -- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991406a270 msgr2=0x7f9914152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f9919775700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991406a270 0x7f9914152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f9919775700 1 -- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991406e3a0 msgr2=0x7f9914152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f9919775700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991406e3a0 0x7f9914152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f9919775700 1 -- 172.21.15.57:0/2074155180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99040094a0 con 0x7f991415cc80 2023-11-28T12:02:38.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f9918f74700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991406a270 0x7f9914152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:38.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f9913fff700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991406e3a0 0x7f9914152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:38.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f9919775700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f991415cc80 0x7f99140b30a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f990c00caa0 tx=0x7f990c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:38.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f99117fa700 1 -- 172.21.15.57:0/2074155180 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00aa80 con 0x7f991415cc80 2023-11-28T12:02:38.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f99117fa700 1 -- 172.21.15.57:0/2074155180 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c00abe0 con 0x7f991415cc80 2023-11-28T12:02:38.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f991b1d8700 1 -- 172.21.15.57:0/2074155180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99140b20b0 con 0x7f991415cc80 2023-11-28T12:02:38.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f99117fa700 1 -- 172.21.15.57:0/2074155180 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c012400 con 0x7f991415cc80 2023-11-28T12:02:38.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.496+0000 7f991b1d8700 1 -- 172.21.15.57:0/2074155180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99140b1dd0 con 0x7f991415cc80 2023-11-28T12:02:38.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.498+0000 7f991b1d8700 1 -- 172.21.15.57:0/2074155180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991406e7e0 con 0x7f991415cc80 2023-11-28T12:02:38.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.498+0000 7f99117fa700 1 -- 172.21.15.57:0/2074155180 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f990c019030 con 0x7f991415cc80 2023-11-28T12:02:38.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.499+0000 7f99117fa700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f98f40682f0 0x7f98f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.499+0000 7f99117fa700 1 -- 172.21.15.57:0/2074155180 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f990c085900 con 0x7f991415cc80 2023-11-28T12:02:38.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.499+0000 7f9918f74700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f98f40682f0 0x7f98f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.500+0000 7f9918f74700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f98f40682f0 0x7f98f406a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9904000c00 tx=0x7f990401c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:38.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.503+0000 7f99117fa700 1 -- 172.21.15.57:0/2074155180 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f990c052120 con 0x7f991415cc80 2023-11-28T12:02:38.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.750+0000 7f991b1d8700 1 -- 172.21.15.57:0/2074155180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99140743f0 con 0x7f991415cc80 2023-11-28T12:02:38.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.751+0000 7f99117fa700 1 -- 172.21.15.57:0/2074155180 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4642 (secure 0 0 0) 0x7f990c055740 con 0x7f991415cc80 2023-11-28T12:02:38.752 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:38.753 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/890913444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":890913444},{"type":"v1","addr":"172.21.15.181:6836","nonce":890913444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/847658150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":847658150},{"type":"v1","addr":"172.21.15.181:6837","nonce":847658150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1379013929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1379013929},{"type":"v1","addr":"172.21.15.57:6835","nonce":1379013929}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:01:43.919415+0000","modified":"2023-11-28T12:01:46.937696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6833/3776085595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3776085595},{"type":"v1","addr":"172.21.15.57:6833","nonce":3776085595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T12:02:38.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.754+0000 7f991b1d8700 1 -- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f98f40682f0 msgr2=0x7f98f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.754+0000 7f991b1d8700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f98f40682f0 0x7f98f406a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9904000c00 tx=0x7f990401c040 comp rx=0 tx=0).stop 2023-11-28T12:02:38.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.754+0000 7f991b1d8700 1 -- 172.21.15.57:0/2074155180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f991415cc80 msgr2=0x7f99140b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.754+0000 7f991b1d8700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f991415cc80 0x7f99140b30a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f990c00caa0 tx=0x7f990c00ce60 comp rx=0 tx=0).stop 2023-11-28T12:02:38.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.754+0000 7f991b1d8700 1 -- 172.21.15.57:0/2074155180 shutdown_connections 2023-11-28T12:02:38.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.754+0000 7f991b1d8700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f98f40682f0 0x7f98f406a7a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.754+0000 7f991b1d8700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f991415cc80 0x7f99140b30a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.754+0000 7f991b1d8700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991406a270 0x7f9914152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.754+0000 7f991b1d8700 1 --2- 172.21.15.57:0/2074155180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991406e3a0 0x7f9914152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.754+0000 7f991b1d8700 1 -- 172.21.15.57:0/2074155180 >> 172.21.15.57:0/2074155180 conn(0x7f991406c010 msgr2=0x7f99140b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:38.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.754+0000 7f991b1d8700 1 -- 172.21.15.57:0/2074155180 shutdown_connections 2023-11-28T12:02:38.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.754+0000 7f991b1d8700 1 -- 172.21.15.57:0/2074155180 wait complete. 2023-11-28T12:02:38.756 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 8 2023-11-28T12:02:38.768 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:38.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.904+0000 7f8bf83e3700 1 Processor -- start 2023-11-28T12:02:38.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.905+0000 7f8bf83e3700 1 -- start start 2023-11-28T12:02:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.905+0000 7f8bf83e3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8bf006e400 0x7f8bf006e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf83e3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf015e010 0x7f8bf00ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf83e3700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf015af30 con 0x7f8bf00d15f0 2023-11-28T12:02:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf83e3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf015a9d0 con 0x7f8bf006e400 2023-11-28T12:02:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf83e3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf01584e0 con 0x7f8bf015e010 2023-11-28T12:02:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf597e700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8bf00d15f0 0x7f8bf015b3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37446/0 (socket says 172.21.15.57:37446) 2023-11-28T12:02:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf597e700 1 -- 172.21.15.57:0/218082916 learned_addr learned my addr 172.21.15.57:0/218082916 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf617f700 1 --2- 172.21.15.57:0/218082916 >> v2:172.21.15.181:3300/0 conn(0x7f8bf006e400 0x7f8bf006e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf6980700 1 --2- 172.21.15.57:0/218082916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf015e010 0x7f8bf00ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf517d700 1 -- 172.21.15.57:0/218082916 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424705044 0 0) 0x7f8bf015af30 con 0x7f8bf00d15f0 2023-11-28T12:02:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf517d700 1 -- 172.21.15.57:0/218082916 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bdc0035b0 con 0x7f8bf00d15f0 2023-11-28T12:02:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf6980700 1 -- 172.21.15.57:0/218082916 >> v2:172.21.15.181:3300/0 conn(0x7f8bf006e400 msgr2=0x7f8bf006e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf6980700 1 --2- 172.21.15.57:0/218082916 >> v2:172.21.15.181:3300/0 conn(0x7f8bf006e400 0x7f8bf006e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf517d700 1 -- 172.21.15.57:0/218082916 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3670870610 0 0) 0x7f8bdc0035b0 con 0x7f8bf00d15f0 2023-11-28T12:02:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf6980700 1 -- 172.21.15.57:0/218082916 >> v1:172.21.15.57:6789/0 conn(0x7f8bf00d15f0 legacy=0x7f8bf015b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf6980700 1 -- 172.21.15.57:0/218082916 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf015ef40 con 0x7f8bf015e010 2023-11-28T12:02:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.906+0000 7f8bf617f700 1 --2- 172.21.15.57:0/218082916 >> v2:172.21.15.181:3300/0 conn(0x7f8bf006e400 0x7f8bf006e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.907+0000 7f8bf6980700 1 --2- 172.21.15.57:0/218082916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf015e010 0x7f8bf00ce970 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009d90 tx=0x7f8be800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb653761d69cebec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.907+0000 7f8bf517d700 1 -- 172.21.15.57:0/218082916 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be800e070 con 0x7f8bf015e010 2023-11-28T12:02:38.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.907+0000 7f8bf517d700 1 -- 172.21.15.57:0/218082916 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be8003a20 con 0x7f8bf015e010 2023-11-28T12:02:38.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.907+0000 7f8bf517d700 1 -- 172.21.15.57:0/218082916 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0001670 con 0x7f8bf015e010 2023-11-28T12:02:38.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.907+0000 7f8bf83e3700 1 -- 172.21.15.57:0/218082916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf015e010 msgr2=0x7f8bf00ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.907+0000 7f8bf83e3700 1 --2- 172.21.15.57:0/218082916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf015e010 0x7f8bf00ce970 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009d90 tx=0x7f8be800b330 comp rx=0 tx=0).stop 2023-11-28T12:02:38.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.907+0000 7f8bf83e3700 1 -- 172.21.15.57:0/218082916 shutdown_connections 2023-11-28T12:02:38.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.907+0000 7f8bf83e3700 1 --2- 172.21.15.57:0/218082916 >> v2:172.21.15.181:3300/0 conn(0x7f8bf006e400 0x7f8bf006e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.907+0000 7f8bf83e3700 1 --2- 172.21.15.57:0/218082916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf015e010 0x7f8bf00ce970 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.907+0000 7f8bf83e3700 1 -- 172.21.15.57:0/218082916 >> 172.21.15.57:0/218082916 conn(0x7f8bf006a270 msgr2=0x7f8bf006a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:38.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.908+0000 7f8bf83e3700 1 -- 172.21.15.57:0/218082916 shutdown_connections 2023-11-28T12:02:38.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.908+0000 7f8bf83e3700 1 -- 172.21.15.57:0/218082916 wait complete. 2023-11-28T12:02:38.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.909+0000 7f8bf83e3700 1 Processor -- start 2023-11-28T12:02:38.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.909+0000 7f8bf83e3700 1 -- start start 2023-11-28T12:02:38.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf83e3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf006e400 0x7f8bf00b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf83e3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf00d15f0 0x7f8bf01526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf83e3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf015e010 0x7f8bf0152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf83e3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00b14c0 con 0x7f8bf006e400 2023-11-28T12:02:38.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf83e3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00d0370 con 0x7f8bf015e010 2023-11-28T12:02:38.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf83e3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00d0650 con 0x7f8bf00d15f0 2023-11-28T12:02:38.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf617f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf006e400 0x7f8bf00b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf617f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf006e400 0x7f8bf00b4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50854/0 (socket says 172.21.15.57:50854) 2023-11-28T12:02:38.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf617f700 1 -- 172.21.15.57:0/3072915650 learned_addr learned my addr 172.21.15.57:0/3072915650 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf597e700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf00d15f0 0x7f8bf01526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf6980700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf015e010 0x7f8bf0152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf617f700 1 -- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf00d15f0 msgr2=0x7f8bf01526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf617f700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf00d15f0 0x7f8bf01526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf617f700 1 -- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf015e010 msgr2=0x7f8bf0152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf617f700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf015e010 0x7f8bf0152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf617f700 1 -- 172.21.15.57:0/3072915650 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec0094a0 con 0x7f8bf006e400 2023-11-28T12:02:38.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf617f700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf006e400 0x7f8bf00b4f20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00ebe0 tx=0x7f8bec00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:38.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8be6ffd700 1 -- 172.21.15.57:0/3072915650 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec00c850 con 0x7f8bf006e400 2023-11-28T12:02:38.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8be6ffd700 1 -- 172.21.15.57:0/3072915650 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec00c9b0 con 0x7f8bf006e400 2023-11-28T12:02:38.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf83e3700 1 -- 172.21.15.57:0/3072915650 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be8009a70 con 0x7f8bf006e400 2023-11-28T12:02:38.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8be6ffd700 1 -- 172.21.15.57:0/3072915650 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec00cd40 con 0x7f8bf006e400 2023-11-28T12:02:38.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.911+0000 7f8bf83e3700 1 -- 172.21.15.57:0/3072915650 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf015e980 con 0x7f8bf006e400 2023-11-28T12:02:38.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.912+0000 7f8bf83e3700 1 -- 172.21.15.57:0/3072915650 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf006ed10 con 0x7f8bf006e400 2023-11-28T12:02:38.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.912+0000 7f8be6ffd700 1 -- 172.21.15.57:0/3072915650 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8bec019030 con 0x7f8bf006e400 2023-11-28T12:02:38.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.913+0000 7f8be6ffd700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8bd40682f0 0x7f8bd406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:38.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.913+0000 7f8be6ffd700 1 -- 172.21.15.57:0/3072915650 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8bec085c60 con 0x7f8bf006e400 2023-11-28T12:02:38.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.913+0000 7f8bf597e700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8bd40682f0 0x7f8bd406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:38.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.913+0000 7f8bf597e700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8bd40682f0 0x7f8bd406a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8be0000f60 tx=0x7f8be0001230 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:38.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:38.917+0000 7f8be6ffd700 1 -- 172.21.15.57:0/3072915650 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8bec052480 con 0x7f8bf006e400 2023-11-28T12:02:39.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.119+0000 7f8bf83e3700 1 -- 172.21.15.57:0/3072915650 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bf015ec60 con 0x7f8bf006e400 2023-11-28T12:02:39.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.121+0000 7f8be6ffd700 1 -- 172.21.15.57:0/3072915650 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17945 (secure 0 0 0) 0x7f8bec017090 con 0x7f8bf006e400 2023-11-28T12:02:39.122 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:39.122 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:02:32.851731+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:39.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.123+0000 7f8bf83e3700 1 -- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8bd40682f0 msgr2=0x7f8bd406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.123+0000 7f8bf83e3700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8bd40682f0 0x7f8bd406a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8be0000f60 tx=0x7f8be0001230 comp rx=0 tx=0).stop 2023-11-28T12:02:39.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.123+0000 7f8bf83e3700 1 -- 172.21.15.57:0/3072915650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf006e400 msgr2=0x7f8bf00b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.123+0000 7f8bf83e3700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf006e400 0x7f8bf00b4f20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00ebe0 tx=0x7f8bec00eef0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.123+0000 7f8bf83e3700 1 -- 172.21.15.57:0/3072915650 shutdown_connections 2023-11-28T12:02:39.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.123+0000 7f8bf83e3700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8bd40682f0 0x7f8bd406a7a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.123+0000 7f8bf83e3700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf015e010 0x7f8bf0152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.123+0000 7f8bf83e3700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bf006e400 0x7f8bf00b4f20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.123+0000 7f8bf83e3700 1 --2- 172.21.15.57:0/3072915650 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf00d15f0 0x7f8bf01526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.123+0000 7f8bf83e3700 1 -- 172.21.15.57:0/3072915650 >> 172.21.15.57:0/3072915650 conn(0x7f8bf006a270 msgr2=0x7f8bf0155e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:39.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.123+0000 7f8bf83e3700 1 -- 172.21.15.57:0/3072915650 shutdown_connections 2023-11-28T12:02:39.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.123+0000 7f8bf83e3700 1 -- 172.21.15.57:0/3072915650 wait complete. 2023-11-28T12:02:39.138 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T12:02:39.138 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:02:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.272+0000 7fe16d6ba700 1 Processor -- start 2023-11-28T12:02:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.272+0000 7fe16d6ba700 1 -- start start 2023-11-28T12:02:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.273+0000 7fe16d6ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16806ab10 0x7fe16806aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.273+0000 7fe16d6ba700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1680d0450 0x7fe168160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.273+0000 7fe16d6ba700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1680d1ee0 con 0x7fe1680cd5a0 2023-11-28T12:02:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.273+0000 7fe16d6ba700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16806b420 con 0x7fe1680d0450 2023-11-28T12:02:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.273+0000 7fe16d6ba700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1680cdb10 con 0x7fe16806ab10 2023-11-28T12:02:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe166ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16806ab10 0x7fe16806aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe1677fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1680d0450 0x7fe168160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe166ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16806ab10 0x7fe16806aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:54568/0 (socket says 172.21.15.57:54568) 2023-11-28T12:02:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe1677fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1680d0450 0x7fe168160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:54616/0 (socket says 172.21.15.57:54616) 2023-11-28T12:02:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe1667fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe1680cd5a0 0x7fe168160300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:56824/0 (socket says 172.21.15.57:56824) 2023-11-28T12:02:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe166ffd700 1 -- 172.21.15.57:0/2576289350 learned_addr learned my addr 172.21.15.57:0/2576289350 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe166ffd700 1 -- 172.21.15.57:0/2576289350 >> v2:172.21.15.181:3300/0 conn(0x7fe1680d0450 msgr2=0x7fe168160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:39.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe166ffd700 1 --2- 172.21.15.57:0/2576289350 >> v2:172.21.15.181:3300/0 conn(0x7fe1680d0450 0x7fe168160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe166ffd700 1 -- 172.21.15.57:0/2576289350 >> v1:172.21.15.181:6789/0 conn(0x7fe1680cd5a0 legacy=0x7fe168160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe166ffd700 1 -- 172.21.15.57:0/2576289350 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680d1890 con 0x7fe16806ab10 2023-11-28T12:02:39.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe166ffd700 1 --2- 172.21.15.57:0/2576289350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16806ab10 0x7fe16806aee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1580098f0 tx=0x7fe158009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41ca04e564e866a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:39.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe165ffb700 1 -- 172.21.15.57:0/2576289350 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158011830 con 0x7fe16806ab10 2023-11-28T12:02:39.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe165ffb700 1 -- 172.21.15.57:0/2576289350 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe158011990 con 0x7fe16806ab10 2023-11-28T12:02:39.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.274+0000 7fe165ffb700 1 -- 172.21.15.57:0/2576289350 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158011d20 con 0x7fe16806ab10 2023-11-28T12:02:39.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.275+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2576289350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16806ab10 msgr2=0x7fe16806aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.275+0000 7fe16d6ba700 1 --2- 172.21.15.57:0/2576289350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16806ab10 0x7fe16806aee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1580098f0 tx=0x7fe158009c00 comp rx=0 tx=0).stop 2023-11-28T12:02:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.275+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2576289350 shutdown_connections 2023-11-28T12:02:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.275+0000 7fe16d6ba700 1 --2- 172.21.15.57:0/2576289350 >> v2:172.21.15.181:3300/0 conn(0x7fe1680d0450 0x7fe168160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.275+0000 7fe16d6ba700 1 --2- 172.21.15.57:0/2576289350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16806ab10 0x7fe16806aee0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.275+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2576289350 >> 172.21.15.57:0/2576289350 conn(0x7fe16806c520 msgr2=0x7fe16806c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.275+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2576289350 shutdown_connections 2023-11-28T12:02:39.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.275+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2576289350 wait complete. 2023-11-28T12:02:39.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.276+0000 7fe16d6ba700 1 Processor -- start 2023-11-28T12:02:39.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.277+0000 7fe16d6ba700 1 -- start start 2023-11-28T12:02:39.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.277+0000 7fe16d6ba700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe16806ab10 0x7fe1680c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:39.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.277+0000 7fe16d6ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1680cd5a0 0x7fe168147c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.277+0000 7fe16d6ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1680d0450 0x7fe1680c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.277+0000 7fe16d6ba700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe168159630 con 0x7fe16806ab10 2023-11-28T12:02:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.277+0000 7fe16d6ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe168158a20 con 0x7fe1680cd5a0 2023-11-28T12:02:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.277+0000 7fe16d6ba700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe168158d00 con 0x7fe1680d0450 2023-11-28T12:02:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.277+0000 7fe166ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe16806ab10 0x7fe1680c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.277+0000 7fe166ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe16806ab10 0x7fe1680c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50866/0 (socket says 172.21.15.57:50866) 2023-11-28T12:02:39.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.277+0000 7fe166ffd700 1 -- 172.21.15.57:0/2234033457 learned_addr learned my addr 172.21.15.57:0/2234033457 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:39.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.278+0000 7fe1667fc700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1680cd5a0 0x7fe168147c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:39.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.278+0000 7fe1677fe700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1680d0450 0x7fe1680c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:39.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.278+0000 7fe166ffd700 1 -- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1680d0450 msgr2=0x7fe1680c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.278+0000 7fe166ffd700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1680d0450 0x7fe1680c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.278+0000 7fe166ffd700 1 -- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1680cd5a0 msgr2=0x7fe168147c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.278+0000 7fe166ffd700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1680cd5a0 0x7fe168147c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.278+0000 7fe166ffd700 1 -- 172.21.15.57:0/2234033457 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe158009550 con 0x7fe16806ab10 2023-11-28T12:02:39.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.278+0000 7fe166ffd700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe16806ab10 0x7fe1680c5e40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe158014040 tx=0x7fe158004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:39.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.278+0000 7fe14f7fe700 1 -- 172.21.15.57:0/2234033457 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158004510 con 0x7fe16806ab10 2023-11-28T12:02:39.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.278+0000 7fe14f7fe700 1 -- 172.21.15.57:0/2234033457 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15801e070 con 0x7fe16806ab10 2023-11-28T12:02:39.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.278+0000 7fe14f7fe700 1 -- 172.21.15.57:0/2234033457 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1580038e0 con 0x7fe16806ab10 2023-11-28T12:02:39.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.278+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2234033457 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16815cad0 con 0x7fe16806ab10 2023-11-28T12:02:39.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.278+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2234033457 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1680d1b70 con 0x7fe16806ab10 2023-11-28T12:02:39.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.279+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2234033457 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1680b1ec0 con 0x7fe16806ab10 2023-11-28T12:02:39.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.280+0000 7fe14f7fe700 1 -- 172.21.15.57:0/2234033457 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe15800f050 con 0x7fe16806ab10 2023-11-28T12:02:39.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.280+0000 7fe14f7fe700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1540682f0 0x7fe15406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:39.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.280+0000 7fe14f7fe700 1 -- 172.21.15.57:0/2234033457 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe158086ca0 con 0x7fe16806ab10 2023-11-28T12:02:39.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.280+0000 7fe1667fc700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1540682f0 0x7fe15406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:39.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.281+0000 7fe1667fc700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1540682f0 0x7fe15406a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe1680b39f0 tx=0x7fe15c007ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:39.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.286+0000 7fe14f7fe700 1 -- 172.21.15.57:0/2234033457 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe158053350 con 0x7fe16806ab10 2023-11-28T12:02:39.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.531+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2234033457 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1680743f0 con 0x7fe16806ab10 2023-11-28T12:02:39.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.532+0000 7fe14f7fe700 1 -- 172.21.15.57:0/2234033457 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4642 (secure 0 0 0) 0x7fe158020090 con 0x7fe16806ab10 2023-11-28T12:02:39.532 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:39.532 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/890913444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":890913444},{"type":"v1","addr":"172.21.15.181:6836","nonce":890913444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/847658150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":847658150},{"type":"v1","addr":"172.21.15.181:6837","nonce":847658150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1379013929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1379013929},{"type":"v1","addr":"172.21.15.57:6835","nonce":1379013929}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:01:43.919415+0000","modified":"2023-11-28T12:01:46.937696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6833/3776085595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3776085595},{"type":"v1","addr":"172.21.15.57:6833","nonce":3776085595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T12:02:39.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.534+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1540682f0 msgr2=0x7fe15406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.534+0000 7fe16d6ba700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1540682f0 0x7fe15406a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe1680b39f0 tx=0x7fe15c007ed0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.534+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2234033457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe16806ab10 msgr2=0x7fe1680c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.534+0000 7fe16d6ba700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe16806ab10 0x7fe1680c5e40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe158014040 tx=0x7fe158004750 comp rx=0 tx=0).stop 2023-11-28T12:02:39.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.534+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2234033457 shutdown_connections 2023-11-28T12:02:39.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.534+0000 7fe16d6ba700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1540682f0 0x7fe15406a7a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.534+0000 7fe16d6ba700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1680d0450 0x7fe1680c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.534+0000 7fe16d6ba700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe16806ab10 0x7fe1680c5e40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.534+0000 7fe16d6ba700 1 --2- 172.21.15.57:0/2234033457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1680cd5a0 0x7fe168147c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.534+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2234033457 >> 172.21.15.57:0/2234033457 conn(0x7fe16806c520 msgr2=0x7fe1680b5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:39.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.534+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2234033457 shutdown_connections 2023-11-28T12:02:39.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.534+0000 7fe16d6ba700 1 -- 172.21.15.57:0/2234033457 wait complete. 2023-11-28T12:02:39.536 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 8 2023-11-28T12:02:39.548 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:02:39.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.683+0000 7f58ccc85700 1 Processor -- start 2023-11-28T12:02:39.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.683+0000 7f58ccc85700 1 -- start start 2023-11-28T12:02:39.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58ccc85700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58c80aeda0 0x7f58c806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:39.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58ccc85700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f58c80cdeb0 0x7f58c80ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:39.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58ccc85700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c806ca90 con 0x7f58c80b1650 2023-11-28T12:02:39.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58ccc85700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c806d330 con 0x7f58c80cdeb0 2023-11-28T12:02:39.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58ccc85700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c80b2010 con 0x7f58c80aeda0 2023-11-28T12:02:39.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58c6d9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f58c80cdeb0 0x7f58c80ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:39.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58c5d9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f58c80b1650 0x7f58c80cd7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37462/0 (socket says 172.21.15.57:37462) 2023-11-28T12:02:39.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58c5d9b700 1 -- 172.21.15.57:0/4022478159 learned_addr learned my addr 172.21.15.57:0/4022478159 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:39.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58c659c700 1 --2- 172.21.15.57:0/4022478159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58c80aeda0 0x7f58c806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:39.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58c6d9d700 1 -- 172.21.15.57:0/4022478159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58c80aeda0 msgr2=0x7f58c806d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:39.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58c6d9d700 1 --2- 172.21.15.57:0/4022478159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58c80aeda0 0x7f58c806d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58c6d9d700 1 -- 172.21.15.57:0/4022478159 >> v1:172.21.15.57:6789/0 conn(0x7f58c80b1650 legacy=0x7f58c80cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58c6d9d700 1 -- 172.21.15.57:0/4022478159 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c806cce0 con 0x7f58c80cdeb0 2023-11-28T12:02:39.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58c559a700 1 -- 172.21.15.57:0/4022478159 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009205714 0 0) 0x7f58c806ca90 con 0x7f58c80b1650 2023-11-28T12:02:39.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58c6d9d700 1 --2- 172.21.15.57:0/4022478159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58c80cdeb0 0x7f58c80ca220 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f58c000bfa0 tx=0x7f58c000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71dcc694e3a2e15 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:39.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.684+0000 7f58c559a700 1 -- 172.21.15.57:0/4022478159 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c0010070 con 0x7f58c80cdeb0 2023-11-28T12:02:39.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.685+0000 7f58c559a700 1 -- 172.21.15.57:0/4022478159 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c000b2e0 con 0x7f58c80cdeb0 2023-11-28T12:02:39.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.685+0000 7f58c559a700 1 -- 172.21.15.57:0/4022478159 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c000dc40 con 0x7f58c80cdeb0 2023-11-28T12:02:39.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.685+0000 7f58ccc85700 1 -- 172.21.15.57:0/4022478159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58c80cdeb0 msgr2=0x7f58c80ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.685+0000 7f58ccc85700 1 --2- 172.21.15.57:0/4022478159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58c80cdeb0 0x7f58c80ca220 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f58c000bfa0 tx=0x7f58c000d330 comp rx=0 tx=0).stop 2023-11-28T12:02:39.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.685+0000 7f58ccc85700 1 -- 172.21.15.57:0/4022478159 shutdown_connections 2023-11-28T12:02:39.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.685+0000 7f58ccc85700 1 --2- 172.21.15.57:0/4022478159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58c80cdeb0 0x7f58c80ca220 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.685+0000 7f58ccc85700 1 --2- 172.21.15.57:0/4022478159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58c80aeda0 0x7f58c806d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.685+0000 7f58ccc85700 1 -- 172.21.15.57:0/4022478159 >> 172.21.15.57:0/4022478159 conn(0x7f58c806bfd0 msgr2=0x7f58c80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:39.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.685+0000 7f58ccc85700 1 -- 172.21.15.57:0/4022478159 shutdown_connections 2023-11-28T12:02:39.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.685+0000 7f58ccc85700 1 -- 172.21.15.57:0/4022478159 wait complete. 2023-11-28T12:02:39.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.687+0000 7f58ccc85700 1 Processor -- start 2023-11-28T12:02:39.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.687+0000 7f58ccc85700 1 -- start start 2023-11-28T12:02:39.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.687+0000 7f58ccc85700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58c80aeda0 0x7f58c8151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:39.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.687+0000 7f58ccc85700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58c80b1650 0x7f58c81eee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:39.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58ccc85700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58c80cdeb0 0x7f58c80b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:39.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58ccc85700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c815c930 con 0x7f58c80b1650 2023-11-28T12:02:39.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58ccc85700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c815c650 con 0x7f58c80aeda0 2023-11-28T12:02:39.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58ccc85700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c81ef9f0 con 0x7f58c80cdeb0 2023-11-28T12:02:39.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58c5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58c80b1650 0x7f58c81eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:39.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58c5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58c80b1650 0x7f58c81eee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50880/0 (socket says 172.21.15.57:50880) 2023-11-28T12:02:39.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58c5d9b700 1 -- 172.21.15.57:0/71144506 learned_addr learned my addr 172.21.15.57:0/71144506 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:39.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58c6d9d700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58c80cdeb0 0x7f58c80b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:39.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58c659c700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58c80aeda0 0x7f58c8151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:39.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58c5d9b700 1 -- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58c80cdeb0 msgr2=0x7f58c80b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58c5d9b700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58c80cdeb0 0x7f58c80b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58c5d9b700 1 -- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58c80aeda0 msgr2=0x7f58c8151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58c5d9b700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58c80aeda0 0x7f58c8151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58c5d9b700 1 -- 172.21.15.57:0/71144506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c000bc80 con 0x7f58c80b1650 2023-11-28T12:02:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58c6d9d700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58c80cdeb0 0x7f58c80b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58c5d9b700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58c80b1650 0x7f58c81eee60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0011e0 tx=0x7f58bc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58b6ffd700 1 -- 172.21.15.57:0/71144506 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc004c80 con 0x7f58c80b1650 2023-11-28T12:02:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58b6ffd700 1 -- 172.21.15.57:0/71144506 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc004e00 con 0x7f58c80b1650 2023-11-28T12:02:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58ccc85700 1 -- 172.21.15.57:0/71144506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c806dd00 con 0x7f58c80b1650 2023-11-28T12:02:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58b6ffd700 1 -- 172.21.15.57:0/71144506 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc007310 con 0x7f58c80b1650 2023-11-28T12:02:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.688+0000 7f58ccc85700 1 -- 172.21.15.57:0/71144506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c806cfc0 con 0x7f58c80b1650 2023-11-28T12:02:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.689+0000 7f58ccc85700 1 -- 172.21.15.57:0/71144506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c80af170 con 0x7f58c80b1650 2023-11-28T12:02:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.690+0000 7f58b6ffd700 1 -- 172.21.15.57:0/71144506 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f58bc01a020 con 0x7f58c80b1650 2023-11-28T12:02:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.690+0000 7f58b6ffd700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f58a80682f0 0x7f58a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.690+0000 7f58b6ffd700 1 -- 172.21.15.57:0/71144506 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f58bc085a10 con 0x7f58c80b1650 2023-11-28T12:02:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.691+0000 7f58c659c700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f58a80682f0 0x7f58a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:39.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.691+0000 7f58c659c700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f58a80682f0 0x7f58a806a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009b70 tx=0x7f58b8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:39.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.695+0000 7f58b6ffd700 1 -- 172.21.15.57:0/71144506 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f58bc052140 con 0x7f58c80b1650 2023-11-28T12:02:39.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.948+0000 7f58ccc85700 1 -- 172.21.15.57:0/71144506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58c81ef850 con 0x7f58c80b1650 2023-11-28T12:02:39.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.948+0000 7f58b6ffd700 1 -- 172.21.15.57:0/71144506 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4642 (secure 0 0 0) 0x7f58c81ef850 con 0x7f58c80b1650 2023-11-28T12:02:39.949 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:39.949 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/890913444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":890913444},{"type":"v1","addr":"172.21.15.181:6836","nonce":890913444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/847658150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":847658150},{"type":"v1","addr":"172.21.15.181:6837","nonce":847658150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1379013929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1379013929},{"type":"v1","addr":"172.21.15.57:6835","nonce":1379013929}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:01:43.919415+0000","modified":"2023-11-28T12:01:46.937696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6833/3776085595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3776085595},{"type":"v1","addr":"172.21.15.57:6833","nonce":3776085595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T12:02:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.951+0000 7f58ccc85700 1 -- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f58a80682f0 msgr2=0x7f58a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.951+0000 7f58ccc85700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f58a80682f0 0x7f58a806a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009b70 tx=0x7f58b8008040 comp rx=0 tx=0).stop 2023-11-28T12:02:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.951+0000 7f58ccc85700 1 -- 172.21.15.57:0/71144506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58c80b1650 msgr2=0x7f58c81eee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.951+0000 7f58ccc85700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58c80b1650 0x7f58c81eee60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0011e0 tx=0x7f58bc00cae0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.951+0000 7f58ccc85700 1 -- 172.21.15.57:0/71144506 shutdown_connections 2023-11-28T12:02:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.951+0000 7f58ccc85700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f58a80682f0 0x7f58a806a7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.951+0000 7f58ccc85700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58c80cdeb0 0x7f58c80b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.951+0000 7f58ccc85700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58c80aeda0 0x7f58c8151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.951+0000 7f58ccc85700 1 --2- 172.21.15.57:0/71144506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58c80b1650 0x7f58c81eee60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.951+0000 7f58ccc85700 1 -- 172.21.15.57:0/71144506 >> 172.21.15.57:0/71144506 conn(0x7f58c806bfd0 msgr2=0x7f58c806e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.951+0000 7f58ccc85700 1 -- 172.21.15.57:0/71144506 shutdown_connections 2023-11-28T12:02:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:39.951+0000 7f58ccc85700 1 -- 172.21.15.57:0/71144506 wait complete. 2023-11-28T12:02:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 8 2023-11-28T12:02:39.966 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:40.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.101+0000 7f8306131700 1 Processor -- start 2023-11-28T12:02:40.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.101+0000 7f8306131700 1 -- start start 2023-11-28T12:02:40.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.102+0000 7f8306131700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f830015cc80 0x7f830006de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:40.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.102+0000 7f8306131700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f830006e3a0 0x7f8300070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:40.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.102+0000 7f8306131700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83000b1810 con 0x7f830006a270 2023-11-28T12:02:40.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.102+0000 7f8306131700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83000b2490 con 0x7f830006e3a0 2023-11-28T12:02:40.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.102+0000 7f8306131700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83000b1af0 con 0x7f830015cc80 2023-11-28T12:02:40.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.102+0000 7f82fffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f830006e3a0 0x7f8300070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:40.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.102+0000 7f82ff7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f830006a270 0x7f830006a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37476/0 (socket says 172.21.15.57:37476) 2023-11-28T12:02:40.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.102+0000 7f82ff7fe700 1 -- 172.21.15.57:0/4020100602 learned_addr learned my addr 172.21.15.57:0/4020100602 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:40.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.102+0000 7f82feffd700 1 --2- 172.21.15.57:0/4020100602 >> v2:172.21.15.181:3300/0 conn(0x7f830015cc80 0x7f830006de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:40.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.103+0000 7f82fffff700 1 -- 172.21.15.57:0/4020100602 >> v2:172.21.15.181:3300/0 conn(0x7f830015cc80 msgr2=0x7f830006de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:40.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.103+0000 7f82fffff700 1 --2- 172.21.15.57:0/4020100602 >> v2:172.21.15.181:3300/0 conn(0x7f830015cc80 0x7f830006de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.103+0000 7f82fffff700 1 -- 172.21.15.57:0/4020100602 >> v1:172.21.15.57:6789/0 conn(0x7f830006a270 legacy=0x7f830006a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:40.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.103+0000 7f82fe7fc700 1 -- 172.21.15.57:0/4020100602 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446295566 0 0) 0x7f83000b1810 con 0x7f830006a270 2023-11-28T12:02:40.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.103+0000 7f82fffff700 1 -- 172.21.15.57:0/4020100602 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83000b1dd0 con 0x7f830006e3a0 2023-11-28T12:02:40.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.103+0000 7f82fffff700 1 --2- 172.21.15.57:0/4020100602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830006e3a0 0x7f8300070d20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f830006c6c0 tx=0x7f82f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2eed22e2e99b426 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:40.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.103+0000 7f82fe7fc700 1 -- 172.21.15.57:0/4020100602 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f800e070 con 0x7f830006e3a0 2023-11-28T12:02:40.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.103+0000 7f82fe7fc700 1 -- 172.21.15.57:0/4020100602 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f800bb20 con 0x7f830006e3a0 2023-11-28T12:02:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.103+0000 7f82fe7fc700 1 -- 172.21.15.57:0/4020100602 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8012430 con 0x7f830006e3a0 2023-11-28T12:02:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.103+0000 7f8306131700 1 -- 172.21.15.57:0/4020100602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830006e3a0 msgr2=0x7f8300070d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.103+0000 7f8306131700 1 --2- 172.21.15.57:0/4020100602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830006e3a0 0x7f8300070d20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f830006c6c0 tx=0x7f82f800b330 comp rx=0 tx=0).stop 2023-11-28T12:02:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.104+0000 7f8306131700 1 -- 172.21.15.57:0/4020100602 shutdown_connections 2023-11-28T12:02:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.104+0000 7f8306131700 1 --2- 172.21.15.57:0/4020100602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830006e3a0 0x7f8300070d20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.104+0000 7f8306131700 1 --2- 172.21.15.57:0/4020100602 >> v2:172.21.15.181:3300/0 conn(0x7f830015cc80 0x7f830006de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.104+0000 7f8306131700 1 -- 172.21.15.57:0/4020100602 >> 172.21.15.57:0/4020100602 conn(0x7f830006c010 msgr2=0x7f83000cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.104+0000 7f8306131700 1 -- 172.21.15.57:0/4020100602 shutdown_connections 2023-11-28T12:02:40.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.104+0000 7f8306131700 1 -- 172.21.15.57:0/4020100602 wait complete. 2023-11-28T12:02:40.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.105+0000 7f8306131700 1 Processor -- start 2023-11-28T12:02:40.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.105+0000 7f8306131700 1 -- start start 2023-11-28T12:02:40.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f8306131700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830006a270 0x7f8300142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:40.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f8306131700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830006e3a0 0x7f83000ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:40.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f8306131700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f830015cc80 0x7f83000c2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:40.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f8306131700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8300142000 con 0x7f830006a270 2023-11-28T12:02:40.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f8306131700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83001531a0 con 0x7f830006e3a0 2023-11-28T12:02:40.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f8306131700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000b5a50 con 0x7f830015cc80 2023-11-28T12:02:40.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f82ff7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830006a270 0x7f8300142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:40.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f82ff7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830006a270 0x7f8300142420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50890/0 (socket says 172.21.15.57:50890) 2023-11-28T12:02:40.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f82ff7fe700 1 -- 172.21.15.57:0/3402474190 learned_addr learned my addr 172.21.15.57:0/3402474190 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:40.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f82fffff700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f830015cc80 0x7f83000c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:40.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f82ff7fe700 1 -- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f830015cc80 msgr2=0x7f83000c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:40.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f82ff7fe700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f830015cc80 0x7f83000c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f82feffd700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830006e3a0 0x7f83000ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:40.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f82ff7fe700 1 -- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830006e3a0 msgr2=0x7f83000ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:40.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f82ff7fe700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830006e3a0 0x7f83000ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f82ff7fe700 1 -- 172.21.15.57:0/3402474190 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f8009a70 con 0x7f830006a270 2023-11-28T12:02:40.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f82fffff700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f830015cc80 0x7f83000c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:40.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.106+0000 7f82feffd700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830006e3a0 0x7f83000ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:40.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.107+0000 7f82ff7fe700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830006a270 0x7f8300142420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c550 tx=0x7f82f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:40.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.107+0000 7f82f7fff700 1 -- 172.21.15.57:0/3402474190 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0004c80 con 0x7f830006a270 2023-11-28T12:02:40.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.107+0000 7f82f7fff700 1 -- 172.21.15.57:0/3402474190 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f0004e00 con 0x7f830006a270 2023-11-28T12:02:40.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.107+0000 7f82f7fff700 1 -- 172.21.15.57:0/3402474190 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0007310 con 0x7f830006a270 2023-11-28T12:02:40.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.107+0000 7f8306131700 1 -- 172.21.15.57:0/3402474190 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830006e7c0 con 0x7f830006a270 2023-11-28T12:02:40.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.107+0000 7f8306131700 1 -- 172.21.15.57:0/3402474190 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83000b20b0 con 0x7f830006a270 2023-11-28T12:02:40.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.108+0000 7f8306131700 1 -- 172.21.15.57:0/3402474190 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830006e950 con 0x7f830006a270 2023-11-28T12:02:40.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.108+0000 7f82f7fff700 1 -- 172.21.15.57:0/3402474190 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f82f001a020 con 0x7f830006a270 2023-11-28T12:02:40.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.109+0000 7f82f7fff700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82e00682f0 0x7f82e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:40.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.109+0000 7f82f7fff700 1 -- 172.21.15.57:0/3402474190 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f82f0085970 con 0x7f830006a270 2023-11-28T12:02:40.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.109+0000 7f82feffd700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82e00682f0 0x7f82e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:40.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.110+0000 7f82feffd700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82e00682f0 0x7f82e006a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8300153770 tx=0x7f82e8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:40.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.114+0000 7f82f7fff700 1 -- 172.21.15.57:0/3402474190 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f830006e950 con 0x7f830006a270 2023-11-28T12:02:40.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.311+0000 7f8306131700 1 -- 172.21.15.57:0/3402474190 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f830006e950 con 0x7f830006a270 2023-11-28T12:02:40.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.312+0000 7f82f7fff700 1 -- 172.21.15.57:0/3402474190 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17945 (secure 0 0 0) 0x7f830006e950 con 0x7f830006a270 2023-11-28T12:02:40.312 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:40.313 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:02:32.851731+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:01:41.687926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T12:01:42.895826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:40.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.314+0000 7f8306131700 1 -- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82e00682f0 msgr2=0x7f82e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:40.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.314+0000 7f8306131700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82e00682f0 0x7f82e006a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8300153770 tx=0x7f82e8008040 comp rx=0 tx=0).stop 2023-11-28T12:02:40.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.314+0000 7f8306131700 1 -- 172.21.15.57:0/3402474190 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830006a270 msgr2=0x7f8300142420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:40.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.314+0000 7f8306131700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830006a270 0x7f8300142420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c550 tx=0x7f82f000c910 comp rx=0 tx=0).stop 2023-11-28T12:02:40.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.314+0000 7f8306131700 1 -- 172.21.15.57:0/3402474190 shutdown_connections 2023-11-28T12:02:40.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.314+0000 7f8306131700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82e00682f0 0x7f82e006a7a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.314+0000 7f8306131700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f830015cc80 0x7f83000c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.314+0000 7f8306131700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830006a270 0x7f8300142420 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.314+0000 7f8306131700 1 --2- 172.21.15.57:0/3402474190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830006e3a0 0x7f83000ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.314+0000 7f8306131700 1 -- 172.21.15.57:0/3402474190 >> 172.21.15.57:0/3402474190 conn(0x7f830006c010 msgr2=0x7f830006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:40.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.314+0000 7f8306131700 1 -- 172.21.15.57:0/3402474190 shutdown_connections 2023-11-28T12:02:40.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.314+0000 7f8306131700 1 -- 172.21.15.57:0/3402474190 wait complete. 2023-11-28T12:02:40.328 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T12:02:40.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.465+0000 7f78a9aac700 1 Processor -- start 2023-11-28T12:02:40.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.465+0000 7f78a9aac700 1 -- start start 2023-11-28T12:02:40.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.466+0000 7f78a9aac700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f78a415fe50 0x7f78a406d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:40.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.466+0000 7f78a9aac700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f78a40b3b80 0x7f78a4160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:40.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.466+0000 7f78a9aac700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a4161370 con 0x7f78a40b27d0 2023-11-28T12:02:40.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.466+0000 7f78a9aac700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a4160220 con 0x7f78a40b3b80 2023-11-28T12:02:40.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.466+0000 7f78a9aac700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a40cf700 con 0x7f78a415fe50 2023-11-28T12:02:40.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.466+0000 7f78a2ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f78a40b3b80 0x7f78a4160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:40.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.466+0000 7f78a3fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f78a40b27d0 0x7f78a40cfb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37484/0 (socket says 172.21.15.57:37484) 2023-11-28T12:02:40.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.466+0000 7f78a2ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f78a40b3b80 0x7f78a4160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50904/0 (socket says 172.21.15.57:50904) 2023-11-28T12:02:40.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.466+0000 7f78a3fff700 1 -- 172.21.15.57:0/2136850846 learned_addr learned my addr 172.21.15.57:0/2136850846 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:40.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.466+0000 7f78a37fe700 1 --2- 172.21.15.57:0/2136850846 >> v2:172.21.15.181:3300/0 conn(0x7f78a415fe50 0x7f78a406d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:40.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.467+0000 7f78a2ffd700 1 -- 172.21.15.57:0/2136850846 >> v2:172.21.15.181:3300/0 conn(0x7f78a415fe50 msgr2=0x7f78a406d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:40.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.467+0000 7f78a2ffd700 1 --2- 172.21.15.57:0/2136850846 >> v2:172.21.15.181:3300/0 conn(0x7f78a415fe50 0x7f78a406d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.467+0000 7f78a2ffd700 1 -- 172.21.15.57:0/2136850846 >> v1:172.21.15.57:6789/0 conn(0x7f78a40b27d0 legacy=0x7f78a40cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:40.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.467+0000 7f78a2ffd700 1 -- 172.21.15.57:0/2136850846 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a40b3fd0 con 0x7f78a40b3b80 2023-11-28T12:02:40.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.467+0000 7f78a37fe700 1 --2- 172.21.15.57:0/2136850846 >> v2:172.21.15.181:3300/0 conn(0x7f78a415fe50 0x7f78a406d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:40.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.467+0000 7f78a27fc700 1 -- 172.21.15.57:0/2136850846 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4086238293 0 0) 0x7f78a4161370 con 0x7f78a40b27d0 2023-11-28T12:02:40.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.467+0000 7f78a2ffd700 1 --2- 172.21.15.57:0/2136850846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78a40b3b80 0x7f78a4160ad0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f78a40d1e30 tx=0x7f788c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a601f96d014d81 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:40.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.467+0000 7f78a27fc700 1 -- 172.21.15.57:0/2136850846 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c011830 con 0x7f78a40b3b80 2023-11-28T12:02:40.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.467+0000 7f78a27fc700 1 -- 172.21.15.57:0/2136850846 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c011990 con 0x7f78a40b3b80 2023-11-28T12:02:40.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.467+0000 7f78a27fc700 1 -- 172.21.15.57:0/2136850846 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c011d20 con 0x7f78a40b3b80 2023-11-28T12:02:40.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.468+0000 7f78a9aac700 1 -- 172.21.15.57:0/2136850846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78a40b3b80 msgr2=0x7f78a4160ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:40.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.468+0000 7f78a9aac700 1 --2- 172.21.15.57:0/2136850846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78a40b3b80 0x7f78a4160ad0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f78a40d1e30 tx=0x7f788c009b20 comp rx=0 tx=0).stop 2023-11-28T12:02:40.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.468+0000 7f78a9aac700 1 -- 172.21.15.57:0/2136850846 shutdown_connections 2023-11-28T12:02:40.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.468+0000 7f78a9aac700 1 --2- 172.21.15.57:0/2136850846 >> v2:172.21.15.181:3300/0 conn(0x7f78a415fe50 0x7f78a406d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.468+0000 7f78a9aac700 1 --2- 172.21.15.57:0/2136850846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78a40b3b80 0x7f78a4160ad0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.468+0000 7f78a9aac700 1 -- 172.21.15.57:0/2136850846 >> 172.21.15.57:0/2136850846 conn(0x7f78a406ea80 msgr2=0x7f78a406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:40.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.468+0000 7f78a9aac700 1 -- 172.21.15.57:0/2136850846 shutdown_connections 2023-11-28T12:02:40.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.468+0000 7f78a9aac700 1 -- 172.21.15.57:0/2136850846 wait complete. 2023-11-28T12:02:40.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.469+0000 7f78a9aac700 1 Processor -- start 2023-11-28T12:02:40.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.469+0000 7f78a9aac700 1 -- start start 2023-11-28T12:02:40.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a9aac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78a40b27d0 0x7f78a40cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:40.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a9aac700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78a40b3b80 0x7f78a40c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:40.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a9aac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78a415fe50 0x7f78a41536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:40.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a9aac700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40c8340 con 0x7f78a40b3b80 2023-11-28T12:02:40.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a9aac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4153c10 con 0x7f78a415fe50 2023-11-28T12:02:40.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a9aac700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40c4820 con 0x7f78a40b27d0 2023-11-28T12:02:40.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78a40b3b80 0x7f78a40c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:40.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78a40b3b80 0x7f78a40c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50918/0 (socket says 172.21.15.57:50918) 2023-11-28T12:02:40.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a2ffd700 1 -- 172.21.15.57:0/3667350957 learned_addr learned my addr 172.21.15.57:0/3667350957 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:40.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a37fe700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78a40b27d0 0x7f78a40cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:40.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a3fff700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78a415fe50 0x7f78a41536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:40.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a2ffd700 1 -- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78a40b27d0 msgr2=0x7f78a40cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:40.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a2ffd700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78a40b27d0 0x7f78a40cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a2ffd700 1 -- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78a415fe50 msgr2=0x7f78a41536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:40.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a2ffd700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78a415fe50 0x7f78a41536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:40.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.470+0000 7f78a2ffd700 1 -- 172.21.15.57:0/3667350957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c0094a0 con 0x7f78a40b3b80 2023-11-28T12:02:40.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.471+0000 7f78a2ffd700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78a40b3b80 0x7f78a40c7e00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f788c014040 tx=0x7f788c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:40.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.471+0000 7f789bfff700 1 -- 172.21.15.57:0/3667350957 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c0038a0 con 0x7f78a40b3b80 2023-11-28T12:02:40.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.471+0000 7f789bfff700 1 -- 172.21.15.57:0/3667350957 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c01e070 con 0x7f78a40b3b80 2023-11-28T12:02:40.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.471+0000 7f789bfff700 1 -- 172.21.15.57:0/3667350957 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c003c60 con 0x7f78a40b3b80 2023-11-28T12:02:40.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.471+0000 7f78a9aac700 1 -- 172.21.15.57:0/3667350957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a40cf0b0 con 0x7f78a40b3b80 2023-11-28T12:02:40.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.471+0000 7f78a9aac700 1 -- 172.21.15.57:0/3667350957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a40cedd0 con 0x7f78a40b3b80 2023-11-28T12:02:40.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.472+0000 7f78a9aac700 1 -- 172.21.15.57:0/3667350957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a40cf390 con 0x7f78a40b3b80 2023-11-28T12:02:40.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.472+0000 7f789bfff700 1 -- 172.21.15.57:0/3667350957 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f788c00f050 con 0x7f78a40b3b80 2023-11-28T12:02:40.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.473+0000 7f789bfff700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f78840682f0 0x7f788406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:40.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.473+0000 7f789bfff700 1 -- 172.21.15.57:0/3667350957 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f788c086db0 con 0x7f78a40b3b80 2023-11-28T12:02:40.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.473+0000 7f78a37fe700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f78840682f0 0x7f788406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:40.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.474+0000 7f78a37fe700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f78840682f0 0x7f788406a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7894006fd0 tx=0x7f7894008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:40.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.478+0000 7f789bfff700 1 -- 172.21.15.57:0/3667350957 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f788c0535d0 con 0x7f78a40b3b80 2023-11-28T12:02:40.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:40.724+0000 7f78a9aac700 1 -- 172.21.15.57:0/3667350957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f78a40743f0 con 0x7f78a40b3b80 2023-11-28T12:02:41.540 INFO:tasks.ceph.mon.a.smithi057.stderr:2023-11-28T12:02:41.540+0000 7fb777740700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T12:02:41.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.544+0000 7f789bfff700 1 -- 172.21.15.57:0/3667350957 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f788c015090 con 0x7f78a40b3b80 2023-11-28T12:02:41.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.547+0000 7f78a9aac700 1 -- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f78840682f0 msgr2=0x7f788406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:41.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.547+0000 7f78a9aac700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f78840682f0 0x7f788406a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7894006fd0 tx=0x7f7894008040 comp rx=0 tx=0).stop 2023-11-28T12:02:41.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.547+0000 7f78a9aac700 1 -- 172.21.15.57:0/3667350957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78a40b3b80 msgr2=0x7f78a40c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:41.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.547+0000 7f78a9aac700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78a40b3b80 0x7f78a40c7e00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f788c014040 tx=0x7f788c004750 comp rx=0 tx=0).stop 2023-11-28T12:02:41.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.547+0000 7f78a9aac700 1 -- 172.21.15.57:0/3667350957 shutdown_connections 2023-11-28T12:02:41.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.547+0000 7f78a9aac700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f78840682f0 0x7f788406a7a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:41.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.547+0000 7f78a9aac700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78a415fe50 0x7f78a41536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:41.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.547+0000 7f78a9aac700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78a40b27d0 0x7f78a40cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:41.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.547+0000 7f78a9aac700 1 --2- 172.21.15.57:0/3667350957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78a40b3b80 0x7f78a40c7e00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:41.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.547+0000 7f78a9aac700 1 -- 172.21.15.57:0/3667350957 >> 172.21.15.57:0/3667350957 conn(0x7f78a406ea80 msgr2=0x7f78a4160900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:41.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.547+0000 7f78a9aac700 1 -- 172.21.15.57:0/3667350957 shutdown_connections 2023-11-28T12:02:41.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.547+0000 7f78a9aac700 1 -- 172.21.15.57:0/3667350957 wait complete. 2023-11-28T12:02:41.549 INFO:tasks.ceph.mds.c.smithi057.stderr: -3163> 2023-11-28T12:01:49.377+0000 7f0635dad700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T12:02:41.549 INFO:teuthology.orchestra.run.smithi057.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T12:02:41.564 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T12:02:41.580 INFO:tasks.ceph.mds.c.smithi057.stdout:starting mds.c at 2023-11-28T12:02:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.705+0000 7fe1d9696700 1 Processor -- start 2023-11-28T12:02:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.705+0000 7fe1d9696700 1 -- start start 2023-11-28T12:02:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.706+0000 7fe1d9696700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe1d4160cb0 0x7fe1d406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.706+0000 7fe1d9696700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d40d2d30 con 0x7fe1d415c8c0 2023-11-28T12:02:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.706+0000 7fe1d9696700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d40d0b40 con 0x7fe1d406b050 2023-11-28T12:02:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.706+0000 7fe1d9696700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d406b490 con 0x7fe1d4160cb0 2023-11-28T12:02:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.706+0000 7fe1d27fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe1d4160cb0 0x7fe1d406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.706+0000 7fe1d2ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe1d415c8c0 0x7fe1d40b5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37494/0 (socket says 172.21.15.57:37494) 2023-11-28T12:02:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.706+0000 7fe1d2ffd700 1 -- 172.21.15.57:0/1243244585 learned_addr learned my addr 172.21.15.57:0/1243244585 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.706+0000 7fe1d27fc700 1 -- 172.21.15.57:0/1243244585 >> v1:172.21.15.181:6789/0 conn(0x7fe1d406b050 legacy=0x7fe1d40d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.706+0000 7fe1d27fc700 1 -- 172.21.15.57:0/1243244585 >> v1:172.21.15.57:6789/0 conn(0x7fe1d415c8c0 legacy=0x7fe1d40b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.706+0000 7fe1d27fc700 1 -- 172.21.15.57:0/1243244585 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d40d2a50 con 0x7fe1d4160cb0 2023-11-28T12:02:41.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.706+0000 7fe1d1ffb700 1 -- 172.21.15.57:0/1243244585 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892372396 0 0) 0x7fe1bc001110 con 0x7fe1d415c8c0 2023-11-28T12:02:41.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.706+0000 7fe1d27fc700 1 --2- 172.21.15.57:0/1243244585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d4160cb0 0x7fe1d406ab10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009840 tx=0x7fe1c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa94e11afe3c9ef8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:41.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.707+0000 7fe1d1ffb700 1 -- 172.21.15.57:0/1243244585 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4011830 con 0x7fe1d4160cb0 2023-11-28T12:02:41.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.707+0000 7fe1d1ffb700 1 -- 172.21.15.57:0/1243244585 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c4011990 con 0x7fe1d4160cb0 2023-11-28T12:02:41.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.707+0000 7fe1d1ffb700 1 -- 172.21.15.57:0/1243244585 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4011d20 con 0x7fe1d4160cb0 2023-11-28T12:02:41.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.707+0000 7fe1d9696700 1 -- 172.21.15.57:0/1243244585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d4160cb0 msgr2=0x7fe1d406ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:41.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.707+0000 7fe1d9696700 1 --2- 172.21.15.57:0/1243244585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d4160cb0 0x7fe1d406ab10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009840 tx=0x7fe1c4009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:41.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.707+0000 7fe1d9696700 1 -- 172.21.15.57:0/1243244585 shutdown_connections 2023-11-28T12:02:41.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.707+0000 7fe1d9696700 1 --2- 172.21.15.57:0/1243244585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d4160cb0 0x7fe1d406ab10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:41.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.707+0000 7fe1d9696700 1 -- 172.21.15.57:0/1243244585 >> 172.21.15.57:0/1243244585 conn(0x7fe1d406c520 msgr2=0x7fe1d406c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:41.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.707+0000 7fe1d9696700 1 -- 172.21.15.57:0/1243244585 shutdown_connections 2023-11-28T12:02:41.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.707+0000 7fe1d9696700 1 -- 172.21.15.57:0/1243244585 wait complete. 2023-11-28T12:02:41.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.709+0000 7fe1d9696700 1 Processor -- start 2023-11-28T12:02:41.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.709+0000 7fe1d9696700 1 -- start start 2023-11-28T12:02:41.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d9696700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d406b050 0x7fe1d41e62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:41.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d9696700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d415c8c0 0x7fe1d414cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:41.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d9696700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d4160cb0 0x7fe1d40b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:41.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d9696700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d414d050 con 0x7fe1d4160cb0 2023-11-28T12:02:41.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d9696700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d415cc90 con 0x7fe1d415c8c0 2023-11-28T12:02:41.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d9696700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d414da40 con 0x7fe1d406b050 2023-11-28T12:02:41.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d4160cb0 0x7fe1d40b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:41.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d4160cb0 0x7fe1d40b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50946/0 (socket says 172.21.15.57:50946) 2023-11-28T12:02:41.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d37fe700 1 -- 172.21.15.57:0/1990541503 learned_addr learned my addr 172.21.15.57:0/1990541503 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:41.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d27fc700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d415c8c0 0x7fe1d414cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:41.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d2ffd700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d406b050 0x7fe1d41e62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:41.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d37fe700 1 -- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d406b050 msgr2=0x7fe1d41e62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:41.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d37fe700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d406b050 0x7fe1d41e62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:41.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d37fe700 1 -- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d415c8c0 msgr2=0x7fe1d414cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:41.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d37fe700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d415c8c0 0x7fe1d414cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:41.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d37fe700 1 -- 172.21.15.57:0/1990541503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c40094a0 con 0x7fe1d4160cb0 2023-11-28T12:02:41.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.710+0000 7fe1d37fe700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d4160cb0 0x7fe1d40b7a50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00cc50 tx=0x7fe1cc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:41.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.711+0000 7fe1cb7fe700 1 -- 172.21.15.57:0/1990541503 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc003c00 con 0x7fe1d4160cb0 2023-11-28T12:02:41.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.711+0000 7fe1cb7fe700 1 -- 172.21.15.57:0/1990541503 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc003d60 con 0x7fe1d4160cb0 2023-11-28T12:02:41.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.711+0000 7fe1d9696700 1 -- 172.21.15.57:0/1990541503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d40cda40 con 0x7fe1d4160cb0 2023-11-28T12:02:41.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.711+0000 7fe1cb7fe700 1 -- 172.21.15.57:0/1990541503 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc00aaf0 con 0x7fe1d4160cb0 2023-11-28T12:02:41.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.711+0000 7fe1d9696700 1 -- 172.21.15.57:0/1990541503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d40cd760 con 0x7fe1d4160cb0 2023-11-28T12:02:41.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.712+0000 7fe1d9696700 1 -- 172.21.15.57:0/1990541503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d4161120 con 0x7fe1d4160cb0 2023-11-28T12:02:41.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.712+0000 7fe1cb7fe700 1 -- 172.21.15.57:0/1990541503 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe1cc010050 con 0x7fe1d4160cb0 2023-11-28T12:02:41.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.713+0000 7fe1cb7fe700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1b40682f0 0x7fe1b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:41.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.713+0000 7fe1cb7fe700 1 -- 172.21.15.57:0/1990541503 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fe1cc085be0 con 0x7fe1d4160cb0 2023-11-28T12:02:41.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.713+0000 7fe1d2ffd700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1b40682f0 0x7fe1b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:41.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.713+0000 7fe1d2ffd700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1b40682f0 0x7fe1b406a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc007e80 tx=0x7fe1bc007e10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:41.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.717+0000 7fe1cb7fe700 1 -- 172.21.15.57:0/1990541503 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe1cc0523b0 con 0x7fe1d4160cb0 2023-11-28T12:02:41.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:41.962+0000 7fe1d9696700 1 -- 172.21.15.57:0/1990541503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe1d4067d70 con 0x7fe1d4160cb0 2023-11-28T12:02:42.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.546+0000 7fe1cb7fe700 1 -- 172.21.15.57:0/1990541503 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe1cc0559d0 con 0x7fe1d4160cb0 2023-11-28T12:02:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.549+0000 7fe1d9696700 1 -- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1b40682f0 msgr2=0x7fe1b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.549+0000 7fe1d9696700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1b40682f0 0x7fe1b406a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc007e80 tx=0x7fe1bc007e10 comp rx=0 tx=0).stop 2023-11-28T12:02:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.549+0000 7fe1d9696700 1 -- 172.21.15.57:0/1990541503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d4160cb0 msgr2=0x7fe1d40b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.549+0000 7fe1d9696700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d4160cb0 0x7fe1d40b7a50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00cc50 tx=0x7fe1cc00cf60 comp rx=0 tx=0).stop 2023-11-28T12:02:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.549+0000 7fe1d9696700 1 -- 172.21.15.57:0/1990541503 shutdown_connections 2023-11-28T12:02:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.549+0000 7fe1d9696700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1b40682f0 0x7fe1b406a7a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.549+0000 7fe1d9696700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1d4160cb0 0x7fe1d40b7a50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.549+0000 7fe1d9696700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1d406b050 0x7fe1d41e62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.549+0000 7fe1d9696700 1 --2- 172.21.15.57:0/1990541503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d415c8c0 0x7fe1d414cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.549+0000 7fe1d9696700 1 -- 172.21.15.57:0/1990541503 >> 172.21.15.57:0/1990541503 conn(0x7fe1d406c520 msgr2=0x7fe1d40b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.549+0000 7fe1d9696700 1 -- 172.21.15.57:0/1990541503 shutdown_connections 2023-11-28T12:02:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.549+0000 7fe1d9696700 1 -- 172.21.15.57:0/1990541503 wait complete. 2023-11-28T12:02:42.565 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T12:02:42.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.752+0000 7f890eb6b700 1 Processor -- start 2023-11-28T12:02:42.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.752+0000 7f890eb6b700 1 -- start start 2023-11-28T12:02:42.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890eb6b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8908157360 0x7f8908161390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:42.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890eb6b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8908071320 0x7f89080ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:42.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890eb6b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8908157730 con 0x7f890815dca0 2023-11-28T12:02:42.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890eb6b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8908160f00 con 0x7f8908157360 2023-11-28T12:02:42.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890eb6b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89080cdba0 con 0x7f8908071320 2023-11-28T12:02:42.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890c907700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8908157360 0x7f8908161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:42.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890d108700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8908071320 0x7f89080ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:42.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890c907700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8908157360 0x7f8908161390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:54712/0 (socket says 172.21.15.57:54712) 2023-11-28T12:02:42.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890c907700 1 -- 172.21.15.57:0/1489781533 learned_addr learned my addr 172.21.15.57:0/1489781533 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:42.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890d108700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8908071320 0x7f89080ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:54634/0 (socket says 172.21.15.57:54634) 2023-11-28T12:02:42.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f8907fff700 1 --1- 172.21.15.57:0/1489781533 >> v1:172.21.15.181:6789/0 conn(0x7f890815dca0 0x7f8908070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:56850/0 (socket says 172.21.15.57:56850) 2023-11-28T12:02:42.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890c907700 1 -- 172.21.15.57:0/1489781533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8908071320 msgr2=0x7f89080ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:42.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890c907700 1 --2- 172.21.15.57:0/1489781533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8908071320 0x7f89080ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:42.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890c907700 1 -- 172.21.15.57:0/1489781533 >> v1:172.21.15.181:6789/0 conn(0x7f890815dca0 legacy=0x7f8908070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:42.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f890c907700 1 -- 172.21.15.57:0/1489781533 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89081608b0 con 0x7f8908157360 2023-11-28T12:02:42.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.753+0000 7f89077fe700 1 -- 172.21.15.57:0/1489781533 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615414771 0 0) 0x7f8908157730 con 0x7f890815dca0 2023-11-28T12:02:42.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.754+0000 7f890c907700 1 --2- 172.21.15.57:0/1489781533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8908157360 0x7f8908161390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0098f0 tx=0x7f88fc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3a0379148c2fe89 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:42.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.754+0000 7f89077fe700 1 -- 172.21.15.57:0/1489781533 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc011830 con 0x7f8908157360 2023-11-28T12:02:42.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.754+0000 7f89077fe700 1 -- 172.21.15.57:0/1489781533 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88fc011990 con 0x7f8908157360 2023-11-28T12:02:42.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.754+0000 7f89077fe700 1 -- 172.21.15.57:0/1489781533 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc011d20 con 0x7f8908157360 2023-11-28T12:02:42.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.754+0000 7f890eb6b700 1 -- 172.21.15.57:0/1489781533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8908157360 msgr2=0x7f8908161390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.754+0000 7f890eb6b700 1 --2- 172.21.15.57:0/1489781533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8908157360 0x7f8908161390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0098f0 tx=0x7f88fc009c00 comp rx=0 tx=0).stop 2023-11-28T12:02:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.754+0000 7f890eb6b700 1 -- 172.21.15.57:0/1489781533 shutdown_connections 2023-11-28T12:02:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.754+0000 7f890eb6b700 1 --2- 172.21.15.57:0/1489781533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8908157360 0x7f8908161390 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.754+0000 7f890eb6b700 1 --2- 172.21.15.57:0/1489781533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8908071320 0x7f89080ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.754+0000 7f890eb6b700 1 -- 172.21.15.57:0/1489781533 >> 172.21.15.57:0/1489781533 conn(0x7f89080cfc30 msgr2=0x7f89080d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.754+0000 7f890eb6b700 1 -- 172.21.15.57:0/1489781533 shutdown_connections 2023-11-28T12:02:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.754+0000 7f890eb6b700 1 -- 172.21.15.57:0/1489781533 wait complete. 2023-11-28T12:02:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.756+0000 7f890eb6b700 1 Processor -- start 2023-11-28T12:02:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.756+0000 7f890eb6b700 1 -- start start 2023-11-28T12:02:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f890eb6b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8908071320 0x7f89081f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:42.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f890eb6b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8908157360 0x7f8908149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f890eb6b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f890815dca0 0x7f8908155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f890eb6b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89081f2ce0 con 0x7f8908157360 2023-11-28T12:02:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f890eb6b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89080b2ba0 con 0x7f8908071320 2023-11-28T12:02:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f890eb6b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89081f6210 con 0x7f890815dca0 2023-11-28T12:02:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f8907fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8908157360 0x7f8908149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f8907fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8908157360 0x7f8908149890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50962/0 (socket says 172.21.15.57:50962) 2023-11-28T12:02:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f8907fff700 1 -- 172.21.15.57:0/765371850 learned_addr learned my addr 172.21.15.57:0/765371850 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f890c907700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8908071320 0x7f89081f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f890d108700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f890815dca0 0x7f8908155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f8907fff700 1 -- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f890815dca0 msgr2=0x7f8908155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f8907fff700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f890815dca0 0x7f8908155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f8907fff700 1 -- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8908071320 msgr2=0x7f89081f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:42.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f8907fff700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8908071320 0x7f89081f27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:42.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f8907fff700 1 -- 172.21.15.57:0/765371850 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc009550 con 0x7f8908157360 2023-11-28T12:02:42.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.757+0000 7f8907fff700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8908157360 0x7f8908149890 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f88f40011e0 tx=0x7f88f400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:42.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.758+0000 7f89057fa700 1 -- 172.21.15.57:0/765371850 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4004c80 con 0x7f8908157360 2023-11-28T12:02:42.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.758+0000 7f89057fa700 1 -- 172.21.15.57:0/765371850 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f4004de0 con 0x7f8908157360 2023-11-28T12:02:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.758+0000 7f890eb6b700 1 -- 172.21.15.57:0/765371850 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89080af410 con 0x7f8908157360 2023-11-28T12:02:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.758+0000 7f89057fa700 1 -- 172.21.15.57:0/765371850 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4007310 con 0x7f8908157360 2023-11-28T12:02:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.758+0000 7f890eb6b700 1 -- 172.21.15.57:0/765371850 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8908160b90 con 0x7f8908157360 2023-11-28T12:02:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.759+0000 7f890eb6b700 1 -- 172.21.15.57:0/765371850 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f890815e0f0 con 0x7f8908157360 2023-11-28T12:02:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.759+0000 7f89057fa700 1 -- 172.21.15.57:0/765371850 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f88f401a020 con 0x7f8908157360 2023-11-28T12:02:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.760+0000 7f89057fa700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f88ec0682f0 0x7f88ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.760+0000 7f89057fa700 1 -- 172.21.15.57:0/765371850 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f88f40858d0 con 0x7f8908157360 2023-11-28T12:02:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.760+0000 7f890c907700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f88ec0682f0 0x7f88ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.761+0000 7f890c907700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f88ec0682f0 0x7f88ec06a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f88fc014040 tx=0x7f88fc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.764+0000 7f89057fa700 1 -- 172.21.15.57:0/765371850 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f88f40520a0 con 0x7f8908157360 2023-11-28T12:02:42.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:42.968+0000 7f890eb6b700 1 -- 172.21.15.57:0/765371850 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f89080743f0 con 0x7f8908157360 2023-11-28T12:02:43.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:43.897+0000 7f89057fa700 1 -- 172.21.15.57:0/765371850 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f88f40556c0 con 0x7f8908157360 2023-11-28T12:02:43.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:43.901+0000 7f890eb6b700 1 -- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f88ec0682f0 msgr2=0x7f88ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:43.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:43.901+0000 7f890eb6b700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f88ec0682f0 0x7f88ec06a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f88fc014040 tx=0x7f88fc008030 comp rx=0 tx=0).stop 2023-11-28T12:02:43.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:43.901+0000 7f890eb6b700 1 -- 172.21.15.57:0/765371850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8908157360 msgr2=0x7f8908149890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:43.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:43.901+0000 7f890eb6b700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8908157360 0x7f8908149890 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f88f40011e0 tx=0x7f88f400ca30 comp rx=0 tx=0).stop 2023-11-28T12:02:43.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:43.902+0000 7f890eb6b700 1 -- 172.21.15.57:0/765371850 shutdown_connections 2023-11-28T12:02:43.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:43.902+0000 7f890eb6b700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f88ec0682f0 0x7f88ec06a7a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:43.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:43.902+0000 7f890eb6b700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f890815dca0 0x7f8908155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:43.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:43.902+0000 7f890eb6b700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8908071320 0x7f89081f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:43.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:43.902+0000 7f890eb6b700 1 --2- 172.21.15.57:0/765371850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8908157360 0x7f8908149890 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:43.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:43.902+0000 7f890eb6b700 1 -- 172.21.15.57:0/765371850 >> 172.21.15.57:0/765371850 conn(0x7f89080cfc30 msgr2=0x7f890815fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:43.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:43.903+0000 7f890eb6b700 1 -- 172.21.15.57:0/765371850 shutdown_connections 2023-11-28T12:02:43.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:43.903+0000 7f890eb6b700 1 -- 172.21.15.57:0/765371850 wait complete. 2023-11-28T12:02:43.906 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' removed 2023-11-28T12:02:43.924 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T12:02:44.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.067+0000 7f7df3d0c700 1 Processor -- start 2023-11-28T12:02:44.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.068+0000 7f7df3d0c700 1 -- start start 2023-11-28T12:02:44.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.068+0000 7f7df3d0c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7dec15c8f0 0x7f7dec06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:44.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.068+0000 7f7df3d0c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dec0ccef0 0x7f7dec06b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:44.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.068+0000 7f7df3d0c700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dec0cbe50 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.068+0000 7f7df3d0c700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dec0cc130 con 0x7f7dec15c8f0 2023-11-28T12:02:44.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.068+0000 7f7df3d0c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dec0cc410 con 0x7f7dec0ccef0 2023-11-28T12:02:44.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df12a7700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7dec15c8f0 0x7f7dec06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:44.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df1aa8700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f7dec0d0cb0 0x7f7dec0d1080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37504/0 (socket says 172.21.15.57:37504) 2023-11-28T12:02:44.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df1aa8700 1 -- 172.21.15.57:0/3682262145 learned_addr learned my addr 172.21.15.57:0/3682262145 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:44.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df22a9700 1 --2- 172.21.15.57:0/3682262145 >> v2:172.21.15.181:3300/0 conn(0x7f7dec0ccef0 0x7f7dec06b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:44.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df12a7700 1 -- 172.21.15.57:0/3682262145 >> v2:172.21.15.181:3300/0 conn(0x7f7dec0ccef0 msgr2=0x7f7dec06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:44.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df12a7700 1 --2- 172.21.15.57:0/3682262145 >> v2:172.21.15.181:3300/0 conn(0x7f7dec0ccef0 0x7f7dec06b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:44.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df12a7700 1 -- 172.21.15.57:0/3682262145 >> v1:172.21.15.57:6789/0 conn(0x7f7dec0d0cb0 legacy=0x7f7dec0d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:44.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df12a7700 1 -- 172.21.15.57:0/3682262145 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dec0cc6f0 con 0x7f7dec15c8f0 2023-11-28T12:02:44.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df0aa6700 1 -- 172.21.15.57:0/3682262145 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2119807187 0 0) 0x7f7dec0cbe50 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df22a9700 1 --2- 172.21.15.57:0/3682262145 >> v2:172.21.15.181:3300/0 conn(0x7f7dec0ccef0 0x7f7dec06b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:44.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df12a7700 1 --2- 172.21.15.57:0/3682262145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec15c8f0 0x7f7dec06b380 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7dec06b220 tx=0x7f7de8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b97f5e1bcc22479 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:44.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df0aa6700 1 -- 172.21.15.57:0/3682262145 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de8011830 con 0x7f7dec15c8f0 2023-11-28T12:02:44.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df0aa6700 1 -- 172.21.15.57:0/3682262145 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de8011990 con 0x7f7dec15c8f0 2023-11-28T12:02:44.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.069+0000 7f7df0aa6700 1 -- 172.21.15.57:0/3682262145 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de8011d20 con 0x7f7dec15c8f0 2023-11-28T12:02:44.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.070+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3682262145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec15c8f0 msgr2=0x7f7dec06b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:44.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.070+0000 7f7df3d0c700 1 --2- 172.21.15.57:0/3682262145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec15c8f0 0x7f7dec06b380 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7dec06b220 tx=0x7f7de8009b20 comp rx=0 tx=0).stop 2023-11-28T12:02:44.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.070+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3682262145 shutdown_connections 2023-11-28T12:02:44.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.070+0000 7f7df3d0c700 1 --2- 172.21.15.57:0/3682262145 >> v2:172.21.15.181:3300/0 conn(0x7f7dec0ccef0 0x7f7dec06b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.070+0000 7f7df3d0c700 1 --2- 172.21.15.57:0/3682262145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec15c8f0 0x7f7dec06b380 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.070+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3682262145 >> 172.21.15.57:0/3682262145 conn(0x7f7dec06ab90 msgr2=0x7f7dec099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.070+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3682262145 shutdown_connections 2023-11-28T12:02:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.070+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3682262145 wait complete. 2023-11-28T12:02:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.071+0000 7f7df3d0c700 1 Processor -- start 2023-11-28T12:02:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.072+0000 7f7df3d0c700 1 -- start start 2023-11-28T12:02:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.072+0000 7f7df3d0c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dec0ccef0 0x7f7dec14a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.072+0000 7f7df3d0c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec0d0cb0 0x7f7dec0b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.072+0000 7f7df3d0c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dec15c8f0 0x7f7dec0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.072+0000 7f7df3d0c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec1581a0 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.072+0000 7f7df3d0c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec0b0360 con 0x7f7dec15c8f0 2023-11-28T12:02:44.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.072+0000 7f7df3d0c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec0b0640 con 0x7f7dec0ccef0 2023-11-28T12:02:44.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.072+0000 7f7df12a7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec0d0cb0 0x7f7dec0b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:44.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.072+0000 7f7df12a7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec0d0cb0 0x7f7dec0b6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50994/0 (socket says 172.21.15.57:50994) 2023-11-28T12:02:44.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.072+0000 7f7df12a7700 1 -- 172.21.15.57:0/3032322707 learned_addr learned my addr 172.21.15.57:0/3032322707 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:44.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7df1aa8700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dec0ccef0 0x7f7dec14a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:44.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7df22a9700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dec15c8f0 0x7f7dec0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:44.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7df12a7700 1 -- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dec0ccef0 msgr2=0x7f7dec14a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:44.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7df12a7700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dec0ccef0 0x7f7dec14a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:44.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7df12a7700 1 -- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dec15c8f0 msgr2=0x7f7dec0c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:44.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7df12a7700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dec15c8f0 0x7f7dec0c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:44.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7df12a7700 1 -- 172.21.15.57:0/3032322707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de80094a0 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7df1aa8700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dec0ccef0 0x7f7dec14a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7df22a9700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dec15c8f0 0x7f7dec0c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7df12a7700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec0d0cb0 0x7f7dec0b6610 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7de8000c00 tx=0x7f7de8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7dde7fc700 1 -- 172.21.15.57:0/3032322707 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de80038a0 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7dde7fc700 1 -- 172.21.15.57:0/3032322707 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de801e070 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3032322707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dec06be00 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7dde7fc700 1 -- 172.21.15.57:0/3032322707 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de8003c40 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.073+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3032322707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dec0cc9d0 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.074+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3032322707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dec0ad980 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.075+0000 7f7dde7fc700 1 -- 172.21.15.57:0/3032322707 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7de800f050 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.075+0000 7f7dde7fc700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7dd00682f0 0x7f7dd006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.075+0000 7f7dde7fc700 1 -- 172.21.15.57:0/3032322707 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f7de8086cb0 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.075+0000 7f7df1aa8700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7dd00682f0 0x7f7dd006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:44.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.076+0000 7f7df1aa8700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7dd00682f0 0x7f7dd006a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7dec147a70 tx=0x7f7de0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:44.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.080+0000 7f7dde7fc700 1 -- 172.21.15.57:0/3032322707 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7de8053630 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.281+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3032322707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7dec0743f0 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.899+0000 7f7dde7fc700 1 -- 172.21.15.57:0/3032322707 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7de8015020 con 0x7f7dec0d0cb0 2023-11-28T12:02:44.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.905+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7dd00682f0 msgr2=0x7f7dd006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:44.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.905+0000 7f7df3d0c700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7dd00682f0 0x7f7dd006a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7dec147a70 tx=0x7f7de0009040 comp rx=0 tx=0).stop 2023-11-28T12:02:44.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.905+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3032322707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec0d0cb0 msgr2=0x7f7dec0b6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:44.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.905+0000 7f7df3d0c700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec0d0cb0 0x7f7dec0b6610 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7de8000c00 tx=0x7f7de8004750 comp rx=0 tx=0).stop 2023-11-28T12:02:44.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.905+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3032322707 shutdown_connections 2023-11-28T12:02:44.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.905+0000 7f7df3d0c700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7dd00682f0 0x7f7dd006a7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:44.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.905+0000 7f7df3d0c700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dec15c8f0 0x7f7dec0c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:44.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.905+0000 7f7df3d0c700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dec0ccef0 0x7f7dec14a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:44.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.905+0000 7f7df3d0c700 1 --2- 172.21.15.57:0/3032322707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec0d0cb0 0x7f7dec0b6610 secure :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7de8000c00 tx=0x7f7de8004750 comp rx=0 tx=0).stop 2023-11-28T12:02:44.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.905+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3032322707 >> 172.21.15.57:0/3032322707 conn(0x7f7dec06ab90 msgr2=0x7f7dec0ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:44.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.905+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3032322707 shutdown_connections 2023-11-28T12:02:44.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:44.905+0000 7f7df3d0c700 1 -- 172.21.15.57:0/3032322707 wait complete. 2023-11-28T12:02:44.907 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' removed 2023-11-28T12:02:44.923 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T12:02:44.924 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T12:02:44.924 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T12:02:44.924 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:02:44.924 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T12:02:44.924 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T12:02:44.924 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T12:02:44.924 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:02:44.925 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T12:02:44.925 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T12:02:44.925 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T12:02:44.925 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:02:44.925 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T12:02:44.926 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T12:02:44.926 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T12:02:44.926 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:02:44.935 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T12:02:44.935 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T12:02:44.936 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T12:02:44.936 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T12:02:44.937 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T12:02:44.937 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T12:02:44.938 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T12:02:44.938 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T12:02:44.978 INFO:tasks.ceph.mds.a:Started 2023-11-28T12:02:44.978 DEBUG:teuthology.parallel:result is None 2023-11-28T12:02:44.979 INFO:tasks.ceph.mds.c:Started 2023-11-28T12:02:44.980 DEBUG:teuthology.parallel:result is None 2023-11-28T12:02:44.980 INFO:tasks.ceph.mds.b:Started 2023-11-28T12:02:44.980 DEBUG:teuthology.parallel:result is None 2023-11-28T12:02:44.981 INFO:tasks.ceph.mds.d:Started 2023-11-28T12:02:44.981 DEBUG:teuthology.parallel:result is None 2023-11-28T12:02:44.982 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T12:02:45.067 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-11-28T12:02:45.068 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-11-28T12:02:45.072 INFO:tasks.ceph.mds.c.smithi057.stdout:starting mds.c at 2023-11-28T12:02:45.073 INFO:tasks.ceph.mds.a.smithi057.stdout:starting mds.a at 2023-11-28T12:02:45.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.131+0000 7f8a97cfa700 1 Processor -- start 2023-11-28T12:02:45.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.132+0000 7f8a97cfa700 1 -- start start 2023-11-28T12:02:45.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.133+0000 7f8a97cfa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8a901651e0 0x7f8a900b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:45.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.133+0000 7f8a97cfa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a900b2d10 0x7f8a900cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:45.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.133+0000 7f8a97cfa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a900b3130 con 0x7f8a90164e10 2023-11-28T12:02:45.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.133+0000 7f8a97cfa700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9006c3c0 con 0x7f8a901651e0 2023-11-28T12:02:45.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.133+0000 7f8a97cfa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900cfea0 con 0x7f8a900b2d10 2023-11-28T12:02:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.134+0000 7f8a96297700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a900b2d10 0x7f8a900cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.134+0000 7f8a95a96700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8a90164e10 0x7f8a90161b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:58450/0 (socket says 172.21.15.57:58450) 2023-11-28T12:02:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.134+0000 7f8a95a96700 1 -- 172.21.15.57:0/3356529063 learned_addr learned my addr 172.21.15.57:0/3356529063 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.134+0000 7f8a94a94700 1 -- 172.21.15.57:0/3356529063 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022267841 0 0) 0x7f8a900b3130 con 0x7f8a90164e10 2023-11-28T12:02:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.134+0000 7f8a94a94700 1 -- 172.21.15.57:0/3356529063 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a7c0035b0 con 0x7f8a90164e10 2023-11-28T12:02:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.134+0000 7f8a96297700 1 -- 172.21.15.57:0/3356529063 >> v2:172.21.15.181:3300/0 conn(0x7f8a901651e0 msgr2=0x7f8a900b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.134+0000 7f8a96297700 1 --2- 172.21.15.57:0/3356529063 >> v2:172.21.15.181:3300/0 conn(0x7f8a901651e0 0x7f8a900b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.134+0000 7f8a96297700 1 -- 172.21.15.57:0/3356529063 >> v1:172.21.15.181:6789/0 conn(0x7f8a90164e10 legacy=0x7f8a90161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.134+0000 7f8a96297700 1 -- 172.21.15.57:0/3356529063 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a90165680 con 0x7f8a900b2d10 2023-11-28T12:02:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.134+0000 7f8a96297700 1 --2- 172.21.15.57:0/3356529063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a900b2d10 0x7f8a900cec30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009840 tx=0x7f8a80009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16b513d539b0985 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:45.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.134+0000 7f8a94a94700 1 -- 172.21.15.57:0/3356529063 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80004230 con 0x7f8a900b2d10 2023-11-28T12:02:45.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.134+0000 7f8a94a94700 1 -- 172.21.15.57:0/3356529063 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a80004390 con 0x7f8a900b2d10 2023-11-28T12:02:45.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.134+0000 7f8a94a94700 1 -- 172.21.15.57:0/3356529063 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a800036a0 con 0x7f8a900b2d10 2023-11-28T12:02:45.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.135+0000 7f8a97cfa700 1 -- 172.21.15.57:0/3356529063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a900b2d10 msgr2=0x7f8a900cec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:45.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.135+0000 7f8a97cfa700 1 --2- 172.21.15.57:0/3356529063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a900b2d10 0x7f8a900cec30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009840 tx=0x7f8a80009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:45.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.136+0000 7f8a97cfa700 1 -- 172.21.15.57:0/3356529063 shutdown_connections 2023-11-28T12:02:45.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.136+0000 7f8a97cfa700 1 --2- 172.21.15.57:0/3356529063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a900b2d10 0x7f8a900cec30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:45.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.136+0000 7f8a97cfa700 1 --2- 172.21.15.57:0/3356529063 >> v2:172.21.15.181:3300/0 conn(0x7f8a901651e0 0x7f8a900b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:45.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.136+0000 7f8a97cfa700 1 -- 172.21.15.57:0/3356529063 >> 172.21.15.57:0/3356529063 conn(0x7f8a9006ea80 msgr2=0x7f8a9006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:45.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.136+0000 7f8a97cfa700 1 -- 172.21.15.57:0/3356529063 shutdown_connections 2023-11-28T12:02:45.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.136+0000 7f8a97cfa700 1 -- 172.21.15.57:0/3356529063 wait complete. 2023-11-28T12:02:45.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.137+0000 7f8a97cfa700 1 Processor -- start 2023-11-28T12:02:45.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.137+0000 7f8a97cfa700 1 -- start start 2023-11-28T12:02:45.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.138+0000 7f8a97cfa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a900b2d10 0x7f8a900c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:45.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.138+0000 7f8a97cfa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a90164e10 0x7f8a90155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:45.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.138+0000 7f8a97cfa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a901651e0 0x7f8a900c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:45.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.138+0000 7f8a97cfa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900c5f30 con 0x7f8a901651e0 2023-11-28T12:02:45.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.138+0000 7f8a97cfa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900c52b0 con 0x7f8a900b2d10 2023-11-28T12:02:45.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.138+0000 7f8a97cfa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900c5590 con 0x7f8a90164e10 2023-11-28T12:02:45.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.139+0000 7f8a95295700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a90164e10 0x7f8a90155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:45.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.139+0000 7f8a95295700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a90164e10 0x7f8a90155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:46582/0 (socket says 172.21.15.57:46582) 2023-11-28T12:02:45.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.139+0000 7f8a95295700 1 -- 172.21.15.57:0/1752069045 learned_addr learned my addr 172.21.15.57:0/1752069045 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:45.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.139+0000 7f8a96297700 1 --2- 172.21.15.57:0/1752069045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a901651e0 0x7f8a900c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:45.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.139+0000 7f8a95295700 1 -- 172.21.15.57:0/1752069045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a900b2d10 msgr2=0x7f8a900c6350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T12:02:45.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.139+0000 7f8a95295700 1 --2- 172.21.15.57:0/1752069045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a900b2d10 0x7f8a900c6350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:45.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.139+0000 7f8a95295700 1 -- 172.21.15.57:0/1752069045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a901651e0 msgr2=0x7f8a900c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:45.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.139+0000 7f8a95295700 1 --2- 172.21.15.57:0/1752069045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a901651e0 0x7f8a900c4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:45.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.139+0000 7f8a95295700 1 -- 172.21.15.57:0/1752069045 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a800094a0 con 0x7f8a90164e10 2023-11-28T12:02:45.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.139+0000 7f8a95295700 1 --2- 172.21.15.57:0/1752069045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a90164e10 0x7f8a90155280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00b530 tx=0x7f8a8c00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:45.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.139+0000 7f8a867fc700 1 -- 172.21.15.57:0/1752069045 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c011830 con 0x7f8a90164e10 2023-11-28T12:02:45.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.139+0000 7f8a867fc700 1 -- 172.21.15.57:0/1752069045 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c011990 con 0x7f8a90164e10 2023-11-28T12:02:45.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.139+0000 7f8a867fc700 1 -- 172.21.15.57:0/1752069045 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c011d20 con 0x7f8a90164e10 2023-11-28T12:02:45.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.140+0000 7f8a97cfa700 1 -- 172.21.15.57:0/1752069045 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9006c940 con 0x7f8a90164e10 2023-11-28T12:02:45.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.140+0000 7f8a97cfa700 1 -- 172.21.15.57:0/1752069045 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9006c660 con 0x7f8a90164e10 2023-11-28T12:02:45.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.141+0000 7f8a97cfa700 1 -- 172.21.15.57:0/1752069045 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9006cc20 con 0x7f8a90164e10 2023-11-28T12:02:45.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.141+0000 7f8a867fc700 1 -- 172.21.15.57:0/1752069045 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8a8c021070 con 0x7f8a90164e10 2023-11-28T12:02:45.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.142+0000 7f8a867fc700 1 --2- 172.21.15.57:0/1752069045 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8a740682f0 0x7f8a7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:45.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.142+0000 7f8a867fc700 1 -- 172.21.15.57:0/1752069045 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8a8c085970 con 0x7f8a90164e10 2023-11-28T12:02:45.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.142+0000 7f8a95a96700 1 --2- 172.21.15.57:0/1752069045 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8a740682f0 0x7f8a7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:45.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.142+0000 7f8a95a96700 1 --2- 172.21.15.57:0/1752069045 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8a740682f0 0x7f8a7406a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8a88002080 tx=0x7f8a88001620 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:45.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.146+0000 7f8a867fc700 1 -- 172.21.15.57:0/1752069045 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8a8c052400 con 0x7f8a90164e10 2023-11-28T12:02:45.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:45.365+0000 7f8a97cfa700 1 -- 172.21.15.57:0/1752069045 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8a900c6070 con 0x7f8a90164e10 2023-11-28T12:02:46.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.123+0000 7f8a867fc700 1 -- 172.21.15.57:0/1752069045 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8a900c6070 con 0x7f8a90164e10 2023-11-28T12:02:46.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.126+0000 7f8a97cfa700 1 -- 172.21.15.57:0/1752069045 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8a740682f0 msgr2=0x7f8a7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:46.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.126+0000 7f8a97cfa700 1 --2- 172.21.15.57:0/1752069045 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8a740682f0 0x7f8a7406a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8a88002080 tx=0x7f8a88001620 comp rx=0 tx=0).stop 2023-11-28T12:02:46.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.126+0000 7f8a97cfa700 1 -- 172.21.15.57:0/1752069045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a90164e10 msgr2=0x7f8a90155280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:46.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.126+0000 7f8a97cfa700 1 --2- 172.21.15.57:0/1752069045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a90164e10 0x7f8a90155280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00b530 tx=0x7f8a8c00b840 comp rx=0 tx=0).stop 2023-11-28T12:02:46.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.126+0000 7f8a97cfa700 1 -- 172.21.15.57:0/1752069045 shutdown_connections 2023-11-28T12:02:46.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.126+0000 7f8a97cfa700 1 --2- 172.21.15.57:0/1752069045 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8a740682f0 0x7f8a7406a7a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.126+0000 7f8a97cfa700 1 --2- 172.21.15.57:0/1752069045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a901651e0 0x7f8a900c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.126+0000 7f8a97cfa700 1 --2- 172.21.15.57:0/1752069045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a900b2d10 0x7f8a900c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.126+0000 7f8a97cfa700 1 --2- 172.21.15.57:0/1752069045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a90164e10 0x7f8a90155280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.126+0000 7f8a97cfa700 1 -- 172.21.15.57:0/1752069045 >> 172.21.15.57:0/1752069045 conn(0x7f8a9006ea80 msgr2=0x7f8a90162c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:46.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.126+0000 7f8a97cfa700 1 -- 172.21.15.57:0/1752069045 shutdown_connections 2023-11-28T12:02:46.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.126+0000 7f8a97cfa700 1 -- 172.21.15.57:0/1752069045 wait complete. 2023-11-28T12:02:46.128 INFO:teuthology.orchestra.run.smithi057.stderr: removed all blocklist entries 2023-11-28T12:02:46.144 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T12:02:46.249 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:02:46.252 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:02:46.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.286+0000 7f81b854e700 1 Processor -- start 2023-11-28T12:02:46.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.286+0000 7f81b854e700 1 -- start start 2023-11-28T12:02:46.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.287+0000 7f81b854e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81b00ce040 0x7f81b015d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:46.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.287+0000 7f81b854e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f81b0142db0 0x7f81b013ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:46.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.287+0000 7f81b854e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b013fc90 con 0x7f81b00b1770 2023-11-28T12:02:46.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.287+0000 7f81b854e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b013f6d0 con 0x7f81b0142db0 2023-11-28T12:02:46.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.287+0000 7f81b854e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b013f9b0 con 0x7f81b00ce040 2023-11-28T12:02:46.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.288+0000 7f81b6aeb700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f81b00b1770 0x7f81b013cd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39522/0 (socket says 172.21.15.57:39522) 2023-11-28T12:02:46.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.288+0000 7f81b6aeb700 1 -- 172.21.15.57:0/2351071346 learned_addr learned my addr 172.21.15.57:0/2351071346 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:46.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.288+0000 7f81b62ea700 1 --2- 172.21.15.57:0/2351071346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81b00ce040 0x7f81b015d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:46.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.288+0000 7f81b5ae9700 1 --2- 172.21.15.57:0/2351071346 >> v2:172.21.15.181:3300/0 conn(0x7f81b0142db0 0x7f81b013ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:46.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.288+0000 7f81b52e8700 1 -- 172.21.15.57:0/2351071346 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150329713 0 0) 0x7f81b013fc90 con 0x7f81b00b1770 2023-11-28T12:02:46.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.288+0000 7f81b52e8700 1 -- 172.21.15.57:0/2351071346 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f819c0035b0 con 0x7f81b00b1770 2023-11-28T12:02:46.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.288+0000 7f81b62ea700 1 -- 172.21.15.57:0/2351071346 >> v2:172.21.15.181:3300/0 conn(0x7f81b0142db0 msgr2=0x7f81b013ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:46.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.288+0000 7f81b52e8700 1 -- 172.21.15.57:0/2351071346 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 675583491 0 0) 0x7f819c0035b0 con 0x7f81b00b1770 2023-11-28T12:02:46.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.288+0000 7f81b62ea700 1 --2- 172.21.15.57:0/2351071346 >> v2:172.21.15.181:3300/0 conn(0x7f81b0142db0 0x7f81b013ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.288+0000 7f81b62ea700 1 -- 172.21.15.57:0/2351071346 >> v1:172.21.15.57:6789/0 conn(0x7f81b00b1770 legacy=0x7f81b013cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:46.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.288+0000 7f81b62ea700 1 -- 172.21.15.57:0/2351071346 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b015d100 con 0x7f81b00ce040 2023-11-28T12:02:46.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.288+0000 7f81b5ae9700 1 --2- 172.21.15.57:0/2351071346 >> v2:172.21.15.181:3300/0 conn(0x7f81b0142db0 0x7f81b013ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:46.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.288+0000 7f81b62ea700 1 --2- 172.21.15.57:0/2351071346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81b00ce040 0x7f81b015d900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009d90 tx=0x7f81ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf91c38677332a12 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:46.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.289+0000 7f81b52e8700 1 -- 172.21.15.57:0/2351071346 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac00e070 con 0x7f81b00ce040 2023-11-28T12:02:46.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.289+0000 7f81b52e8700 1 -- 172.21.15.57:0/2351071346 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac003a20 con 0x7f81b00ce040 2023-11-28T12:02:46.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.289+0000 7f81b52e8700 1 -- 172.21.15.57:0/2351071346 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8001670 con 0x7f81b00ce040 2023-11-28T12:02:46.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.289+0000 7f81b854e700 1 -- 172.21.15.57:0/2351071346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81b00ce040 msgr2=0x7f81b015d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:46.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.289+0000 7f81b854e700 1 --2- 172.21.15.57:0/2351071346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81b00ce040 0x7f81b015d900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009d90 tx=0x7f81ac00b330 comp rx=0 tx=0).stop 2023-11-28T12:02:46.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.289+0000 7f81b854e700 1 -- 172.21.15.57:0/2351071346 shutdown_connections 2023-11-28T12:02:46.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.289+0000 7f81b854e700 1 --2- 172.21.15.57:0/2351071346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81b00ce040 0x7f81b015d900 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.289+0000 7f81b854e700 1 --2- 172.21.15.57:0/2351071346 >> v2:172.21.15.181:3300/0 conn(0x7f81b0142db0 0x7f81b013ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.289+0000 7f81b854e700 1 -- 172.21.15.57:0/2351071346 >> 172.21.15.57:0/2351071346 conn(0x7f81b006c010 msgr2=0x7f81b00cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:46.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.289+0000 7f81b854e700 1 -- 172.21.15.57:0/2351071346 shutdown_connections 2023-11-28T12:02:46.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.289+0000 7f81b854e700 1 -- 172.21.15.57:0/2351071346 wait complete. 2023-11-28T12:02:46.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.291+0000 7f81b854e700 1 Processor -- start 2023-11-28T12:02:46.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.291+0000 7f81b854e700 1 -- start start 2023-11-28T12:02:46.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.291+0000 7f81b854e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b00b1770 0x7f81b01f0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:46.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.291+0000 7f81b854e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81b00ce040 0x7f81b00b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:46.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.291+0000 7f81b854e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81b0142db0 0x7f81b0159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:46.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.291+0000 7f81b854e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b015a470 con 0x7f81b00b1770 2023-11-28T12:02:46.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.291+0000 7f81b854e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b015a750 con 0x7f81b00ce040 2023-11-28T12:02:46.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.291+0000 7f81b854e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b015aa30 con 0x7f81b0142db0 2023-11-28T12:02:46.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.291+0000 7f81b62ea700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b00b1770 0x7f81b01f0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:46.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.291+0000 7f81b62ea700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b00b1770 0x7f81b01f0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44924/0 (socket says 172.21.15.57:44924) 2023-11-28T12:02:46.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81b62ea700 1 -- 172.21.15.57:0/1306965409 learned_addr learned my addr 172.21.15.57:0/1306965409 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:46.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81b6aeb700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81b0142db0 0x7f81b0159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:46.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81b5ae9700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81b00ce040 0x7f81b00b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:46.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81b62ea700 1 -- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81b0142db0 msgr2=0x7f81b0159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:46.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81b62ea700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81b0142db0 0x7f81b0159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81b62ea700 1 -- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81b00ce040 msgr2=0x7f81b00b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:46.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81b62ea700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81b00ce040 0x7f81b00b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81b62ea700 1 -- 172.21.15.57:0/1306965409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a40094a0 con 0x7f81b00b1770 2023-11-28T12:02:46.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81b62ea700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b00b1770 0x7f81b01f0a40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f81b013faf0 tx=0x7f81ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:46.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81a2ffd700 1 -- 172.21.15.57:0/1306965409 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0044c0 con 0x7f81b00b1770 2023-11-28T12:02:46.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81a2ffd700 1 -- 172.21.15.57:0/1306965409 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac004660 con 0x7f81b00b1770 2023-11-28T12:02:46.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81b854e700 1 -- 172.21.15.57:0/1306965409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ac009a70 con 0x7f81b00b1770 2023-11-28T12:02:46.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81a2ffd700 1 -- 172.21.15.57:0/1306965409 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac012430 con 0x7f81b00b1770 2023-11-28T12:02:46.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.292+0000 7f81b854e700 1 -- 172.21.15.57:0/1306965409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b00ce410 con 0x7f81b00b1770 2023-11-28T12:02:46.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.293+0000 7f81b854e700 1 -- 172.21.15.57:0/1306965409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b015d3e0 con 0x7f81b00b1770 2023-11-28T12:02:46.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.294+0000 7f81a2ffd700 1 -- 172.21.15.57:0/1306965409 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f81ac02c070 con 0x7f81b00b1770 2023-11-28T12:02:46.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.294+0000 7f81a2ffd700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f81940682f0 0x7f819406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:46.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.294+0000 7f81a2ffd700 1 -- 172.21.15.57:0/1306965409 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f81ac0873a0 con 0x7f81b00b1770 2023-11-28T12:02:46.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.294+0000 7f81b5ae9700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f81940682f0 0x7f819406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:46.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.295+0000 7f81b5ae9700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f81940682f0 0x7f819406a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f81b006d020 tx=0x7f81a4009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:46.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.299+0000 7f81a2ffd700 1 -- 172.21.15.57:0/1306965409 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f81ac053f50 con 0x7f81b00b1770 2023-11-28T12:02:46.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.581+0000 7f81b854e700 1 -- 172.21.15.57:0/1306965409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f81b00ce5a0 con 0x7f81b00b1770 2023-11-28T12:02:46.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.582+0000 7f81a2ffd700 1 -- 172.21.15.57:0/1306965409 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f81ac057570 con 0x7f81b00b1770 2023-11-28T12:02:46.583 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:46.583 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:02:46.583 INFO:teuthology.orchestra.run.smithi057.stdout: "auth_dump": [ 2023-11-28T12:02:46.583 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.584 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.a", 2023-11-28T12:02:46.584 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVlFqRBJBAAxYPlgmjdxnRzi0h9JBB1BQ==", 2023-11-28T12:02:46.584 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.584 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:02:46.584 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:02:46.584 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:02:46.584 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.584 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.584 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.584 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.584 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.b", 2023-11-28T12:02:46.584 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVl0d0ZMhAAykXowuVhIqRSjBr/XMvanw==", 2023-11-28T12:02:46.585 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.585 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:02:46.585 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:02:46.585 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:02:46.585 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.585 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.585 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.585 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.585 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.c", 2023-11-28T12:02:46.585 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVl4Z2qLBAACYGN7Glu6Bv2yLio50flgQ==", 2023-11-28T12:02:46.585 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.585 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:02:46.586 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:02:46.586 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:02:46.586 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.586 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.586 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.586 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.586 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.d", 2023-11-28T12:02:46.586 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVl2P5jORAAR7iZXovJiiCXuxlaaZJ75Q==", 2023-11-28T12:02:46.586 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.586 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:02:46.586 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:02:46.586 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:02:46.586 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.587 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.587 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.587 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.587 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.0", 2023-11-28T12:02:46.587 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQB31mVlUtyoIRAAVGG65BAriZ4e4QHUYCJXdg==", 2023-11-28T12:02:46.587 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.587 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:02:46.587 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:02:46.587 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.587 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.587 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.587 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.588 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.1", 2023-11-28T12:02:46.588 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQB61mVlNA98IRAArcXcx4glfEvQ1gxAXQYxHA==", 2023-11-28T12:02:46.588 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.588 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:02:46.588 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:02:46.588 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.588 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.588 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.588 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.588 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.2", 2023-11-28T12:02:46.588 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQB91mVlzthPIhAAFgsWUTI/2kmVCVMgcZrIdA==", 2023-11-28T12:02:46.588 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.589 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:02:46.589 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:02:46.589 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.589 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.589 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.589 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.589 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.3", 2023-11-28T12:02:46.589 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCA1mVlZPuzJBAAyAAUBUEitzTq7fPdxj5gzQ==", 2023-11-28T12:02:46.589 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.589 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:02:46.589 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:02:46.589 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.589 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.590 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.590 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.590 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.4", 2023-11-28T12:02:46.590 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCG1mVlWibrKhAAzuTgZ4ayPw3imRALW/Y4FQ==", 2023-11-28T12:02:46.590 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.590 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:02:46.590 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:02:46.590 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.590 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.590 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.590 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.590 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.5", 2023-11-28T12:02:46.591 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCJ1mVlnX7bKBAAYYUYhDSDTVKkrzVKw78+Ug==", 2023-11-28T12:02:46.591 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.591 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:02:46.591 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:02:46.591 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.591 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.591 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.591 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.591 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.6", 2023-11-28T12:02:46.591 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCM1mVlE7aUJxAAXM3axU5A9IEXt2/pNcvJ9Q==", 2023-11-28T12:02:46.591 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.591 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:02:46.591 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:02:46.592 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.592 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.592 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.592 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.592 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.7", 2023-11-28T12:02:46.592 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCP1mVlSApqJhAA93hHHLl+o8cKTgdCaG+0Bg==", 2023-11-28T12:02:46.592 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.592 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:02:46.592 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:02:46.592 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.592 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.592 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.592 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.593 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.0", 2023-11-28T12:02:46.593 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBx1mVlmhvFAhAAnKh5ldWFcp+P85m8HR2vug==", 2023-11-28T12:02:46.593 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.593 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:02:46.593 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-11-28T12:02:46.593 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-11-28T12:02:46.593 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-11-28T12:02:46.593 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.593 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.593 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.593 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.1", 2023-11-28T12:02:46.593 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBx1mVlMEgaChAAGCjKh3RcPIVp2UWKGqmqRA==", 2023-11-28T12:02:46.594 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.594 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:02:46.594 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-11-28T12:02:46.594 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-11-28T12:02:46.594 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-11-28T12:02:46.594 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.594 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.594 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.594 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.2", 2023-11-28T12:02:46.594 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBx1mVlqGCwBhAA6ZJBICxFbxIjYlgA80ekuA==", 2023-11-28T12:02:46.594 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.594 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:02:46.594 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-11-28T12:02:46.595 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-11-28T12:02:46.595 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-11-28T12:02:46.595 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.595 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.595 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.595 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.3", 2023-11-28T12:02:46.595 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBx1mVldQXGDxAAkY9/3er6vMwer5oWETJvfg==", 2023-11-28T12:02:46.595 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.595 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:02:46.595 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-11-28T12:02:46.595 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-11-28T12:02:46.595 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-11-28T12:02:46.596 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.596 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.596 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.596 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.admin", 2023-11-28T12:02:46.596 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVlzqiCAxAAIOe5RWQ+gxWKaK64A5jhlA==", 2023-11-28T12:02:46.596 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.596 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-11-28T12:02:46.596 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:02:46.596 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:02:46.596 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.596 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.596 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.596 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.597 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.bootstrap-mds", 2023-11-28T12:02:46.597 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCV1mVlQSMBLRAAAvnU3UaVPvShp923u0BrfQ==", 2023-11-28T12:02:46.597 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.597 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile bootstrap-mds" 2023-11-28T12:02:46.597 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.597 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.597 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.597 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.bootstrap-mgr", 2023-11-28T12:02:46.597 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCV1mVlvS8BLRAAaRpQCTfc95/KCN4Hl6dJDg==", 2023-11-28T12:02:46.597 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.597 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-28T12:02:46.597 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.597 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.598 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.598 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.bootstrap-osd", 2023-11-28T12:02:46.598 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCV1mVlCjsBLRAALuVhODS2c2FlUW+O61YD+g==", 2023-11-28T12:02:46.598 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.598 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile bootstrap-osd" 2023-11-28T12:02:46.598 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.598 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.598 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.598 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.bootstrap-rbd", 2023-11-28T12:02:46.598 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCV1mVlDkgBLRAAwCxRtoZee/veP5hmbsHMXw==", 2023-11-28T12:02:46.598 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.598 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-28T12:02:46.599 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.599 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.599 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.599 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-28T12:02:46.599 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCV1mVlRFIBLRAA8bKhVmwRMeIXaVuQsM7V8Q==", 2023-11-28T12:02:46.599 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.599 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-28T12:02:46.599 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.599 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.599 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.599 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.bootstrap-rgw", 2023-11-28T12:02:46.599 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCV1mVlgFsBLRAAfvyAYbNrC9ep3UrlVUHd9A==", 2023-11-28T12:02:46.600 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.600 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-28T12:02:46.600 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.600 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.600 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.600 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mgr.x", 2023-11-28T12:02:46.600 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVl+yMTIBAAUgQtxbubBwh+1DLDFEaSKQ==", 2023-11-28T12:02:46.600 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.600 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-11-28T12:02:46.601 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile mgr", 2023-11-28T12:02:46.601 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.601 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.601 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:46.601 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:46.601 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mgr.y", 2023-11-28T12:02:46.601 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVlEPXqGxAAxVwmP5CKh4D1Hv+kYm7fxg==", 2023-11-28T12:02:46.601 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:02:46.601 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-11-28T12:02:46.601 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile mgr", 2023-11-28T12:02:46.601 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:02:46.601 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.601 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:46.602 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-11-28T12:02:46.602 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:02:46.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.584+0000 7f81b854e700 1 -- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f81940682f0 msgr2=0x7f819406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:46.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.584+0000 7f81b854e700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f81940682f0 0x7f819406a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f81b006d020 tx=0x7f81a4009210 comp rx=0 tx=0).stop 2023-11-28T12:02:46.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.584+0000 7f81b854e700 1 -- 172.21.15.57:0/1306965409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b00b1770 msgr2=0x7f81b01f0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:46.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.584+0000 7f81b854e700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b00b1770 0x7f81b01f0a40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f81b013faf0 tx=0x7f81ac004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.584+0000 7f81b854e700 1 -- 172.21.15.57:0/1306965409 shutdown_connections 2023-11-28T12:02:46.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.584+0000 7f81b854e700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f81940682f0 0x7f819406a7a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.584+0000 7f81b854e700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81b0142db0 0x7f81b0159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.584+0000 7f81b854e700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81b00b1770 0x7f81b01f0a40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.584+0000 7f81b854e700 1 --2- 172.21.15.57:0/1306965409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81b00ce040 0x7f81b00b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.584+0000 7f81b854e700 1 -- 172.21.15.57:0/1306965409 >> 172.21.15.57:0/1306965409 conn(0x7f81b006c010 msgr2=0x7f81b0155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:46.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.584+0000 7f81b854e700 1 -- 172.21.15.57:0/1306965409 shutdown_connections 2023-11-28T12:02:46.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.585+0000 7f81b854e700 1 -- 172.21.15.57:0/1306965409 wait complete. 2023-11-28T12:02:46.604 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-28T12:02:46.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.782+0000 7fe898809700 1 Processor -- start 2023-11-28T12:02:46.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.782+0000 7fe898809700 1 -- start start 2023-11-28T12:02:46.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.783+0000 7fe898809700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8940b2ee0 0x7fe8940b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.783+0000 7fe898809700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe894161370 con 0x7fe89415fe50 2023-11-28T12:02:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.783+0000 7fe898809700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8940cf700 con 0x7fe89406d170 2023-11-28T12:02:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.783+0000 7fe898809700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89406d5c0 con 0x7fe8940b2ee0 2023-11-28T12:02:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.784+0000 7fe89259c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe89415fe50 0x7fe894160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39524/0 (socket says 172.21.15.57:39524) 2023-11-28T12:02:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.784+0000 7fe89259c700 1 -- 172.21.15.57:0/568738830 learned_addr learned my addr 172.21.15.57:0/568738830 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.784+0000 7fe892d9d700 1 --2- 172.21.15.57:0/568738830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8940b2ee0 0x7fe8940b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.784+0000 7fe89159a700 1 -- 172.21.15.57:0/568738830 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954012433 0 0) 0x7fe894161370 con 0x7fe89415fe50 2023-11-28T12:02:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.784+0000 7fe89159a700 1 -- 172.21.15.57:0/568738830 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8800035b0 con 0x7fe89415fe50 2023-11-28T12:02:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.784+0000 7fe89159a700 1 -- 172.21.15.57:0/568738830 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149594196 0 0) 0x7fe8940cf700 con 0x7fe89406d170 2023-11-28T12:02:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.784+0000 7fe89159a700 1 -- 172.21.15.57:0/568738830 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe894161370 con 0x7fe89406d170 2023-11-28T12:02:46.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.784+0000 7fe892d9d700 1 -- 172.21.15.57:0/568738830 >> v1:172.21.15.181:6789/0 conn(0x7fe89406d170 legacy=0x7fe8940b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:46.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.784+0000 7fe892d9d700 1 -- 172.21.15.57:0/568738830 >> v1:172.21.15.57:6789/0 conn(0x7fe89415fe50 legacy=0x7fe894160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:46.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.784+0000 7fe892d9d700 1 -- 172.21.15.57:0/568738830 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8940cedd0 con 0x7fe8940b2ee0 2023-11-28T12:02:46.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.784+0000 7fe892d9d700 1 --2- 172.21.15.57:0/568738830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8940b2ee0 0x7fe8940b3b80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009dc0 tx=0x7fe88c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8a4bca6cc2b19ed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:46.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.785+0000 7fe89159a700 1 -- 172.21.15.57:0/568738830 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c00e070 con 0x7fe8940b2ee0 2023-11-28T12:02:46.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.785+0000 7fe89159a700 1 -- 172.21.15.57:0/568738830 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c00bb20 con 0x7fe8940b2ee0 2023-11-28T12:02:46.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.785+0000 7fe89159a700 1 -- 172.21.15.57:0/568738830 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c012430 con 0x7fe8940b2ee0 2023-11-28T12:02:46.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.785+0000 7fe898809700 1 -- 172.21.15.57:0/568738830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8940b2ee0 msgr2=0x7fe8940b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:46.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.785+0000 7fe898809700 1 --2- 172.21.15.57:0/568738830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8940b2ee0 0x7fe8940b3b80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009dc0 tx=0x7fe88c00b330 comp rx=0 tx=0).stop 2023-11-28T12:02:46.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.785+0000 7fe898809700 1 -- 172.21.15.57:0/568738830 shutdown_connections 2023-11-28T12:02:46.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.785+0000 7fe898809700 1 --2- 172.21.15.57:0/568738830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8940b2ee0 0x7fe8940b3b80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.785+0000 7fe898809700 1 -- 172.21.15.57:0/568738830 >> 172.21.15.57:0/568738830 conn(0x7fe89406ea80 msgr2=0x7fe89406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:46.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.785+0000 7fe898809700 1 -- 172.21.15.57:0/568738830 shutdown_connections 2023-11-28T12:02:46.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.785+0000 7fe898809700 1 -- 172.21.15.57:0/568738830 wait complete. 2023-11-28T12:02:46.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.787+0000 7fe898809700 1 Processor -- start 2023-11-28T12:02:46.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.787+0000 7fe898809700 1 -- start start 2023-11-28T12:02:46.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe898809700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe89406d170 0x7fe8940ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:46.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe898809700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8940b2ee0 0x7fe8940ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:46.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe898809700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89415fe50 0x7fe89415c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:46.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe898809700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe894157ae0 con 0x7fe89415fe50 2023-11-28T12:02:46.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe898809700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8940cc200 con 0x7fe89406d170 2023-11-28T12:02:46.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe898809700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8940c96c0 con 0x7fe8940b2ee0 2023-11-28T12:02:46.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe892d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89415fe50 0x7fe89415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:46.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe892d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89415fe50 0x7fe89415c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44930/0 (socket says 172.21.15.57:44930) 2023-11-28T12:02:46.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe892d9d700 1 -- 172.21.15.57:0/798962410 learned_addr learned my addr 172.21.15.57:0/798962410 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:46.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe891d9b700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8940b2ee0 0x7fe8940ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:46.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe89259c700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe89406d170 0x7fe8940ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:46.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe892d9d700 1 -- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8940b2ee0 msgr2=0x7fe8940ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:46.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe892d9d700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8940b2ee0 0x7fe8940ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe892d9d700 1 -- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe89406d170 msgr2=0x7fe8940ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:46.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe892d9d700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe89406d170 0x7fe8940ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:46.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe892d9d700 1 -- 172.21.15.57:0/798962410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c009a70 con 0x7fe89415fe50 2023-11-28T12:02:46.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe892d9d700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89415fe50 0x7fe89415c240 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00b370 tx=0x7fe88c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:46.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe88affd700 1 -- 172.21.15.57:0/798962410 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c01b810 con 0x7fe89415fe50 2023-11-28T12:02:46.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe88affd700 1 -- 172.21.15.57:0/798962410 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c020070 con 0x7fe89415fe50 2023-11-28T12:02:46.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe898809700 1 -- 172.21.15.57:0/798962410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8940cf390 con 0x7fe89415fe50 2023-11-28T12:02:46.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.788+0000 7fe88affd700 1 -- 172.21.15.57:0/798962410 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c01bbf0 con 0x7fe89415fe50 2023-11-28T12:02:46.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.789+0000 7fe898809700 1 -- 172.21.15.57:0/798962410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8940cf0b0 con 0x7fe89415fe50 2023-11-28T12:02:46.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.790+0000 7fe898809700 1 -- 172.21.15.57:0/798962410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8940cfb90 con 0x7fe89415fe50 2023-11-28T12:02:46.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.790+0000 7fe88affd700 1 -- 172.21.15.57:0/798962410 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe88c01bd50 con 0x7fe89415fe50 2023-11-28T12:02:46.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.790+0000 7fe88affd700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe8740682f0 0x7fe87406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:46.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.790+0000 7fe88affd700 1 -- 172.21.15.57:0/798962410 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe88c022070 con 0x7fe89415fe50 2023-11-28T12:02:46.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.791+0000 7fe89259c700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe8740682f0 0x7fe87406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:46.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.791+0000 7fe89259c700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe8740682f0 0x7fe87406a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe87c007fd0 tx=0x7fe87c007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:46.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:46.795+0000 7fe88affd700 1 -- 172.21.15.57:0/798962410 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe88c054a00 con 0x7fe89415fe50 2023-11-28T12:02:47.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.081+0000 7fe898809700 1 -- 172.21.15.57:0/798962410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fe8940743f0 con 0x7fe89415fe50 2023-11-28T12:02:47.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.084+0000 7fe88affd700 1 -- 172.21.15.57:0/798962410 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe88c02c0a0 con 0x7fe89415fe50 2023-11-28T12:02:47.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.087+0000 7fe898809700 1 -- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe8740682f0 msgr2=0x7fe87406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.087+0000 7fe898809700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe8740682f0 0x7fe87406a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe87c007fd0 tx=0x7fe87c007de0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.087+0000 7fe898809700 1 -- 172.21.15.57:0/798962410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89415fe50 msgr2=0x7fe89415c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.087+0000 7fe898809700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89415fe50 0x7fe89415c240 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00b370 tx=0x7fe88c004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.087+0000 7fe898809700 1 -- 172.21.15.57:0/798962410 shutdown_connections 2023-11-28T12:02:47.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.087+0000 7fe898809700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe8740682f0 0x7fe87406a7a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.087+0000 7fe898809700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89415fe50 0x7fe89415c240 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.087+0000 7fe898809700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe89406d170 0x7fe8940ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.087+0000 7fe898809700 1 --2- 172.21.15.57:0/798962410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8940b2ee0 0x7fe8940ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.087+0000 7fe898809700 1 -- 172.21.15.57:0/798962410 >> 172.21.15.57:0/798962410 conn(0x7fe89406ea80 msgr2=0x7fe89415dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:47.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.087+0000 7fe898809700 1 -- 172.21.15.57:0/798962410 shutdown_connections 2023-11-28T12:02:47.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.087+0000 7fe898809700 1 -- 172.21.15.57:0/798962410 wait complete. 2023-11-28T12:02:47.102 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-28T12:02:47.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.236+0000 7f4b17a9e700 1 Processor -- start 2023-11-28T12:02:47.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.237+0000 7f4b17a9e700 1 -- start start 2023-11-28T12:02:47.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b17a9e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b100b2ee0 0x7f4b100d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:47.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b17a9e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1015a9c0 con 0x7f4b1015a5f0 2023-11-28T12:02:47.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b17a9e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b100c7b80 con 0x7f4b100cc200 2023-11-28T12:02:47.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b17a9e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b100cc990 con 0x7f4b100b2ee0 2023-11-28T12:02:47.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b1583a700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4b1015a5f0 0x7f4b100b27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39530/0 (socket says 172.21.15.57:39530) 2023-11-28T12:02:47.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b1583a700 1 -- 172.21.15.57:0/812507478 learned_addr learned my addr 172.21.15.57:0/812507478 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:47.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b15039700 1 --2- 172.21.15.57:0/812507478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b100b2ee0 0x7f4b100d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:47.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561096494 0 0) 0x7f4b1015a9c0 con 0x7f4b1015a5f0 2023-11-28T12:02:47.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4afc0035b0 con 0x7f4b1015a5f0 2023-11-28T12:02:47.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1107906643 0 0) 0x7f4afc0035b0 con 0x7f4b1015a5f0 2023-11-28T12:02:47.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b100b2ee0 msgr2=0x7f4b100d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:47.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b14838700 1 --2- 172.21.15.57:0/812507478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b100b2ee0 0x7f4b100d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 >> v1:172.21.15.181:6789/0 conn(0x7f4b100cc200 legacy=0x7f4b100ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b100ccc70 con 0x7f4b1015a5f0 2023-11-28T12:02:47.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.238+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f4b0c001670 con 0x7f4b1015a5f0 2023-11-28T12:02:47.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 >> v1:172.21.15.57:6789/0 conn(0x7f4b1015a5f0 legacy=0x7f4b100b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b14838700 1 --2- 172.21.15.57:0/812507478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b100b2ee0 0x7f4afc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:47.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b14838700 1 --2- 172.21.15.57:0/812507478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc003f70 0x7f4afc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:47.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b14838700 1 --2- 172.21.15.57:0/812507478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4afc004d90 0x7f4afc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:47.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b100cc990 con 0x7f4afc004d90 2023-11-28T12:02:47.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc00ba40 con 0x7f4afc003f70 2023-11-28T12:02:47.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc00bc00 con 0x7f4b100b2ee0 2023-11-28T12:02:47.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b1603b700 1 --2- 172.21.15.57:0/812507478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4afc004d90 0x7f4afc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:47.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b1583a700 1 --2- 172.21.15.57:0/812507478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b100b2ee0 0x7f4afc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:47.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b15039700 1 --2- 172.21.15.57:0/812507478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc003f70 0x7f4afc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:47.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b1603b700 1 -- 172.21.15.57:0/812507478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b100b2ee0 msgr2=0x7f4afc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:47.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b1603b700 1 --2- 172.21.15.57:0/812507478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b100b2ee0 0x7f4afc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b1603b700 1 -- 172.21.15.57:0/812507478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc003f70 msgr2=0x7f4afc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:47.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b1603b700 1 --2- 172.21.15.57:0/812507478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc003f70 0x7f4afc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b1603b700 1 -- 172.21.15.57:0/812507478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc0035b0 con 0x7f4afc004d90 2023-11-28T12:02:47.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b1603b700 1 --2- 172.21.15.57:0/812507478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4afc004d90 0x7f4afc005200 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800c240 tx=0x7f4afc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5fdde7ee440ca1d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:47.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08001e70 con 0x7f4afc004d90 2023-11-28T12:02:47.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0800cde0 con 0x7f4afc004d90 2023-11-28T12:02:47.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.239+0000 7f4b14838700 1 -- 172.21.15.57:0/812507478 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08012660 con 0x7f4afc004d90 2023-11-28T12:02:47.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.240+0000 7f4b17a9e700 1 -- 172.21.15.57:0/812507478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4afc004d90 msgr2=0x7f4afc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.240+0000 7f4b17a9e700 1 --2- 172.21.15.57:0/812507478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4afc004d90 0x7f4afc005200 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800c240 tx=0x7f4afc005880 comp rx=0 tx=0).stop 2023-11-28T12:02:47.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.240+0000 7f4b1603b700 1 -- 172.21.15.57:0/812507478 reap_dead start 2023-11-28T12:02:47.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.240+0000 7f4b17a9e700 1 -- 172.21.15.57:0/812507478 shutdown_connections 2023-11-28T12:02:47.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.240+0000 7f4b17a9e700 1 -- 172.21.15.57:0/812507478 >> 172.21.15.57:0/812507478 conn(0x7f4b1006ea80 msgr2=0x7f4b1006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:47.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.240+0000 7f4b17a9e700 1 -- 172.21.15.57:0/812507478 shutdown_connections 2023-11-28T12:02:47.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.240+0000 7f4b17a9e700 1 -- 172.21.15.57:0/812507478 wait complete. 2023-11-28T12:02:47.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.241+0000 7f4b17a9e700 1 Processor -- start 2023-11-28T12:02:47.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.241+0000 7f4b17a9e700 1 -- start start 2023-11-28T12:02:47.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.242+0000 7f4b17a9e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc004d90 0x7f4b101f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:47.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.242+0000 7f4b17a9e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b100a8ef0 0x7f4b1006c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:47.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.242+0000 7f4b17a9e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b10158c30 0x7f4b101590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:47.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.242+0000 7f4b17a9e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b101595e0 con 0x7f4b100a8ef0 2023-11-28T12:02:47.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.242+0000 7f4b17a9e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b100a9360 con 0x7f4afc004d90 2023-11-28T12:02:47.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.242+0000 7f4b17a9e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b100a95f0 con 0x7f4b10158c30 2023-11-28T12:02:47.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.242+0000 7f4b15039700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b100a8ef0 0x7f4b1006c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:47.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.242+0000 7f4b15039700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b100a8ef0 0x7f4b1006c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44942/0 (socket says 172.21.15.57:44942) 2023-11-28T12:02:47.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.242+0000 7f4b15039700 1 -- 172.21.15.57:0/4127543593 learned_addr learned my addr 172.21.15.57:0/4127543593 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.242+0000 7f4b1603b700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b10158c30 0x7f4b101590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.242+0000 7f4b1583a700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc004d90 0x7f4b101f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.243+0000 7f4b15039700 1 -- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b10158c30 msgr2=0x7f4b101590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.243+0000 7f4b15039700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b10158c30 0x7f4b101590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.243+0000 7f4b15039700 1 -- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc004d90 msgr2=0x7f4b101f4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.243+0000 7f4b15039700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc004d90 0x7f4b101f4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.243+0000 7f4b15039700 1 -- 172.21.15.57:0/4127543593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b100cd230 con 0x7f4b100a8ef0 2023-11-28T12:02:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.243+0000 7f4b1583a700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc004d90 0x7f4b101f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.243+0000 7f4b15039700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b100a8ef0 0x7f4b1006c220 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4b000119c0 tx=0x7f4b00011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.243+0000 7f4b067fc700 1 -- 172.21.15.57:0/4127543593 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00004240 con 0x7f4b100a8ef0 2023-11-28T12:02:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.243+0000 7f4b067fc700 1 -- 172.21.15.57:0/4127543593 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b000043a0 con 0x7f4b100a8ef0 2023-11-28T12:02:47.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.243+0000 7f4b17a9e700 1 -- 172.21.15.57:0/4127543593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b100ccf50 con 0x7f4b100a8ef0 2023-11-28T12:02:47.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.243+0000 7f4b067fc700 1 -- 172.21.15.57:0/4127543593 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000f730 con 0x7f4b100a8ef0 2023-11-28T12:02:47.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.243+0000 7f4b17a9e700 1 -- 172.21.15.57:0/4127543593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b100cd510 con 0x7f4b100a8ef0 2023-11-28T12:02:47.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.244+0000 7f4b17a9e700 1 -- 172.21.15.57:0/4127543593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1006c760 con 0x7f4b100a8ef0 2023-11-28T12:02:47.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.245+0000 7f4b067fc700 1 -- 172.21.15.57:0/4127543593 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4b00025070 con 0x7f4b100a8ef0 2023-11-28T12:02:47.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.245+0000 7f4b067fc700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4af40682f0 0x7f4af406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:47.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.245+0000 7f4b067fc700 1 -- 172.21.15.57:0/4127543593 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4b00085d60 con 0x7f4b100a8ef0 2023-11-28T12:02:47.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.245+0000 7f4b1583a700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4af40682f0 0x7f4af406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:47.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.246+0000 7f4b1583a700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4af40682f0 0x7f4af406a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c001eb0 tx=0x7f4b0c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:47.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.250+0000 7f4b067fc700 1 -- 172.21.15.57:0/4127543593 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4b00052910 con 0x7f4b100a8ef0 2023-11-28T12:02:47.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.539+0000 7f4b17a9e700 1 -- 172.21.15.57:0/4127543593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f4b100743f0 con 0x7f4b100a8ef0 2023-11-28T12:02:47.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.547+0000 7f4b067fc700 1 -- 172.21.15.57:0/4127543593 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4b000190a0 con 0x7f4b100a8ef0 2023-11-28T12:02:47.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.550+0000 7f4b17a9e700 1 -- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4af40682f0 msgr2=0x7f4af406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.550+0000 7f4b17a9e700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4af40682f0 0x7f4af406a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c001eb0 tx=0x7f4b0c009040 comp rx=0 tx=0).stop 2023-11-28T12:02:47.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.550+0000 7f4b17a9e700 1 -- 172.21.15.57:0/4127543593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b100a8ef0 msgr2=0x7f4b1006c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.550+0000 7f4b17a9e700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b100a8ef0 0x7f4b1006c220 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4b000119c0 tx=0x7f4b00011d80 comp rx=0 tx=0).stop 2023-11-28T12:02:47.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.550+0000 7f4b17a9e700 1 -- 172.21.15.57:0/4127543593 shutdown_connections 2023-11-28T12:02:47.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.550+0000 7f4b17a9e700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4af40682f0 0x7f4af406a7a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.550+0000 7f4b17a9e700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b10158c30 0x7f4b101590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.550+0000 7f4b17a9e700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc004d90 0x7f4b101f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.550+0000 7f4b17a9e700 1 --2- 172.21.15.57:0/4127543593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b100a8ef0 0x7f4b1006c220 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.550+0000 7f4b17a9e700 1 -- 172.21.15.57:0/4127543593 >> 172.21.15.57:0/4127543593 conn(0x7f4b1006ea80 msgr2=0x7f4b101569d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:47.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.550+0000 7f4b17a9e700 1 -- 172.21.15.57:0/4127543593 shutdown_connections 2023-11-28T12:02:47.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.550+0000 7f4b17a9e700 1 -- 172.21.15.57:0/4127543593 wait complete. 2023-11-28T12:02:47.565 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-28T12:02:47.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.700+0000 7f5139324700 1 Processor -- start 2023-11-28T12:02:47.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.701+0000 7f5139324700 1 -- start start 2023-11-28T12:02:47.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.701+0000 7f5139324700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f513415fe50 0x7f513406d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:47.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.701+0000 7f5139324700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5134161370 con 0x7f51340b27d0 2023-11-28T12:02:47.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.701+0000 7f5139324700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51340cf700 con 0x7f51340b3b80 2023-11-28T12:02:47.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.701+0000 7f5139324700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5134160220 con 0x7f513415fe50 2023-11-28T12:02:47.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.702+0000 7f51337fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f51340b27d0 0x7f51340cfb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39532/0 (socket says 172.21.15.57:39532) 2023-11-28T12:02:47.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.702+0000 7f51337fe700 1 -- 172.21.15.57:0/184550938 learned_addr learned my addr 172.21.15.57:0/184550938 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:47.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.702+0000 7f5132ffd700 1 --2- 172.21.15.57:0/184550938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f513415fe50 0x7f513406d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:47.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.702+0000 7f5131ffb700 1 -- 172.21.15.57:0/184550938 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196794896 0 0) 0x7f5134161370 con 0x7f51340b27d0 2023-11-28T12:02:47.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.702+0000 7f5131ffb700 1 -- 172.21.15.57:0/184550938 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f511c0035b0 con 0x7f51340b27d0 2023-11-28T12:02:47.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.702+0000 7f5131ffb700 1 -- 172.21.15.57:0/184550938 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3968405757 0 0) 0x7f51340cf700 con 0x7f51340b3b80 2023-11-28T12:02:47.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.702+0000 7f5131ffb700 1 -- 172.21.15.57:0/184550938 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5134161370 con 0x7f51340b3b80 2023-11-28T12:02:47.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.702+0000 7f5132ffd700 1 -- 172.21.15.57:0/184550938 >> v1:172.21.15.181:6789/0 conn(0x7f51340b3b80 legacy=0x7f5134160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.702+0000 7f5132ffd700 1 -- 172.21.15.57:0/184550938 >> v1:172.21.15.57:6789/0 conn(0x7f51340b27d0 legacy=0x7f51340cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.702+0000 7f5132ffd700 1 -- 172.21.15.57:0/184550938 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51340b3fd0 con 0x7f513415fe50 2023-11-28T12:02:47.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.702+0000 7f5132ffd700 1 --2- 172.21.15.57:0/184550938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f513415fe50 0x7f513406d170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5124009870 tx=0x7f5124009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26eba036d318de79 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:47.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.702+0000 7f5131ffb700 1 -- 172.21.15.57:0/184550938 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124011830 con 0x7f513415fe50 2023-11-28T12:02:47.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.703+0000 7f5131ffb700 1 -- 172.21.15.57:0/184550938 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5124011990 con 0x7f513415fe50 2023-11-28T12:02:47.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.703+0000 7f5131ffb700 1 -- 172.21.15.57:0/184550938 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124011d20 con 0x7f513415fe50 2023-11-28T12:02:47.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.703+0000 7f5139324700 1 -- 172.21.15.57:0/184550938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f513415fe50 msgr2=0x7f513406d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.703+0000 7f5139324700 1 --2- 172.21.15.57:0/184550938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f513415fe50 0x7f513406d170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5124009870 tx=0x7f5124009b80 comp rx=0 tx=0).stop 2023-11-28T12:02:47.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.703+0000 7f5139324700 1 -- 172.21.15.57:0/184550938 shutdown_connections 2023-11-28T12:02:47.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.703+0000 7f5139324700 1 --2- 172.21.15.57:0/184550938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f513415fe50 0x7f513406d170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.703+0000 7f5139324700 1 -- 172.21.15.57:0/184550938 >> 172.21.15.57:0/184550938 conn(0x7f513406ea80 msgr2=0x7f513406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:47.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.703+0000 7f5139324700 1 -- 172.21.15.57:0/184550938 shutdown_connections 2023-11-28T12:02:47.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.703+0000 7f5139324700 1 -- 172.21.15.57:0/184550938 wait complete. 2023-11-28T12:02:47.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.705+0000 7f5139324700 1 Processor -- start 2023-11-28T12:02:47.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.705+0000 7f5139324700 1 -- start start 2023-11-28T12:02:47.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.705+0000 7f5139324700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51340b27d0 0x7f513415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:47.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.705+0000 7f5139324700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51340b3b80 0x7f51340aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:47.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.705+0000 7f5139324700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f513415fe50 0x7f513415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:47.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.705+0000 7f5139324700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513415a9a0 con 0x7f513415fe50 2023-11-28T12:02:47.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.705+0000 7f5139324700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51340adeb0 con 0x7f51340b27d0 2023-11-28T12:02:47.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.705+0000 7f5139324700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51340ae190 con 0x7f51340b3b80 2023-11-28T12:02:47.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f51337fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f513415fe50 0x7f513415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:47.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f51337fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f513415fe50 0x7f513415ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44948/0 (socket says 172.21.15.57:44948) 2023-11-28T12:02:47.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f51337fe700 1 -- 172.21.15.57:0/1299612836 learned_addr learned my addr 172.21.15.57:0/1299612836 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:47.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f51327fc700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51340b3b80 0x7f51340aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:47.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f5132ffd700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51340b27d0 0x7f513415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:47.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f51337fe700 1 -- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51340b3b80 msgr2=0x7f51340aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f51337fe700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51340b3b80 0x7f51340aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f51337fe700 1 -- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51340b27d0 msgr2=0x7f513415a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:47.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f51337fe700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51340b27d0 0x7f513415a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:47.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f51337fe700 1 -- 172.21.15.57:0/1299612836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240094a0 con 0x7f513415fe50 2023-11-28T12:02:47.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f5132ffd700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51340b27d0 0x7f513415a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:47.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f51337fe700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f513415fe50 0x7f513415ab80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f512c00caf0 tx=0x7f512c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:47.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f511b7fe700 1 -- 172.21.15.57:0/1299612836 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c00aad0 con 0x7f513415fe50 2023-11-28T12:02:47.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f511b7fe700 1 -- 172.21.15.57:0/1299612836 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c00ac30 con 0x7f513415fe50 2023-11-28T12:02:47.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f5139324700 1 -- 172.21.15.57:0/1299612836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51340cf0b0 con 0x7f513415fe50 2023-11-28T12:02:47.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f511b7fe700 1 -- 172.21.15.57:0/1299612836 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c012400 con 0x7f513415fe50 2023-11-28T12:02:47.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.706+0000 7f5139324700 1 -- 172.21.15.57:0/1299612836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51340cedd0 con 0x7f513415fe50 2023-11-28T12:02:47.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.707+0000 7f5139324700 1 -- 172.21.15.57:0/1299612836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51340cf390 con 0x7f513415fe50 2023-11-28T12:02:47.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.708+0000 7f511b7fe700 1 -- 172.21.15.57:0/1299612836 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f512c019030 con 0x7f513415fe50 2023-11-28T12:02:47.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.708+0000 7f511b7fe700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f51200682f0 0x7f512006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:47.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.708+0000 7f511b7fe700 1 -- 172.21.15.57:0/1299612836 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f512c0853a0 con 0x7f513415fe50 2023-11-28T12:02:47.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.708+0000 7f5132ffd700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f51200682f0 0x7f512006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:47.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.709+0000 7f5132ffd700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f51200682f0 0x7f512006a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f513406d4f0 tx=0x7f512401c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:47.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.713+0000 7f511b7fe700 1 -- 172.21.15.57:0/1299612836 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f512c04b5e0 con 0x7f513415fe50 2023-11-28T12:02:47.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.996+0000 7f5139324700 1 -- 172.21.15.57:0/1299612836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f51340743f0 con 0x7f513415fe50 2023-11-28T12:02:48.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:47.999+0000 7f511b7fe700 1 -- 172.21.15.57:0/1299612836 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f512c052000 con 0x7f513415fe50 2023-11-28T12:02:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.002+0000 7f5139324700 1 -- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f51200682f0 msgr2=0x7f512006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.002+0000 7f5139324700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f51200682f0 0x7f512006a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f513406d4f0 tx=0x7f512401c040 comp rx=0 tx=0).stop 2023-11-28T12:02:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.002+0000 7f5139324700 1 -- 172.21.15.57:0/1299612836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f513415fe50 msgr2=0x7f513415ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.002+0000 7f5139324700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f513415fe50 0x7f513415ab80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f512c00caf0 tx=0x7f512c00ceb0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.002+0000 7f5139324700 1 -- 172.21.15.57:0/1299612836 shutdown_connections 2023-11-28T12:02:48.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.002+0000 7f5139324700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f51200682f0 0x7f512006a7a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.002+0000 7f5139324700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f513415fe50 0x7f513415ab80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.002+0000 7f5139324700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51340b27d0 0x7f513415a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.002+0000 7f5139324700 1 --2- 172.21.15.57:0/1299612836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51340b3b80 0x7f51340aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.002+0000 7f5139324700 1 -- 172.21.15.57:0/1299612836 >> 172.21.15.57:0/1299612836 conn(0x7f513406ea80 msgr2=0x7f5134160b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:48.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.002+0000 7f5139324700 1 -- 172.21.15.57:0/1299612836 shutdown_connections 2023-11-28T12:02:48.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.002+0000 7f5139324700 1 -- 172.21.15.57:0/1299612836 wait complete. 2023-11-28T12:02:48.016 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-28T12:02:48.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.152+0000 7fdceac48700 1 Processor -- start 2023-11-28T12:02:48.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.152+0000 7fdceac48700 1 -- start start 2023-11-28T12:02:48.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.153+0000 7fdceac48700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce415c810 0x7fdce4160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:48.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.153+0000 7fdceac48700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce41607a0 con 0x7fdce40718e0 2023-11-28T12:02:48.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.153+0000 7fdceac48700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce4160a50 con 0x7fdce406a270 2023-11-28T12:02:48.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.153+0000 7fdceac48700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce415cbe0 con 0x7fdce415c810 2023-11-28T12:02:48.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.153+0000 7fdce3fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fdce40718e0 0x7fdce406c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39542/0 (socket says 172.21.15.57:39542) 2023-11-28T12:02:48.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.153+0000 7fdce3fff700 1 -- 172.21.15.57:0/2492791567 learned_addr learned my addr 172.21.15.57:0/2492791567 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:48.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.153+0000 7fdce89e4700 1 --2- 172.21.15.57:0/2492791567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce415c810 0x7fdce4160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:48.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.153+0000 7fdce37fe700 1 -- 172.21.15.57:0/2492791567 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96353156 0 0) 0x7fdce41607a0 con 0x7fdce40718e0 2023-11-28T12:02:48.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.153+0000 7fdce37fe700 1 -- 172.21.15.57:0/2492791567 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd40035b0 con 0x7fdce40718e0 2023-11-28T12:02:48.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.154+0000 7fdce37fe700 1 -- 172.21.15.57:0/2492791567 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034263455 0 0) 0x7fdce4160a50 con 0x7fdce406a270 2023-11-28T12:02:48.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.154+0000 7fdce37fe700 1 -- 172.21.15.57:0/2492791567 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce41607a0 con 0x7fdce406a270 2023-11-28T12:02:48.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.154+0000 7fdce37fe700 1 -- 172.21.15.57:0/2492791567 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3132120604 0 0) 0x7fdcd40035b0 con 0x7fdce40718e0 2023-11-28T12:02:48.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.154+0000 7fdce89e4700 1 -- 172.21.15.57:0/2492791567 >> v1:172.21.15.181:6789/0 conn(0x7fdce406a270 legacy=0x7fdce40cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.154+0000 7fdce89e4700 1 -- 172.21.15.57:0/2492791567 >> v1:172.21.15.57:6789/0 conn(0x7fdce40718e0 legacy=0x7fdce406c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.154+0000 7fdce89e4700 1 -- 172.21.15.57:0/2492791567 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce4160d30 con 0x7fdce415c810 2023-11-28T12:02:48.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.154+0000 7fdce89e4700 1 --2- 172.21.15.57:0/2492791567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce415c810 0x7fdce4160260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009870 tx=0x7fdcd8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0a8b94167a685a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:48.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.154+0000 7fdce37fe700 1 -- 172.21.15.57:0/2492791567 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0001670 con 0x7fdce415c810 2023-11-28T12:02:48.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.154+0000 7fdce37fe700 1 -- 172.21.15.57:0/2492791567 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd8004030 con 0x7fdce415c810 2023-11-28T12:02:48.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.154+0000 7fdce37fe700 1 -- 172.21.15.57:0/2492791567 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80043c0 con 0x7fdce415c810 2023-11-28T12:02:48.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.155+0000 7fdceac48700 1 -- 172.21.15.57:0/2492791567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce415c810 msgr2=0x7fdce4160260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.155+0000 7fdceac48700 1 --2- 172.21.15.57:0/2492791567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce415c810 0x7fdce4160260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009870 tx=0x7fdcd8009b80 comp rx=0 tx=0).stop 2023-11-28T12:02:48.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.155+0000 7fdceac48700 1 -- 172.21.15.57:0/2492791567 shutdown_connections 2023-11-28T12:02:48.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.155+0000 7fdceac48700 1 --2- 172.21.15.57:0/2492791567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce415c810 0x7fdce4160260 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.155+0000 7fdceac48700 1 -- 172.21.15.57:0/2492791567 >> 172.21.15.57:0/2492791567 conn(0x7fdce406c010 msgr2=0x7fdce40cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:48.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.155+0000 7fdceac48700 1 -- 172.21.15.57:0/2492791567 shutdown_connections 2023-11-28T12:02:48.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.155+0000 7fdceac48700 1 -- 172.21.15.57:0/2492791567 wait complete. 2023-11-28T12:02:48.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.156+0000 7fdceac48700 1 Processor -- start 2023-11-28T12:02:48.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.156+0000 7fdceac48700 1 -- start start 2023-11-28T12:02:48.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdceac48700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce406a270 0x7fdce40c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:48.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdceac48700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce40718e0 0x7fdce41f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:48.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdceac48700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce415c810 0x7fdce40c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:48.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdceac48700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce4156560 con 0x7fdce40718e0 2023-11-28T12:02:48.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdceac48700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce40c7c90 con 0x7fdce415c810 2023-11-28T12:02:48.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdceac48700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce40c3700 con 0x7fdce406a270 2023-11-28T12:02:48.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdce3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce40718e0 0x7fdce41f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:48.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdce3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce40718e0 0x7fdce41f18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44964/0 (socket says 172.21.15.57:44964) 2023-11-28T12:02:48.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdce3fff700 1 -- 172.21.15.57:0/3943721049 learned_addr learned my addr 172.21.15.57:0/3943721049 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:48.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdce91e5700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce415c810 0x7fdce40c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:48.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdce89e4700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce406a270 0x7fdce40c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:48.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdce3fff700 1 -- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce406a270 msgr2=0x7fdce40c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdce3fff700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce406a270 0x7fdce40c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdce3fff700 1 -- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce415c810 msgr2=0x7fdce40c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdce3fff700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce415c810 0x7fdce40c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.157+0000 7fdce3fff700 1 -- 172.21.15.57:0/3943721049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd80094a0 con 0x7fdce40718e0 2023-11-28T12:02:48.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.158+0000 7fdce3fff700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce40718e0 0x7fdce41f18f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000c750 tx=0x7fdcd000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:48.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.158+0000 7fdce17fa700 1 -- 172.21.15.57:0/3943721049 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0004c80 con 0x7fdce40718e0 2023-11-28T12:02:48.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.158+0000 7fdce17fa700 1 -- 172.21.15.57:0/3943721049 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd0004de0 con 0x7fdce40718e0 2023-11-28T12:02:48.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.158+0000 7fdceac48700 1 -- 172.21.15.57:0/3943721049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce40ceb20 con 0x7fdce40718e0 2023-11-28T12:02:48.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.158+0000 7fdce17fa700 1 -- 172.21.15.57:0/3943721049 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0007310 con 0x7fdce40718e0 2023-11-28T12:02:48.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.158+0000 7fdceac48700 1 -- 172.21.15.57:0/3943721049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce4161010 con 0x7fdce40718e0 2023-11-28T12:02:48.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.159+0000 7fdceac48700 1 -- 172.21.15.57:0/3943721049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce406cf90 con 0x7fdce40718e0 2023-11-28T12:02:48.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.159+0000 7fdce17fa700 1 -- 172.21.15.57:0/3943721049 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdcd001a020 con 0x7fdce40718e0 2023-11-28T12:02:48.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.160+0000 7fdce17fa700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdcc80682f0 0x7fdcc806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:48.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.160+0000 7fdce17fa700 1 -- 172.21.15.57:0/3943721049 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdcd0085630 con 0x7fdce40718e0 2023-11-28T12:02:48.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.160+0000 7fdce89e4700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdcc80682f0 0x7fdcc806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:48.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.161+0000 7fdce89e4700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdcc80682f0 0x7fdcc806a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80097c0 tx=0x7fdcd8013040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:48.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.164+0000 7fdce17fa700 1 -- 172.21.15.57:0/3943721049 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdcd00521e0 con 0x7fdce40718e0 2023-11-28T12:02:48.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.449+0000 7fdceac48700 1 -- 172.21.15.57:0/3943721049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fdce40743f0 con 0x7fdce40718e0 2023-11-28T12:02:48.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.457+0000 7fdce17fa700 1 -- 172.21.15.57:0/3943721049 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdcd0085020 con 0x7fdce40718e0 2023-11-28T12:02:48.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.459+0000 7fdceac48700 1 -- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdcc80682f0 msgr2=0x7fdcc806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.459+0000 7fdceac48700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdcc80682f0 0x7fdcc806a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80097c0 tx=0x7fdcd8013040 comp rx=0 tx=0).stop 2023-11-28T12:02:48.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.459+0000 7fdceac48700 1 -- 172.21.15.57:0/3943721049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce40718e0 msgr2=0x7fdce41f18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.459+0000 7fdceac48700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce40718e0 0x7fdce41f18f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000c750 tx=0x7fdcd000cb10 comp rx=0 tx=0).stop 2023-11-28T12:02:48.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.460+0000 7fdceac48700 1 -- 172.21.15.57:0/3943721049 shutdown_connections 2023-11-28T12:02:48.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.460+0000 7fdceac48700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdcc80682f0 0x7fdcc806a7a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.460+0000 7fdceac48700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce415c810 0x7fdce40c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.460+0000 7fdceac48700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce406a270 0x7fdce40c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.460+0000 7fdceac48700 1 --2- 172.21.15.57:0/3943721049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdce40718e0 0x7fdce41f18f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.460+0000 7fdceac48700 1 -- 172.21.15.57:0/3943721049 >> 172.21.15.57:0/3943721049 conn(0x7fdce406c010 msgr2=0x7fdce40cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:48.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.460+0000 7fdceac48700 1 -- 172.21.15.57:0/3943721049 shutdown_connections 2023-11-28T12:02:48.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.460+0000 7fdceac48700 1 -- 172.21.15.57:0/3943721049 wait complete. 2023-11-28T12:02:48.474 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-28T12:02:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.610+0000 7f75d78aa700 1 Processor -- start 2023-11-28T12:02:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.610+0000 7f75d78aa700 1 -- start start 2023-11-28T12:02:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.611+0000 7f75d78aa700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f75d00b1650 0x7f75d00b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.611+0000 7f75d78aa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d01599b0 0x7f75d015d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.611+0000 7f75d78aa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d0159ec0 con 0x7f75d00af0a0 2023-11-28T12:02:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.611+0000 7f75d78aa700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d015aa60 con 0x7f75d00b1650 2023-11-28T12:02:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.611+0000 7f75d78aa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d00b1f60 con 0x7f75d01599b0 2023-11-28T12:02:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.611+0000 7f75d5646700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f75d00b1650 0x7f75d00b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.611+0000 7f75d5646700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f75d00b1650 0x7f75d00b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44966/0 (socket says 172.21.15.57:44966) 2023-11-28T12:02:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.611+0000 7f75d5646700 1 -- 172.21.15.57:0/1221766209 learned_addr learned my addr 172.21.15.57:0/1221766209 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.611+0000 7f75d4e45700 1 --2- 172.21.15.57:0/1221766209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d01599b0 0x7f75d015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.611+0000 7f75d5646700 1 -- 172.21.15.57:0/1221766209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d01599b0 msgr2=0x7f75d015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.611+0000 7f75d5646700 1 --2- 172.21.15.57:0/1221766209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d01599b0 0x7f75d015d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.611+0000 7f75d5646700 1 -- 172.21.15.57:0/1221766209 >> v1:172.21.15.181:6789/0 conn(0x7f75d00af0a0 legacy=0x7f75d015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.611+0000 7f75d5646700 1 -- 172.21.15.57:0/1221766209 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d015d8a0 con 0x7f75d00b1650 2023-11-28T12:02:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.612+0000 7f75d5646700 1 --2- 172.21.15.57:0/1221766209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75d00b1650 0x7f75d00b1a20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0098f0 tx=0x7f75cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d623c632355ea0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.612+0000 7f75c7fff700 1 -- 172.21.15.57:0/1221766209 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc011830 con 0x7f75d00b1650 2023-11-28T12:02:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.612+0000 7f75c7fff700 1 -- 172.21.15.57:0/1221766209 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc011990 con 0x7f75d00b1650 2023-11-28T12:02:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.612+0000 7f75c7fff700 1 -- 172.21.15.57:0/1221766209 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc011d20 con 0x7f75d00b1650 2023-11-28T12:02:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.612+0000 7f75d78aa700 1 -- 172.21.15.57:0/1221766209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75d00b1650 msgr2=0x7f75d00b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.612+0000 7f75d78aa700 1 --2- 172.21.15.57:0/1221766209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75d00b1650 0x7f75d00b1a20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0098f0 tx=0x7f75cc009c00 comp rx=0 tx=0).stop 2023-11-28T12:02:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.612+0000 7f75d78aa700 1 -- 172.21.15.57:0/1221766209 shutdown_connections 2023-11-28T12:02:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.612+0000 7f75d78aa700 1 --2- 172.21.15.57:0/1221766209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75d00b1650 0x7f75d00b1a20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.612+0000 7f75d78aa700 1 --2- 172.21.15.57:0/1221766209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d01599b0 0x7f75d015d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.612+0000 7f75d78aa700 1 -- 172.21.15.57:0/1221766209 >> 172.21.15.57:0/1221766209 conn(0x7f75d006c010 msgr2=0x7f75d00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:48.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.612+0000 7f75d78aa700 1 -- 172.21.15.57:0/1221766209 shutdown_connections 2023-11-28T12:02:48.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.613+0000 7f75d78aa700 1 -- 172.21.15.57:0/1221766209 wait complete. 2023-11-28T12:02:48.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.614+0000 7f75d78aa700 1 Processor -- start 2023-11-28T12:02:48.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.614+0000 7f75d78aa700 1 -- start start 2023-11-28T12:02:48.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.614+0000 7f75d78aa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75d00af0a0 0x7f75d014f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:48.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d78aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d00b1650 0x7f75d0166df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:48.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d78aa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d0165390 0x7f75d00cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:48.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d78aa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d014fe30 con 0x7f75d00af0a0 2023-11-28T12:02:48.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d78aa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d0165d50 con 0x7f75d00b1650 2023-11-28T12:02:48.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d78aa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d0166030 con 0x7f75d0165390 2023-11-28T12:02:48.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d5646700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75d00af0a0 0x7f75d014f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:48.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d5646700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75d00af0a0 0x7f75d014f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44972/0 (socket says 172.21.15.57:44972) 2023-11-28T12:02:48.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d5646700 1 -- 172.21.15.57:0/425204792 learned_addr learned my addr 172.21.15.57:0/425204792 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:48.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d5e47700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d0165390 0x7f75d00cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:48.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d4e45700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d00b1650 0x7f75d0166df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:48.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d5646700 1 -- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d0165390 msgr2=0x7f75d00cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d5646700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d0165390 0x7f75d00cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d5646700 1 -- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d00b1650 msgr2=0x7f75d0166df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d5646700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d00b1650 0x7f75d0166df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d5646700 1 -- 172.21.15.57:0/425204792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc009550 con 0x7f75d00af0a0 2023-11-28T12:02:48.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d5646700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75d00af0a0 0x7f75d014f8f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009c40 tx=0x7f75cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:48.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75c5ffb700 1 -- 172.21.15.57:0/425204792 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc004510 con 0x7f75d00af0a0 2023-11-28T12:02:48.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75c5ffb700 1 -- 172.21.15.57:0/425204792 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc01e070 con 0x7f75d00af0a0 2023-11-28T12:02:48.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75d78aa700 1 -- 172.21.15.57:0/425204792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d015de60 con 0x7f75d00af0a0 2023-11-28T12:02:48.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.615+0000 7f75c5ffb700 1 -- 172.21.15.57:0/425204792 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0038e0 con 0x7f75d00af0a0 2023-11-28T12:02:48.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.616+0000 7f75d78aa700 1 -- 172.21.15.57:0/425204792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d015db80 con 0x7f75d00af0a0 2023-11-28T12:02:48.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.617+0000 7f75d78aa700 1 -- 172.21.15.57:0/425204792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d015d590 con 0x7f75d00af0a0 2023-11-28T12:02:48.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.617+0000 7f75c5ffb700 1 -- 172.21.15.57:0/425204792 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f75cc00f050 con 0x7f75d00af0a0 2023-11-28T12:02:48.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.617+0000 7f75c5ffb700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f75b4068320 0x7f75b406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:48.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.618+0000 7f75c5ffb700 1 -- 172.21.15.57:0/425204792 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f75cc086830 con 0x7f75d00af0a0 2023-11-28T12:02:48.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.618+0000 7f75d4e45700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f75b4068320 0x7f75b406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:48.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.618+0000 7f75d4e45700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f75b4068320 0x7f75b406a7d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f75d0155a20 tx=0x7f75c0009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:48.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.622+0000 7f75c5ffb700 1 -- 172.21.15.57:0/425204792 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f75cc053360 con 0x7f75d00af0a0 2023-11-28T12:02:48.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.909+0000 7f75d78aa700 1 -- 172.21.15.57:0/425204792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f75d00743f0 con 0x7f75d00af0a0 2023-11-28T12:02:48.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.917+0000 7f75c5ffb700 1 -- 172.21.15.57:0/425204792 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f75cc0200a0 con 0x7f75d00af0a0 2023-11-28T12:02:48.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.920+0000 7f75d78aa700 1 -- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f75b4068320 msgr2=0x7f75b406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.920+0000 7f75d78aa700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f75b4068320 0x7f75b406a7d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f75d0155a20 tx=0x7f75c0009210 comp rx=0 tx=0).stop 2023-11-28T12:02:48.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.920+0000 7f75d78aa700 1 -- 172.21.15.57:0/425204792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75d00af0a0 msgr2=0x7f75d014f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:48.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.920+0000 7f75d78aa700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75d00af0a0 0x7f75d014f8f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009c40 tx=0x7f75cc004750 comp rx=0 tx=0).stop 2023-11-28T12:02:48.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.920+0000 7f75d78aa700 1 -- 172.21.15.57:0/425204792 shutdown_connections 2023-11-28T12:02:48.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.920+0000 7f75d78aa700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f75b4068320 0x7f75b406a7d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.920+0000 7f75d78aa700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d0165390 0x7f75d00cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.920+0000 7f75d78aa700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f75d00af0a0 0x7f75d014f8f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.920+0000 7f75d78aa700 1 --2- 172.21.15.57:0/425204792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d00b1650 0x7f75d0166df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:48.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.920+0000 7f75d78aa700 1 -- 172.21.15.57:0/425204792 >> 172.21.15.57:0/425204792 conn(0x7f75d006c010 msgr2=0x7f75d00cd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:48.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.920+0000 7f75d78aa700 1 -- 172.21.15.57:0/425204792 shutdown_connections 2023-11-28T12:02:48.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:48.920+0000 7f75d78aa700 1 -- 172.21.15.57:0/425204792 wait complete. 2023-11-28T12:02:48.935 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-28T12:02:49.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.071+0000 7f2ea1741700 1 Processor -- start 2023-11-28T12:02:49.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.071+0000 7f2ea1741700 1 -- start start 2023-11-28T12:02:49.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.072+0000 7f2ea1741700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c15a4d0 0x7f2e9c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.072+0000 7f2ea1741700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2e9c06a7b0 0x7f2e9c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.072+0000 7f2ea1741700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c06ac00 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.072+0000 7f2ea1741700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c15e0d0 con 0x7f2e9c06a7b0 2023-11-28T12:02:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.072+0000 7f2ea1741700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0cc020 con 0x7f2e9c15a4d0 2023-11-28T12:02:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.072+0000 7f2e9a7fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2e9c06a7b0 0x7f2e9c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.072+0000 7f2e9b7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2e9c0cdd50 0x7f2e9c0b1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39556/0 (socket says 172.21.15.57:39556) 2023-11-28T12:02:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.072+0000 7f2e9b7fe700 1 -- 172.21.15.57:0/1392703175 learned_addr learned my addr 172.21.15.57:0/1392703175 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.072+0000 7f2e9a7fc700 1 --2- 172.21.15.57:0/1392703175 >> v2:172.21.15.57:3300/0 conn(0x7f2e9c06a7b0 0x7f2e9c0cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44976/0 (socket says 172.21.15.57:44976) 2023-11-28T12:02:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.072+0000 7f2e9affd700 1 --2- 172.21.15.57:0/1392703175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c15a4d0 0x7f2e9c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.073+0000 7f2e9a7fc700 1 -- 172.21.15.57:0/1392703175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c15a4d0 msgr2=0x7f2e9c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.073+0000 7f2e9a7fc700 1 --2- 172.21.15.57:0/1392703175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c15a4d0 0x7f2e9c06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.073+0000 7f2e9a7fc700 1 -- 172.21.15.57:0/1392703175 >> v1:172.21.15.57:6789/0 conn(0x7f2e9c0cdd50 legacy=0x7f2e9c0b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.073+0000 7f2e99ffb700 1 -- 172.21.15.57:0/1392703175 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670500853 0 0) 0x7f2e94001490 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.073+0000 7f2e9a7fc700 1 -- 172.21.15.57:0/1392703175 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0ce560 con 0x7f2e9c06a7b0 2023-11-28T12:02:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.073+0000 7f2e9affd700 1 --2- 172.21.15.57:0/1392703175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c15a4d0 0x7f2e9c06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.073+0000 7f2e9a7fc700 1 --2- 172.21.15.57:0/1392703175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e9c06a7b0 0x7f2e9c0cd810 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009840 tx=0x7f2e8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b420ce415b09e964 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.073+0000 7f2e99ffb700 1 -- 172.21.15.57:0/1392703175 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c011830 con 0x7f2e9c06a7b0 2023-11-28T12:02:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.073+0000 7f2e99ffb700 1 -- 172.21.15.57:0/1392703175 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c011990 con 0x7f2e9c06a7b0 2023-11-28T12:02:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.073+0000 7f2e99ffb700 1 -- 172.21.15.57:0/1392703175 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c011d20 con 0x7f2e9c06a7b0 2023-11-28T12:02:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.074+0000 7f2ea1741700 1 -- 172.21.15.57:0/1392703175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e9c06a7b0 msgr2=0x7f2e9c0cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.074+0000 7f2ea1741700 1 --2- 172.21.15.57:0/1392703175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e9c06a7b0 0x7f2e9c0cd810 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009840 tx=0x7f2e8c009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:49.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.074+0000 7f2ea1741700 1 -- 172.21.15.57:0/1392703175 shutdown_connections 2023-11-28T12:02:49.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.074+0000 7f2ea1741700 1 --2- 172.21.15.57:0/1392703175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c15a4d0 0x7f2e9c06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.074+0000 7f2ea1741700 1 --2- 172.21.15.57:0/1392703175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e9c06a7b0 0x7f2e9c0cd810 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.074+0000 7f2ea1741700 1 -- 172.21.15.57:0/1392703175 >> 172.21.15.57:0/1392703175 conn(0x7f2e9c06c010 msgr2=0x7f2e9c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:49.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.074+0000 7f2ea1741700 1 -- 172.21.15.57:0/1392703175 shutdown_connections 2023-11-28T12:02:49.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.074+0000 7f2ea1741700 1 -- 172.21.15.57:0/1392703175 wait complete. 2023-11-28T12:02:49.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.075+0000 7f2ea1741700 1 Processor -- start 2023-11-28T12:02:49.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.075+0000 7f2ea1741700 1 -- start start 2023-11-28T12:02:49.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.076+0000 7f2ea1741700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c06a7b0 0x7f2e9c153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.076+0000 7f2ea1741700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e9c0cdd50 0x7f2e9c15acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.076+0000 7f2ea1741700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e9c15a4d0 0x7f2e9c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.076+0000 7f2ea1741700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c15cca0 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.076+0000 7f2ea1741700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c15cf40 con 0x7f2e9c15a4d0 2023-11-28T12:02:49.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.076+0000 7f2ea1741700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c15d220 con 0x7f2e9c06a7b0 2023-11-28T12:02:49.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.076+0000 7f2e9a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e9c0cdd50 0x7f2e9c15acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.076+0000 7f2e9a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e9c0cdd50 0x7f2e9c15acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44978/0 (socket says 172.21.15.57:44978) 2023-11-28T12:02:49.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.076+0000 7f2e9a7fc700 1 -- 172.21.15.57:0/3151373952 learned_addr learned my addr 172.21.15.57:0/3151373952 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:49.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.076+0000 7f2e9affd700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c06a7b0 0x7f2e9c153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.076+0000 7f2e9b7fe700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e9c15a4d0 0x7f2e9c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.077+0000 7f2e9a7fc700 1 -- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c06a7b0 msgr2=0x7f2e9c153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.077+0000 7f2e9a7fc700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c06a7b0 0x7f2e9c153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.077+0000 7f2e9a7fc700 1 -- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e9c15a4d0 msgr2=0x7f2e9c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.077+0000 7f2e9a7fc700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e9c15a4d0 0x7f2e9c15b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.077+0000 7f2e9a7fc700 1 -- 172.21.15.57:0/3151373952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0094a0 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.077+0000 7f2e9b7fe700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e9c15a4d0 0x7f2e9c15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:49.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.077+0000 7f2e9a7fc700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e9c0cdd50 0x7f2e9c15acb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009b90 tx=0x7f2e8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:49.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.077+0000 7f2e937fe700 1 -- 172.21.15.57:0/3151373952 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0045d0 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.077+0000 7f2e937fe700 1 -- 172.21.15.57:0/3151373952 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c01f070 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.077+0000 7f2ea1741700 1 -- 172.21.15.57:0/3151373952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c15e600 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.077+0000 7f2e937fe700 1 -- 172.21.15.57:0/3151373952 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0038c0 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.077+0000 7f2ea1741700 1 -- 172.21.15.57:0/3151373952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c15e320 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.078+0000 7f2ea1741700 1 -- 172.21.15.57:0/3151373952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c15e8e0 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.078+0000 7f2e937fe700 1 -- 172.21.15.57:0/3151373952 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2e8c00f050 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.079+0000 7f2e937fe700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2e7c0682f0 0x7f2e7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.079+0000 7f2e937fe700 1 -- 172.21.15.57:0/3151373952 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2e8c021070 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.079+0000 7f2e9affd700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2e7c0682f0 0x7f2e7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.080+0000 7f2e9affd700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2e7c0682f0 0x7f2e7c06a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400c940 tx=0x7f2e84008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:49.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.083+0000 7f2e937fe700 1 -- 172.21.15.57:0/3151373952 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2e8c053e60 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.367+0000 7f2ea1741700 1 -- 172.21.15.57:0/3151373952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f2e9c0c79f0 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.371+0000 7f2e937fe700 1 -- 172.21.15.57:0/3151373952 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2e9c0c79f0 con 0x7f2e9c0cdd50 2023-11-28T12:02:49.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.373+0000 7f2ea1741700 1 -- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2e7c0682f0 msgr2=0x7f2e7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.373+0000 7f2ea1741700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2e7c0682f0 0x7f2e7c06a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400c940 tx=0x7f2e84008040 comp rx=0 tx=0).stop 2023-11-28T12:02:49.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.373+0000 7f2ea1741700 1 -- 172.21.15.57:0/3151373952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e9c0cdd50 msgr2=0x7f2e9c15acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.373+0000 7f2ea1741700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e9c0cdd50 0x7f2e9c15acb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009b90 tx=0x7f2e8c004750 comp rx=0 tx=0).stop 2023-11-28T12:02:49.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.373+0000 7f2ea1741700 1 -- 172.21.15.57:0/3151373952 shutdown_connections 2023-11-28T12:02:49.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.373+0000 7f2ea1741700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2e7c0682f0 0x7f2e7c06a7a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.373+0000 7f2ea1741700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e9c15a4d0 0x7f2e9c15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.373+0000 7f2ea1741700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c06a7b0 0x7f2e9c153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.373+0000 7f2ea1741700 1 --2- 172.21.15.57:0/3151373952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e9c0cdd50 0x7f2e9c15acb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.373+0000 7f2ea1741700 1 -- 172.21.15.57:0/3151373952 >> 172.21.15.57:0/3151373952 conn(0x7f2e9c06c010 msgr2=0x7f2e9c154a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:49.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.373+0000 7f2ea1741700 1 -- 172.21.15.57:0/3151373952 shutdown_connections 2023-11-28T12:02:49.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.373+0000 7f2ea1741700 1 -- 172.21.15.57:0/3151373952 wait complete. 2023-11-28T12:02:49.388 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:49.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.523+0000 7fcae4b77700 1 Processor -- start 2023-11-28T12:02:49.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.523+0000 7fcae4b77700 1 -- start start 2023-11-28T12:02:49.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcae4b77700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcae006ab10 0x7fcae006aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcae4b77700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcae00cd5a0 0x7fcae0160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcae4b77700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae00b22a0 0x7fcae0160840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcae4b77700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae00d1ee0 con 0x7fcae006ab10 2023-11-28T12:02:49.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcae4b77700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae006b420 con 0x7fcae00cd5a0 2023-11-28T12:02:49.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcae4b77700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00cdb10 con 0x7fcae00b22a0 2023-11-28T12:02:49.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcade59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcae006ab10 0x7fcae006aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcade59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcae006ab10 0x7fcae006aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44980/0 (socket says 172.21.15.57:44980) 2023-11-28T12:02:49.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcade59c700 1 -- 172.21.15.57:0/1861359188 learned_addr learned my addr 172.21.15.57:0/1861359188 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:49.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcaded9d700 1 --2- 172.21.15.57:0/1861359188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae00b22a0 0x7fcae0160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcad7fff700 1 --2- 172.21.15.57:0/1861359188 >> v2:172.21.15.181:3300/0 conn(0x7fcae00cd5a0 0x7fcae0160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcade59c700 1 -- 172.21.15.57:0/1861359188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae00b22a0 msgr2=0x7fcae0160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:49.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcade59c700 1 --2- 172.21.15.57:0/1861359188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae00b22a0 0x7fcae0160840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcade59c700 1 -- 172.21.15.57:0/1861359188 >> v2:172.21.15.181:3300/0 conn(0x7fcae00cd5a0 msgr2=0x7fcae0160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:49.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcade59c700 1 --2- 172.21.15.57:0/1861359188 >> v2:172.21.15.181:3300/0 conn(0x7fcae00cd5a0 0x7fcae0160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.524+0000 7fcade59c700 1 -- 172.21.15.57:0/1861359188 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae00d1890 con 0x7fcae006ab10 2023-11-28T12:02:49.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.525+0000 7fcaded9d700 1 --2- 172.21.15.57:0/1861359188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae00b22a0 0x7fcae0160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:49.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.525+0000 7fcad7fff700 1 --2- 172.21.15.57:0/1861359188 >> v2:172.21.15.181:3300/0 conn(0x7fcae00cd5a0 0x7fcae0160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:49.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.525+0000 7fcade59c700 1 --2- 172.21.15.57:0/1861359188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcae006ab10 0x7fcae006aee0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcae00d0a60 tx=0x7fcad0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df348ae968491970 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:49.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.525+0000 7fcaddd9b700 1 -- 172.21.15.57:0/1861359188 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad000b870 con 0x7fcae006ab10 2023-11-28T12:02:49.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.525+0000 7fcaddd9b700 1 -- 172.21.15.57:0/1861359188 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad000b9d0 con 0x7fcae006ab10 2023-11-28T12:02:49.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.525+0000 7fcaddd9b700 1 -- 172.21.15.57:0/1861359188 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad000bd60 con 0x7fcae006ab10 2023-11-28T12:02:49.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.525+0000 7fcae4b77700 1 -- 172.21.15.57:0/1861359188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcae006ab10 msgr2=0x7fcae006aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.525+0000 7fcae4b77700 1 --2- 172.21.15.57:0/1861359188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcae006ab10 0x7fcae006aee0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcae00d0a60 tx=0x7fcad0009bd0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.525+0000 7fcae4b77700 1 -- 172.21.15.57:0/1861359188 shutdown_connections 2023-11-28T12:02:49.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.525+0000 7fcae4b77700 1 --2- 172.21.15.57:0/1861359188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae00b22a0 0x7fcae0160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.525+0000 7fcae4b77700 1 --2- 172.21.15.57:0/1861359188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcae006ab10 0x7fcae006aee0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.525+0000 7fcae4b77700 1 --2- 172.21.15.57:0/1861359188 >> v2:172.21.15.181:3300/0 conn(0x7fcae00cd5a0 0x7fcae0160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.525+0000 7fcae4b77700 1 -- 172.21.15.57:0/1861359188 >> 172.21.15.57:0/1861359188 conn(0x7fcae006c520 msgr2=0x7fcae006c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:49.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.526+0000 7fcae4b77700 1 -- 172.21.15.57:0/1861359188 shutdown_connections 2023-11-28T12:02:49.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.526+0000 7fcae4b77700 1 -- 172.21.15.57:0/1861359188 wait complete. 2023-11-28T12:02:49.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.527+0000 7fcae4b77700 1 Processor -- start 2023-11-28T12:02:49.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.527+0000 7fcae4b77700 1 -- start start 2023-11-28T12:02:49.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcae4b77700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae006ab10 0x7fcae01575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcae4b77700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae00b22a0 0x7fcae00cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcae4b77700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcae00cd5a0 0x7fcae00cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcae4b77700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae01f2a70 con 0x7fcae00cd5a0 2023-11-28T12:02:49.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcae4b77700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00cbd80 con 0x7fcae006ab10 2023-11-28T12:02:49.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcae4b77700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00cc060 con 0x7fcae00b22a0 2023-11-28T12:02:49.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcaded9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcae00cd5a0 0x7fcae00cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcaded9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcae00cd5a0 0x7fcae00cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44984/0 (socket says 172.21.15.57:44984) 2023-11-28T12:02:49.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcaded9d700 1 -- 172.21.15.57:0/864000651 learned_addr learned my addr 172.21.15.57:0/864000651 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:49.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcad7fff700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae00b22a0 0x7fcae00cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcade59c700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae006ab10 0x7fcae01575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcaded9d700 1 -- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae00b22a0 msgr2=0x7fcae00cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcaded9d700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae00b22a0 0x7fcae00cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcaded9d700 1 -- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae006ab10 msgr2=0x7fcae01575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcaded9d700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae006ab10 0x7fcae01575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.528+0000 7fcaded9d700 1 -- 172.21.15.57:0/864000651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad0009550 con 0x7fcae00cd5a0 2023-11-28T12:02:49.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.529+0000 7fcaded9d700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcae00cd5a0 0x7fcae00cb6e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcad8007ef0 tx=0x7fcad800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:49.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.529+0000 7fcad6ffd700 1 -- 172.21.15.57:0/864000651 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad800dd80 con 0x7fcae00cd5a0 2023-11-28T12:02:49.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.529+0000 7fcad6ffd700 1 -- 172.21.15.57:0/864000651 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad800f040 con 0x7fcae00cd5a0 2023-11-28T12:02:49.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.529+0000 7fcad6ffd700 1 -- 172.21.15.57:0/864000651 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8003c60 con 0x7fcae00cd5a0 2023-11-28T12:02:49.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.529+0000 7fcae4b77700 1 -- 172.21.15.57:0/864000651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae015cad0 con 0x7fcae00cd5a0 2023-11-28T12:02:49.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.529+0000 7fcae4b77700 1 -- 172.21.15.57:0/864000651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae00d1b70 con 0x7fcae00cd5a0 2023-11-28T12:02:49.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.530+0000 7fcae4b77700 1 -- 172.21.15.57:0/864000651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae00b1ec0 con 0x7fcae00cd5a0 2023-11-28T12:02:49.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.531+0000 7fcad6ffd700 1 -- 172.21.15.57:0/864000651 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcad801c030 con 0x7fcae00cd5a0 2023-11-28T12:02:49.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.531+0000 7fcad6ffd700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcac00682f0 0x7fcac006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.531+0000 7fcad6ffd700 1 -- 172.21.15.57:0/864000651 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcad8085cb0 con 0x7fcae00cd5a0 2023-11-28T12:02:49.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.532+0000 7fcade59c700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcac00682f0 0x7fcac006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.532+0000 7fcade59c700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcac00682f0 0x7fcac006a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcad0014040 tx=0x7fcad0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:49.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.535+0000 7fcad6ffd700 1 -- 172.21.15.57:0/864000651 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcad8052900 con 0x7fcae00cd5a0 2023-11-28T12:02:49.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.738+0000 7fcae4b77700 1 -- 172.21.15.57:0/864000651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcae00743f0 con 0x7fcae00cd5a0 2023-11-28T12:02:49.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.739+0000 7fcad6ffd700 1 -- 172.21.15.57:0/864000651 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+14209 (secure 0 0 0) 0x7fcad8055f20 con 0x7fcae00cd5a0 2023-11-28T12:02:49.740 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:49.740 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":31,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:02:46.115743+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:49.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.741+0000 7fcae4b77700 1 -- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcac00682f0 msgr2=0x7fcac006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.741+0000 7fcae4b77700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcac00682f0 0x7fcac006a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcad0014040 tx=0x7fcad0008030 comp rx=0 tx=0).stop 2023-11-28T12:02:49.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.741+0000 7fcae4b77700 1 -- 172.21.15.57:0/864000651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcae00cd5a0 msgr2=0x7fcae00cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.741+0000 7fcae4b77700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcae00cd5a0 0x7fcae00cb6e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcad8007ef0 tx=0x7fcad800d3b0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.741+0000 7fcae4b77700 1 -- 172.21.15.57:0/864000651 shutdown_connections 2023-11-28T12:02:49.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.741+0000 7fcae4b77700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcac00682f0 0x7fcac006a7a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.741+0000 7fcae4b77700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcae00cd5a0 0x7fcae00cb6e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.741+0000 7fcae4b77700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae006ab10 0x7fcae01575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.741+0000 7fcae4b77700 1 --2- 172.21.15.57:0/864000651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae00b22a0 0x7fcae00cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.741+0000 7fcae4b77700 1 -- 172.21.15.57:0/864000651 >> 172.21.15.57:0/864000651 conn(0x7fcae006c520 msgr2=0x7fcae00cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:49.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.741+0000 7fcae4b77700 1 -- 172.21.15.57:0/864000651 shutdown_connections 2023-11-28T12:02:49.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.741+0000 7fcae4b77700 1 -- 172.21.15.57:0/864000651 wait complete. 2023-11-28T12:02:49.756 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:02:49.757 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:02:49.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.895+0000 7fa7be401700 1 Processor -- start 2023-11-28T12:02:49.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.895+0000 7fa7be401700 1 -- start start 2023-11-28T12:02:49.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.896+0000 7fa7be401700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b806d170 0x7fa7b80b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.896+0000 7fa7be401700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa7b80b27d0 0x7fa7b80b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.896+0000 7fa7be401700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b8161370 con 0x7fa7b815fe50 2023-11-28T12:02:49.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.896+0000 7fa7be401700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b806d5c0 con 0x7fa7b80b27d0 2023-11-28T12:02:49.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.896+0000 7fa7be401700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b80b3190 con 0x7fa7b806d170 2023-11-28T12:02:49.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.896+0000 7fa7bc99e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa7b80b27d0 0x7fa7b80b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.896+0000 7fa7b7fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa7b815fe50 0x7fa7b8160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39564/0 (socket says 172.21.15.57:39564) 2023-11-28T12:02:49.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.896+0000 7fa7b7fff700 1 -- 172.21.15.57:0/2636320399 learned_addr learned my addr 172.21.15.57:0/2636320399 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:49.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.896+0000 7fa7b77fe700 1 --2- 172.21.15.57:0/2636320399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b806d170 0x7fa7b80b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.896+0000 7fa7bc99e700 1 -- 172.21.15.57:0/2636320399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b806d170 msgr2=0x7fa7b80b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:49.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.896+0000 7fa7bc99e700 1 --2- 172.21.15.57:0/2636320399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b806d170 0x7fa7b80b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.896+0000 7fa7bc99e700 1 -- 172.21.15.57:0/2636320399 >> v1:172.21.15.57:6789/0 conn(0x7fa7b815fe50 legacy=0x7fa7b8160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.896+0000 7fa7bc99e700 1 -- 172.21.15.57:0/2636320399 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b80cf700 con 0x7fa7b80b27d0 2023-11-28T12:02:49.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.897+0000 7fa7bc99e700 1 --2- 172.21.15.57:0/2636320399 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b80b27d0 0x7fa7b80b2c50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009d90 tx=0x7fa7b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e113244fdb444a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:49.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.897+0000 7fa7b6ffd700 1 -- 172.21.15.57:0/2636320399 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b000e070 con 0x7fa7b80b27d0 2023-11-28T12:02:49.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.897+0000 7fa7b6ffd700 1 -- 172.21.15.57:0/2636320399 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b000bb20 con 0x7fa7b80b27d0 2023-11-28T12:02:49.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.897+0000 7fa7b6ffd700 1 -- 172.21.15.57:0/2636320399 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0012430 con 0x7fa7b80b27d0 2023-11-28T12:02:49.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.897+0000 7fa7be401700 1 -- 172.21.15.57:0/2636320399 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b80b27d0 msgr2=0x7fa7b80b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.897+0000 7fa7be401700 1 --2- 172.21.15.57:0/2636320399 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b80b27d0 0x7fa7b80b2c50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009d90 tx=0x7fa7b000b330 comp rx=0 tx=0).stop 2023-11-28T12:02:49.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.897+0000 7fa7be401700 1 -- 172.21.15.57:0/2636320399 shutdown_connections 2023-11-28T12:02:49.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.897+0000 7fa7be401700 1 --2- 172.21.15.57:0/2636320399 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b80b27d0 0x7fa7b80b2c50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.897+0000 7fa7be401700 1 --2- 172.21.15.57:0/2636320399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b806d170 0x7fa7b80b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.897+0000 7fa7be401700 1 -- 172.21.15.57:0/2636320399 >> 172.21.15.57:0/2636320399 conn(0x7fa7b806ea80 msgr2=0x7fa7b806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:49.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.897+0000 7fa7be401700 1 -- 172.21.15.57:0/2636320399 shutdown_connections 2023-11-28T12:02:49.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.897+0000 7fa7be401700 1 -- 172.21.15.57:0/2636320399 wait complete. 2023-11-28T12:02:49.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.899+0000 7fa7be401700 1 Processor -- start 2023-11-28T12:02:49.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.899+0000 7fa7be401700 1 -- start start 2023-11-28T12:02:49.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7be401700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b806d170 0x7fa7b80af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7be401700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7b80b27d0 0x7fa7b80ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7be401700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b815fe50 0x7fa7b8159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7be401700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b80cda20 con 0x7fa7b815fe50 2023-11-28T12:02:49.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7be401700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b80cd460 con 0x7fa7b80b27d0 2023-11-28T12:02:49.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7be401700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b80cd740 con 0x7fa7b806d170 2023-11-28T12:02:49.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7bc99e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b815fe50 0x7fa7b8159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7bc99e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b815fe50 0x7fa7b8159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44998/0 (socket says 172.21.15.57:44998) 2023-11-28T12:02:49.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7bc99e700 1 -- 172.21.15.57:0/2639080427 learned_addr learned my addr 172.21.15.57:0/2639080427 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7b77fe700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7b80b27d0 0x7fa7b80ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7b7fff700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b806d170 0x7fa7b80af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7bc99e700 1 -- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b806d170 msgr2=0x7fa7b80af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7bc99e700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b806d170 0x7fa7b80af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7bc99e700 1 -- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7b80b27d0 msgr2=0x7fa7b80ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7bc99e700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7b80b27d0 0x7fa7b80ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.900+0000 7fa7bc99e700 1 -- 172.21.15.57:0/2639080427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b0009a70 con 0x7fa7b815fe50 2023-11-28T12:02:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.901+0000 7fa7bc99e700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b815fe50 0x7fa7b8159c00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0014040 tx=0x7fa7b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.901+0000 7fa7b4ff9700 1 -- 172.21.15.57:0/2639080427 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0012430 con 0x7fa7b815fe50 2023-11-28T12:02:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.901+0000 7fa7b4ff9700 1 -- 172.21.15.57:0/2639080427 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b0020070 con 0x7fa7b815fe50 2023-11-28T12:02:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.901+0000 7fa7be401700 1 -- 172.21.15.57:0/2639080427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b80cf0b0 con 0x7fa7b815fe50 2023-11-28T12:02:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.901+0000 7fa7b4ff9700 1 -- 172.21.15.57:0/2639080427 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0003ea0 con 0x7fa7b815fe50 2023-11-28T12:02:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.901+0000 7fa7be401700 1 -- 172.21.15.57:0/2639080427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b80cedd0 con 0x7fa7b815fe50 2023-11-28T12:02:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.902+0000 7fa7be401700 1 -- 172.21.15.57:0/2639080427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b80cf390 con 0x7fa7b815fe50 2023-11-28T12:02:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.902+0000 7fa7b4ff9700 1 -- 172.21.15.57:0/2639080427 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa7b0004040 con 0x7fa7b815fe50 2023-11-28T12:02:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.903+0000 7fa7b4ff9700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa7a40682f0 0x7fa7a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.903+0000 7fa7b4ff9700 1 -- 172.21.15.57:0/2639080427 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa7b0022070 con 0x7fa7b815fe50 2023-11-28T12:02:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.903+0000 7fa7b7fff700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa7a40682f0 0x7fa7a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.904+0000 7fa7b7fff700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa7a40682f0 0x7fa7a406a7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8005910 tx=0x7fa7a8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:49.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:49.907+0000 7fa7b4ff9700 1 -- 172.21.15.57:0/2639080427 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa7b0054810 con 0x7fa7b815fe50 2023-11-28T12:02:50.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.105+0000 7fa7be401700 1 -- 172.21.15.57:0/2639080427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa7b80672a0 con 0x7fa7b815fe50 2023-11-28T12:02:50.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.106+0000 7fa7b4ff9700 1 -- 172.21.15.57:0/2639080427 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v31) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa7b0057e30 con 0x7fa7b815fe50 2023-11-28T12:02:50.107 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:02:50.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.108+0000 7fa7be401700 1 -- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa7a40682f0 msgr2=0x7fa7a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.108+0000 7fa7be401700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa7a40682f0 0x7fa7a406a7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8005910 tx=0x7fa7a8009040 comp rx=0 tx=0).stop 2023-11-28T12:02:50.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.108+0000 7fa7be401700 1 -- 172.21.15.57:0/2639080427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b815fe50 msgr2=0x7fa7b8159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.108+0000 7fa7be401700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b815fe50 0x7fa7b8159c00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0014040 tx=0x7fa7b0004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.108+0000 7fa7be401700 1 -- 172.21.15.57:0/2639080427 shutdown_connections 2023-11-28T12:02:50.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.108+0000 7fa7be401700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa7a40682f0 0x7fa7a406a7a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.108+0000 7fa7be401700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b815fe50 0x7fa7b8159c00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.108+0000 7fa7be401700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7b806d170 0x7fa7b80af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.108+0000 7fa7be401700 1 --2- 172.21.15.57:0/2639080427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7b80b27d0 0x7fa7b80ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.108+0000 7fa7be401700 1 -- 172.21.15.57:0/2639080427 >> 172.21.15.57:0/2639080427 conn(0x7fa7b806ea80 msgr2=0x7fa7b80c8980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:50.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.108+0000 7fa7be401700 1 -- 172.21.15.57:0/2639080427 shutdown_connections 2023-11-28T12:02:50.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.108+0000 7fa7be401700 1 -- 172.21.15.57:0/2639080427 wait complete. 2023-11-28T12:02:50.123 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:02:50.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.259+0000 7fee6b1ad700 1 Processor -- start 2023-11-28T12:02:50.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.259+0000 7fee6b1ad700 1 -- start start 2023-11-28T12:02:50.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.260+0000 7fee6b1ad700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fee640b1650 0x7fee640b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.260+0000 7fee6b1ad700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee64159ec0 con 0x7fee640af0a0 2023-11-28T12:02:50.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.260+0000 7fee6b1ad700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6415aa60 con 0x7fee641599b0 2023-11-28T12:02:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.260+0000 7fee6b1ad700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee640b1f60 con 0x7fee640b1650 2023-11-28T12:02:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.260+0000 7fee6974a700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fee640af0a0 0x7fee6415a350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39570/0 (socket says 172.21.15.57:39570) 2023-11-28T12:02:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.260+0000 7fee6974a700 1 -- 172.21.15.57:0/2698626641 learned_addr learned my addr 172.21.15.57:0/2698626641 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.260+0000 7fee68f49700 1 --2- 172.21.15.57:0/2698626641 >> v2:172.21.15.181:3300/0 conn(0x7fee640b1650 0x7fee640b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.261+0000 7fee637fe700 1 -- 172.21.15.57:0/2698626641 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522623342 0 0) 0x7fee64159ec0 con 0x7fee640af0a0 2023-11-28T12:02:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.261+0000 7fee637fe700 1 -- 172.21.15.57:0/2698626641 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee500035b0 con 0x7fee640af0a0 2023-11-28T12:02:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.261+0000 7fee637fe700 1 -- 172.21.15.57:0/2698626641 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261964660 0 0) 0x7fee6415aa60 con 0x7fee641599b0 2023-11-28T12:02:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.261+0000 7fee637fe700 1 -- 172.21.15.57:0/2698626641 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee64159ec0 con 0x7fee641599b0 2023-11-28T12:02:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.261+0000 7fee637fe700 1 -- 172.21.15.57:0/2698626641 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4012966114 0 0) 0x7fee500035b0 con 0x7fee640af0a0 2023-11-28T12:02:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.261+0000 7fee68f49700 1 -- 172.21.15.57:0/2698626641 >> v1:172.21.15.181:6789/0 conn(0x7fee641599b0 legacy=0x7fee6415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.261+0000 7fee68f49700 1 -- 172.21.15.57:0/2698626641 >> v1:172.21.15.57:6789/0 conn(0x7fee640af0a0 legacy=0x7fee6415a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.261+0000 7fee68f49700 1 -- 172.21.15.57:0/2698626641 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6415d8a0 con 0x7fee640b1650 2023-11-28T12:02:50.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.261+0000 7fee68f49700 1 --2- 172.21.15.57:0/2698626641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee640b1650 0x7fee640b1a20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fee54009840 tx=0x7fee54009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ee930af03536a02 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:50.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.261+0000 7fee637fe700 1 -- 172.21.15.57:0/2698626641 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0019f0 con 0x7fee640b1650 2023-11-28T12:02:50.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.261+0000 7fee637fe700 1 -- 172.21.15.57:0/2698626641 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee54004030 con 0x7fee640b1650 2023-11-28T12:02:50.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.261+0000 7fee637fe700 1 -- 172.21.15.57:0/2698626641 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee540043c0 con 0x7fee640b1650 2023-11-28T12:02:50.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.262+0000 7fee6b1ad700 1 -- 172.21.15.57:0/2698626641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee640b1650 msgr2=0x7fee640b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.262+0000 7fee6b1ad700 1 --2- 172.21.15.57:0/2698626641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee640b1650 0x7fee640b1a20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fee54009840 tx=0x7fee54009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:50.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.262+0000 7fee6b1ad700 1 -- 172.21.15.57:0/2698626641 shutdown_connections 2023-11-28T12:02:50.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.262+0000 7fee6b1ad700 1 --2- 172.21.15.57:0/2698626641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee640b1650 0x7fee640b1a20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.262+0000 7fee6b1ad700 1 -- 172.21.15.57:0/2698626641 >> 172.21.15.57:0/2698626641 conn(0x7fee6406c010 msgr2=0x7fee640cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:50.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.262+0000 7fee6b1ad700 1 -- 172.21.15.57:0/2698626641 shutdown_connections 2023-11-28T12:02:50.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.262+0000 7fee6b1ad700 1 -- 172.21.15.57:0/2698626641 wait complete. 2023-11-28T12:02:50.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.263+0000 7fee6b1ad700 1 Processor -- start 2023-11-28T12:02:50.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.263+0000 7fee6b1ad700 1 -- start start 2023-11-28T12:02:50.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.264+0000 7fee6b1ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee640af0a0 0x7fee6414ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.264+0000 7fee6b1ad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee640b1650 0x7fee640b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.264+0000 7fee6b1ad700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee640b47d0 0x7fee6415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.264+0000 7fee6b1ad700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee641eef20 con 0x7fee640b47d0 2023-11-28T12:02:50.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.264+0000 7fee6b1ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee641e2ef0 con 0x7fee640af0a0 2023-11-28T12:02:50.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.264+0000 7fee6b1ad700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee641e31d0 con 0x7fee640b1650 2023-11-28T12:02:50.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.264+0000 7fee6974a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee640b47d0 0x7fee6415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.264+0000 7fee6974a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee640b47d0 0x7fee6415ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45014/0 (socket says 172.21.15.57:45014) 2023-11-28T12:02:50.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.264+0000 7fee6974a700 1 -- 172.21.15.57:0/10263654 learned_addr learned my addr 172.21.15.57:0/10263654 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:50.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee68f49700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee640af0a0 0x7fee6414ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee63fff700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee640b1650 0x7fee640b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee6974a700 1 -- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee640b1650 msgr2=0x7fee640b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee6974a700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee640b1650 0x7fee640b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee6974a700 1 -- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee640af0a0 msgr2=0x7fee6414ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee6974a700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee640af0a0 0x7fee6414ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee6974a700 1 -- 172.21.15.57:0/10263654 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee540094a0 con 0x7fee640b47d0 2023-11-28T12:02:50.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee63fff700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee640b1650 0x7fee640b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:50.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee6974a700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee640b47d0 0x7fee6415ac10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00cc70 tx=0x7fee5c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:50.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee617fa700 1 -- 172.21.15.57:0/10263654 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c00ade0 con 0x7fee640b47d0 2023-11-28T12:02:50.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee617fa700 1 -- 172.21.15.57:0/10263654 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c00ce70 con 0x7fee640b47d0 2023-11-28T12:02:50.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee6b1ad700 1 -- 172.21.15.57:0/10263654 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6415de60 con 0x7fee640b47d0 2023-11-28T12:02:50.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee617fa700 1 -- 172.21.15.57:0/10263654 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0072f0 con 0x7fee640b47d0 2023-11-28T12:02:50.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.265+0000 7fee6b1ad700 1 -- 172.21.15.57:0/10263654 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6415db80 con 0x7fee640b47d0 2023-11-28T12:02:50.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.266+0000 7fee6b1ad700 1 -- 172.21.15.57:0/10263654 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee640743f0 con 0x7fee640b47d0 2023-11-28T12:02:50.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.267+0000 7fee617fa700 1 -- 172.21.15.57:0/10263654 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fee5c01a030 con 0x7fee640b47d0 2023-11-28T12:02:50.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.267+0000 7fee617fa700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee440682f0 0x7fee4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.267+0000 7fee617fa700 1 -- 172.21.15.57:0/10263654 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fee5c085720 con 0x7fee640b47d0 2023-11-28T12:02:50.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.267+0000 7fee68f49700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee440682f0 0x7fee4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.268+0000 7fee68f49700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee440682f0 0x7fee4406a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fee540090b0 tx=0x7fee54008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:50.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.271+0000 7fee617fa700 1 -- 172.21.15.57:0/10263654 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fee5c0522d0 con 0x7fee640b47d0 2023-11-28T12:02:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.471+0000 7fee6b1ad700 1 -- 172.21.15.57:0/10263654 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee6406cd10 con 0x7fee640b47d0 2023-11-28T12:02:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.471+0000 7fee617fa700 1 -- 172.21.15.57:0/10263654 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee5c0180a0 con 0x7fee640b47d0 2023-11-28T12:02:50.473 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:02:50.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.474+0000 7fee6b1ad700 1 -- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee440682f0 msgr2=0x7fee4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.474+0000 7fee6b1ad700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee440682f0 0x7fee4406a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fee540090b0 tx=0x7fee54008030 comp rx=0 tx=0).stop 2023-11-28T12:02:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.474+0000 7fee6b1ad700 1 -- 172.21.15.57:0/10263654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee640b47d0 msgr2=0x7fee6415ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.474+0000 7fee6b1ad700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee640b47d0 0x7fee6415ac10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00cc70 tx=0x7fee5c00a340 comp rx=0 tx=0).stop 2023-11-28T12:02:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.474+0000 7fee6b1ad700 1 -- 172.21.15.57:0/10263654 shutdown_connections 2023-11-28T12:02:50.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.474+0000 7fee6b1ad700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee440682f0 0x7fee4406a7a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.474+0000 7fee6b1ad700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee640b47d0 0x7fee6415ac10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.474+0000 7fee6b1ad700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee640af0a0 0x7fee6414ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.474+0000 7fee6b1ad700 1 --2- 172.21.15.57:0/10263654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee640b1650 0x7fee640b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.474+0000 7fee6b1ad700 1 -- 172.21.15.57:0/10263654 >> 172.21.15.57:0/10263654 conn(0x7fee6406c010 msgr2=0x7fee6406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:50.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.474+0000 7fee6b1ad700 1 -- 172.21.15.57:0/10263654 shutdown_connections 2023-11-28T12:02:50.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.474+0000 7fee6b1ad700 1 -- 172.21.15.57:0/10263654 wait complete. 2023-11-28T12:02:50.489 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T12:02:50.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.624+0000 7fa2cb069700 1 Processor -- start 2023-11-28T12:02:50.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.624+0000 7fa2cb069700 1 -- start start 2023-11-28T12:02:50.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.625+0000 7fa2cb069700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2c4142db0 0x7fa2c413ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.625+0000 7fa2cb069700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa2c40b1770 0x7fa2c40b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.625+0000 7fa2cb069700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c415e010 con 0x7fa2c40ce040 2023-11-28T12:02:50.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.625+0000 7fa2cb069700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c40b2130 con 0x7fa2c40b1770 2023-11-28T12:02:50.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.625+0000 7fa2cb069700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c413fc90 con 0x7fa2c4142db0 2023-11-28T12:02:50.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.625+0000 7fa2c8e05700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa2c40ce040 0x7fa2c415d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39574/0 (socket says 172.21.15.57:39574) 2023-11-28T12:02:50.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.625+0000 7fa2c8e05700 1 -- 172.21.15.57:0/4147487554 learned_addr learned my addr 172.21.15.57:0/4147487554 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:50.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.625+0000 7fa2c3fff700 1 --2- 172.21.15.57:0/4147487554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2c4142db0 0x7fa2c413ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.625+0000 7fa2c9606700 1 --2- 172.21.15.57:0/4147487554 >> v2:172.21.15.181:3300/0 conn(0x7fa2c40b1770 0x7fa2c40b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.625+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950449051 0 0) 0x7fa2b8001110 con 0x7fa2c40ce040 2023-11-28T12:02:50.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.625+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2b40035b0 con 0x7fa2c40ce040 2023-11-28T12:02:50.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4000059309 0 0) 0x7fa2b40035b0 con 0x7fa2c40ce040 2023-11-28T12:02:50.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2c4142db0 msgr2=0x7fa2c413ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:50.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 --2- 172.21.15.57:0/4147487554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2c4142db0 0x7fa2c413ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 >> v2:172.21.15.181:3300/0 conn(0x7fa2c40b1770 msgr2=0x7fa2c40b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:50.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 --2- 172.21.15.57:0/4147487554 >> v2:172.21.15.181:3300/0 conn(0x7fa2c40b1770 0x7fa2c40b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c413f6d0 con 0x7fa2c40ce040 2023-11-28T12:02:50.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fa2b8001800 con 0x7fa2c40ce040 2023-11-28T12:02:50.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 >> v1:172.21.15.57:6789/0 conn(0x7fa2c40ce040 legacy=0x7fa2c415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 --2- 172.21.15.57:0/4147487554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b4003a00 0x7fa2b4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 --2- 172.21.15.57:0/4147487554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2b4004430 0x7fa2b40048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 --2- 172.21.15.57:0/4147487554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2c4142db0 0x7fa2b4004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40b2130 con 0x7fa2b4004430 2023-11-28T12:02:50.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c413fc90 con 0x7fa2b4003a00 2023-11-28T12:02:50.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b400ba10 con 0x7fa2c4142db0 2023-11-28T12:02:50.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c3fff700 1 --2- 172.21.15.57:0/4147487554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2b4004430 0x7fa2b40048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c8e05700 1 --2- 172.21.15.57:0/4147487554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b4003a00 0x7fa2b4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c9606700 1 --2- 172.21.15.57:0/4147487554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2c4142db0 0x7fa2b4004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c3fff700 1 -- 172.21.15.57:0/4147487554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2c4142db0 msgr2=0x7fa2b4004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:50.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c3fff700 1 --2- 172.21.15.57:0/4147487554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2c4142db0 0x7fa2b4004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c3fff700 1 -- 172.21.15.57:0/4147487554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b4003a00 msgr2=0x7fa2b4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:50.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c3fff700 1 --2- 172.21.15.57:0/4147487554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b4003a00 0x7fa2b4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c3fff700 1 -- 172.21.15.57:0/4147487554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b40035b0 con 0x7fa2b4004430 2023-11-28T12:02:50.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.626+0000 7fa2c3fff700 1 --2- 172.21.15.57:0/4147487554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2b4004430 0x7fa2b40048f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009b60 tx=0x7fa2b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31f33681e55c893 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:50.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.627+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000e6c0 con 0x7fa2b4004430 2023-11-28T12:02:50.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.627+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b000e820 con 0x7fa2b4004430 2023-11-28T12:02:50.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.627+0000 7fa2c37fe700 1 -- 172.21.15.57:0/4147487554 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000eb90 con 0x7fa2b4004430 2023-11-28T12:02:50.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.627+0000 7fa2cb069700 1 -- 172.21.15.57:0/4147487554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2b4004430 msgr2=0x7fa2b40048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.627+0000 7fa2cb069700 1 --2- 172.21.15.57:0/4147487554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2b4004430 0x7fa2b40048f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009b60 tx=0x7fa2b4005880 comp rx=0 tx=0).stop 2023-11-28T12:02:50.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.627+0000 7fa2c9606700 1 -- 172.21.15.57:0/4147487554 reap_dead start 2023-11-28T12:02:50.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.627+0000 7fa2cb069700 1 -- 172.21.15.57:0/4147487554 shutdown_connections 2023-11-28T12:02:50.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.627+0000 7fa2cb069700 1 -- 172.21.15.57:0/4147487554 >> 172.21.15.57:0/4147487554 conn(0x7fa2c406c010 msgr2=0x7fa2c40cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:50.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.627+0000 7fa2cb069700 1 -- 172.21.15.57:0/4147487554 shutdown_connections 2023-11-28T12:02:50.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.627+0000 7fa2cb069700 1 -- 172.21.15.57:0/4147487554 wait complete. 2023-11-28T12:02:50.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.628+0000 7fa2cb069700 1 Processor -- start 2023-11-28T12:02:50.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.628+0000 7fa2cb069700 1 -- start start 2023-11-28T12:02:50.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.629+0000 7fa2cb069700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b4004430 0x7fa2c40c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.629+0000 7fa2cb069700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2c40c61a0 0x7fa2c40c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.629+0000 7fa2cb069700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2c40c7d10 0x7fa2c41577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.629+0000 7fa2cb069700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40c9d10 con 0x7fa2c40c7d10 2023-11-28T12:02:50.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.629+0000 7fa2cb069700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40c8770 con 0x7fa2b4004430 2023-11-28T12:02:50.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.629+0000 7fa2cb069700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40c8a10 con 0x7fa2c40c61a0 2023-11-28T12:02:50.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c9606700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2c40c7d10 0x7fa2c41577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c9606700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2c40c7d10 0x7fa2c41577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45032/0 (socket says 172.21.15.57:45032) 2023-11-28T12:02:50.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c9606700 1 -- 172.21.15.57:0/490488973 learned_addr learned my addr 172.21.15.57:0/490488973 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:50.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c3fff700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2c40c61a0 0x7fa2c40c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c8e05700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b4004430 0x7fa2c40c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c9606700 1 -- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2c40c61a0 msgr2=0x7fa2c40c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c9606700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2c40c61a0 0x7fa2c40c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c9606700 1 -- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b4004430 msgr2=0x7fa2c40c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c9606700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b4004430 0x7fa2c40c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c9606700 1 -- 172.21.15.57:0/490488973 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c415d100 con 0x7fa2c40c7d10 2023-11-28T12:02:50.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c9606700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2c40c7d10 0x7fa2c41577b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40cc740 tx=0x7fa2bc011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:50.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c17fa700 1 -- 172.21.15.57:0/490488973 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00e040 con 0x7fa2c40c7d10 2023-11-28T12:02:50.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c17fa700 1 -- 172.21.15.57:0/490488973 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc00f040 con 0x7fa2c40c7d10 2023-11-28T12:02:50.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2cb069700 1 -- 172.21.15.57:0/490488973 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c413f9b0 con 0x7fa2c40c7d10 2023-11-28T12:02:50.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2c17fa700 1 -- 172.21.15.57:0/490488973 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00fe40 con 0x7fa2c40c7d10 2023-11-28T12:02:50.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.630+0000 7fa2cb069700 1 -- 172.21.15.57:0/490488973 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c415d3e0 con 0x7fa2c40c7d10 2023-11-28T12:02:50.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.631+0000 7fa2cb069700 1 -- 172.21.15.57:0/490488973 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c40743f0 con 0x7fa2c40c7d10 2023-11-28T12:02:50.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.632+0000 7fa2c17fa700 1 -- 172.21.15.57:0/490488973 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa2bc015430 con 0x7fa2c40c7d10 2023-11-28T12:02:50.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.632+0000 7fa2c17fa700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2a80682f0 0x7fa2a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:50.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.632+0000 7fa2c17fa700 1 -- 172.21.15.57:0/490488973 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa2bc086370 con 0x7fa2c40c7d10 2023-11-28T12:02:50.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.633+0000 7fa2c8e05700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2a80682f0 0x7fa2a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:50.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.633+0000 7fa2c8e05700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2a80682f0 0x7fa2a806a7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80012b0 tx=0x7fa2b8007b00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:50.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.637+0000 7fa2c17fa700 1 -- 172.21.15.57:0/490488973 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa2bc052ec0 con 0x7fa2c40c7d10 2023-11-28T12:02:50.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.841+0000 7fa2cb069700 1 -- 172.21.15.57:0/490488973 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa2c40b5c70 con 0x7fa2c40c7d10 2023-11-28T12:02:50.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.841+0000 7fa2c17fa700 1 -- 172.21.15.57:0/490488973 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa2bc0564e0 con 0x7fa2c40c7d10 2023-11-28T12:02:50.843 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:50.843 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2023-11-28T12:02:50.843 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:50.843 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 1, 2023-11-28T12:02:50.843 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": "rbd" 2023-11-28T12:02:50.843 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:02:50.843 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:02:50.843 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 2, 2023-11-28T12:02:50.844 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": ".mgr" 2023-11-28T12:02:50.844 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:02:50.844 INFO:teuthology.orchestra.run.smithi057.stdout:] 2023-11-28T12:02:50.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.844+0000 7fa2cb069700 1 -- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2a80682f0 msgr2=0x7fa2a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.844+0000 7fa2cb069700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2a80682f0 0x7fa2a806a7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80012b0 tx=0x7fa2b8007b00 comp rx=0 tx=0).stop 2023-11-28T12:02:50.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.844+0000 7fa2cb069700 1 -- 172.21.15.57:0/490488973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2c40c7d10 msgr2=0x7fa2c41577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:50.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.844+0000 7fa2cb069700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2c40c7d10 0x7fa2c41577b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40cc740 tx=0x7fa2bc011f60 comp rx=0 tx=0).stop 2023-11-28T12:02:50.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.844+0000 7fa2cb069700 1 -- 172.21.15.57:0/490488973 shutdown_connections 2023-11-28T12:02:50.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.844+0000 7fa2cb069700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2a80682f0 0x7fa2a806a7a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.844+0000 7fa2cb069700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2c40c7d10 0x7fa2c41577b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.844+0000 7fa2cb069700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b4004430 0x7fa2c40c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.844+0000 7fa2cb069700 1 --2- 172.21.15.57:0/490488973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2c40c61a0 0x7fa2c40c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:50.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.844+0000 7fa2cb069700 1 -- 172.21.15.57:0/490488973 >> 172.21.15.57:0/490488973 conn(0x7fa2c406c010 msgr2=0x7fa2c4070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:50.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.844+0000 7fa2cb069700 1 -- 172.21.15.57:0/490488973 shutdown_connections 2023-11-28T12:02:50.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.844+0000 7fa2cb069700 1 -- 172.21.15.57:0/490488973 wait complete. 2023-11-28T12:02:50.858 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T12:02:50.859 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T12:02:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.998+0000 7f9b0c060700 1 Processor -- start 2023-11-28T12:02:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.998+0000 7f9b0c060700 1 -- start start 2023-11-28T12:02:51.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.999+0000 7f9b0c060700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b040b22e0 0x7f9b040d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:51.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.999+0000 7f9b0c060700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9b040d1270 0x7f9b040b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:51.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.999+0000 7f9b0c060700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b040b32b0 con 0x7f9b0415cce0 2023-11-28T12:02:51.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.999+0000 7f9b0c060700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b040d1690 con 0x7f9b040d1270 2023-11-28T12:02:51.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.999+0000 7f9b0c060700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b040b26e0 con 0x7f9b040b22e0 2023-11-28T12:02:51.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.999+0000 7f9b0a5fd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9b040d1270 0x7f9b040b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:51.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.999+0000 7f9b09dfc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f9b0415cce0 0x7f9b040b1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39590/0 (socket says 172.21.15.57:39590) 2023-11-28T12:02:51.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.999+0000 7f9b09dfc700 1 -- 172.21.15.57:0/1955849405 learned_addr learned my addr 172.21.15.57:0/1955849405 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.999+0000 7f9b095fb700 1 --2- 172.21.15.57:0/1955849405 >> v2:172.21.15.181:3300/0 conn(0x7f9b040b22e0 0x7f9b040d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:50.999+0000 7f9b08dfa700 1 -- 172.21.15.57:0/1955849405 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265613783 0 0) 0x7f9b040b32b0 con 0x7f9b0415cce0 2023-11-28T12:02:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.000+0000 7f9b08dfa700 1 -- 172.21.15.57:0/1955849405 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9af00035b0 con 0x7f9b0415cce0 2023-11-28T12:02:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.000+0000 7f9b0a5fd700 1 -- 172.21.15.57:0/1955849405 >> v2:172.21.15.181:3300/0 conn(0x7f9b040b22e0 msgr2=0x7f9b040d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.000+0000 7f9b0a5fd700 1 --2- 172.21.15.57:0/1955849405 >> v2:172.21.15.181:3300/0 conn(0x7f9b040b22e0 0x7f9b040d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.000+0000 7f9b0a5fd700 1 -- 172.21.15.57:0/1955849405 >> v1:172.21.15.57:6789/0 conn(0x7f9b0415cce0 legacy=0x7f9b040b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.000+0000 7f9b0a5fd700 1 -- 172.21.15.57:0/1955849405 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b040b1e60 con 0x7f9b040d1270 2023-11-28T12:02:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.000+0000 7f9b095fb700 1 --2- 172.21.15.57:0/1955849405 >> v2:172.21.15.181:3300/0 conn(0x7f9b040b22e0 0x7f9b040d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.000+0000 7f9b0a5fd700 1 --2- 172.21.15.57:0/1955849405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b040d1270 0x7f9b040b3740 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9b0406d450 tx=0x7f9afc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413817f28e7f34 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.000+0000 7f9b08dfa700 1 -- 172.21.15.57:0/1955849405 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc00e070 con 0x7f9b040d1270 2023-11-28T12:02:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.000+0000 7f9b08dfa700 1 -- 172.21.15.57:0/1955849405 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc00bb20 con 0x7f9b040d1270 2023-11-28T12:02:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.000+0000 7f9b08dfa700 1 -- 172.21.15.57:0/1955849405 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc012430 con 0x7f9b040d1270 2023-11-28T12:02:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.000+0000 7f9b0c060700 1 -- 172.21.15.57:0/1955849405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b040d1270 msgr2=0x7f9b040b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.001+0000 7f9b0c060700 1 --2- 172.21.15.57:0/1955849405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b040d1270 0x7f9b040b3740 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9b0406d450 tx=0x7f9afc00b330 comp rx=0 tx=0).stop 2023-11-28T12:02:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.001+0000 7f9b0c060700 1 -- 172.21.15.57:0/1955849405 shutdown_connections 2023-11-28T12:02:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.001+0000 7f9b0c060700 1 --2- 172.21.15.57:0/1955849405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b040d1270 0x7f9b040b3740 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.001+0000 7f9b0c060700 1 --2- 172.21.15.57:0/1955849405 >> v2:172.21.15.181:3300/0 conn(0x7f9b040b22e0 0x7f9b040d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.001+0000 7f9b0c060700 1 -- 172.21.15.57:0/1955849405 >> 172.21.15.57:0/1955849405 conn(0x7f9b0409c4d0 msgr2=0x7f9b040d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.001+0000 7f9b0c060700 1 -- 172.21.15.57:0/1955849405 shutdown_connections 2023-11-28T12:02:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.001+0000 7f9b0c060700 1 -- 172.21.15.57:0/1955849405 wait complete. 2023-11-28T12:02:51.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.002+0000 7f9b0c060700 1 Processor -- start 2023-11-28T12:02:51.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.002+0000 7f9b0c060700 1 -- start start 2023-11-28T12:02:51.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.003+0000 7f9b0c060700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b040d1270 0x7f9b040b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:51.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.003+0000 7f9b0c060700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b0415cce0 0x7f9b0415b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:51.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.003+0000 7f9b0c060700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b0415bcb0 0x7f9b041f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:51.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.003+0000 7f9b0c060700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0415c150 con 0x7f9b0415cce0 2023-11-28T12:02:51.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.003+0000 7f9b0c060700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b041f6870 con 0x7f9b040d1270 2023-11-28T12:02:51.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.003+0000 7f9b0c060700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b041f5ab0 con 0x7f9b0415bcb0 2023-11-28T12:02:51.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9b095fb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b0415cce0 0x7f9b0415b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:51.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9b095fb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b0415cce0 0x7f9b0415b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45050/0 (socket says 172.21.15.57:45050) 2023-11-28T12:02:51.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9b095fb700 1 -- 172.21.15.57:0/3356888605 learned_addr learned my addr 172.21.15.57:0/3356888605 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:51.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9b09dfc700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b040d1270 0x7f9b040b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:51.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9b0a5fd700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b0415bcb0 0x7f9b041f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:51.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9b095fb700 1 -- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b0415bcb0 msgr2=0x7f9b041f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:51.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9b095fb700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b0415bcb0 0x7f9b041f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:51.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9b095fb700 1 -- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b040d1270 msgr2=0x7f9b040b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:51.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9b095fb700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b040d1270 0x7f9b040b8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:51.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9b095fb700 1 -- 172.21.15.57:0/3356888605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc009a70 con 0x7f9b0415cce0 2023-11-28T12:02:51.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9b095fb700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b0415cce0 0x7f9b0415b770 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e810 tx=0x7f9af800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:51.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9af67fc700 1 -- 172.21.15.57:0/3356888605 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af800c020 con 0x7f9b0415cce0 2023-11-28T12:02:51.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9af67fc700 1 -- 172.21.15.57:0/3356888605 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af8004250 con 0x7f9b0415cce0 2023-11-28T12:02:51.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9b0c060700 1 -- 172.21.15.57:0/3356888605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b040b2c60 con 0x7f9b0415cce0 2023-11-28T12:02:51.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9af67fc700 1 -- 172.21.15.57:0/3356888605 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80036a0 con 0x7f9b0415cce0 2023-11-28T12:02:51.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.004+0000 7f9b0c060700 1 -- 172.21.15.57:0/3356888605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b040b2980 con 0x7f9b0415cce0 2023-11-28T12:02:51.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.005+0000 7f9b0c060700 1 -- 172.21.15.57:0/3356888605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b040b2f40 con 0x7f9b0415cce0 2023-11-28T12:02:51.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.006+0000 7f9af67fc700 1 -- 172.21.15.57:0/3356888605 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9af8003800 con 0x7f9b0415cce0 2023-11-28T12:02:51.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.006+0000 7f9af67fc700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ae80682f0 0x7f9ae806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:51.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.007+0000 7f9af67fc700 1 -- 172.21.15.57:0/3356888605 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9af80854d0 con 0x7f9b0415cce0 2023-11-28T12:02:51.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.007+0000 7f9b09dfc700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ae80682f0 0x7f9ae806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:51.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.007+0000 7f9b09dfc700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ae80682f0 0x7f9ae806a7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9b00007fd0 tx=0x7f9b00007dc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:51.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.011+0000 7f9af67fc700 1 -- 172.21.15.57:0/3356888605 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9af8052080 con 0x7f9b0415cce0 2023-11-28T12:02:51.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.213+0000 7f9b0c060700 1 -- 172.21.15.57:0/3356888605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9b040743f0 con 0x7f9b0415cce0 2023-11-28T12:02:51.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.948+0000 7f9af67fc700 1 -- 172.21.15.57:0/3356888605 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9af80556a0 con 0x7f9b0415cce0 2023-11-28T12:02:51.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.952+0000 7f9b0c060700 1 -- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ae80682f0 msgr2=0x7f9ae806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:51.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.952+0000 7f9b0c060700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ae80682f0 0x7f9ae806a7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9b00007fd0 tx=0x7f9b00007dc0 comp rx=0 tx=0).stop 2023-11-28T12:02:51.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.952+0000 7f9b0c060700 1 -- 172.21.15.57:0/3356888605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b0415cce0 msgr2=0x7f9b0415b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:51.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.952+0000 7f9b0c060700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b0415cce0 0x7f9b0415b770 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e810 tx=0x7f9af800eb20 comp rx=0 tx=0).stop 2023-11-28T12:02:51.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.953+0000 7f9b0c060700 1 -- 172.21.15.57:0/3356888605 shutdown_connections 2023-11-28T12:02:51.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.953+0000 7f9b0c060700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ae80682f0 0x7f9ae806a7a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:51.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.953+0000 7f9b0c060700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b0415bcb0 0x7f9b041f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:51.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.953+0000 7f9b0c060700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b040d1270 0x7f9b040b8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:51.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.953+0000 7f9b0c060700 1 --2- 172.21.15.57:0/3356888605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b0415cce0 0x7f9b0415b770 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:51.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.953+0000 7f9b0c060700 1 -- 172.21.15.57:0/3356888605 >> 172.21.15.57:0/3356888605 conn(0x7f9b0409c4d0 msgr2=0x7f9b040afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:51.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.954+0000 7f9b0c060700 1 -- 172.21.15.57:0/3356888605 shutdown_connections 2023-11-28T12:02:51.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:51.954+0000 7f9b0c060700 1 -- 172.21.15.57:0/3356888605 wait complete. 2023-11-28T12:02:51.957 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' created 2023-11-28T12:02:51.971 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T12:02:52.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.116+0000 7f618da87700 1 Processor -- start 2023-11-28T12:02:52.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.116+0000 7f618da87700 1 -- start start 2023-11-28T12:02:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.117+0000 7f618da87700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f618806a270 0x7f618806a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.117+0000 7f618da87700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618806abd0 con 0x7f618815c810 2023-11-28T12:02:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.117+0000 7f618da87700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6188160a50 con 0x7f61880718e0 2023-11-28T12:02:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.117+0000 7f618da87700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618815cbe0 con 0x7f618806a270 2023-11-28T12:02:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.117+0000 7f61877fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f618815c810 0x7f6188160260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39600/0 (socket says 172.21.15.57:39600) 2023-11-28T12:02:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.117+0000 7f61877fe700 1 -- 172.21.15.57:0/3208642432 learned_addr learned my addr 172.21.15.57:0/3208642432 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262776190 0 0) 0x7f618806abd0 con 0x7f618815c810 2023-11-28T12:02:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61740035b0 con 0x7f618815c810 2023-11-28T12:02:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f6187fff700 1 --2- 172.21.15.57:0/3208642432 >> v2:172.21.15.181:3300/0 conn(0x7f618806a270 0x7f618806a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4218389712 0 0) 0x7f61740035b0 con 0x7f618815c810 2023-11-28T12:02:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 >> v2:172.21.15.181:3300/0 conn(0x7f618806a270 msgr2=0x7f618806a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 --2- 172.21.15.57:0/3208642432 >> v2:172.21.15.181:3300/0 conn(0x7f618806a270 0x7f618806a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 >> v1:172.21.15.181:6789/0 conn(0x7f61880718e0 legacy=0x7f618806c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6188160d30 con 0x7f618815c810 2023-11-28T12:02:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f6178001670 con 0x7f618815c810 2023-11-28T12:02:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 >> v1:172.21.15.57:6789/0 conn(0x7f618815c810 legacy=0x7f6188160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 --2- 172.21.15.57:0/3208642432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6174003a00 0x7f6174003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 --2- 172.21.15.57:0/3208642432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6174004850 0x7f6174004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 --2- 172.21.15.57:0/3208642432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174005200 0x7f617400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618815cbe0 con 0x7f6174004850 2023-11-28T12:02:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617400c080 con 0x7f6174005200 2023-11-28T12:02:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617400c240 con 0x7f6174003a00 2023-11-28T12:02:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f6186ffd700 1 --2- 172.21.15.57:0/3208642432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6174004850 0x7f6174004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f61877fe700 1 --2- 172.21.15.57:0/3208642432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6174003a00 0x7f6174003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f6187fff700 1 --2- 172.21.15.57:0/3208642432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174005200 0x7f617400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f6186ffd700 1 -- 172.21.15.57:0/3208642432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6174003a00 msgr2=0x7f6174003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f6186ffd700 1 --2- 172.21.15.57:0/3208642432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6174003a00 0x7f6174003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f6186ffd700 1 -- 172.21.15.57:0/3208642432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174005200 msgr2=0x7f617400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f6186ffd700 1 --2- 172.21.15.57:0/3208642432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174005200 0x7f617400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f6186ffd700 1 -- 172.21.15.57:0/3208642432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61740035b0 con 0x7f6174004850 2023-11-28T12:02:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.118+0000 7f6187fff700 1 -- 172.21.15.57:0/3208642432 reap_dead start 2023-11-28T12:02:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.119+0000 7f6186ffd700 1 --2- 172.21.15.57:0/3208642432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6174004850 0x7f6174004cc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f61700017c0 tx=0x7f6174005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8711a6049310a0e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.119+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700052d0 con 0x7f6174004850 2023-11-28T12:02:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.119+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6170005450 con 0x7f6174004850 2023-11-28T12:02:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.119+0000 7f61867fc700 1 -- 172.21.15.57:0/3208642432 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700038c0 con 0x7f6174004850 2023-11-28T12:02:52.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.119+0000 7f618da87700 1 -- 172.21.15.57:0/3208642432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6174004850 msgr2=0x7f6174004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:52.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.119+0000 7f618da87700 1 --2- 172.21.15.57:0/3208642432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6174004850 0x7f6174004cc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f61700017c0 tx=0x7f6174005880 comp rx=0 tx=0).stop 2023-11-28T12:02:52.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.119+0000 7f618da87700 1 -- 172.21.15.57:0/3208642432 shutdown_connections 2023-11-28T12:02:52.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.119+0000 7f618da87700 1 --2- 172.21.15.57:0/3208642432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6174004850 0x7f6174004cc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:52.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.119+0000 7f618da87700 1 -- 172.21.15.57:0/3208642432 >> 172.21.15.57:0/3208642432 conn(0x7f618806c010 msgr2=0x7f61880cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:52.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.119+0000 7f618da87700 1 -- 172.21.15.57:0/3208642432 shutdown_connections 2023-11-28T12:02:52.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.119+0000 7f618da87700 1 -- 172.21.15.57:0/3208642432 wait complete. 2023-11-28T12:02:52.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.121+0000 7f618da87700 1 Processor -- start 2023-11-28T12:02:52.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.121+0000 7f618da87700 1 -- start start 2023-11-28T12:02:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f618da87700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174004850 0x7f61881f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f618da87700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61880c76b0 0x7f61880b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f618da87700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f618815ae50 0x7f618815b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f618da87700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618815b830 con 0x7f618815ae50 2023-11-28T12:02:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f618da87700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61880c8830 con 0x7f6174004850 2023-11-28T12:02:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f618da87700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61880c7c20 con 0x7f61880c76b0 2023-11-28T12:02:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f6187fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f618815ae50 0x7f618815b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f6187fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f618815ae50 0x7f618815b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45062/0 (socket says 172.21.15.57:45062) 2023-11-28T12:02:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f6187fff700 1 -- 172.21.15.57:0/3740492006 learned_addr learned my addr 172.21.15.57:0/3740492006 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f6186ffd700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61880c76b0 0x7f61880b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f61877fe700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174004850 0x7f61881f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f6187fff700 1 -- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61880c76b0 msgr2=0x7f61880b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f6187fff700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61880c76b0 0x7f61880b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:52.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f6187fff700 1 -- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174004850 msgr2=0x7f61881f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:52.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f6187fff700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174004850 0x7f61881f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:52.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f6187fff700 1 -- 172.21.15.57:0/3740492006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61880ceb20 con 0x7f618815ae50 2023-11-28T12:02:52.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f6187fff700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f618815ae50 0x7f618815b2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6188161320 tx=0x7f6180011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:52.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.122+0000 7f617ffff700 1 -- 172.21.15.57:0/3740492006 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000e040 con 0x7f618815ae50 2023-11-28T12:02:52.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.123+0000 7f617ffff700 1 -- 172.21.15.57:0/3740492006 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618000f040 con 0x7f618815ae50 2023-11-28T12:02:52.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.123+0000 7f618da87700 1 -- 172.21.15.57:0/3740492006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6188161010 con 0x7f618815ae50 2023-11-28T12:02:52.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.123+0000 7f617ffff700 1 -- 172.21.15.57:0/3740492006 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000fc10 con 0x7f618815ae50 2023-11-28T12:02:52.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.123+0000 7f618da87700 1 -- 172.21.15.57:0/3740492006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61880ce740 con 0x7f618815ae50 2023-11-28T12:02:52.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.124+0000 7f618da87700 1 -- 172.21.15.57:0/3740492006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61881f1e10 con 0x7f618815ae50 2023-11-28T12:02:52.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.124+0000 7f617ffff700 1 -- 172.21.15.57:0/3740492006 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f618000fd70 con 0x7f618815ae50 2023-11-28T12:02:52.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.125+0000 7f617ffff700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f61680682f0 0x7f616806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:52.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.125+0000 7f617ffff700 1 -- 172.21.15.57:0/3740492006 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f61800860c0 con 0x7f618815ae50 2023-11-28T12:02:52.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.125+0000 7f61877fe700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f61680682f0 0x7f616806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:52.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.126+0000 7f61877fe700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f61680682f0 0x7f616806a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6178001c90 tx=0x7f6178001b20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:52.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.130+0000 7f617ffff700 1 -- 172.21.15.57:0/3740492006 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6180052b00 con 0x7f618815ae50 2023-11-28T12:02:52.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.331+0000 7f618da87700 1 -- 172.21.15.57:0/3740492006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f61880743f0 con 0x7f618815ae50 2023-11-28T12:02:52.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.959+0000 7f617ffff700 1 -- 172.21.15.57:0/3740492006 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6180056120 con 0x7f618815ae50 2023-11-28T12:02:52.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.967+0000 7f618da87700 1 -- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f61680682f0 msgr2=0x7f616806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:52.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.967+0000 7f618da87700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f61680682f0 0x7f616806a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6178001c90 tx=0x7f6178001b20 comp rx=0 tx=0).stop 2023-11-28T12:02:52.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.967+0000 7f618da87700 1 -- 172.21.15.57:0/3740492006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f618815ae50 msgr2=0x7f618815b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:52.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.967+0000 7f618da87700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f618815ae50 0x7f618815b2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6188161320 tx=0x7f6180011ed0 comp rx=0 tx=0).stop 2023-11-28T12:02:52.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.968+0000 7f618da87700 1 -- 172.21.15.57:0/3740492006 shutdown_connections 2023-11-28T12:02:52.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.968+0000 7f618da87700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f61680682f0 0x7f616806a7a0 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6178001c90 tx=0x7f6178001b20 comp rx=0 tx=0).stop 2023-11-28T12:02:52.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.968+0000 7f618da87700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f618815ae50 0x7f618815b2f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:52.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.968+0000 7f618da87700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174004850 0x7f61881f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:52.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.968+0000 7f618da87700 1 --2- 172.21.15.57:0/3740492006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61880c76b0 0x7f61880b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:52.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.968+0000 7f618da87700 1 -- 172.21.15.57:0/3740492006 >> 172.21.15.57:0/3740492006 conn(0x7f618806c010 msgr2=0x7f61880ca7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:52.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.969+0000 7f618da87700 1 -- 172.21.15.57:0/3740492006 shutdown_connections 2023-11-28T12:02:53.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:52.971+0000 7f618da87700 1 -- 172.21.15.57:0/3740492006 wait complete. 2023-11-28T12:02:53.000 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' created 2023-11-28T12:02:53.000 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T12:02:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.142+0000 7f91ae7ff700 1 Processor -- start 2023-11-28T12:02:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.142+0000 7f91ae7ff700 1 -- start start 2023-11-28T12:02:53.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91ae7ff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91a806d170 0x7f91a80b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91ae7ff700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a8161370 con 0x7f91a815fe50 2023-11-28T12:02:53.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91ae7ff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a80cf700 con 0x7f91a80b27d0 2023-11-28T12:02:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91ae7ff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a806d5c0 con 0x7f91a806d170 2023-11-28T12:02:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91a7fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f91a815fe50 0x7f91a8160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39612/0 (socket says 172.21.15.57:39612) 2023-11-28T12:02:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91a7fff700 1 -- 172.21.15.57:0/875130006 learned_addr learned my addr 172.21.15.57:0/875130006 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91a77fe700 1 --2- 172.21.15.57:0/875130006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91a806d170 0x7f91a80b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123173829 0 0) 0x7f91a8161370 con 0x7f91a815fe50 2023-11-28T12:02:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91940035b0 con 0x7f91a815fe50 2023-11-28T12:02:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1055140249 0 0) 0x7f91940035b0 con 0x7f91a815fe50 2023-11-28T12:02:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91a806d170 msgr2=0x7f91a80b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91a6ffd700 1 --2- 172.21.15.57:0/875130006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91a806d170 0x7f91a80b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 >> v1:172.21.15.181:6789/0 conn(0x7f91a80b27d0 legacy=0x7f91a80cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a80cedd0 con 0x7f91a815fe50 2023-11-28T12:02:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.143+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f9198001670 con 0x7f91a815fe50 2023-11-28T12:02:53.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 >> v1:172.21.15.57:6789/0 conn(0x7f91a815fe50 legacy=0x7f91a8160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a6ffd700 1 --2- 172.21.15.57:0/875130006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9194003a30 0x7f9194003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a6ffd700 1 --2- 172.21.15.57:0/875130006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91940043d0 0x7f9194004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a6ffd700 1 --2- 172.21.15.57:0/875130006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91a806d170 0x7f91940050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919400ba10 con 0x7f9194003a30 2023-11-28T12:02:53.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919400bb50 con 0x7f91940043d0 2023-11-28T12:02:53.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919400bce0 con 0x7f91a806d170 2023-11-28T12:02:53.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a7fff700 1 --2- 172.21.15.57:0/875130006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9194003a30 0x7f9194003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a77fe700 1 --2- 172.21.15.57:0/875130006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91940043d0 0x7f9194004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91acd9c700 1 --2- 172.21.15.57:0/875130006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91a806d170 0x7f91940050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a7fff700 1 -- 172.21.15.57:0/875130006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91a806d170 msgr2=0x7f91940050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:53.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a7fff700 1 --2- 172.21.15.57:0/875130006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91a806d170 0x7f91940050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a7fff700 1 -- 172.21.15.57:0/875130006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91940043d0 msgr2=0x7f9194004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:53.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a7fff700 1 --2- 172.21.15.57:0/875130006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91940043d0 0x7f9194004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a7fff700 1 -- 172.21.15.57:0/875130006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91940035b0 con 0x7f9194003a30 2023-11-28T12:02:53.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a7fff700 1 --2- 172.21.15.57:0/875130006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9194003a30 0x7f9194003e00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f919800c240 tx=0x7f9194005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e7dab287efe8af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:53.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919800cd80 con 0x7f9194003a30 2023-11-28T12:02:53.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9198005960 con 0x7f9194003a30 2023-11-28T12:02:53.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.144+0000 7f91a6ffd700 1 -- 172.21.15.57:0/875130006 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9198005d10 con 0x7f9194003a30 2023-11-28T12:02:53.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.145+0000 7f91ae7ff700 1 -- 172.21.15.57:0/875130006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9194003a30 msgr2=0x7f9194003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.145+0000 7f91ae7ff700 1 --2- 172.21.15.57:0/875130006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9194003a30 0x7f9194003e00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f919800c240 tx=0x7f9194005880 comp rx=0 tx=0).stop 2023-11-28T12:02:53.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.145+0000 7f91acd9c700 1 -- 172.21.15.57:0/875130006 reap_dead start 2023-11-28T12:02:53.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.145+0000 7f91ae7ff700 1 -- 172.21.15.57:0/875130006 shutdown_connections 2023-11-28T12:02:53.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.145+0000 7f91ae7ff700 1 -- 172.21.15.57:0/875130006 >> 172.21.15.57:0/875130006 conn(0x7f91a806ea80 msgr2=0x7f91a806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:53.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.145+0000 7f91ae7ff700 1 -- 172.21.15.57:0/875130006 shutdown_connections 2023-11-28T12:02:53.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.145+0000 7f91ae7ff700 1 -- 172.21.15.57:0/875130006 wait complete. 2023-11-28T12:02:53.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.146+0000 7f91ae7ff700 1 Processor -- start 2023-11-28T12:02:53.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.146+0000 7f91ae7ff700 1 -- start start 2023-11-28T12:02:53.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.147+0000 7f91ae7ff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9194003a30 0x7f91a80ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.147+0000 7f91ae7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91a80cce00 0x7f91a81f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.147+0000 7f91ae7ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91a81f25b0 0x7f91a81f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.147+0000 7f91ae7ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a80cdd10 con 0x7f91a81f25b0 2023-11-28T12:02:53.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.147+0000 7f91ae7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a80c7c60 con 0x7f91a80cce00 2023-11-28T12:02:53.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.147+0000 7f91ae7ff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a80cd270 con 0x7f9194003a30 2023-11-28T12:02:53.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.147+0000 7f91acd9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91a81f25b0 0x7f91a81f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.147+0000 7f91acd9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91a81f25b0 0x7f91a81f2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45082/0 (socket says 172.21.15.57:45082) 2023-11-28T12:02:53.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.147+0000 7f91acd9c700 1 -- 172.21.15.57:0/4035884992 learned_addr learned my addr 172.21.15.57:0/4035884992 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:53.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91a77fe700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91a80cce00 0x7f91a81f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91a7fff700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9194003a30 0x7f91a80ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91acd9c700 1 -- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9194003a30 msgr2=0x7f91a80ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91acd9c700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9194003a30 0x7f91a80ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91acd9c700 1 -- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91a80cce00 msgr2=0x7f91a81f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91acd9c700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91a80cce00 0x7f91a81f2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91acd9c700 1 -- 172.21.15.57:0/4035884992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a80cf390 con 0x7f91a81f25b0 2023-11-28T12:02:53.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91a7fff700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9194003a30 0x7f91a80ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:53.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91a77fe700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91a80cce00 0x7f91a81f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:53.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91acd9c700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91a81f25b0 0x7f91a81f2a30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f91a8072a50 tx=0x7f91a000c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:53.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91a4ff9700 1 -- 172.21.15.57:0/4035884992 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0003c10 con 0x7f91a81f25b0 2023-11-28T12:02:53.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91a4ff9700 1 -- 172.21.15.57:0/4035884992 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a0003d70 con 0x7f91a81f25b0 2023-11-28T12:02:53.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91ae7ff700 1 -- 172.21.15.57:0/4035884992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a80cf0b0 con 0x7f91a81f25b0 2023-11-28T12:02:53.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91a4ff9700 1 -- 172.21.15.57:0/4035884992 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a000c9a0 con 0x7f91a81f25b0 2023-11-28T12:02:53.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.148+0000 7f91ae7ff700 1 -- 172.21.15.57:0/4035884992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a80b3b80 con 0x7f91a81f25b0 2023-11-28T12:02:53.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.149+0000 7f91ae7ff700 1 -- 172.21.15.57:0/4035884992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a80743f0 con 0x7f91a81f25b0 2023-11-28T12:02:53.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.150+0000 7f91a4ff9700 1 -- 172.21.15.57:0/4035884992 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f91a000cb00 con 0x7f91a81f25b0 2023-11-28T12:02:53.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.150+0000 7f91a4ff9700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f91880682f0 0x7f918806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.150+0000 7f91a4ff9700 1 -- 172.21.15.57:0/4035884992 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f91a0085ef0 con 0x7f91a81f25b0 2023-11-28T12:02:53.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.151+0000 7f91a7fff700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f91880682f0 0x7f918806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.151+0000 7f91a7fff700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f91880682f0 0x7f918806a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f91980018b0 tx=0x7f919800a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:53.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.155+0000 7f91a4ff9700 1 -- 172.21.15.57:0/4035884992 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f91a00527b0 con 0x7f91a81f25b0 2023-11-28T12:02:53.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.400+0000 7f91ae7ff700 1 -- 172.21.15.57:0/4035884992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f91a806ad90 con 0x7f91a81f25b0 2023-11-28T12:02:53.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.408+0000 7f91a4ff9700 1 -- 172.21.15.57:0/4035884992 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f91a0055dd0 con 0x7f91a81f25b0 2023-11-28T12:02:53.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.413+0000 7f91ae7ff700 1 -- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f91880682f0 msgr2=0x7f918806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.414+0000 7f91ae7ff700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f91880682f0 0x7f918806a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f91980018b0 tx=0x7f919800a000 comp rx=0 tx=0).stop 2023-11-28T12:02:53.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.414+0000 7f91ae7ff700 1 -- 172.21.15.57:0/4035884992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91a81f25b0 msgr2=0x7f91a81f2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.414+0000 7f91ae7ff700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91a81f25b0 0x7f91a81f2a30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f91a8072a50 tx=0x7f91a000c0e0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.414+0000 7f91ae7ff700 1 -- 172.21.15.57:0/4035884992 shutdown_connections 2023-11-28T12:02:53.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.414+0000 7f91ae7ff700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f91880682f0 0x7f918806a7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.414+0000 7f91ae7ff700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91a81f25b0 0x7f91a81f2a30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.414+0000 7f91ae7ff700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9194003a30 0x7f91a80ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.414+0000 7f91ae7ff700 1 --2- 172.21.15.57:0/4035884992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91a80cce00 0x7f91a81f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.414+0000 7f91ae7ff700 1 -- 172.21.15.57:0/4035884992 >> 172.21.15.57:0/4035884992 conn(0x7f91a806ea80 msgr2=0x7f91a8160900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:53.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.414+0000 7f91ae7ff700 1 -- 172.21.15.57:0/4035884992 shutdown_connections 2023-11-28T12:02:53.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.414+0000 7f91ae7ff700 1 -- 172.21.15.57:0/4035884992 wait complete. 2023-11-28T12:02:53.423 INFO:teuthology.orchestra.run.smithi057.stderr:new fs with metadata pool 5 and data pool 6 2023-11-28T12:02:53.441 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:53.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.574+0000 7feb7d312700 1 Processor -- start 2023-11-28T12:02:53.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.574+0000 7feb7d312700 1 -- start start 2023-11-28T12:02:53.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.575+0000 7feb7d312700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb7806e400 0x7feb780b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.575+0000 7feb7d312700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7feb78160e70 0x7feb7815c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.575+0000 7feb7d312700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7806e900 con 0x7feb780d0c60 2023-11-28T12:02:53.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.575+0000 7feb7d312700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb78160390 con 0x7feb78160e70 2023-11-28T12:02:53.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.575+0000 7feb7d312700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb78160670 con 0x7feb7806e400 2023-11-28T12:02:53.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.575+0000 7feb777fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7feb78160e70 0x7feb7815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.575+0000 7feb76ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7feb780d0c60 0x7feb780d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39628/0 (socket says 172.21.15.57:39628) 2023-11-28T12:02:53.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.575+0000 7feb76ffd700 1 -- 172.21.15.57:0/3240501639 learned_addr learned my addr 172.21.15.57:0/3240501639 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:53.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.576+0000 7feb767fc700 1 --2- 172.21.15.57:0/3240501639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb7806e400 0x7feb780b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.576+0000 7feb777fe700 1 -- 172.21.15.57:0/3240501639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb7806e400 msgr2=0x7feb780b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:53.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.576+0000 7feb777fe700 1 --2- 172.21.15.57:0/3240501639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb7806e400 0x7feb780b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.576+0000 7feb777fe700 1 -- 172.21.15.57:0/3240501639 >> v1:172.21.15.57:6789/0 conn(0x7feb780d0c60 legacy=0x7feb780d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.576+0000 7feb75ffb700 1 -- 172.21.15.57:0/3240501639 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331930827 0 0) 0x7feb7806e900 con 0x7feb780d0c60 2023-11-28T12:02:53.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.576+0000 7feb777fe700 1 -- 172.21.15.57:0/3240501639 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb78160950 con 0x7feb78160e70 2023-11-28T12:02:53.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.576+0000 7feb777fe700 1 --2- 172.21.15.57:0/3240501639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb78160e70 0x7feb7815c970 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7feb70009d90 tx=0x7feb7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0139cd04efeec4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:53.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.576+0000 7feb75ffb700 1 -- 172.21.15.57:0/3240501639 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7000e070 con 0x7feb78160e70 2023-11-28T12:02:53.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.576+0000 7feb75ffb700 1 -- 172.21.15.57:0/3240501639 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7000bbd0 con 0x7feb78160e70 2023-11-28T12:02:53.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.576+0000 7feb75ffb700 1 -- 172.21.15.57:0/3240501639 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70012430 con 0x7feb78160e70 2023-11-28T12:02:53.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.577+0000 7feb7d312700 1 -- 172.21.15.57:0/3240501639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb78160e70 msgr2=0x7feb7815c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.577+0000 7feb7d312700 1 --2- 172.21.15.57:0/3240501639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb78160e70 0x7feb7815c970 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7feb70009d90 tx=0x7feb7000b330 comp rx=0 tx=0).stop 2023-11-28T12:02:53.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.577+0000 7feb7d312700 1 -- 172.21.15.57:0/3240501639 shutdown_connections 2023-11-28T12:02:53.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.577+0000 7feb7d312700 1 --2- 172.21.15.57:0/3240501639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb78160e70 0x7feb7815c970 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.577+0000 7feb7d312700 1 --2- 172.21.15.57:0/3240501639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb7806e400 0x7feb780b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.577+0000 7feb7d312700 1 -- 172.21.15.57:0/3240501639 >> 172.21.15.57:0/3240501639 conn(0x7feb7806a270 msgr2=0x7feb7806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:53.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.577+0000 7feb7d312700 1 -- 172.21.15.57:0/3240501639 shutdown_connections 2023-11-28T12:02:53.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.577+0000 7feb7d312700 1 -- 172.21.15.57:0/3240501639 wait complete. 2023-11-28T12:02:53.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.578+0000 7feb7d312700 1 Processor -- start 2023-11-28T12:02:53.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.578+0000 7feb7d312700 1 -- start start 2023-11-28T12:02:53.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb7d312700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb7806e400 0x7feb781569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb7d312700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb780d0c60 0x7feb78155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb7d312700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb78160e70 0x7feb78153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb7d312700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb780b1820 con 0x7feb7806e400 2023-11-28T12:02:53.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb7d312700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb781562c0 con 0x7feb780d0c60 2023-11-28T12:02:53.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb7d312700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb780b1a70 con 0x7feb78160e70 2023-11-28T12:02:53.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb76ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb7806e400 0x7feb781569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb76ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb7806e400 0x7feb781569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45102/0 (socket says 172.21.15.57:45102) 2023-11-28T12:02:53.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb76ffd700 1 -- 172.21.15.57:0/666290780 learned_addr learned my addr 172.21.15.57:0/666290780 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:53.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb76ffd700 1 -- 172.21.15.57:0/666290780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb78160e70 msgr2=0x7feb78153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb777fe700 1 --2- 172.21.15.57:0/666290780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb78160e70 0x7feb78153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb76ffd700 1 --2- 172.21.15.57:0/666290780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb78160e70 0x7feb78153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb76ffd700 1 -- 172.21.15.57:0/666290780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb780d0c60 msgr2=0x7feb78155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.579+0000 7feb76ffd700 1 --2- 172.21.15.57:0/666290780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb780d0c60 0x7feb78155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.580+0000 7feb76ffd700 1 -- 172.21.15.57:0/666290780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb70009a70 con 0x7feb7806e400 2023-11-28T12:02:53.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.580+0000 7feb76ffd700 1 --2- 172.21.15.57:0/666290780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb7806e400 0x7feb781569e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00c750 tx=0x7feb6c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:53.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.580+0000 7feb5f7fe700 1 -- 172.21.15.57:0/666290780 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c004c80 con 0x7feb7806e400 2023-11-28T12:02:53.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.580+0000 7feb5f7fe700 1 -- 172.21.15.57:0/666290780 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6c004e00 con 0x7feb7806e400 2023-11-28T12:02:53.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.580+0000 7feb7d312700 1 -- 172.21.15.57:0/666290780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7815ceb0 con 0x7feb7806e400 2023-11-28T12:02:53.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.580+0000 7feb5f7fe700 1 -- 172.21.15.57:0/666290780 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c007310 con 0x7feb7806e400 2023-11-28T12:02:53.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.580+0000 7feb7d312700 1 -- 172.21.15.57:0/666290780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb780d0740 con 0x7feb7806e400 2023-11-28T12:02:53.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.581+0000 7feb7d312700 1 -- 172.21.15.57:0/666290780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7815d040 con 0x7feb7806e400 2023-11-28T12:02:53.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.582+0000 7feb5f7fe700 1 -- 172.21.15.57:0/666290780 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feb6c01a020 con 0x7feb7806e400 2023-11-28T12:02:53.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.582+0000 7feb5f7fe700 1 --2- 172.21.15.57:0/666290780 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb640682f0 0x7feb6406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.582+0000 7feb5f7fe700 1 -- 172.21.15.57:0/666290780 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feb6c0859c0 con 0x7feb7806e400 2023-11-28T12:02:53.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.582+0000 7feb767fc700 1 --2- 172.21.15.57:0/666290780 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb640682f0 0x7feb6406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.583+0000 7feb767fc700 1 --2- 172.21.15.57:0/666290780 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb640682f0 0x7feb6406a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb780cf4a0 tx=0x7feb68008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:53.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.586+0000 7feb5f7fe700 1 -- 172.21.15.57:0/666290780 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feb6c0522d0 con 0x7feb7806e400 2023-11-28T12:02:53.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.785+0000 7feb7d312700 1 -- 172.21.15.57:0/666290780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb780d0150 con 0x7feb7806e400 2023-11-28T12:02:53.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.787+0000 7feb5f7fe700 1 -- 172.21.15.57:0/666290780 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+17934 (secure 0 0 0) 0x7feb6c0558f0 con 0x7feb7806e400 2023-11-28T12:02:53.788 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:53.788 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":34,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:02:53.402057+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:02:51.245666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T12:02:52.363863+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:53.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.789+0000 7feb7d312700 1 -- 172.21.15.57:0/666290780 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb640682f0 msgr2=0x7feb6406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.789+0000 7feb7d312700 1 --2- 172.21.15.57:0/666290780 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb640682f0 0x7feb6406a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb780cf4a0 tx=0x7feb68008040 comp rx=0 tx=0).stop 2023-11-28T12:02:53.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.789+0000 7feb7d312700 1 -- 172.21.15.57:0/666290780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb7806e400 msgr2=0x7feb781569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.789+0000 7feb7d312700 1 --2- 172.21.15.57:0/666290780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb7806e400 0x7feb781569e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00c750 tx=0x7feb6c00cb10 comp rx=0 tx=0).stop 2023-11-28T12:02:53.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.789+0000 7feb7d312700 1 -- 172.21.15.57:0/666290780 shutdown_connections 2023-11-28T12:02:53.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.789+0000 7feb7d312700 1 --2- 172.21.15.57:0/666290780 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb640682f0 0x7feb6406a7a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.789+0000 7feb7d312700 1 --2- 172.21.15.57:0/666290780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb78160e70 0x7feb78153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.789+0000 7feb7d312700 1 --2- 172.21.15.57:0/666290780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb7806e400 0x7feb781569e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.789+0000 7feb7d312700 1 --2- 172.21.15.57:0/666290780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb780d0c60 0x7feb78155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.789+0000 7feb7d312700 1 -- 172.21.15.57:0/666290780 >> 172.21.15.57:0/666290780 conn(0x7feb7806a270 msgr2=0x7feb7806cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:53.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.789+0000 7feb7d312700 1 -- 172.21.15.57:0/666290780 shutdown_connections 2023-11-28T12:02:53.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.789+0000 7feb7d312700 1 -- 172.21.15.57:0/666290780 wait complete. 2023-11-28T12:02:53.805 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:53.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.939+0000 7f6b653fa700 1 Processor -- start 2023-11-28T12:02:53.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.939+0000 7f6b653fa700 1 -- start start 2023-11-28T12:02:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.940+0000 7f6b653fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b60071f70 0x7f6b600cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.940+0000 7f6b653fa700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f6b600b4a30 0x7f6b600b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.940+0000 7f6b653fa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b600cfb00 con 0x7f6b600b4660 2023-11-28T12:02:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.940+0000 7f6b653fa700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6015aa20 con 0x7f6b600b4a30 2023-11-28T12:02:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.940+0000 7f6b653fa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b600cf4d0 con 0x7f6b60071f70 2023-11-28T12:02:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.940+0000 7f6b5f7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f6b600b4a30 0x7f6b600b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.940+0000 7f6b5f7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f6b600b4a30 0x7f6b600b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45112/0 (socket says 172.21.15.57:45112) 2023-11-28T12:02:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.940+0000 7f6b5f7fe700 1 -- 172.21.15.57:0/982796674 learned_addr learned my addr 172.21.15.57:0/982796674 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.940+0000 7f6b5e7fc700 1 --2- 172.21.15.57:0/982796674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b60071f70 0x7f6b600cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.941+0000 7f6b5f7fe700 1 -- 172.21.15.57:0/982796674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b60071f70 msgr2=0x7f6b600cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.941+0000 7f6b5f7fe700 1 --2- 172.21.15.57:0/982796674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b60071f70 0x7f6b600cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.941+0000 7f6b5f7fe700 1 -- 172.21.15.57:0/982796674 >> v1:172.21.15.181:6789/0 conn(0x7f6b600b4660 legacy=0x7f6b6006b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.941+0000 7f6b5f7fe700 1 -- 172.21.15.57:0/982796674 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b60072550 con 0x7f6b600b4a30 2023-11-28T12:02:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.941+0000 7f6b5f7fe700 1 --2- 172.21.15.57:0/982796674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b600b4a30 0x7f6b600b4eb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009e50 tx=0x7f6b5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69ba293ed318348 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.941+0000 7f6b5dffb700 1 -- 172.21.15.57:0/982796674 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800e070 con 0x7f6b600b4a30 2023-11-28T12:02:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.941+0000 7f6b5dffb700 1 -- 172.21.15.57:0/982796674 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5800b910 con 0x7f6b600b4a30 2023-11-28T12:02:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.941+0000 7f6b5dffb700 1 -- 172.21.15.57:0/982796674 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800bcc0 con 0x7f6b600b4a30 2023-11-28T12:02:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.942+0000 7f6b653fa700 1 -- 172.21.15.57:0/982796674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b600b4a30 msgr2=0x7f6b600b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.942+0000 7f6b653fa700 1 --2- 172.21.15.57:0/982796674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b600b4a30 0x7f6b600b4eb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009e50 tx=0x7f6b5800b330 comp rx=0 tx=0).stop 2023-11-28T12:02:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.942+0000 7f6b653fa700 1 -- 172.21.15.57:0/982796674 shutdown_connections 2023-11-28T12:02:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.942+0000 7f6b653fa700 1 --2- 172.21.15.57:0/982796674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b600b4a30 0x7f6b600b4eb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.942+0000 7f6b653fa700 1 --2- 172.21.15.57:0/982796674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b60071f70 0x7f6b600cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.942+0000 7f6b653fa700 1 -- 172.21.15.57:0/982796674 >> 172.21.15.57:0/982796674 conn(0x7f6b600b2fc0 msgr2=0x7f6b60071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.942+0000 7f6b653fa700 1 -- 172.21.15.57:0/982796674 shutdown_connections 2023-11-28T12:02:53.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.942+0000 7f6b653fa700 1 -- 172.21.15.57:0/982796674 wait complete. 2023-11-28T12:02:53.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.943+0000 7f6b653fa700 1 Processor -- start 2023-11-28T12:02:53.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.943+0000 7f6b653fa700 1 -- start start 2023-11-28T12:02:53.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b653fa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b60071f70 0x7f6b60157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b653fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b600b4660 0x7f6b60159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b653fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b600b4a30 0x7f6b600afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b653fa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b60159ab0 con 0x7f6b60071f70 2023-11-28T12:02:53.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b653fa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b60159d90 con 0x7f6b600b4a30 2023-11-28T12:02:53.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b653fa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6015a070 con 0x7f6b600b4660 2023-11-28T12:02:53.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b5effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b60071f70 0x7f6b60157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b5effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b60071f70 0x7f6b60157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45116/0 (socket says 172.21.15.57:45116) 2023-11-28T12:02:53.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b5effd700 1 -- 172.21.15.57:0/3671185873 learned_addr learned my addr 172.21.15.57:0/3671185873 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:53.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b5e7fc700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b600b4660 0x7f6b60159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b5f7fe700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b600b4a30 0x7f6b600afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b5effd700 1 -- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b600b4660 msgr2=0x7f6b60159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b5effd700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b600b4660 0x7f6b60159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b5effd700 1 -- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b600b4a30 msgr2=0x7f6b600afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:53.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b5effd700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b600b4a30 0x7f6b600afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:53.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.944+0000 7f6b5effd700 1 -- 172.21.15.57:0/3671185873 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b58009b30 con 0x7f6b60071f70 2023-11-28T12:02:53.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.945+0000 7f6b5effd700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b60071f70 0x7f6b60157660 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000c4e0 tx=0x7f6b5000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:53.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.945+0000 7f6b577fe700 1 -- 172.21.15.57:0/3671185873 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b500052d0 con 0x7f6b60071f70 2023-11-28T12:02:53.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.945+0000 7f6b577fe700 1 -- 172.21.15.57:0/3671185873 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b50005430 con 0x7f6b60071f70 2023-11-28T12:02:53.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.945+0000 7f6b653fa700 1 -- 172.21.15.57:0/3671185873 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6015f170 con 0x7f6b60071f70 2023-11-28T12:02:53.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.945+0000 7f6b577fe700 1 -- 172.21.15.57:0/3671185873 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b500038c0 con 0x7f6b60071f70 2023-11-28T12:02:53.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.945+0000 7f6b653fa700 1 -- 172.21.15.57:0/3671185873 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b600cf790 con 0x7f6b60071f70 2023-11-28T12:02:53.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.946+0000 7f6b653fa700 1 -- 172.21.15.57:0/3671185873 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6015f300 con 0x7f6b60071f70 2023-11-28T12:02:53.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.946+0000 7f6b577fe700 1 -- 172.21.15.57:0/3671185873 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6b50019030 con 0x7f6b60071f70 2023-11-28T12:02:53.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.947+0000 7f6b577fe700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6b40068320 0x7f6b4006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:53.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.947+0000 7f6b577fe700 1 -- 172.21.15.57:0/3671185873 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6b50084fe0 con 0x7f6b60071f70 2023-11-28T12:02:53.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.947+0000 7f6b5e7fc700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6b40068320 0x7f6b4006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:53.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.947+0000 7f6b5e7fc700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6b40068320 0x7f6b4006a7d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6b601561d0 tx=0x7f6b48009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:53.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:53.951+0000 7f6b577fe700 1 -- 172.21.15.57:0/3671185873 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6b50051750 con 0x7f6b60071f70 2023-11-28T12:02:54.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.151+0000 7f6b653fa700 1 -- 172.21.15.57:0/3671185873 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b600743f0 con 0x7f6b60071f70 2023-11-28T12:02:54.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.152+0000 7f6b577fe700 1 -- 172.21.15.57:0/3671185873 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+17934 (secure 0 0 0) 0x7f6b50017090 con 0x7f6b60071f70 2023-11-28T12:02:54.155 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:54.155 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":34,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:02:53.402057+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:02:51.245666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T12:02:52.363863+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:54.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.154+0000 7f6b653fa700 1 -- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6b40068320 msgr2=0x7f6b4006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:54.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.154+0000 7f6b653fa700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6b40068320 0x7f6b4006a7d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6b601561d0 tx=0x7f6b48009210 comp rx=0 tx=0).stop 2023-11-28T12:02:54.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.154+0000 7f6b653fa700 1 -- 172.21.15.57:0/3671185873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b60071f70 msgr2=0x7f6b60157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:54.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.154+0000 7f6b653fa700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b60071f70 0x7f6b60157660 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000c4e0 tx=0x7f6b5000c7f0 comp rx=0 tx=0).stop 2023-11-28T12:02:54.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.154+0000 7f6b653fa700 1 -- 172.21.15.57:0/3671185873 shutdown_connections 2023-11-28T12:02:54.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.154+0000 7f6b653fa700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6b40068320 0x7f6b4006a7d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:54.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.154+0000 7f6b653fa700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b600b4a30 0x7f6b600afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:54.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.154+0000 7f6b653fa700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b60071f70 0x7f6b60157660 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:54.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.154+0000 7f6b653fa700 1 --2- 172.21.15.57:0/3671185873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b600b4660 0x7f6b60159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:54.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.154+0000 7f6b653fa700 1 -- 172.21.15.57:0/3671185873 >> 172.21.15.57:0/3671185873 conn(0x7f6b600b2fc0 msgr2=0x7f6b6006d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:54.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.154+0000 7f6b653fa700 1 -- 172.21.15.57:0/3671185873 shutdown_connections 2023-11-28T12:02:54.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.154+0000 7f6b653fa700 1 -- 172.21.15.57:0/3671185873 wait complete. 2023-11-28T12:02:54.169 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T12:02:54.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.307+0000 7f8cab1b8700 1 Processor -- start 2023-11-28T12:02:54.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.307+0000 7f8cab1b8700 1 -- start start 2023-11-28T12:02:54.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.308+0000 7f8cab1b8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ca415a4d0 0x7f8ca406a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:54.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.308+0000 7f8cab1b8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8ca40cdf20 0x7f8ca40cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:54.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.308+0000 7f8cab1b8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca406ac00 con 0x7f8ca406a7b0 2023-11-28T12:02:54.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.308+0000 7f8cab1b8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca40ce360 con 0x7f8ca40cdf20 2023-11-28T12:02:54.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.308+0000 7f8cab1b8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca415e0d0 con 0x7f8ca415a4d0 2023-11-28T12:02:54.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.308+0000 7f8ca8f54700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ca415a4d0 0x7f8ca406a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:54.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.308+0000 7f8ca9755700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8ca40cdf20 0x7f8ca40cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:54.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.308+0000 7f8ca8f54700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ca415a4d0 0x7f8ca406a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:46868/0 (socket says 172.21.15.57:46868) 2023-11-28T12:02:54.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.308+0000 7f8ca9755700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8ca40cdf20 0x7f8ca40cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:37776/0 (socket says 172.21.15.57:37776) 2023-11-28T12:02:54.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.308+0000 7f8ca8f54700 1 -- 172.21.15.57:0/3638289883 learned_addr learned my addr 172.21.15.57:0/3638289883 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:54.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.308+0000 7f8ca3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8ca406a7b0 0x7f8ca40cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:58516/0 (socket says 172.21.15.57:58516) 2023-11-28T12:02:54.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.308+0000 7f8ca8f54700 1 -- 172.21.15.57:0/3638289883 >> v2:172.21.15.181:3300/0 conn(0x7f8ca40cdf20 msgr2=0x7f8ca40cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:54.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.308+0000 7f8ca8f54700 1 --2- 172.21.15.57:0/3638289883 >> v2:172.21.15.181:3300/0 conn(0x7f8ca40cdf20 0x7f8ca40cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:54.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.309+0000 7f8ca8f54700 1 -- 172.21.15.57:0/3638289883 >> v1:172.21.15.181:6789/0 conn(0x7f8ca406a7b0 legacy=0x7f8ca40cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:54.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.309+0000 7f8ca8f54700 1 -- 172.21.15.57:0/3638289883 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca40ce560 con 0x7f8ca415a4d0 2023-11-28T12:02:54.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.309+0000 7f8ca9755700 1 --2- 172.21.15.57:0/3638289883 >> v2:172.21.15.181:3300/0 conn(0x7f8ca40cdf20 0x7f8ca40cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:54.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.309+0000 7f8ca8f54700 1 --2- 172.21.15.57:0/3638289883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ca415a4d0 0x7f8ca406a270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c980098f0 tx=0x7f8c98009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f0a057979ef147 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:54.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.309+0000 7f8ca37fe700 1 -- 172.21.15.57:0/3638289883 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98011830 con 0x7f8ca415a4d0 2023-11-28T12:02:54.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.309+0000 7f8ca37fe700 1 -- 172.21.15.57:0/3638289883 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c98011990 con 0x7f8ca415a4d0 2023-11-28T12:02:54.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.309+0000 7f8ca37fe700 1 -- 172.21.15.57:0/3638289883 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98011d20 con 0x7f8ca415a4d0 2023-11-28T12:02:54.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.310+0000 7f8cab1b8700 1 -- 172.21.15.57:0/3638289883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ca415a4d0 msgr2=0x7f8ca406a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:54.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.310+0000 7f8cab1b8700 1 --2- 172.21.15.57:0/3638289883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ca415a4d0 0x7f8ca406a270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c980098f0 tx=0x7f8c98009c00 comp rx=0 tx=0).stop 2023-11-28T12:02:54.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.310+0000 7f8cab1b8700 1 -- 172.21.15.57:0/3638289883 shutdown_connections 2023-11-28T12:02:54.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.310+0000 7f8cab1b8700 1 --2- 172.21.15.57:0/3638289883 >> v2:172.21.15.181:3300/0 conn(0x7f8ca40cdf20 0x7f8ca40cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:54.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.310+0000 7f8cab1b8700 1 --2- 172.21.15.57:0/3638289883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ca415a4d0 0x7f8ca406a270 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:54.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.310+0000 7f8cab1b8700 1 -- 172.21.15.57:0/3638289883 >> 172.21.15.57:0/3638289883 conn(0x7f8ca406c010 msgr2=0x7f8ca40cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:54.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.310+0000 7f8cab1b8700 1 -- 172.21.15.57:0/3638289883 shutdown_connections 2023-11-28T12:02:54.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.310+0000 7f8cab1b8700 1 -- 172.21.15.57:0/3638289883 wait complete. 2023-11-28T12:02:54.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.311+0000 7f8cab1b8700 1 Processor -- start 2023-11-28T12:02:54.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.311+0000 7f8cab1b8700 1 -- start start 2023-11-28T12:02:54.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8cab1b8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca406a7b0 0x7f8ca406c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:54.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8cab1b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ca40cdf20 0x7f8ca414f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:54.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8cab1b8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ca415a4d0 0x7f8ca40c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:54.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8cab1b8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40b54e0 con 0x7f8ca406a7b0 2023-11-28T12:02:54.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8cab1b8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca406cf80 con 0x7f8ca40cdf20 2023-11-28T12:02:54.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8cab1b8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca406d260 con 0x7f8ca415a4d0 2023-11-28T12:02:54.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8ca8f54700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca406a7b0 0x7f8ca406c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:54.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8ca8f54700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca406a7b0 0x7f8ca406c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45130/0 (socket says 172.21.15.57:45130) 2023-11-28T12:02:54.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8ca8f54700 1 -- 172.21.15.57:0/4185585037 learned_addr learned my addr 172.21.15.57:0/4185585037 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:54.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8ca9755700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ca415a4d0 0x7f8ca40c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:54.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8ca3fff700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ca40cdf20 0x7f8ca414f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:54.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8ca8f54700 1 -- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ca415a4d0 msgr2=0x7f8ca40c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:54.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8ca8f54700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ca415a4d0 0x7f8ca40c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:54.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8ca8f54700 1 -- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ca40cdf20 msgr2=0x7f8ca414f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:54.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8ca8f54700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ca40cdf20 0x7f8ca414f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:54.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8ca8f54700 1 -- 172.21.15.57:0/4185585037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c009a70 con 0x7f8ca406a7b0 2023-11-28T12:02:54.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.312+0000 7f8ca9755700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ca415a4d0 0x7f8ca40c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:54.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.313+0000 7f8ca8f54700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca406a7b0 0x7f8ca406c880 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8c98000c00 tx=0x7f8c98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:54.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.313+0000 7f8ca17fa700 1 -- 172.21.15.57:0/4185585037 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98004530 con 0x7f8ca406a7b0 2023-11-28T12:02:54.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.313+0000 7f8ca17fa700 1 -- 172.21.15.57:0/4185585037 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9801f070 con 0x7f8ca406a7b0 2023-11-28T12:02:54.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.313+0000 7f8cab1b8700 1 -- 172.21.15.57:0/4185585037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c98009550 con 0x7f8ca406a7b0 2023-11-28T12:02:54.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.313+0000 7f8ca17fa700 1 -- 172.21.15.57:0/4185585037 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c980038e0 con 0x7f8ca406a7b0 2023-11-28T12:02:54.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.313+0000 7f8cab1b8700 1 -- 172.21.15.57:0/4185585037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca415e600 con 0x7f8ca406a7b0 2023-11-28T12:02:54.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.314+0000 7f8cab1b8700 1 -- 172.21.15.57:0/4185585037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca415e320 con 0x7f8ca406a7b0 2023-11-28T12:02:54.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.314+0000 7f8ca17fa700 1 -- 172.21.15.57:0/4185585037 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8c9802a020 con 0x7f8ca406a7b0 2023-11-28T12:02:54.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.315+0000 7f8ca17fa700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c880682f0 0x7f8c8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:54.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.315+0000 7f8ca17fa700 1 -- 172.21.15.57:0/4185585037 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8c98087290 con 0x7f8ca406a7b0 2023-11-28T12:02:54.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.315+0000 7f8ca3fff700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c880682f0 0x7f8c8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:54.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.316+0000 7f8ca3fff700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c880682f0 0x7f8c8806a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c90000f60 tx=0x7f8c90009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:54.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.319+0000 7f8ca17fa700 1 -- 172.21.15.57:0/4185585037 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8c98053af0 con 0x7f8ca406a7b0 2023-11-28T12:02:54.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:54.569+0000 7f8cab1b8700 1 -- 172.21.15.57:0/4185585037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8ca415e8e0 con 0x7f8ca406a7b0 2023-11-28T12:02:55.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.420+0000 7f8ca17fa700 1 -- 172.21.15.57:0/4185585037 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8c98057110 con 0x7f8ca406a7b0 2023-11-28T12:02:55.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.423+0000 7f8cab1b8700 1 -- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c880682f0 msgr2=0x7f8c8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:55.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.423+0000 7f8cab1b8700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c880682f0 0x7f8c8806a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c90000f60 tx=0x7f8c90009040 comp rx=0 tx=0).stop 2023-11-28T12:02:55.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.423+0000 7f8cab1b8700 1 -- 172.21.15.57:0/4185585037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca406a7b0 msgr2=0x7f8ca406c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:55.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.423+0000 7f8cab1b8700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca406a7b0 0x7f8ca406c880 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8c98000c00 tx=0x7f8c98004750 comp rx=0 tx=0).stop 2023-11-28T12:02:55.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.423+0000 7f8cab1b8700 1 -- 172.21.15.57:0/4185585037 shutdown_connections 2023-11-28T12:02:55.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.423+0000 7f8cab1b8700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c880682f0 0x7f8c8806a7a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:55.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.423+0000 7f8cab1b8700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ca415a4d0 0x7f8ca40c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:55.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.423+0000 7f8cab1b8700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca406a7b0 0x7f8ca406c880 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:55.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.423+0000 7f8cab1b8700 1 --2- 172.21.15.57:0/4185585037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ca40cdf20 0x7f8ca414f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:55.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.423+0000 7f8cab1b8700 1 -- 172.21.15.57:0/4185585037 >> 172.21.15.57:0/4185585037 conn(0x7f8ca406c010 msgr2=0x7f8ca40b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:55.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.423+0000 7f8cab1b8700 1 -- 172.21.15.57:0/4185585037 shutdown_connections 2023-11-28T12:02:55.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.423+0000 7f8cab1b8700 1 -- 172.21.15.57:0/4185585037 wait complete. 2023-11-28T12:02:55.440 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T12:02:55.440 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T12:02:55.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.582+0000 7f38dce72700 1 Processor -- start 2023-11-28T12:02:55.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.582+0000 7f38dce72700 1 -- start start 2023-11-28T12:02:55.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.583+0000 7f38dce72700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f38d80af0a0 0x7f38d815a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:55.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.583+0000 7f38dce72700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d8159ec0 con 0x7f38d81599b0 2023-11-28T12:02:55.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.583+0000 7f38dce72700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d815d8a0 con 0x7f38d80b1650 2023-11-28T12:02:55.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.583+0000 7f38dce72700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d815db80 con 0x7f38d80af0a0 2023-11-28T12:02:55.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.583+0000 7f38d6d9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f38d80af0a0 0x7f38d815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:55.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.583+0000 7f38d5d9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f38d81599b0 0x7f38d815d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47364/0 (socket says 172.21.15.57:47364) 2023-11-28T12:02:55.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.583+0000 7f38d5d9b700 1 -- 172.21.15.57:0/106767926 learned_addr learned my addr 172.21.15.57:0/106767926 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:55.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.583+0000 7f38d6d9d700 1 -- 172.21.15.57:0/106767926 >> v1:172.21.15.181:6789/0 conn(0x7f38d80b1650 legacy=0x7f38d80b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:55.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.583+0000 7f38d6d9d700 1 -- 172.21.15.57:0/106767926 >> v1:172.21.15.57:6789/0 conn(0x7f38d81599b0 legacy=0x7f38d815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:55.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.583+0000 7f38d6d9d700 1 -- 172.21.15.57:0/106767926 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d815de60 con 0x7f38d80af0a0 2023-11-28T12:02:55.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.583+0000 7f38d559a700 1 -- 172.21.15.57:0/106767926 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045823428 0 0) 0x7f38d8159ec0 con 0x7f38d81599b0 2023-11-28T12:02:55.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.584+0000 7f38d6d9d700 1 --2- 172.21.15.57:0/106767926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d80af0a0 0x7f38d815a350 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009d90 tx=0x7f38d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44b6513f287de08 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:55.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.584+0000 7f38d559a700 1 -- 172.21.15.57:0/106767926 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d000e070 con 0x7f38d80af0a0 2023-11-28T12:02:55.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.584+0000 7f38d559a700 1 -- 172.21.15.57:0/106767926 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d000bb90 con 0x7f38d80af0a0 2023-11-28T12:02:55.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.584+0000 7f38d559a700 1 -- 172.21.15.57:0/106767926 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0003a20 con 0x7f38d80af0a0 2023-11-28T12:02:55.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.584+0000 7f38dce72700 1 -- 172.21.15.57:0/106767926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d80af0a0 msgr2=0x7f38d815a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:55.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.584+0000 7f38dce72700 1 --2- 172.21.15.57:0/106767926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d80af0a0 0x7f38d815a350 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009d90 tx=0x7f38d000b330 comp rx=0 tx=0).stop 2023-11-28T12:02:55.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.584+0000 7f38dce72700 1 -- 172.21.15.57:0/106767926 shutdown_connections 2023-11-28T12:02:55.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.584+0000 7f38dce72700 1 --2- 172.21.15.57:0/106767926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d80af0a0 0x7f38d815a350 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:55.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.584+0000 7f38dce72700 1 -- 172.21.15.57:0/106767926 >> 172.21.15.57:0/106767926 conn(0x7f38d806c010 msgr2=0x7f38d80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:55.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.584+0000 7f38dce72700 1 -- 172.21.15.57:0/106767926 shutdown_connections 2023-11-28T12:02:55.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.584+0000 7f38dce72700 1 -- 172.21.15.57:0/106767926 wait complete. 2023-11-28T12:02:55.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.586+0000 7f38dce72700 1 Processor -- start 2023-11-28T12:02:55.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.586+0000 7f38dce72700 1 -- start start 2023-11-28T12:02:55.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38dce72700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38d80af0a0 0x7f38d8152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:55.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38dce72700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d80b1650 0x7f38d80b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:55.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38dce72700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d8155870 0x7f38d80af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:55.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38dce72700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80b55a0 con 0x7f38d80b1650 2023-11-28T12:02:55.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38dce72700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80b4990 con 0x7f38d8155870 2023-11-28T12:02:55.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38dce72700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80b4c70 con 0x7f38d80af0a0 2023-11-28T12:02:55.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38d5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d80b1650 0x7f38d80b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:55.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38d5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d80b1650 0x7f38d80b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55844/0 (socket says 172.21.15.57:55844) 2023-11-28T12:02:55.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38d5d9b700 1 -- 172.21.15.57:0/3321837132 learned_addr learned my addr 172.21.15.57:0/3321837132 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:55.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38d659c700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38d80af0a0 0x7f38d8152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:55.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38d6d9d700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d8155870 0x7f38d80af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:55.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38d5d9b700 1 -- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38d80af0a0 msgr2=0x7f38d8152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:55.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38d5d9b700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38d80af0a0 0x7f38d8152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:55.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38d5d9b700 1 -- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d8155870 msgr2=0x7f38d80af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:55.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38d5d9b700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d8155870 0x7f38d80af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:55.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38d5d9b700 1 -- 172.21.15.57:0/3321837132 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d0009a70 con 0x7f38d80b1650 2023-11-28T12:02:55.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.587+0000 7f38d5d9b700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d80b1650 0x7f38d80b3f90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c460 tx=0x7f38cc00c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:55.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.588+0000 7f38c6ffd700 1 -- 172.21.15.57:0/3321837132 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc004c80 con 0x7f38d80b1650 2023-11-28T12:02:55.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.588+0000 7f38c6ffd700 1 -- 172.21.15.57:0/3321837132 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc004e00 con 0x7f38d80b1650 2023-11-28T12:02:55.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.588+0000 7f38dce72700 1 -- 172.21.15.57:0/3321837132 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d815aa20 con 0x7f38d80b1650 2023-11-28T12:02:55.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.588+0000 7f38c6ffd700 1 -- 172.21.15.57:0/3321837132 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc007310 con 0x7f38d80b1650 2023-11-28T12:02:55.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.588+0000 7f38dce72700 1 -- 172.21.15.57:0/3321837132 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d815a890 con 0x7f38d80b1650 2023-11-28T12:02:55.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.589+0000 7f38dce72700 1 -- 172.21.15.57:0/3321837132 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d80743f0 con 0x7f38d80b1650 2023-11-28T12:02:55.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.589+0000 7f38c6ffd700 1 -- 172.21.15.57:0/3321837132 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f38cc01a020 con 0x7f38d80b1650 2023-11-28T12:02:55.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.589+0000 7f38c6ffd700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f38b80682f0 0x7f38b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:55.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.590+0000 7f38c6ffd700 1 -- 172.21.15.57:0/3321837132 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38cc085690 con 0x7f38d80b1650 2023-11-28T12:02:55.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.590+0000 7f38d659c700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f38b80682f0 0x7f38b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:55.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.590+0000 7f38d659c700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f38b80682f0 0x7f38b806a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f38d80cfaa0 tx=0x7f38c8007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:55.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.594+0000 7f38c6ffd700 1 -- 172.21.15.57:0/3321837132 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f38cc051ef0 con 0x7f38d80b1650 2023-11-28T12:02:55.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:55.842+0000 7f38dce72700 1 -- 172.21.15.57:0/3321837132 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f38d806cd10 con 0x7f38d80b1650 2023-11-28T12:02:55.915 INFO:tasks.ceph.mds.a.smithi057.stderr:2023-11-28T12:02:55.914+0000 7f8a54e21700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T12:02:56.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.547+0000 7f38c6ffd700 1 -- 172.21.15.57:0/3321837132 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f38cc055510 con 0x7f38d80b1650 2023-11-28T12:02:56.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.550+0000 7f38dce72700 1 -- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f38b80682f0 msgr2=0x7f38b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.550+0000 7f38dce72700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f38b80682f0 0x7f38b806a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f38d80cfaa0 tx=0x7f38c8007ec0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.550+0000 7f38dce72700 1 -- 172.21.15.57:0/3321837132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d80b1650 msgr2=0x7f38d80b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.550+0000 7f38dce72700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d80b1650 0x7f38d80b3f90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c460 tx=0x7f38cc00c770 comp rx=0 tx=0).stop 2023-11-28T12:02:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.550+0000 7f38dce72700 1 -- 172.21.15.57:0/3321837132 shutdown_connections 2023-11-28T12:02:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.550+0000 7f38dce72700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f38b80682f0 0x7f38b806a7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.550+0000 7f38dce72700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d8155870 0x7f38d80af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.550+0000 7f38dce72700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38d80af0a0 0x7f38d8152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.550+0000 7f38dce72700 1 --2- 172.21.15.57:0/3321837132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38d80b1650 0x7f38d80b3f90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.550+0000 7f38dce72700 1 -- 172.21.15.57:0/3321837132 >> 172.21.15.57:0/3321837132 conn(0x7f38d806c010 msgr2=0x7f38d80b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.550+0000 7f38dce72700 1 -- 172.21.15.57:0/3321837132 shutdown_connections 2023-11-28T12:02:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.550+0000 7f38dce72700 1 -- 172.21.15.57:0/3321837132 wait complete. 2023-11-28T12:02:56.567 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T12:02:56.567 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:02:56.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.705+0000 7f4bfb8a5700 1 Processor -- start 2023-11-28T12:02:56.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.705+0000 7f4bfb8a5700 1 -- start start 2023-11-28T12:02:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.706+0000 7f4bfb8a5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf415cc80 0x7f4bf406de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.706+0000 7f4bfb8a5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4bf406e3a0 0x7f4bf4070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.706+0000 7f4bfb8a5700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf40b1810 con 0x7f4bf406a270 2023-11-28T12:02:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.706+0000 7f4bfb8a5700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf40b2490 con 0x7f4bf406e3a0 2023-11-28T12:02:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.706+0000 7f4bfb8a5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40b1af0 con 0x7f4bf415cc80 2023-11-28T12:02:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.706+0000 7f4bf9641700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4bf406a270 0x7f4bf406a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47376/0 (socket says 172.21.15.57:47376) 2023-11-28T12:02:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.706+0000 7f4bf9641700 1 -- 172.21.15.57:0/36427000 learned_addr learned my addr 172.21.15.57:0/36427000 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.706+0000 7f4bf9e42700 1 --2- 172.21.15.57:0/36427000 >> v2:172.21.15.57:3300/0 conn(0x7f4bf406e3a0 0x7f4bf4070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.706+0000 7f4bf8e40700 1 --2- 172.21.15.57:0/36427000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf415cc80 0x7f4bf406de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.707+0000 7f4bf9e42700 1 -- 172.21.15.57:0/36427000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf415cc80 msgr2=0x7f4bf406de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.707+0000 7f4bf9e42700 1 --2- 172.21.15.57:0/36427000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf415cc80 0x7f4bf406de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.707+0000 7f4bf9e42700 1 -- 172.21.15.57:0/36427000 >> v1:172.21.15.57:6789/0 conn(0x7f4bf406a270 legacy=0x7f4bf406a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.707+0000 7f4bf9e42700 1 -- 172.21.15.57:0/36427000 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf40b1dd0 con 0x7f4bf406e3a0 2023-11-28T12:02:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.707+0000 7f4bf3fff700 1 -- 172.21.15.57:0/36427000 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357079741 0 0) 0x7f4bf40b1810 con 0x7f4bf406a270 2023-11-28T12:02:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.707+0000 7f4bf9e42700 1 --2- 172.21.15.57:0/36427000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bf406e3a0 0x7f4bf4070d20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009840 tx=0x7f4bec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfc724ae57721b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.707+0000 7f4bf3fff700 1 -- 172.21.15.57:0/36427000 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec011830 con 0x7f4bf406e3a0 2023-11-28T12:02:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.707+0000 7f4bf3fff700 1 -- 172.21.15.57:0/36427000 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec011990 con 0x7f4bf406e3a0 2023-11-28T12:02:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.707+0000 7f4bf3fff700 1 -- 172.21.15.57:0/36427000 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec011d20 con 0x7f4bf406e3a0 2023-11-28T12:02:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.708+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/36427000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bf406e3a0 msgr2=0x7f4bf4070d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:56.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.708+0000 7f4bfb8a5700 1 --2- 172.21.15.57:0/36427000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bf406e3a0 0x7f4bf4070d20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009840 tx=0x7f4bec009b50 comp rx=0 tx=0).stop 2023-11-28T12:02:56.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.708+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/36427000 shutdown_connections 2023-11-28T12:02:56.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.708+0000 7f4bfb8a5700 1 --2- 172.21.15.57:0/36427000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bf406e3a0 0x7f4bf4070d20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.708+0000 7f4bfb8a5700 1 --2- 172.21.15.57:0/36427000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf415cc80 0x7f4bf406de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.708+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/36427000 >> 172.21.15.57:0/36427000 conn(0x7f4bf406c010 msgr2=0x7f4bf40cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:56.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.708+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/36427000 shutdown_connections 2023-11-28T12:02:56.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.708+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/36427000 wait complete. 2023-11-28T12:02:56.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.709+0000 7f4bfb8a5700 1 Processor -- start 2023-11-28T12:02:56.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.709+0000 7f4bfb8a5700 1 -- start start 2023-11-28T12:02:56.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bfb8a5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bf406a270 0x7f4bf4152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:56.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bfb8a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf406e3a0 0x7f4bf40b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:56.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bfb8a5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf415cc80 0x7f4bf40b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:56.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bfb8a5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40c9120 con 0x7f4bf406a270 2023-11-28T12:02:56.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bfb8a5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40c9980 con 0x7f4bf406e3a0 2023-11-28T12:02:56.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bfb8a5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40b57e0 con 0x7f4bf415cc80 2023-11-28T12:02:56.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bf9641700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bf406a270 0x7f4bf4152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:56.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bf9641700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bf406a270 0x7f4bf4152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55868/0 (socket says 172.21.15.57:55868) 2023-11-28T12:02:56.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bf9641700 1 -- 172.21.15.57:0/2826073782 learned_addr learned my addr 172.21.15.57:0/2826073782 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:56.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bf9e42700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf415cc80 0x7f4bf40b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:56.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bf8e40700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf406e3a0 0x7f4bf40b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:56.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bf9641700 1 -- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf415cc80 msgr2=0x7f4bf40b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:56.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bf9641700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf415cc80 0x7f4bf40b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bf9641700 1 -- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf406e3a0 msgr2=0x7f4bf40b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:56.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bf9641700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf406e3a0 0x7f4bf40b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bf9641700 1 -- 172.21.15.57:0/2826073782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec0094a0 con 0x7f4bf406a270 2023-11-28T12:02:56.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.710+0000 7f4bf9641700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bf406a270 0x7f4bf4152e00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4be400cc20 tx=0x7f4be400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:56.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.711+0000 7f4bf1ffb700 1 -- 172.21.15.57:0/2826073782 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be400ac20 con 0x7f4bf406a270 2023-11-28T12:02:56.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.711+0000 7f4bf1ffb700 1 -- 172.21.15.57:0/2826073782 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be400ad80 con 0x7f4bf406a270 2023-11-28T12:02:56.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.711+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/2826073782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf406e7e0 con 0x7f4bf406a270 2023-11-28T12:02:56.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.711+0000 7f4bf1ffb700 1 -- 172.21.15.57:0/2826073782 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4007310 con 0x7f4bf406a270 2023-11-28T12:02:56.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.711+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/2826073782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf40b20b0 con 0x7f4bf406a270 2023-11-28T12:02:56.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.712+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/2826073782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf415daa0 con 0x7f4bf406a270 2023-11-28T12:02:56.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.712+0000 7f4bf1ffb700 1 -- 172.21.15.57:0/2826073782 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4be4010070 con 0x7f4bf406a270 2023-11-28T12:02:56.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.713+0000 7f4bf1ffb700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4be00682f0 0x7f4be006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:56.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.713+0000 7f4bf1ffb700 1 -- 172.21.15.57:0/2826073782 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4be4085660 con 0x7f4bf406a270 2023-11-28T12:02:56.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.713+0000 7f4bf8e40700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4be00682f0 0x7f4be006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:56.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.714+0000 7f4bf8e40700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4be00682f0 0x7f4be006a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4bf41f5ec0 tx=0x7f4be8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:56.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.717+0000 7f4bf1ffb700 1 -- 172.21.15.57:0/2826073782 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4be4051ec0 con 0x7f4bf406a270 2023-11-28T12:02:56.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.966+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/2826073782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bf40743f0 con 0x7f4bf406a270 2023-11-28T12:02:56.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.966+0000 7f4bf1ffb700 1 -- 172.21.15.57:0/2826073782 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f4be40554e0 con 0x7f4bf406a270 2023-11-28T12:02:56.968 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:56.968 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/340292109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":340292109},{"type":"v1","addr":"172.21.15.181:6836","nonce":340292109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2442321825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2442321825},{"type":"v1","addr":"172.21.15.181:6837","nonce":2442321825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/3452242211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3452242211},{"type":"v1","addr":"172.21.15.57:6833","nonce":3452242211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:02:53.402391+0000","modified":"2023-11-28T12:02:56.540229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.57:6835/968299041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":968299041},{"type":"v1","addr":"172.21.15.57:6835","nonce":968299041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T12:02:56.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.969+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4be00682f0 msgr2=0x7f4be006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:56.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.969+0000 7f4bfb8a5700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4be00682f0 0x7f4be006a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4bf41f5ec0 tx=0x7f4be8008040 comp rx=0 tx=0).stop 2023-11-28T12:02:56.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.969+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/2826073782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bf406a270 msgr2=0x7f4bf4152e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:56.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.969+0000 7f4bfb8a5700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bf406a270 0x7f4bf4152e00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4be400cc20 tx=0x7f4be400a060 comp rx=0 tx=0).stop 2023-11-28T12:02:56.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.969+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/2826073782 shutdown_connections 2023-11-28T12:02:56.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.969+0000 7f4bfb8a5700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4be00682f0 0x7f4be006a7a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.969+0000 7f4bfb8a5700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf415cc80 0x7f4bf40b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.969+0000 7f4bfb8a5700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bf406a270 0x7f4bf4152e00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.969+0000 7f4bfb8a5700 1 --2- 172.21.15.57:0/2826073782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bf406e3a0 0x7f4bf40b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:56.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.969+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/2826073782 >> 172.21.15.57:0/2826073782 conn(0x7f4bf406c010 msgr2=0x7f4bf40add10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:56.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.969+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/2826073782 shutdown_connections 2023-11-28T12:02:56.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:56.969+0000 7f4bfb8a5700 1 -- 172.21.15.57:0/2826073782 wait complete. 2023-11-28T12:02:56.973 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 16 2023-11-28T12:02:56.984 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:02:57.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.117+0000 7faca3e30700 1 Processor -- start 2023-11-28T12:02:57.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.117+0000 7faca3e30700 1 -- start start 2023-11-28T12:02:57.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.118+0000 7faca3e30700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9c142db0 0x7fac9c13ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.118+0000 7faca3e30700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac9c0b1770 0x7fac9c0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.118+0000 7faca3e30700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c15e010 con 0x7fac9c0ce040 2023-11-28T12:02:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.118+0000 7faca3e30700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9c0b2130 con 0x7fac9c0b1770 2023-11-28T12:02:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.118+0000 7faca3e30700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c13fc90 con 0x7fac9c142db0 2023-11-28T12:02:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.118+0000 7faca23cd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac9c0b1770 0x7fac9c0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.118+0000 7faca13cb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9c142db0 0x7fac9c13ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.118+0000 7faca23cd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac9c0b1770 0x7fac9c0b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:44046/0 (socket says 172.21.15.57:44046) 2023-11-28T12:02:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.118+0000 7faca23cd700 1 -- 172.21.15.57:0/95795403 learned_addr learned my addr 172.21.15.57:0/95795403 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.118+0000 7faca1bcc700 1 --1- 172.21.15.57:0/95795403 >> v1:172.21.15.181:6789/0 conn(0x7fac9c0ce040 0x7fac9c15d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:35824/0 (socket says 172.21.15.57:35824) 2023-11-28T12:02:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.119+0000 7faca23cd700 1 -- 172.21.15.57:0/95795403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9c142db0 msgr2=0x7fac9c13ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.119+0000 7faca23cd700 1 --2- 172.21.15.57:0/95795403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9c142db0 0x7fac9c13ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.119+0000 7faca23cd700 1 -- 172.21.15.57:0/95795403 >> v1:172.21.15.181:6789/0 conn(0x7fac9c0ce040 legacy=0x7fac9c15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.119+0000 7faca23cd700 1 -- 172.21.15.57:0/95795403 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c13f6d0 con 0x7fac9c0b1770 2023-11-28T12:02:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.119+0000 7faca13cb700 1 --2- 172.21.15.57:0/95795403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9c142db0 0x7fac9c13ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:02:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.119+0000 7faca0bca700 1 -- 172.21.15.57:0/95795403 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570796797 0 0) 0x7fac98001110 con 0x7fac9c0ce040 2023-11-28T12:02:57.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.119+0000 7faca23cd700 1 --2- 172.21.15.57:0/95795403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c0b1770 0x7fac9c0b1bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fac94009e50 tx=0x7fac9400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52e46a9dd897074c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:57.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.119+0000 7faca0bca700 1 -- 172.21.15.57:0/95795403 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9400e070 con 0x7fac9c0b1770 2023-11-28T12:02:57.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.119+0000 7faca0bca700 1 -- 172.21.15.57:0/95795403 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9400b910 con 0x7fac9c0b1770 2023-11-28T12:02:57.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.119+0000 7faca0bca700 1 -- 172.21.15.57:0/95795403 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9400bcc0 con 0x7fac9c0b1770 2023-11-28T12:02:57.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.120+0000 7faca3e30700 1 -- 172.21.15.57:0/95795403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c0b1770 msgr2=0x7fac9c0b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.120+0000 7faca3e30700 1 --2- 172.21.15.57:0/95795403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c0b1770 0x7fac9c0b1bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fac94009e50 tx=0x7fac9400b330 comp rx=0 tx=0).stop 2023-11-28T12:02:57.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.120+0000 7faca3e30700 1 -- 172.21.15.57:0/95795403 shutdown_connections 2023-11-28T12:02:57.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.120+0000 7faca3e30700 1 --2- 172.21.15.57:0/95795403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c0b1770 0x7fac9c0b1bf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.120+0000 7faca3e30700 1 --2- 172.21.15.57:0/95795403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9c142db0 0x7fac9c13ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.120+0000 7faca3e30700 1 -- 172.21.15.57:0/95795403 >> 172.21.15.57:0/95795403 conn(0x7fac9c06c010 msgr2=0x7fac9c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:57.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.120+0000 7faca3e30700 1 -- 172.21.15.57:0/95795403 shutdown_connections 2023-11-28T12:02:57.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.120+0000 7faca3e30700 1 -- 172.21.15.57:0/95795403 wait complete. 2023-11-28T12:02:57.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.121+0000 7faca3e30700 1 Processor -- start 2023-11-28T12:02:57.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.121+0000 7faca3e30700 1 -- start start 2023-11-28T12:02:57.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.122+0000 7faca3e30700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9c0b1770 0x7fac9c0b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.122+0000 7faca3e30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c0ce040 0x7fac9c1efed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.122+0000 7faca3e30700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac9c142db0 0x7fac9c0b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.122+0000 7faca3e30700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1f0410 con 0x7fac9c142db0 2023-11-28T12:02:57.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.122+0000 7faca3e30700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1f0b90 con 0x7fac9c0ce040 2023-11-28T12:02:57.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.122+0000 7faca3e30700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1f0e70 con 0x7fac9c0b1770 2023-11-28T12:02:57.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.122+0000 7faca23cd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac9c142db0 0x7fac9c0b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.122+0000 7faca23cd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac9c142db0 0x7fac9c0b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55870/0 (socket says 172.21.15.57:55870) 2023-11-28T12:02:57.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.122+0000 7faca23cd700 1 -- 172.21.15.57:0/2915567739 learned_addr learned my addr 172.21.15.57:0/2915567739 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:57.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.122+0000 7faca1bcc700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9c0b1770 0x7fac9c0b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.122+0000 7faca13cb700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c0ce040 0x7fac9c1efed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.123+0000 7faca23cd700 1 -- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9c0b1770 msgr2=0x7fac9c0b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.123+0000 7faca23cd700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9c0b1770 0x7fac9c0b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.123+0000 7faca23cd700 1 -- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c0ce040 msgr2=0x7fac9c1efed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.123+0000 7faca23cd700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c0ce040 0x7fac9c1efed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.123+0000 7faca23cd700 1 -- 172.21.15.57:0/2915567739 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac900094a0 con 0x7fac9c142db0 2023-11-28T12:02:57.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.123+0000 7faca23cd700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac9c142db0 0x7fac9c0b36b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fac9400b680 tx=0x7fac94004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:57.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.123+0000 7fac8e7fc700 1 -- 172.21.15.57:0/2915567739 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac940099b0 con 0x7fac9c142db0 2023-11-28T12:02:57.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.123+0000 7fac8e7fc700 1 -- 172.21.15.57:0/2915567739 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac940092e0 con 0x7fac9c142db0 2023-11-28T12:02:57.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.123+0000 7faca3e30700 1 -- 172.21.15.57:0/2915567739 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac94009b30 con 0x7fac9c142db0 2023-11-28T12:02:57.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.123+0000 7fac8e7fc700 1 -- 172.21.15.57:0/2915567739 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9401b650 con 0x7fac9c142db0 2023-11-28T12:02:57.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.123+0000 7faca3e30700 1 -- 172.21.15.57:0/2915567739 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c15d100 con 0x7fac9c142db0 2023-11-28T12:02:57.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.124+0000 7faca3e30700 1 -- 172.21.15.57:0/2915567739 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9c13f9b0 con 0x7fac9c142db0 2023-11-28T12:02:57.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.125+0000 7fac8e7fc700 1 -- 172.21.15.57:0/2915567739 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fac9402b070 con 0x7fac9c142db0 2023-11-28T12:02:57.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.125+0000 7fac8e7fc700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fac800682f0 0x7fac8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.125+0000 7fac8e7fc700 1 -- 172.21.15.57:0/2915567739 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fac940874e0 con 0x7fac9c142db0 2023-11-28T12:02:57.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.125+0000 7faca1bcc700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fac800682f0 0x7fac8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.126+0000 7faca1bcc700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fac800682f0 0x7fac8006a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fac98000f60 tx=0x7fac98007a40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:57.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.130+0000 7fac8e7fc700 1 -- 172.21.15.57:0/2915567739 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fac94053ce0 con 0x7fac9c142db0 2023-11-28T12:02:57.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.327+0000 7faca3e30700 1 -- 172.21.15.57:0/2915567739 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac9c15d3e0 con 0x7fac9c142db0 2023-11-28T12:02:57.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.328+0000 7fac8e7fc700 1 -- 172.21.15.57:0/2915567739 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+17921 (secure 0 0 0) 0x7fac94057300 con 0x7fac9c142db0 2023-11-28T12:02:57.330 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:57.330 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":35,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:02:54.404560+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:02:51.245666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T12:02:52.363863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:02:57.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.331+0000 7faca3e30700 1 -- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fac800682f0 msgr2=0x7fac8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.331+0000 7faca3e30700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fac800682f0 0x7fac8006a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fac98000f60 tx=0x7fac98007a40 comp rx=0 tx=0).stop 2023-11-28T12:02:57.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.331+0000 7faca3e30700 1 -- 172.21.15.57:0/2915567739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac9c142db0 msgr2=0x7fac9c0b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.331+0000 7faca3e30700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac9c142db0 0x7fac9c0b36b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fac9400b680 tx=0x7fac94004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.331+0000 7faca3e30700 1 -- 172.21.15.57:0/2915567739 shutdown_connections 2023-11-28T12:02:57.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.331+0000 7faca3e30700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fac800682f0 0x7fac8006a7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.331+0000 7faca3e30700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac9c142db0 0x7fac9c0b36b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.331+0000 7faca3e30700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9c0b1770 0x7fac9c0b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.331+0000 7faca3e30700 1 --2- 172.21.15.57:0/2915567739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c0ce040 0x7fac9c1efed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.331+0000 7faca3e30700 1 -- 172.21.15.57:0/2915567739 >> 172.21.15.57:0/2915567739 conn(0x7fac9c06c010 msgr2=0x7fac9c140900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:57.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.331+0000 7faca3e30700 1 -- 172.21.15.57:0/2915567739 shutdown_connections 2023-11-28T12:02:57.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.331+0000 7faca3e30700 1 -- 172.21.15.57:0/2915567739 wait complete. 2023-11-28T12:02:57.346 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T12:02:57.346 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:02:57.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.480+0000 7f556bcaa700 1 Processor -- start 2023-11-28T12:02:57.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.481+0000 7f556bcaa700 1 -- start start 2023-11-28T12:02:57.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.481+0000 7f556bcaa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f556406b7b0 0x7f5564072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.481+0000 7f556bcaa700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55640726c0 con 0x7f55640ad1f0 2023-11-28T12:02:57.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.482+0000 7f556bcaa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f556406bc20 con 0x7f55640b0880 2023-11-28T12:02:57.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.482+0000 7f556bcaa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55640b43f0 con 0x7f556406b7b0 2023-11-28T12:02:57.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.482+0000 7f556a247700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f55640ad1f0 0x7f55640af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47378/0 (socket says 172.21.15.57:47378) 2023-11-28T12:02:57.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.482+0000 7f556a247700 1 -- 172.21.15.57:0/3658484962 learned_addr learned my addr 172.21.15.57:0/3658484962 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:57.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.482+0000 7f5569245700 1 --2- 172.21.15.57:0/3658484962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f556406b7b0 0x7f5564072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.482+0000 7f5568a44700 1 -- 172.21.15.57:0/3658484962 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912485913 0 0) 0x7f55640726c0 con 0x7f55640ad1f0 2023-11-28T12:02:57.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.482+0000 7f5568a44700 1 -- 172.21.15.57:0/3658484962 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55500035b0 con 0x7f55640ad1f0 2023-11-28T12:02:57.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.482+0000 7f5568a44700 1 -- 172.21.15.57:0/3658484962 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764833386 0 0) 0x7f5560001110 con 0x7f55640b0880 2023-11-28T12:02:57.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.482+0000 7f5569245700 1 -- 172.21.15.57:0/3658484962 >> v1:172.21.15.181:6789/0 conn(0x7f55640b0880 legacy=0x7f55640aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.482+0000 7f5569245700 1 -- 172.21.15.57:0/3658484962 >> v1:172.21.15.57:6789/0 conn(0x7f55640ad1f0 legacy=0x7f55640af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.482+0000 7f5569245700 1 -- 172.21.15.57:0/3658484962 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55640b46d0 con 0x7f556406b7b0 2023-11-28T12:02:57.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.482+0000 7f5569245700 1 --2- 172.21.15.57:0/3658484962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f556406b7b0 0x7f5564072180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5554009870 tx=0x7f5554009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6e8e5046c873fcb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:57.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.483+0000 7f5568a44700 1 -- 172.21.15.57:0/3658484962 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554011830 con 0x7f556406b7b0 2023-11-28T12:02:57.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.483+0000 7f5568a44700 1 -- 172.21.15.57:0/3658484962 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5554011990 con 0x7f556406b7b0 2023-11-28T12:02:57.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.483+0000 7f5568a44700 1 -- 172.21.15.57:0/3658484962 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554011d20 con 0x7f556406b7b0 2023-11-28T12:02:57.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.483+0000 7f556bcaa700 1 -- 172.21.15.57:0/3658484962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f556406b7b0 msgr2=0x7f5564072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.483+0000 7f556bcaa700 1 --2- 172.21.15.57:0/3658484962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f556406b7b0 0x7f5564072180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5554009870 tx=0x7f5554009b80 comp rx=0 tx=0).stop 2023-11-28T12:02:57.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.483+0000 7f556bcaa700 1 -- 172.21.15.57:0/3658484962 shutdown_connections 2023-11-28T12:02:57.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.483+0000 7f556bcaa700 1 --2- 172.21.15.57:0/3658484962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f556406b7b0 0x7f5564072180 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.483+0000 7f556bcaa700 1 -- 172.21.15.57:0/3658484962 >> 172.21.15.57:0/3658484962 conn(0x7f55640b2fc0 msgr2=0x7f5564071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:57.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.483+0000 7f556bcaa700 1 -- 172.21.15.57:0/3658484962 shutdown_connections 2023-11-28T12:02:57.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.483+0000 7f556bcaa700 1 -- 172.21.15.57:0/3658484962 wait complete. 2023-11-28T12:02:57.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.485+0000 7f556bcaa700 1 Processor -- start 2023-11-28T12:02:57.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.485+0000 7f556bcaa700 1 -- start start 2023-11-28T12:02:57.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.485+0000 7f556bcaa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f556406b7b0 0x7f5564098800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.485+0000 7f556bcaa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55640ad1f0 0x7f55641f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.485+0000 7f556bcaa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55640b0880 0x7f55641f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.485+0000 7f556bcaa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556409d330 con 0x7f55640b0880 2023-11-28T12:02:57.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.485+0000 7f556bcaa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5564098d40 con 0x7f556406b7b0 2023-11-28T12:02:57.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.485+0000 7f556bcaa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556409cc90 con 0x7f55640ad1f0 2023-11-28T12:02:57.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f556a247700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55640b0880 0x7f55641f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f556a247700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55640b0880 0x7f55641f59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55874/0 (socket says 172.21.15.57:55874) 2023-11-28T12:02:57.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f556a247700 1 -- 172.21.15.57:0/790292759 learned_addr learned my addr 172.21.15.57:0/790292759 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:57.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f5569245700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55640ad1f0 0x7f55641f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f5569a46700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f556406b7b0 0x7f5564098800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f556a247700 1 -- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55640ad1f0 msgr2=0x7f55641f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f556a247700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55640ad1f0 0x7f55641f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f556a247700 1 -- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f556406b7b0 msgr2=0x7f5564098800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f556a247700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f556406b7b0 0x7f5564098800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f556a247700 1 -- 172.21.15.57:0/790292759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55540094a0 con 0x7f55640b0880 2023-11-28T12:02:57.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f556a247700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55640b0880 0x7f55641f59d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f555c00cb40 tx=0x7f555c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:57.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f555a7fc700 1 -- 172.21.15.57:0/790292759 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c00ab50 con 0x7f55640b0880 2023-11-28T12:02:57.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f555a7fc700 1 -- 172.21.15.57:0/790292759 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555c00acb0 con 0x7f55640b0880 2023-11-28T12:02:57.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f555a7fc700 1 -- 172.21.15.57:0/790292759 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c012620 con 0x7f55640b0880 2023-11-28T12:02:57.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f556bcaa700 1 -- 172.21.15.57:0/790292759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55640b4fe0 con 0x7f55640b0880 2023-11-28T12:02:57.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.486+0000 7f556bcaa700 1 -- 172.21.15.57:0/790292759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55640b4d00 con 0x7f55640b0880 2023-11-28T12:02:57.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.487+0000 7f556bcaa700 1 -- 172.21.15.57:0/790292759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55640b49b0 con 0x7f55640b0880 2023-11-28T12:02:57.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.488+0000 7f555a7fc700 1 -- 172.21.15.57:0/790292759 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f555c019030 con 0x7f55640b0880 2023-11-28T12:02:57.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.488+0000 7f555a7fc700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f55480682f0 0x7f554806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.488+0000 7f555a7fc700 1 -- 172.21.15.57:0/790292759 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f555c085960 con 0x7f55640b0880 2023-11-28T12:02:57.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.488+0000 7f5569a46700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f55480682f0 0x7f554806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.489+0000 7f5569a46700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f55480682f0 0x7f554806a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5560007fd0 tx=0x7f5560009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:57.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.493+0000 7f555a7fc700 1 -- 172.21.15.57:0/790292759 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f555c0521c0 con 0x7f55640b0880 2023-11-28T12:02:57.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.688+0000 7f556bcaa700 1 -- 172.21.15.57:0/790292759 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5564152b50 con 0x7f55480682f0 2023-11-28T12:02:57.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.697+0000 7f555a7fc700 1 -- 172.21.15.57:0/790292759 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312176 (secure 0 0 0) 0x7f5564152b50 con 0x7f55480682f0 2023-11-28T12:02:57.704 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:57.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.701+0000 7f556bcaa700 1 -- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f55480682f0 msgr2=0x7f554806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.701+0000 7f556bcaa700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f55480682f0 0x7f554806a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5560007fd0 tx=0x7f5560009040 comp rx=0 tx=0).stop 2023-11-28T12:02:57.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.701+0000 7f556bcaa700 1 -- 172.21.15.57:0/790292759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55640b0880 msgr2=0x7f55641f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.701+0000 7f556bcaa700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55640b0880 0x7f55641f59d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f555c00cb40 tx=0x7f555c00cf00 comp rx=0 tx=0).stop 2023-11-28T12:02:57.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.701+0000 7f556bcaa700 1 -- 172.21.15.57:0/790292759 shutdown_connections 2023-11-28T12:02:57.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.701+0000 7f556bcaa700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f55480682f0 0x7f554806a7a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.701+0000 7f556bcaa700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55640b0880 0x7f55641f59d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.701+0000 7f556bcaa700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f556406b7b0 0x7f5564098800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.701+0000 7f556bcaa700 1 --2- 172.21.15.57:0/790292759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55640ad1f0 0x7f55641f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.701+0000 7f556bcaa700 1 -- 172.21.15.57:0/790292759 >> 172.21.15.57:0/790292759 conn(0x7f55640b2fc0 msgr2=0x7f55640ac1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:57.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.701+0000 7f556bcaa700 1 -- 172.21.15.57:0/790292759 shutdown_connections 2023-11-28T12:02:57.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.701+0000 7f556bcaa700 1 -- 172.21.15.57:0/790292759 wait complete. 2023-11-28T12:02:57.708 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:02:57.715 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2023-11-28T12:02:57.678087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":69672,"kb_used_data":4328,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":754905048,"statfs":{"total":773094113280,"available":773022769152,"internally_reserved":0,"allocated":4431872,"data_stored":2507388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.729966"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413650+0000","last_change":"2023-11-28T12:02:53.423945+0000","last_active":"2023-11-28T12:02:54.413650+0000","last_peered":"2023-11-28T12:02:54.413650+0000","last_clean":"2023-11-28T12:02:54.413650+0000","last_became_active":"2023-11-28T12:02:53.423658+0000","last_became_peered":"2023-11-28T12:02:53.423658+0000","last_unstale":"2023-11-28T12:02:54.413650+0000","last_undegraded":"2023-11-28T12:02:54.413650+0000","last_fullsized":"2023-11-28T12:02:54.413650+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:33:11.245893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413510+0000","last_change":"2023-11-28T12:02:53.423076+0000","last_active":"2023-11-28T12:02:54.413510+0000","last_peered":"2023-11-28T12:02:54.413510+0000","last_clean":"2023-11-28T12:02:54.413510+0000","last_became_active":"2023-11-28T12:02:53.422775+0000","last_became_peered":"2023-11-28T12:02:53.422775+0000","last_unstale":"2023-11-28T12:02:54.413510+0000","last_undegraded":"2023-11-28T12:02:54.413510+0000","last_fullsized":"2023-11-28T12:02:54.413510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:30.182850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456914+0000","last_change":"2023-11-28T12:02:53.429327+0000","last_active":"2023-11-28T12:02:54.456914+0000","last_peered":"2023-11-28T12:02:54.456914+0000","last_clean":"2023-11-28T12:02:54.456914+0000","last_became_active":"2023-11-28T12:02:53.428973+0000","last_became_peered":"2023-11-28T12:02:53.428973+0000","last_unstale":"2023-11-28T12:02:54.456914+0000","last_undegraded":"2023-11-28T12:02:54.456914+0000","last_fullsized":"2023-11-28T12:02:54.456914+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:40.204459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456545+0000","last_change":"2023-11-28T12:02:53.429854+0000","last_active":"2023-11-28T12:02:54.456545+0000","last_peered":"2023-11-28T12:02:54.456545+0000","last_clean":"2023-11-28T12:02:54.456545+0000","last_became_active":"2023-11-28T12:02:53.429596+0000","last_became_peered":"2023-11-28T12:02:53.429596+0000","last_unstale":"2023-11-28T12:02:54.456545+0000","last_undegraded":"2023-11-28T12:02:54.456545+0000","last_fullsized":"2023-11-28T12:02:54.456545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:21:32.592938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456030+0000","last_change":"2023-11-28T12:02:53.427968+0000","last_active":"2023-11-28T12:02:54.456030+0000","last_peered":"2023-11-28T12:02:54.456030+0000","last_clean":"2023-11-28T12:02:54.456030+0000","last_became_active":"2023-11-28T12:02:53.427203+0000","last_became_peered":"2023-11-28T12:02:53.427203+0000","last_unstale":"2023-11-28T12:02:54.456030+0000","last_undegraded":"2023-11-28T12:02:54.456030+0000","last_fullsized":"2023-11-28T12:02:54.456030+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:46.539464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456359+0000","last_change":"2023-11-28T12:02:53.432557+0000","last_active":"2023-11-28T12:02:54.456359+0000","last_peered":"2023-11-28T12:02:54.456359+0000","last_clean":"2023-11-28T12:02:54.456359+0000","last_became_active":"2023-11-28T12:02:53.432226+0000","last_became_peered":"2023-11-28T12:02:53.432226+0000","last_unstale":"2023-11-28T12:02:54.456359+0000","last_undegraded":"2023-11-28T12:02:54.456359+0000","last_fullsized":"2023-11-28T12:02:54.456359+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:28:43.717539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457348+0000","last_change":"2023-11-28T12:02:53.433399+0000","last_active":"2023-11-28T12:02:54.457348+0000","last_peered":"2023-11-28T12:02:54.457348+0000","last_clean":"2023-11-28T12:02:54.457348+0000","last_became_active":"2023-11-28T12:02:53.433145+0000","last_became_peered":"2023-11-28T12:02:53.433145+0000","last_unstale":"2023-11-28T12:02:54.457348+0000","last_undegraded":"2023-11-28T12:02:54.457348+0000","last_fullsized":"2023-11-28T12:02:54.457348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:51.792418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413017+0000","last_change":"2023-11-28T12:02:53.431533+0000","last_active":"2023-11-28T12:02:54.413017+0000","last_peered":"2023-11-28T12:02:54.413017+0000","last_clean":"2023-11-28T12:02:54.413017+0000","last_became_active":"2023-11-28T12:02:53.430385+0000","last_became_peered":"2023-11-28T12:02:53.430385+0000","last_unstale":"2023-11-28T12:02:54.413017+0000","last_undegraded":"2023-11-28T12:02:54.413017+0000","last_fullsized":"2023-11-28T12:02:54.413017+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:53.807130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457162+0000","last_change":"2023-11-28T12:02:53.430266+0000","last_active":"2023-11-28T12:02:54.457162+0000","last_peered":"2023-11-28T12:02:54.457162+0000","last_clean":"2023-11-28T12:02:54.457162+0000","last_became_active":"2023-11-28T12:02:53.429884+0000","last_became_peered":"2023-11-28T12:02:53.429884+0000","last_unstale":"2023-11-28T12:02:54.457162+0000","last_undegraded":"2023-11-28T12:02:54.457162+0000","last_fullsized":"2023-11-28T12:02:54.457162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:31.438055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456702+0000","last_change":"2023-11-28T12:02:53.430543+0000","last_active":"2023-11-28T12:02:54.456702+0000","last_peered":"2023-11-28T12:02:54.456702+0000","last_clean":"2023-11-28T12:02:54.456702+0000","last_became_active":"2023-11-28T12:02:53.430163+0000","last_became_peered":"2023-11-28T12:02:53.430163+0000","last_unstale":"2023-11-28T12:02:54.456702+0000","last_undegraded":"2023-11-28T12:02:54.456702+0000","last_fullsized":"2023-11-28T12:02:54.456702+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:25:36.902715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456604+0000","last_change":"2023-11-28T12:02:53.431815+0000","last_active":"2023-11-28T12:02:54.456604+0000","last_peered":"2023-11-28T12:02:54.456604+0000","last_clean":"2023-11-28T12:02:54.456604+0000","last_became_active":"2023-11-28T12:02:53.431487+0000","last_became_peered":"2023-11-28T12:02:53.431487+0000","last_unstale":"2023-11-28T12:02:54.456604+0000","last_undegraded":"2023-11-28T12:02:54.456604+0000","last_fullsized":"2023-11-28T12:02:54.456604+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:58:27.198338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456794+0000","last_change":"2023-11-28T12:02:53.434210+0000","last_active":"2023-11-28T12:02:54.456794+0000","last_peered":"2023-11-28T12:02:54.456794+0000","last_clean":"2023-11-28T12:02:54.456794+0000","last_became_active":"2023-11-28T12:02:53.433887+0000","last_became_peered":"2023-11-28T12:02:53.433887+0000","last_unstale":"2023-11-28T12:02:54.456794+0000","last_undegraded":"2023-11-28T12:02:54.456794+0000","last_fullsized":"2023-11-28T12:02:54.456794+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:51.792418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456303+0000","last_change":"2023-11-28T12:02:53.429928+0000","last_active":"2023-11-28T12:02:54.456303+0000","last_peered":"2023-11-28T12:02:54.456303+0000","last_clean":"2023-11-28T12:02:54.456303+0000","last_became_active":"2023-11-28T12:02:53.427946+0000","last_became_peered":"2023-11-28T12:02:53.427946+0000","last_unstale":"2023-11-28T12:02:54.456303+0000","last_undegraded":"2023-11-28T12:02:54.456303+0000","last_fullsized":"2023-11-28T12:02:54.456303+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:57:29.151797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413769+0000","last_change":"2023-11-28T12:02:53.424388+0000","last_active":"2023-11-28T12:02:54.413769+0000","last_peered":"2023-11-28T12:02:54.413769+0000","last_clean":"2023-11-28T12:02:54.413769+0000","last_became_active":"2023-11-28T12:02:53.423710+0000","last_became_peered":"2023-11-28T12:02:53.423710+0000","last_unstale":"2023-11-28T12:02:54.413769+0000","last_undegraded":"2023-11-28T12:02:54.413769+0000","last_fullsized":"2023-11-28T12:02:54.413769+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:52.828071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457260+0000","last_change":"2023-11-28T12:02:53.429927+0000","last_active":"2023-11-28T12:02:54.457260+0000","last_peered":"2023-11-28T12:02:54.457260+0000","last_clean":"2023-11-28T12:02:54.457260+0000","last_became_active":"2023-11-28T12:02:53.429643+0000","last_became_peered":"2023-11-28T12:02:53.429643+0000","last_unstale":"2023-11-28T12:02:54.457260+0000","last_undegraded":"2023-11-28T12:02:54.457260+0000","last_fullsized":"2023-11-28T12:02:54.457260+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:29.971331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.455991+0000","last_change":"2023-11-28T12:02:53.431690+0000","last_active":"2023-11-28T12:02:54.455991+0000","last_peered":"2023-11-28T12:02:54.455991+0000","last_clean":"2023-11-28T12:02:54.455991+0000","last_became_active":"2023-11-28T12:02:53.431250+0000","last_became_peered":"2023-11-28T12:02:53.431250+0000","last_unstale":"2023-11-28T12:02:54.455991+0000","last_undegraded":"2023-11-28T12:02:54.455991+0000","last_fullsized":"2023-11-28T12:02:54.455991+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:57:29.151797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457284+0000","last_change":"2023-11-28T12:02:53.430870+0000","last_active":"2023-11-28T12:02:54.457284+0000","last_peered":"2023-11-28T12:02:54.457284+0000","last_clean":"2023-11-28T12:02:54.457284+0000","last_became_active":"2023-11-28T12:02:53.430603+0000","last_became_peered":"2023-11-28T12:02:53.430603+0000","last_unstale":"2023-11-28T12:02:54.457284+0000","last_undegraded":"2023-11-28T12:02:54.457284+0000","last_fullsized":"2023-11-28T12:02:54.457284+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:25:36.902715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413213+0000","last_change":"2023-11-28T12:02:53.424074+0000","last_active":"2023-11-28T12:02:54.413213+0000","last_peered":"2023-11-28T12:02:54.413213+0000","last_clean":"2023-11-28T12:02:54.413213+0000","last_became_active":"2023-11-28T12:02:53.423755+0000","last_became_peered":"2023-11-28T12:02:53.423755+0000","last_unstale":"2023-11-28T12:02:54.413213+0000","last_undegraded":"2023-11-28T12:02:54.413213+0000","last_fullsized":"2023-11-28T12:02:54.413213+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:57:32.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456421+0000","last_change":"2023-11-28T12:02:53.423211+0000","last_active":"2023-11-28T12:02:54.456421+0000","last_peered":"2023-11-28T12:02:54.456421+0000","last_clean":"2023-11-28T12:02:54.456421+0000","last_became_active":"2023-11-28T12:02:53.422900+0000","last_became_peered":"2023-11-28T12:02:53.422900+0000","last_unstale":"2023-11-28T12:02:54.456421+0000","last_undegraded":"2023-11-28T12:02:54.456421+0000","last_fullsized":"2023-11-28T12:02:54.456421+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:59:07.837003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.412905+0000","last_change":"2023-11-28T12:02:53.429830+0000","last_active":"2023-11-28T12:02:54.412905+0000","last_peered":"2023-11-28T12:02:54.412905+0000","last_clean":"2023-11-28T12:02:54.412905+0000","last_became_active":"2023-11-28T12:02:53.429418+0000","last_became_peered":"2023-11-28T12:02:53.429418+0000","last_unstale":"2023-11-28T12:02:54.412905+0000","last_undegraded":"2023-11-28T12:02:54.412905+0000","last_fullsized":"2023-11-28T12:02:54.412905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:33.639319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456538+0000","last_change":"2023-11-28T12:02:53.428372+0000","last_active":"2023-11-28T12:02:54.456538+0000","last_peered":"2023-11-28T12:02:54.456538+0000","last_clean":"2023-11-28T12:02:54.456538+0000","last_became_active":"2023-11-28T12:02:53.428091+0000","last_became_peered":"2023-11-28T12:02:53.428091+0000","last_unstale":"2023-11-28T12:02:54.456538+0000","last_undegraded":"2023-11-28T12:02:54.456538+0000","last_fullsized":"2023-11-28T12:02:54.456538+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:29:16.559693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.412920+0000","last_change":"2023-11-28T12:02:53.425945+0000","last_active":"2023-11-28T12:02:54.412920+0000","last_peered":"2023-11-28T12:02:54.412920+0000","last_clean":"2023-11-28T12:02:54.412920+0000","last_became_active":"2023-11-28T12:02:53.424911+0000","last_became_peered":"2023-11-28T12:02:53.424911+0000","last_unstale":"2023-11-28T12:02:54.412920+0000","last_undegraded":"2023-11-28T12:02:54.412920+0000","last_fullsized":"2023-11-28T12:02:54.412920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:04.964774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456151+0000","last_change":"2023-11-28T12:02:53.428719+0000","last_active":"2023-11-28T12:02:54.456151+0000","last_peered":"2023-11-28T12:02:54.456151+0000","last_clean":"2023-11-28T12:02:54.456151+0000","last_became_active":"2023-11-28T12:02:53.428464+0000","last_became_peered":"2023-11-28T12:02:53.428464+0000","last_unstale":"2023-11-28T12:02:54.456151+0000","last_undegraded":"2023-11-28T12:02:54.456151+0000","last_fullsized":"2023-11-28T12:02:54.456151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:38.587636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.455912+0000","last_change":"2023-11-28T12:02:53.422609+0000","last_active":"2023-11-28T12:02:54.455912+0000","last_peered":"2023-11-28T12:02:54.455912+0000","last_clean":"2023-11-28T12:02:54.455912+0000","last_became_active":"2023-11-28T12:02:53.422326+0000","last_became_peered":"2023-11-28T12:02:53.422326+0000","last_unstale":"2023-11-28T12:02:54.455912+0000","last_undegraded":"2023-11-28T12:02:54.455912+0000","last_fullsized":"2023-11-28T12:02:54.455912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:58.499744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413394+0000","last_change":"2023-11-28T12:02:53.423455+0000","last_active":"2023-11-28T12:02:54.413394+0000","last_peered":"2023-11-28T12:02:54.413394+0000","last_clean":"2023-11-28T12:02:54.413394+0000","last_became_active":"2023-11-28T12:02:53.423172+0000","last_became_peered":"2023-11-28T12:02:53.423172+0000","last_unstale":"2023-11-28T12:02:54.413394+0000","last_undegraded":"2023-11-28T12:02:54.413394+0000","last_fullsized":"2023-11-28T12:02:54.413394+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:54.855708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456586+0000","last_change":"2023-11-28T12:02:53.432363+0000","last_active":"2023-11-28T12:02:54.456586+0000","last_peered":"2023-11-28T12:02:54.456586+0000","last_clean":"2023-11-28T12:02:54.456586+0000","last_became_active":"2023-11-28T12:02:53.432047+0000","last_became_peered":"2023-11-28T12:02:53.432047+0000","last_unstale":"2023-11-28T12:02:54.456586+0000","last_undegraded":"2023-11-28T12:02:54.456586+0000","last_fullsized":"2023-11-28T12:02:54.456586+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:40.682921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413244+0000","last_change":"2023-11-28T12:02:53.423976+0000","last_active":"2023-11-28T12:02:54.413244+0000","last_peered":"2023-11-28T12:02:54.413244+0000","last_clean":"2023-11-28T12:02:54.413244+0000","last_became_active":"2023-11-28T12:02:53.423299+0000","last_became_peered":"2023-11-28T12:02:53.423299+0000","last_unstale":"2023-11-28T12:02:54.413244+0000","last_undegraded":"2023-11-28T12:02:54.413244+0000","last_fullsized":"2023-11-28T12:02:54.413244+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:27.463305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456349+0000","last_change":"2023-11-28T12:02:53.433435+0000","last_active":"2023-11-28T12:02:54.456349+0000","last_peered":"2023-11-28T12:02:54.456349+0000","last_clean":"2023-11-28T12:02:54.456349+0000","last_became_active":"2023-11-28T12:02:53.433130+0000","last_became_peered":"2023-11-28T12:02:53.433130+0000","last_unstale":"2023-11-28T12:02:54.456349+0000","last_undegraded":"2023-11-28T12:02:54.456349+0000","last_fullsized":"2023-11-28T12:02:54.456349+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:33.606174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.412858+0000","last_change":"2023-11-28T12:02:53.424327+0000","last_active":"2023-11-28T12:02:54.412858+0000","last_peered":"2023-11-28T12:02:54.412858+0000","last_clean":"2023-11-28T12:02:54.412858+0000","last_became_active":"2023-11-28T12:02:53.422570+0000","last_became_peered":"2023-11-28T12:02:53.422570+0000","last_unstale":"2023-11-28T12:02:54.412858+0000","last_undegraded":"2023-11-28T12:02:54.412858+0000","last_fullsized":"2023-11-28T12:02:54.412858+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:09:07.273671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.412877+0000","last_change":"2023-11-28T12:02:53.434373+0000","last_active":"2023-11-28T12:02:54.412877+0000","last_peered":"2023-11-28T12:02:54.412877+0000","last_clean":"2023-11-28T12:02:54.412877+0000","last_became_active":"2023-11-28T12:02:53.434064+0000","last_became_peered":"2023-11-28T12:02:53.434064+0000","last_unstale":"2023-11-28T12:02:54.412877+0000","last_undegraded":"2023-11-28T12:02:54.412877+0000","last_fullsized":"2023-11-28T12:02:54.412877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:09:11.895327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456078+0000","last_change":"2023-11-28T12:02:53.420458+0000","last_active":"2023-11-28T12:02:54.456078+0000","last_peered":"2023-11-28T12:02:54.456078+0000","last_clean":"2023-11-28T12:02:54.456078+0000","last_became_active":"2023-11-28T12:02:53.420107+0000","last_became_peered":"2023-11-28T12:02:53.420107+0000","last_unstale":"2023-11-28T12:02:54.456078+0000","last_undegraded":"2023-11-28T12:02:54.456078+0000","last_fullsized":"2023-11-28T12:02:54.456078+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:04.866139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456033+0000","last_change":"2023-11-28T12:02:53.431783+0000","last_active":"2023-11-28T12:02:54.456033+0000","last_peered":"2023-11-28T12:02:54.456033+0000","last_clean":"2023-11-28T12:02:54.456033+0000","last_became_active":"2023-11-28T12:02:53.431523+0000","last_became_peered":"2023-11-28T12:02:53.431523+0000","last_unstale":"2023-11-28T12:02:54.456033+0000","last_undegraded":"2023-11-28T12:02:54.456033+0000","last_fullsized":"2023-11-28T12:02:54.456033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:04.866139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456658+0000","last_change":"2023-11-28T12:02:53.431006+0000","last_active":"2023-11-28T12:02:54.456658+0000","last_peered":"2023-11-28T12:02:54.456658+0000","last_clean":"2023-11-28T12:02:54.456658+0000","last_became_active":"2023-11-28T12:02:53.430747+0000","last_became_peered":"2023-11-28T12:02:53.430747+0000","last_unstale":"2023-11-28T12:02:54.456658+0000","last_undegraded":"2023-11-28T12:02:54.456658+0000","last_fullsized":"2023-11-28T12:02:54.456658+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:33.606174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"34'4","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413445+0000","last_change":"2023-11-28T12:02:52.970699+0000","last_active":"2023-11-28T12:02:54.413445+0000","last_peered":"2023-11-28T12:02:54.413445+0000","last_clean":"2023-11-28T12:02:54.413445+0000","last_became_active":"2023-11-28T12:02:52.970418+0000","last_became_peered":"2023-11-28T12:02:52.970418+0000","last_unstale":"2023-11-28T12:02:54.413445+0000","last_undegraded":"2023-11-28T12:02:54.413445+0000","last_fullsized":"2023-11-28T12:02:54.413445+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:59:06.822931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456775+0000","last_change":"2023-11-28T12:02:53.430375+0000","last_active":"2023-11-28T12:02:54.456775+0000","last_peered":"2023-11-28T12:02:54.456775+0000","last_clean":"2023-11-28T12:02:54.456775+0000","last_became_active":"2023-11-28T12:02:53.430109+0000","last_became_peered":"2023-11-28T12:02:53.430109+0000","last_unstale":"2023-11-28T12:02:54.456775+0000","last_undegraded":"2023-11-28T12:02:54.456775+0000","last_fullsized":"2023-11-28T12:02:54.456775+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:02.067084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456902+0000","last_change":"2023-11-28T12:02:52.971272+0000","last_active":"2023-11-28T12:02:54.456902+0000","last_peered":"2023-11-28T12:02:54.456902+0000","last_clean":"2023-11-28T12:02:54.456902+0000","last_became_active":"2023-11-28T12:02:52.970916+0000","last_became_peered":"2023-11-28T12:02:52.970916+0000","last_unstale":"2023-11-28T12:02:54.456902+0000","last_undegraded":"2023-11-28T12:02:54.456902+0000","last_fullsized":"2023-11-28T12:02:54.456902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:00:43.937765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413926+0000","last_change":"2023-11-28T12:02:53.423719+0000","last_active":"2023-11-28T12:02:54.413926+0000","last_peered":"2023-11-28T12:02:54.413926+0000","last_clean":"2023-11-28T12:02:54.413926+0000","last_became_active":"2023-11-28T12:02:53.423453+0000","last_became_peered":"2023-11-28T12:02:53.423453+0000","last_unstale":"2023-11-28T12:02:54.413926+0000","last_undegraded":"2023-11-28T12:02:54.413926+0000","last_fullsized":"2023-11-28T12:02:54.413926+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:30.218802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456924+0000","last_change":"2023-11-28T12:02:52.973621+0000","last_active":"2023-11-28T12:02:54.456924+0000","last_peered":"2023-11-28T12:02:54.456924+0000","last_clean":"2023-11-28T12:02:54.456924+0000","last_became_active":"2023-11-28T12:02:52.973239+0000","last_became_peered":"2023-11-28T12:02:52.973239+0000","last_unstale":"2023-11-28T12:02:54.456924+0000","last_undegraded":"2023-11-28T12:02:54.456924+0000","last_fullsized":"2023-11-28T12:02:54.456924+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:32.625247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413467+0000","last_change":"2023-11-28T12:02:53.423815+0000","last_active":"2023-11-28T12:02:54.413467+0000","last_peered":"2023-11-28T12:02:54.413467+0000","last_clean":"2023-11-28T12:02:54.413467+0000","last_became_active":"2023-11-28T12:02:53.423526+0000","last_became_peered":"2023-11-28T12:02:53.423526+0000","last_unstale":"2023-11-28T12:02:54.413467+0000","last_undegraded":"2023-11-28T12:02:54.413467+0000","last_fullsized":"2023-11-28T12:02:54.413467+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:04.964774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413820+0000","last_change":"2023-11-28T12:02:52.973240+0000","last_active":"2023-11-28T12:02:54.413820+0000","last_peered":"2023-11-28T12:02:54.413820+0000","last_clean":"2023-11-28T12:02:54.413820+0000","last_became_active":"2023-11-28T12:02:52.972819+0000","last_became_peered":"2023-11-28T12:02:52.972819+0000","last_unstale":"2023-11-28T12:02:54.413820+0000","last_undegraded":"2023-11-28T12:02:54.413820+0000","last_fullsized":"2023-11-28T12:02:54.413820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:48.088028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.455937+0000","last_change":"2023-11-28T12:02:53.422051+0000","last_active":"2023-11-28T12:02:54.455937+0000","last_peered":"2023-11-28T12:02:54.455937+0000","last_clean":"2023-11-28T12:02:54.455937+0000","last_became_active":"2023-11-28T12:02:53.421718+0000","last_became_peered":"2023-11-28T12:02:53.421718+0000","last_unstale":"2023-11-28T12:02:54.455937+0000","last_undegraded":"2023-11-28T12:02:54.455937+0000","last_fullsized":"2023-11-28T12:02:54.455937+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:57:29.151797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456414+0000","last_change":"2023-11-28T12:02:52.980130+0000","last_active":"2023-11-28T12:02:54.456414+0000","last_peered":"2023-11-28T12:02:54.456414+0000","last_clean":"2023-11-28T12:02:54.456414+0000","last_became_active":"2023-11-28T12:02:52.979862+0000","last_became_peered":"2023-11-28T12:02:52.979862+0000","last_unstale":"2023-11-28T12:02:54.456414+0000","last_undegraded":"2023-11-28T12:02:54.456414+0000","last_fullsized":"2023-11-28T12:02:54.456414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:57:31.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457267+0000","last_change":"2023-11-28T12:02:53.429333+0000","last_active":"2023-11-28T12:02:54.457267+0000","last_peered":"2023-11-28T12:02:54.457267+0000","last_clean":"2023-11-28T12:02:54.457267+0000","last_became_active":"2023-11-28T12:02:53.428978+0000","last_became_peered":"2023-11-28T12:02:53.428978+0000","last_unstale":"2023-11-28T12:02:54.457267+0000","last_undegraded":"2023-11-28T12:02:54.457267+0000","last_fullsized":"2023-11-28T12:02:54.457267+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:49.938647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456413+0000","last_change":"2023-11-28T12:02:52.976849+0000","last_active":"2023-11-28T12:02:54.456413+0000","last_peered":"2023-11-28T12:02:54.456413+0000","last_clean":"2023-11-28T12:02:54.456413+0000","last_became_active":"2023-11-28T12:02:52.976552+0000","last_became_peered":"2023-11-28T12:02:52.976552+0000","last_unstale":"2023-11-28T12:02:54.456413+0000","last_undegraded":"2023-11-28T12:02:54.456413+0000","last_fullsized":"2023-11-28T12:02:54.456413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:26.679201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413624+0000","last_change":"2023-11-28T12:02:53.431299+0000","last_active":"2023-11-28T12:02:54.413624+0000","last_peered":"2023-11-28T12:02:54.413624+0000","last_clean":"2023-11-28T12:02:54.413624+0000","last_became_active":"2023-11-28T12:02:53.431025+0000","last_became_peered":"2023-11-28T12:02:53.431025+0000","last_unstale":"2023-11-28T12:02:54.413624+0000","last_undegraded":"2023-11-28T12:02:54.413624+0000","last_fullsized":"2023-11-28T12:02:54.413624+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:33.520090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456423+0000","last_change":"2023-11-28T12:02:52.977272+0000","last_active":"2023-11-28T12:02:54.456423+0000","last_peered":"2023-11-28T12:02:54.456423+0000","last_clean":"2023-11-28T12:02:54.456423+0000","last_became_active":"2023-11-28T12:02:52.976980+0000","last_became_peered":"2023-11-28T12:02:52.976980+0000","last_unstale":"2023-11-28T12:02:54.456423+0000","last_undegraded":"2023-11-28T12:02:54.456423+0000","last_fullsized":"2023-11-28T12:02:54.456423+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:04.556861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413309+0000","last_change":"2023-11-28T12:02:53.420853+0000","last_active":"2023-11-28T12:02:54.413309+0000","last_peered":"2023-11-28T12:02:54.413309+0000","last_clean":"2023-11-28T12:02:54.413309+0000","last_became_active":"2023-11-28T12:02:53.420410+0000","last_became_peered":"2023-11-28T12:02:53.420410+0000","last_unstale":"2023-11-28T12:02:54.413309+0000","last_undegraded":"2023-11-28T12:02:54.413309+0000","last_fullsized":"2023-11-28T12:02:54.413309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:22:19.291125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.412617+0000","last_change":"2023-11-28T12:02:52.966640+0000","last_active":"2023-11-28T12:02:54.412617+0000","last_peered":"2023-11-28T12:02:54.412617+0000","last_clean":"2023-11-28T12:02:54.412617+0000","last_became_active":"2023-11-28T12:02:52.966367+0000","last_became_peered":"2023-11-28T12:02:52.966367+0000","last_unstale":"2023-11-28T12:02:54.412617+0000","last_undegraded":"2023-11-28T12:02:54.412617+0000","last_fullsized":"2023-11-28T12:02:54.412617+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:48.924576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456642+0000","last_change":"2023-11-28T12:02:53.429341+0000","last_active":"2023-11-28T12:02:54.456642+0000","last_peered":"2023-11-28T12:02:54.456642+0000","last_clean":"2023-11-28T12:02:54.456642+0000","last_became_active":"2023-11-28T12:02:53.425238+0000","last_became_peered":"2023-11-28T12:02:53.425238+0000","last_unstale":"2023-11-28T12:02:54.456642+0000","last_undegraded":"2023-11-28T12:02:54.456642+0000","last_fullsized":"2023-11-28T12:02:54.456642+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:58.447498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413577+0000","last_change":"2023-11-28T12:02:52.975369+0000","last_active":"2023-11-28T12:02:54.413577+0000","last_peered":"2023-11-28T12:02:54.413577+0000","last_clean":"2023-11-28T12:02:54.413577+0000","last_became_active":"2023-11-28T12:02:52.975069+0000","last_became_peered":"2023-11-28T12:02:52.975069+0000","last_unstale":"2023-11-28T12:02:54.413577+0000","last_undegraded":"2023-11-28T12:02:54.413577+0000","last_fullsized":"2023-11-28T12:02:54.413577+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:37.573564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456722+0000","last_change":"2023-11-28T12:02:53.421660+0000","last_active":"2023-11-28T12:02:54.456722+0000","last_peered":"2023-11-28T12:02:54.456722+0000","last_clean":"2023-11-28T12:02:54.456722+0000","last_became_active":"2023-11-28T12:02:53.421331+0000","last_became_peered":"2023-11-28T12:02:53.421331+0000","last_unstale":"2023-11-28T12:02:54.456722+0000","last_undegraded":"2023-11-28T12:02:54.456722+0000","last_fullsized":"2023-11-28T12:02:54.456722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:29.971331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456922+0000","last_change":"2023-11-28T12:02:52.978378+0000","last_active":"2023-11-28T12:02:54.456922+0000","last_peered":"2023-11-28T12:02:54.456922+0000","last_clean":"2023-11-28T12:02:54.456922+0000","last_became_active":"2023-11-28T12:02:52.978129+0000","last_became_peered":"2023-11-28T12:02:52.978129+0000","last_unstale":"2023-11-28T12:02:54.456922+0000","last_undegraded":"2023-11-28T12:02:54.456922+0000","last_fullsized":"2023-11-28T12:02:54.456922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:04.556861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456843+0000","last_change":"2023-11-28T12:02:53.421981+0000","last_active":"2023-11-28T12:02:54.456843+0000","last_peered":"2023-11-28T12:02:54.456843+0000","last_clean":"2023-11-28T12:02:54.456843+0000","last_became_active":"2023-11-28T12:02:53.421724+0000","last_became_peered":"2023-11-28T12:02:53.421724+0000","last_unstale":"2023-11-28T12:02:54.456843+0000","last_undegraded":"2023-11-28T12:02:54.456843+0000","last_fullsized":"2023-11-28T12:02:54.456843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:34.704261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456806+0000","last_change":"2023-11-28T12:02:52.977588+0000","last_active":"2023-11-28T12:02:54.456806+0000","last_peered":"2023-11-28T12:02:54.456806+0000","last_clean":"2023-11-28T12:02:54.456806+0000","last_became_active":"2023-11-28T12:02:52.977272+0000","last_became_peered":"2023-11-28T12:02:52.977272+0000","last_unstale":"2023-11-28T12:02:54.456806+0000","last_undegraded":"2023-11-28T12:02:54.456806+0000","last_fullsized":"2023-11-28T12:02:54.456806+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:50.678023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413276+0000","last_change":"2023-11-28T12:02:53.431964+0000","last_active":"2023-11-28T12:02:54.413276+0000","last_peered":"2023-11-28T12:02:54.413276+0000","last_clean":"2023-11-28T12:02:54.413276+0000","last_became_active":"2023-11-28T12:02:53.431359+0000","last_became_peered":"2023-11-28T12:02:53.431359+0000","last_unstale":"2023-11-28T12:02:54.413276+0000","last_undegraded":"2023-11-28T12:02:54.413276+0000","last_fullsized":"2023-11-28T12:02:54.413276+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:09.337505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"34'6","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413542+0000","last_change":"2023-11-28T12:02:52.971689+0000","last_active":"2023-11-28T12:02:54.413542+0000","last_peered":"2023-11-28T12:02:54.413542+0000","last_clean":"2023-11-28T12:02:54.413542+0000","last_became_active":"2023-11-28T12:02:52.971397+0000","last_became_peered":"2023-11-28T12:02:52.971397+0000","last_unstale":"2023-11-28T12:02:54.413542+0000","last_undegraded":"2023-11-28T12:02:54.413542+0000","last_fullsized":"2023-11-28T12:02:54.413542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:03.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456207+0000","last_change":"2023-11-28T12:02:53.428756+0000","last_active":"2023-11-28T12:02:54.456207+0000","last_peered":"2023-11-28T12:02:54.456207+0000","last_clean":"2023-11-28T12:02:54.456207+0000","last_became_active":"2023-11-28T12:02:53.428439+0000","last_became_peered":"2023-11-28T12:02:53.428439+0000","last_unstale":"2023-11-28T12:02:54.456207+0000","last_undegraded":"2023-11-28T12:02:54.456207+0000","last_fullsized":"2023-11-28T12:02:54.456207+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:58:27.198338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413304+0000","last_change":"2023-11-28T12:02:52.972607+0000","last_active":"2023-11-28T12:02:54.413304+0000","last_peered":"2023-11-28T12:02:54.413304+0000","last_clean":"2023-11-28T12:02:54.413304+0000","last_became_active":"2023-11-28T12:02:52.972323+0000","last_became_peered":"2023-11-28T12:02:52.972323+0000","last_unstale":"2023-11-28T12:02:54.413304+0000","last_undegraded":"2023-11-28T12:02:54.413304+0000","last_fullsized":"2023-11-28T12:02:54.413304+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:48.088028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413176+0000","last_change":"2023-11-28T12:02:53.423855+0000","last_active":"2023-11-28T12:02:54.413176+0000","last_peered":"2023-11-28T12:02:54.413176+0000","last_clean":"2023-11-28T12:02:54.413176+0000","last_became_active":"2023-11-28T12:02:53.423561+0000","last_became_peered":"2023-11-28T12:02:53.423561+0000","last_unstale":"2023-11-28T12:02:54.413176+0000","last_undegraded":"2023-11-28T12:02:54.413176+0000","last_fullsized":"2023-11-28T12:02:54.413176+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:52.468097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456441+0000","last_change":"2023-11-28T12:02:52.975551+0000","last_active":"2023-11-28T12:02:54.456441+0000","last_peered":"2023-11-28T12:02:54.456441+0000","last_clean":"2023-11-28T12:02:54.456441+0000","last_became_active":"2023-11-28T12:02:52.975144+0000","last_became_peered":"2023-11-28T12:02:52.975144+0000","last_unstale":"2023-11-28T12:02:54.456441+0000","last_undegraded":"2023-11-28T12:02:54.456441+0000","last_fullsized":"2023-11-28T12:02:54.456441+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:50.678023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456133+0000","last_change":"2023-11-28T12:02:53.425607+0000","last_active":"2023-11-28T12:02:54.456133+0000","last_peered":"2023-11-28T12:02:54.456133+0000","last_clean":"2023-11-28T12:02:54.456133+0000","last_became_active":"2023-11-28T12:02:53.423207+0000","last_became_peered":"2023-11-28T12:02:53.423207+0000","last_unstale":"2023-11-28T12:02:54.456133+0000","last_undegraded":"2023-11-28T12:02:54.456133+0000","last_fullsized":"2023-11-28T12:02:54.456133+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:49.938647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413548+0000","last_change":"2023-11-28T12:02:52.969795+0000","last_active":"2023-11-28T12:02:54.413548+0000","last_peered":"2023-11-28T12:02:54.413548+0000","last_clean":"2023-11-28T12:02:54.413548+0000","last_became_active":"2023-11-28T12:02:52.969515+0000","last_became_peered":"2023-11-28T12:02:52.969515+0000","last_unstale":"2023-11-28T12:02:54.413548+0000","last_undegraded":"2023-11-28T12:02:54.413548+0000","last_fullsized":"2023-11-28T12:02:54.413548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:57.485672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413178+0000","last_change":"2023-11-28T12:02:53.427776+0000","last_active":"2023-11-28T12:02:54.413178+0000","last_peered":"2023-11-28T12:02:54.413178+0000","last_clean":"2023-11-28T12:02:54.413178+0000","last_became_active":"2023-11-28T12:02:53.427494+0000","last_became_peered":"2023-11-28T12:02:53.427494+0000","last_unstale":"2023-11-28T12:02:54.413178+0000","last_undegraded":"2023-11-28T12:02:54.413178+0000","last_fullsized":"2023-11-28T12:02:54.413178+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:33.520090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"34'8","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456102+0000","last_change":"2023-11-28T12:02:52.976780+0000","last_active":"2023-11-28T12:02:54.456102+0000","last_peered":"2023-11-28T12:02:54.456102+0000","last_clean":"2023-11-28T12:02:54.456102+0000","last_became_active":"2023-11-28T12:02:52.976507+0000","last_became_peered":"2023-11-28T12:02:52.976507+0000","last_unstale":"2023-11-28T12:02:54.456102+0000","last_undegraded":"2023-11-28T12:02:54.456102+0000","last_fullsized":"2023-11-28T12:02:54.456102+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:32.625247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413434+0000","last_change":"2023-11-28T12:02:53.419256+0000","last_active":"2023-11-28T12:02:54.413434+0000","last_peered":"2023-11-28T12:02:54.413434+0000","last_clean":"2023-11-28T12:02:54.413434+0000","last_became_active":"2023-11-28T12:02:53.418953+0000","last_became_peered":"2023-11-28T12:02:53.418953+0000","last_unstale":"2023-11-28T12:02:54.413434+0000","last_undegraded":"2023-11-28T12:02:54.413434+0000","last_fullsized":"2023-11-28T12:02:54.413434+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:52.828071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456714+0000","last_change":"2023-11-28T12:02:52.977172+0000","last_active":"2023-11-28T12:02:54.456714+0000","last_peered":"2023-11-28T12:02:54.456714+0000","last_clean":"2023-11-28T12:02:54.456714+0000","last_became_active":"2023-11-28T12:02:52.976724+0000","last_became_peered":"2023-11-28T12:02:52.976724+0000","last_unstale":"2023-11-28T12:02:54.456714+0000","last_undegraded":"2023-11-28T12:02:54.456714+0000","last_fullsized":"2023-11-28T12:02:54.456714+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:26.679201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456482+0000","last_change":"2023-11-28T12:02:53.428368+0000","last_active":"2023-11-28T12:02:54.456482+0000","last_peered":"2023-11-28T12:02:54.456482+0000","last_clean":"2023-11-28T12:02:54.456482+0000","last_became_active":"2023-11-28T12:02:53.428058+0000","last_became_peered":"2023-11-28T12:02:53.428058+0000","last_unstale":"2023-11-28T12:02:54.456482+0000","last_undegraded":"2023-11-28T12:02:54.456482+0000","last_fullsized":"2023-11-28T12:02:54.456482+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:49.102100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.414034+0000","last_change":"2023-11-28T12:02:52.975455+0000","last_active":"2023-11-28T12:02:54.414034+0000","last_peered":"2023-11-28T12:02:54.414034+0000","last_clean":"2023-11-28T12:02:54.414034+0000","last_became_active":"2023-11-28T12:02:52.975190+0000","last_became_peered":"2023-11-28T12:02:52.975190+0000","last_unstale":"2023-11-28T12:02:54.414034+0000","last_undegraded":"2023-11-28T12:02:54.414034+0000","last_fullsized":"2023-11-28T12:02:54.414034+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:29:15.545621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413981+0000","last_change":"2023-11-28T12:02:53.420843+0000","last_active":"2023-11-28T12:02:54.413981+0000","last_peered":"2023-11-28T12:02:54.413981+0000","last_clean":"2023-11-28T12:02:54.413981+0000","last_became_active":"2023-11-28T12:02:53.420573+0000","last_became_peered":"2023-11-28T12:02:53.420573+0000","last_unstale":"2023-11-28T12:02:54.413981+0000","last_undegraded":"2023-11-28T12:02:54.413981+0000","last_fullsized":"2023-11-28T12:02:54.413981+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:30.182850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456491+0000","last_change":"2023-11-28T12:02:52.976363+0000","last_active":"2023-11-28T12:02:54.456491+0000","last_peered":"2023-11-28T12:02:54.456491+0000","last_clean":"2023-11-28T12:02:54.456491+0000","last_became_active":"2023-11-28T12:02:52.975991+0000","last_became_peered":"2023-11-28T12:02:52.975991+0000","last_unstale":"2023-11-28T12:02:54.456491+0000","last_undegraded":"2023-11-28T12:02:54.456491+0000","last_fullsized":"2023-11-28T12:02:54.456491+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.406926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456847+0000","last_change":"2023-11-28T12:02:53.428127+0000","last_active":"2023-11-28T12:02:54.456847+0000","last_peered":"2023-11-28T12:02:54.456847+0000","last_clean":"2023-11-28T12:02:54.456847+0000","last_became_active":"2023-11-28T12:02:53.427562+0000","last_became_peered":"2023-11-28T12:02:53.427562+0000","last_unstale":"2023-11-28T12:02:54.456847+0000","last_undegraded":"2023-11-28T12:02:54.456847+0000","last_fullsized":"2023-11-28T12:02:54.456847+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:55.967764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457303+0000","last_change":"2023-11-28T12:02:52.971336+0000","last_active":"2023-11-28T12:02:54.457303+0000","last_peered":"2023-11-28T12:02:54.457303+0000","last_clean":"2023-11-28T12:02:54.457303+0000","last_became_active":"2023-11-28T12:02:52.971068+0000","last_became_peered":"2023-11-28T12:02:52.971068+0000","last_unstale":"2023-11-28T12:02:54.457303+0000","last_undegraded":"2023-11-28T12:02:54.457303+0000","last_fullsized":"2023-11-28T12:02:54.457303+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:26.679201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456066+0000","last_change":"2023-11-28T12:02:53.431509+0000","last_active":"2023-11-28T12:02:54.456066+0000","last_peered":"2023-11-28T12:02:54.456066+0000","last_clean":"2023-11-28T12:02:54.456066+0000","last_became_active":"2023-11-28T12:02:53.430231+0000","last_became_peered":"2023-11-28T12:02:53.430231+0000","last_unstale":"2023-11-28T12:02:54.456066+0000","last_undegraded":"2023-11-28T12:02:54.456066+0000","last_fullsized":"2023-11-28T12:02:54.456066+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:34.704261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456132+0000","last_change":"2023-11-28T12:02:52.973243+0000","last_active":"2023-11-28T12:02:54.456132+0000","last_peered":"2023-11-28T12:02:54.456132+0000","last_clean":"2023-11-28T12:02:54.456132+0000","last_became_active":"2023-11-28T12:02:52.972875+0000","last_became_peered":"2023-11-28T12:02:52.972875+0000","last_unstale":"2023-11-28T12:02:54.456132+0000","last_undegraded":"2023-11-28T12:02:54.456132+0000","last_fullsized":"2023-11-28T12:02:54.456132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:39:56.165145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456017+0000","last_change":"2023-11-28T12:02:53.427161+0000","last_active":"2023-11-28T12:02:54.456017+0000","last_peered":"2023-11-28T12:02:54.456017+0000","last_clean":"2023-11-28T12:02:54.456017+0000","last_became_active":"2023-11-28T12:02:53.426793+0000","last_became_peered":"2023-11-28T12:02:53.426793+0000","last_unstale":"2023-11-28T12:02:54.456017+0000","last_undegraded":"2023-11-28T12:02:54.456017+0000","last_fullsized":"2023-11-28T12:02:54.456017+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:33.606174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456302+0000","last_change":"2023-11-28T12:02:52.978335+0000","last_active":"2023-11-28T12:02:54.456302+0000","last_peered":"2023-11-28T12:02:54.456302+0000","last_clean":"2023-11-28T12:02:54.456302+0000","last_became_active":"2023-11-28T12:02:52.978042+0000","last_became_peered":"2023-11-28T12:02:52.978042+0000","last_unstale":"2023-11-28T12:02:54.456302+0000","last_undegraded":"2023-11-28T12:02:54.456302+0000","last_fullsized":"2023-11-28T12:02:54.456302+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:21:46.803306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457146+0000","last_change":"2023-11-28T12:02:53.428424+0000","last_active":"2023-11-28T12:02:54.457146+0000","last_peered":"2023-11-28T12:02:54.457146+0000","last_clean":"2023-11-28T12:02:54.457146+0000","last_became_active":"2023-11-28T12:02:53.428118+0000","last_became_peered":"2023-11-28T12:02:53.428118+0000","last_unstale":"2023-11-28T12:02:54.457146+0000","last_undegraded":"2023-11-28T12:02:54.457146+0000","last_fullsized":"2023-11-28T12:02:54.457146+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:02.067084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456196+0000","last_change":"2023-11-28T12:02:52.975477+0000","last_active":"2023-11-28T12:02:54.456196+0000","last_peered":"2023-11-28T12:02:54.456196+0000","last_clean":"2023-11-28T12:02:54.456196+0000","last_became_active":"2023-11-28T12:02:52.975205+0000","last_became_peered":"2023-11-28T12:02:52.975205+0000","last_unstale":"2023-11-28T12:02:54.456196+0000","last_undegraded":"2023-11-28T12:02:54.456196+0000","last_fullsized":"2023-11-28T12:02:54.456196+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:50.678023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413190+0000","last_change":"2023-11-28T12:02:53.412062+0000","last_active":"2023-11-28T12:02:54.413190+0000","last_peered":"2023-11-28T12:02:54.413190+0000","last_clean":"2023-11-28T12:02:54.413190+0000","last_became_active":"2023-11-28T12:02:53.411795+0000","last_became_peered":"2023-11-28T12:02:53.411795+0000","last_unstale":"2023-11-28T12:02:54.413190+0000","last_undegraded":"2023-11-28T12:02:54.413190+0000","last_fullsized":"2023-11-28T12:02:54.413190+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:09.337505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457063+0000","last_change":"2023-11-28T12:02:52.978001+0000","last_active":"2023-11-28T12:02:54.457063+0000","last_peered":"2023-11-28T12:02:54.457063+0000","last_clean":"2023-11-28T12:02:54.457063+0000","last_became_active":"2023-11-28T12:02:52.977718+0000","last_became_peered":"2023-11-28T12:02:52.977718+0000","last_unstale":"2023-11-28T12:02:54.457063+0000","last_undegraded":"2023-11-28T12:02:54.457063+0000","last_fullsized":"2023-11-28T12:02:54.457063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:00.920928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413740+0000","last_change":"2023-11-28T12:02:53.412545+0000","last_active":"2023-11-28T12:02:54.413740+0000","last_peered":"2023-11-28T12:02:54.413740+0000","last_clean":"2023-11-28T12:02:54.413740+0000","last_became_active":"2023-11-28T12:02:53.412237+0000","last_became_peered":"2023-11-28T12:02:53.412237+0000","last_unstale":"2023-11-28T12:02:54.413740+0000","last_undegraded":"2023-11-28T12:02:54.413740+0000","last_fullsized":"2023-11-28T12:02:54.413740+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:31.438055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413218+0000","last_change":"2023-11-28T12:02:52.964204+0000","last_active":"2023-11-28T12:02:54.413218+0000","last_peered":"2023-11-28T12:02:54.413218+0000","last_clean":"2023-11-28T12:02:54.413218+0000","last_became_active":"2023-11-28T12:02:52.963932+0000","last_became_peered":"2023-11-28T12:02:52.963932+0000","last_unstale":"2023-11-28T12:02:54.413218+0000","last_undegraded":"2023-11-28T12:02:54.413218+0000","last_fullsized":"2023-11-28T12:02:54.413218+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:03.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.409022+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:02:53.409022+0000","last_peered":"2023-11-28T12:02:53.409022+0000","last_clean":"2023-11-28T12:02:53.409022+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:02:53.409022+0000","last_undegraded":"2023-11-28T12:02:53.409022+0000","last_fullsized":"2023-11-28T12:02:53.409022+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456473+0000","last_change":"2023-11-28T12:02:53.422244+0000","last_active":"2023-11-28T12:02:54.456473+0000","last_peered":"2023-11-28T12:02:54.456473+0000","last_clean":"2023-11-28T12:02:54.456473+0000","last_became_active":"2023-11-28T12:02:53.421984+0000","last_became_peered":"2023-11-28T12:02:53.421984+0000","last_unstale":"2023-11-28T12:02:54.456473+0000","last_undegraded":"2023-11-28T12:02:54.456473+0000","last_fullsized":"2023-11-28T12:02:54.456473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:49.102100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413070+0000","last_change":"2023-11-28T12:02:52.970359+0000","last_active":"2023-11-28T12:02:54.413070+0000","last_peered":"2023-11-28T12:02:54.413070+0000","last_clean":"2023-11-28T12:02:54.413070+0000","last_became_active":"2023-11-28T12:02:52.970057+0000","last_became_peered":"2023-11-28T12:02:52.970057+0000","last_unstale":"2023-11-28T12:02:54.413070+0000","last_undegraded":"2023-11-28T12:02:54.413070+0000","last_fullsized":"2023-11-28T12:02:54.413070+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:01.053013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":29,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.408303+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:02:53.408303+0000","last_peered":"2023-11-28T12:02:53.408303+0000","last_clean":"2023-11-28T12:02:53.408303+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:02:53.408303+0000","last_undegraded":"2023-11-28T12:02:53.408303+0000","last_fullsized":"2023-11-28T12:02:53.408303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457187+0000","last_change":"2023-11-28T12:02:53.430056+0000","last_active":"2023-11-28T12:02:54.457187+0000","last_peered":"2023-11-28T12:02:54.457187+0000","last_clean":"2023-11-28T12:02:54.457187+0000","last_became_active":"2023-11-28T12:02:53.429788+0000","last_became_peered":"2023-11-28T12:02:53.429788+0000","last_unstale":"2023-11-28T12:02:54.457187+0000","last_undegraded":"2023-11-28T12:02:54.457187+0000","last_fullsized":"2023-11-28T12:02:54.457187+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:38.587636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456866+0000","last_change":"2023-11-28T12:02:52.977390+0000","last_active":"2023-11-28T12:02:54.456866+0000","last_peered":"2023-11-28T12:02:54.456866+0000","last_clean":"2023-11-28T12:02:54.456866+0000","last_became_active":"2023-11-28T12:02:52.977100+0000","last_became_peered":"2023-11-28T12:02:52.977100+0000","last_unstale":"2023-11-28T12:02:54.456866+0000","last_undegraded":"2023-11-28T12:02:54.456866+0000","last_fullsized":"2023-11-28T12:02:54.456866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:32.625247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.414654+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:02:53.414654+0000","last_peered":"2023-11-28T12:02:53.414654+0000","last_clean":"2023-11-28T12:02:53.414654+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:02:53.414654+0000","last_undegraded":"2023-11-28T12:02:53.414654+0000","last_fullsized":"2023-11-28T12:02:53.414654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":174,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.409457+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:02:53.409457+0000","last_peered":"2023-11-28T12:02:53.409457+0000","last_clean":"2023-11-28T12:02:53.409457+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:02:53.409457+0000","last_undegraded":"2023-11-28T12:02:53.409457+0000","last_fullsized":"2023-11-28T12:02:53.409457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456821+0000","last_change":"2023-11-28T12:02:53.422423+0000","last_active":"2023-11-28T12:02:54.456821+0000","last_peered":"2023-11-28T12:02:54.456821+0000","last_clean":"2023-11-28T12:02:54.456821+0000","last_became_active":"2023-11-28T12:02:53.420944+0000","last_became_peered":"2023-11-28T12:02:53.420944+0000","last_unstale":"2023-11-28T12:02:54.456821+0000","last_undegraded":"2023-11-28T12:02:54.456821+0000","last_fullsized":"2023-11-28T12:02:54.456821+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:29:16.559693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:55.080492+0000","last_change":"2023-11-28T12:02:52.976180+0000","last_active":"2023-11-28T12:02:55.080492+0000","last_peered":"2023-11-28T12:02:55.080492+0000","last_clean":"2023-11-28T12:02:55.080492+0000","last_became_active":"2023-11-28T12:02:52.975850+0000","last_became_peered":"2023-11-28T12:02:52.975850+0000","last_unstale":"2023-11-28T12:02:55.080492+0000","last_undegraded":"2023-11-28T12:02:55.080492+0000","last_fullsized":"2023-11-28T12:02:55.080492+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:16.116447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.408998+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:02:53.408998+0000","last_peered":"2023-11-28T12:02:53.408998+0000","last_clean":"2023-11-28T12:02:53.408998+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:02:53.408998+0000","last_undegraded":"2023-11-28T12:02:53.408998+0000","last_fullsized":"2023-11-28T12:02:53.408998+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456705+0000","last_change":"2023-11-28T12:02:53.433298+0000","last_active":"2023-11-28T12:02:54.456705+0000","last_peered":"2023-11-28T12:02:54.456705+0000","last_clean":"2023-11-28T12:02:54.456705+0000","last_became_active":"2023-11-28T12:02:53.432985+0000","last_became_peered":"2023-11-28T12:02:53.432985+0000","last_unstale":"2023-11-28T12:02:54.456705+0000","last_undegraded":"2023-11-28T12:02:54.456705+0000","last_fullsized":"2023-11-28T12:02:54.456705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:34.704261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456757+0000","last_change":"2023-11-28T12:02:52.977173+0000","last_active":"2023-11-28T12:02:54.456757+0000","last_peered":"2023-11-28T12:02:54.456757+0000","last_clean":"2023-11-28T12:02:54.456757+0000","last_became_active":"2023-11-28T12:02:52.976856+0000","last_became_peered":"2023-11-28T12:02:52.976856+0000","last_unstale":"2023-11-28T12:02:54.456757+0000","last_undegraded":"2023-11-28T12:02:54.456757+0000","last_fullsized":"2023-11-28T12:02:54.456757+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:57:31.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.409355+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:02:53.409355+0000","last_peered":"2023-11-28T12:02:53.409355+0000","last_clean":"2023-11-28T12:02:53.409355+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:02:53.409355+0000","last_undegraded":"2023-11-28T12:02:53.409355+0000","last_fullsized":"2023-11-28T12:02:53.409355+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413412+0000","last_change":"2023-11-28T12:02:53.411661+0000","last_active":"2023-11-28T12:02:54.413412+0000","last_peered":"2023-11-28T12:02:54.413412+0000","last_clean":"2023-11-28T12:02:54.413412+0000","last_became_active":"2023-11-28T12:02:53.411367+0000","last_became_peered":"2023-11-28T12:02:53.411367+0000","last_unstale":"2023-11-28T12:02:54.413412+0000","last_undegraded":"2023-11-28T12:02:54.413412+0000","last_fullsized":"2023-11-28T12:02:54.413412+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:34.704261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456996+0000","last_change":"2023-11-28T12:02:52.972309+0000","last_active":"2023-11-28T12:02:54.456996+0000","last_peered":"2023-11-28T12:02:54.456996+0000","last_clean":"2023-11-28T12:02:54.456996+0000","last_became_active":"2023-11-28T12:02:52.971984+0000","last_became_peered":"2023-11-28T12:02:52.971984+0000","last_unstale":"2023-11-28T12:02:54.456996+0000","last_undegraded":"2023-11-28T12:02:54.456996+0000","last_fullsized":"2023-11-28T12:02:54.456996+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:54:35.436932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.422184+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:02:53.422184+0000","last_peered":"2023-11-28T12:02:53.422184+0000","last_clean":"2023-11-28T12:02:53.422184+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:02:53.422184+0000","last_undegraded":"2023-11-28T12:02:53.422184+0000","last_fullsized":"2023-11-28T12:02:53.422184+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456336+0000","last_change":"2023-11-28T12:02:53.419953+0000","last_active":"2023-11-28T12:02:54.456336+0000","last_peered":"2023-11-28T12:02:54.456336+0000","last_clean":"2023-11-28T12:02:54.456336+0000","last_became_active":"2023-11-28T12:02:53.419617+0000","last_became_peered":"2023-11-28T12:02:53.419617+0000","last_unstale":"2023-11-28T12:02:54.456336+0000","last_undegraded":"2023-11-28T12:02:54.456336+0000","last_fullsized":"2023-11-28T12:02:54.456336+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:17:04.593407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.412912+0000","last_change":"2023-11-28T12:02:52.970894+0000","last_active":"2023-11-28T12:02:54.412912+0000","last_peered":"2023-11-28T12:02:54.412912+0000","last_clean":"2023-11-28T12:02:54.412912+0000","last_became_active":"2023-11-28T12:02:52.970072+0000","last_became_peered":"2023-11-28T12:02:52.970072+0000","last_unstale":"2023-11-28T12:02:54.412912+0000","last_undegraded":"2023-11-28T12:02:54.412912+0000","last_fullsized":"2023-11-28T12:02:54.412912+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:55.568072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.415975+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:02:53.415975+0000","last_peered":"2023-11-28T12:02:53.415975+0000","last_clean":"2023-11-28T12:02:53.415975+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:02:53.415975+0000","last_undegraded":"2023-11-28T12:02:53.415975+0000","last_fullsized":"2023-11-28T12:02:53.415975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456101+0000","last_change":"2023-11-28T12:02:53.435138+0000","last_active":"2023-11-28T12:02:54.456101+0000","last_peered":"2023-11-28T12:02:54.456101+0000","last_clean":"2023-11-28T12:02:54.456101+0000","last_became_active":"2023-11-28T12:02:53.434289+0000","last_became_peered":"2023-11-28T12:02:53.434289+0000","last_unstale":"2023-11-28T12:02:54.456101+0000","last_undegraded":"2023-11-28T12:02:54.456101+0000","last_fullsized":"2023-11-28T12:02:54.456101+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:25:36.902715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457292+0000","last_change":"2023-11-28T12:02:52.968838+0000","last_active":"2023-11-28T12:02:54.457292+0000","last_peered":"2023-11-28T12:02:54.457292+0000","last_clean":"2023-11-28T12:02:54.457292+0000","last_became_active":"2023-11-28T12:02:52.968475+0000","last_became_peered":"2023-11-28T12:02:52.968475+0000","last_unstale":"2023-11-28T12:02:54.457292+0000","last_undegraded":"2023-11-28T12:02:54.457292+0000","last_fullsized":"2023-11-28T12:02:54.457292+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:57.068408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.416348+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:02:53.416348+0000","last_peered":"2023-11-28T12:02:53.416348+0000","last_clean":"2023-11-28T12:02:53.416348+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:02:53.416348+0000","last_undegraded":"2023-11-28T12:02:53.416348+0000","last_fullsized":"2023-11-28T12:02:53.416348+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9568,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362272,"statfs":{"total":96636764160,"available":96626966528,"internally_reserved":0,"allocated":1007616,"data_stored":757235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.67100000000000004}]},{"osd":1,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.7460000000000004,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.69599999999999995}]},{"osd":2,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":7.7789999999999999,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68300000000000005}]},{"osd":3,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.8199999999999998,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.65500000000000003}]},{"osd":4,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.439}]},{"osd":5,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47099999999999997}]},{"osd":6,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.49199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":401408,"data_stored":165815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.66500000000000004}]},{"osd":1,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.60299999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.69299999999999995}]},{"osd":3,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.70799999999999996}]},{"osd":4,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.49099999999999999}]},{"osd":5,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53000000000000003}]},{"osd":7,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.55100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9492,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362348,"statfs":{"total":96636764160,"available":96627044352,"internally_reserved":0,"allocated":995328,"data_stored":755707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.33600000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.45100000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.29799999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.66000000000000003}]},{"osd":5,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.67700000000000005}]},{"osd":6,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.60099999999999998}]},{"osd":7,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.55300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8400,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363440,"statfs":{"total":96636764160,"available":96628162560,"internally_reserved":0,"allocated":401408,"data_stored":165283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":4.4950000000000001,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.46500000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":4.5369999999999999,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.48599999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.5999999999999996,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.67100000000000004}]},{"osd":5,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.752},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.70499999999999996}]},{"osd":6,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.5709999999999997,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.623},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.72799999999999998}]},{"osd":7,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.60799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":401408,"data_stored":165319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.34399999999999997}]},{"osd":1,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.30599999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.38400000000000001}]},{"osd":4,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47399999999999998}]},{"osd":5,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.502},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57199999999999995}]},{"osd":6,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54700000000000004}]},{"osd":7,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.52300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8540,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363300,"statfs":{"total":96636764160,"available":96628019200,"internally_reserved":0,"allocated":413696,"data_stored":166831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.371}]},{"osd":1,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.49199999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.53500000000000003}]},{"osd":4,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59799999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8532,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363308,"statfs":{"total":96636764160,"available":96628027392,"internally_reserved":0,"allocated":405504,"data_stored":165837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.56899999999999995}]},{"osd":2,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.60099999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.61799999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69699999999999995}]},{"osd":7,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8468,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363372,"statfs":{"total":96636764160,"available":96628092928,"internally_reserved":0,"allocated":405504,"data_stored":165361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":9.0630000000000006,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.41899999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.50900000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55400000000000005}]},{"osd":4,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.47199999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57699999999999996}]},{"osd":7,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.624},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:02:57.721 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:02:57.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.893+0000 7fe205dd5700 1 Processor -- start 2023-11-28T12:02:57.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.893+0000 7fe205dd5700 1 -- start start 2023-11-28T12:02:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe205dd5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe20006e400 0x7fe2000b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe205dd5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe200160e70 0x7fe20015c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe205dd5700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20006e900 con 0x7fe2000d0c60 2023-11-28T12:02:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe205dd5700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe200160390 con 0x7fe20006e400 2023-11-28T12:02:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe205dd5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe200160670 con 0x7fe200160e70 2023-11-28T12:02:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe1feffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe20006e400 0x7fe2000b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe1ff7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe2000d0c60 0x7fe2000d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47380/0 (socket says 172.21.15.57:47380) 2023-11-28T12:02:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe1ff7fe700 1 -- 172.21.15.57:0/976096906 learned_addr learned my addr 172.21.15.57:0/976096906 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:57.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe1fffff700 1 --2- 172.21.15.57:0/976096906 >> v2:172.21.15.181:3300/0 conn(0x7fe200160e70 0x7fe20015c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe1fe7fc700 1 -- 172.21.15.57:0/976096906 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800276796 0 0) 0x7fe20006e900 con 0x7fe2000d0c60 2023-11-28T12:02:57.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe1fe7fc700 1 -- 172.21.15.57:0/976096906 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e80035b0 con 0x7fe2000d0c60 2023-11-28T12:02:57.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe1feffd700 1 -- 172.21.15.57:0/976096906 >> v2:172.21.15.181:3300/0 conn(0x7fe200160e70 msgr2=0x7fe20015c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:57.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe1feffd700 1 --2- 172.21.15.57:0/976096906 >> v2:172.21.15.181:3300/0 conn(0x7fe200160e70 0x7fe20015c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe1feffd700 1 -- 172.21.15.57:0/976096906 >> v1:172.21.15.57:6789/0 conn(0x7fe2000d0c60 legacy=0x7fe2000d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.894+0000 7fe1feffd700 1 -- 172.21.15.57:0/976096906 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe200160950 con 0x7fe20006e400 2023-11-28T12:02:57.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.895+0000 7fe1feffd700 1 --2- 172.21.15.57:0/976096906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20006e400 0x7fe2000b5be0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe200068250 tx=0x7fe1f4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2486aefd7a7af68b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:57.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.895+0000 7fe1fe7fc700 1 -- 172.21.15.57:0/976096906 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4011830 con 0x7fe20006e400 2023-11-28T12:02:57.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.895+0000 7fe1fe7fc700 1 -- 172.21.15.57:0/976096906 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f4011990 con 0x7fe20006e400 2023-11-28T12:02:57.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.895+0000 7fe1fe7fc700 1 -- 172.21.15.57:0/976096906 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4011d20 con 0x7fe20006e400 2023-11-28T12:02:57.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.895+0000 7fe205dd5700 1 -- 172.21.15.57:0/976096906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20006e400 msgr2=0x7fe2000b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.895+0000 7fe205dd5700 1 --2- 172.21.15.57:0/976096906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20006e400 0x7fe2000b5be0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe200068250 tx=0x7fe1f4009b20 comp rx=0 tx=0).stop 2023-11-28T12:02:57.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.895+0000 7fe205dd5700 1 -- 172.21.15.57:0/976096906 shutdown_connections 2023-11-28T12:02:57.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.895+0000 7fe205dd5700 1 --2- 172.21.15.57:0/976096906 >> v2:172.21.15.181:3300/0 conn(0x7fe200160e70 0x7fe20015c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.895+0000 7fe205dd5700 1 --2- 172.21.15.57:0/976096906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20006e400 0x7fe2000b5be0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.895+0000 7fe205dd5700 1 -- 172.21.15.57:0/976096906 >> 172.21.15.57:0/976096906 conn(0x7fe20006a270 msgr2=0x7fe20006a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:57.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.895+0000 7fe205dd5700 1 -- 172.21.15.57:0/976096906 shutdown_connections 2023-11-28T12:02:57.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.895+0000 7fe205dd5700 1 -- 172.21.15.57:0/976096906 wait complete. 2023-11-28T12:02:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.897+0000 7fe205dd5700 1 Processor -- start 2023-11-28T12:02:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.897+0000 7fe205dd5700 1 -- start start 2023-11-28T12:02:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.898+0000 7fe205dd5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe20006e400 0x7fe200152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.898+0000 7fe205dd5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2000d0c60 0x7fe2001e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.898+0000 7fe205dd5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe200160e70 0x7fe2001e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.898+0000 7fe205dd5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000cd1c0 con 0x7fe2000d0c60 2023-11-28T12:02:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.898+0000 7fe205dd5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2001f20f0 con 0x7fe20006e400 2023-11-28T12:02:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.898+0000 7fe205dd5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000cac60 con 0x7fe200160e70 2023-11-28T12:02:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.898+0000 7fe1feffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2000d0c60 0x7fe2001e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.898+0000 7fe1feffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2000d0c60 0x7fe2001e7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55900/0 (socket says 172.21.15.57:55900) 2023-11-28T12:02:57.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.898+0000 7fe1feffd700 1 -- 172.21.15.57:0/407556957 learned_addr learned my addr 172.21.15.57:0/407556957 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:57.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.899+0000 7fe1fffff700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe200160e70 0x7fe2001e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.899+0000 7fe1ff7fe700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe20006e400 0x7fe200152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.899+0000 7fe1feffd700 1 -- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe200160e70 msgr2=0x7fe2001e8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.899+0000 7fe1feffd700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe200160e70 0x7fe2001e8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.899+0000 7fe1feffd700 1 -- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe20006e400 msgr2=0x7fe200152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:57.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.899+0000 7fe1feffd700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe20006e400 0x7fe200152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:57.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.899+0000 7fe1feffd700 1 -- 172.21.15.57:0/407556957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f40094a0 con 0x7fe2000d0c60 2023-11-28T12:02:57.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.899+0000 7fe1feffd700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2000d0c60 0x7fe2001e7bc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009b60 tx=0x7fe1f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:57.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.899+0000 7fe1e7fff700 1 -- 172.21.15.57:0/407556957 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40038a0 con 0x7fe2000d0c60 2023-11-28T12:02:57.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.899+0000 7fe1e7fff700 1 -- 172.21.15.57:0/407556957 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f401f070 con 0x7fe2000d0c60 2023-11-28T12:02:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.899+0000 7fe205dd5700 1 -- 172.21.15.57:0/407556957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20015ceb0 con 0x7fe2000d0c60 2023-11-28T12:02:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.899+0000 7fe1e7fff700 1 -- 172.21.15.57:0/407556957 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4003c00 con 0x7fe2000d0c60 2023-11-28T12:02:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.899+0000 7fe205dd5700 1 -- 172.21.15.57:0/407556957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2000d0740 con 0x7fe2000d0c60 2023-11-28T12:02:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.900+0000 7fe205dd5700 1 -- 172.21.15.57:0/407556957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20015d040 con 0x7fe2000d0c60 2023-11-28T12:02:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.901+0000 7fe1e7fff700 1 -- 172.21.15.57:0/407556957 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe1f400f050 con 0x7fe2000d0c60 2023-11-28T12:02:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.901+0000 7fe1e7fff700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1ec0682f0 0x7fe1ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.901+0000 7fe1e7fff700 1 -- 172.21.15.57:0/407556957 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe1f4021070 con 0x7fe2000d0c60 2023-11-28T12:02:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.901+0000 7fe1ff7fe700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1ec0682f0 0x7fe1ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.902+0000 7fe1ff7fe700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1ec0682f0 0x7fe1ec06a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00011e0 tx=0x7fe1f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:57.905+0000 7fe1e7fff700 1 -- 172.21.15.57:0/407556957 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe1f4053dc0 con 0x7fe2000d0c60 2023-11-28T12:02:58.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.103+0000 7fe205dd5700 1 -- 172.21.15.57:0/407556957 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2001f1f80 con 0x7fe1ec0682f0 2023-11-28T12:02:58.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.111+0000 7fe1e7fff700 1 -- 172.21.15.57:0/407556957 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312176 (secure 0 0 0) 0x7fe2001f1f80 con 0x7fe1ec0682f0 2023-11-28T12:02:58.115 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:58.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.115+0000 7fe205dd5700 1 -- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1ec0682f0 msgr2=0x7fe1ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:58.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.115+0000 7fe205dd5700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1ec0682f0 0x7fe1ec06a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00011e0 tx=0x7fe1f0009040 comp rx=0 tx=0).stop 2023-11-28T12:02:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.115+0000 7fe205dd5700 1 -- 172.21.15.57:0/407556957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2000d0c60 msgr2=0x7fe2001e7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.115+0000 7fe205dd5700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2000d0c60 0x7fe2001e7bc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009b60 tx=0x7fe1f4004750 comp rx=0 tx=0).stop 2023-11-28T12:02:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.115+0000 7fe205dd5700 1 -- 172.21.15.57:0/407556957 shutdown_connections 2023-11-28T12:02:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.115+0000 7fe205dd5700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1ec0682f0 0x7fe1ec06a7a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.115+0000 7fe205dd5700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe200160e70 0x7fe2001e8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.115+0000 7fe205dd5700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe20006e400 0x7fe200152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.115+0000 7fe205dd5700 1 --2- 172.21.15.57:0/407556957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2000d0c60 0x7fe2001e7bc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.115+0000 7fe205dd5700 1 -- 172.21.15.57:0/407556957 >> 172.21.15.57:0/407556957 conn(0x7fe20006a270 msgr2=0x7fe20006f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:58.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.115+0000 7fe205dd5700 1 -- 172.21.15.57:0/407556957 shutdown_connections 2023-11-28T12:02:58.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.115+0000 7fe205dd5700 1 -- 172.21.15.57:0/407556957 wait complete. 2023-11-28T12:02:58.124 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:02:58.130 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2023-11-28T12:02:57.678087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":69672,"kb_used_data":4328,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":754905048,"statfs":{"total":773094113280,"available":773022769152,"internally_reserved":0,"allocated":4431872,"data_stored":2507388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.729966"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413650+0000","last_change":"2023-11-28T12:02:53.423945+0000","last_active":"2023-11-28T12:02:54.413650+0000","last_peered":"2023-11-28T12:02:54.413650+0000","last_clean":"2023-11-28T12:02:54.413650+0000","last_became_active":"2023-11-28T12:02:53.423658+0000","last_became_peered":"2023-11-28T12:02:53.423658+0000","last_unstale":"2023-11-28T12:02:54.413650+0000","last_undegraded":"2023-11-28T12:02:54.413650+0000","last_fullsized":"2023-11-28T12:02:54.413650+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:33:11.245893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413510+0000","last_change":"2023-11-28T12:02:53.423076+0000","last_active":"2023-11-28T12:02:54.413510+0000","last_peered":"2023-11-28T12:02:54.413510+0000","last_clean":"2023-11-28T12:02:54.413510+0000","last_became_active":"2023-11-28T12:02:53.422775+0000","last_became_peered":"2023-11-28T12:02:53.422775+0000","last_unstale":"2023-11-28T12:02:54.413510+0000","last_undegraded":"2023-11-28T12:02:54.413510+0000","last_fullsized":"2023-11-28T12:02:54.413510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:30.182850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456914+0000","last_change":"2023-11-28T12:02:53.429327+0000","last_active":"2023-11-28T12:02:54.456914+0000","last_peered":"2023-11-28T12:02:54.456914+0000","last_clean":"2023-11-28T12:02:54.456914+0000","last_became_active":"2023-11-28T12:02:53.428973+0000","last_became_peered":"2023-11-28T12:02:53.428973+0000","last_unstale":"2023-11-28T12:02:54.456914+0000","last_undegraded":"2023-11-28T12:02:54.456914+0000","last_fullsized":"2023-11-28T12:02:54.456914+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:40.204459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456545+0000","last_change":"2023-11-28T12:02:53.429854+0000","last_active":"2023-11-28T12:02:54.456545+0000","last_peered":"2023-11-28T12:02:54.456545+0000","last_clean":"2023-11-28T12:02:54.456545+0000","last_became_active":"2023-11-28T12:02:53.429596+0000","last_became_peered":"2023-11-28T12:02:53.429596+0000","last_unstale":"2023-11-28T12:02:54.456545+0000","last_undegraded":"2023-11-28T12:02:54.456545+0000","last_fullsized":"2023-11-28T12:02:54.456545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:21:32.592938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456030+0000","last_change":"2023-11-28T12:02:53.427968+0000","last_active":"2023-11-28T12:02:54.456030+0000","last_peered":"2023-11-28T12:02:54.456030+0000","last_clean":"2023-11-28T12:02:54.456030+0000","last_became_active":"2023-11-28T12:02:53.427203+0000","last_became_peered":"2023-11-28T12:02:53.427203+0000","last_unstale":"2023-11-28T12:02:54.456030+0000","last_undegraded":"2023-11-28T12:02:54.456030+0000","last_fullsized":"2023-11-28T12:02:54.456030+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:46.539464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456359+0000","last_change":"2023-11-28T12:02:53.432557+0000","last_active":"2023-11-28T12:02:54.456359+0000","last_peered":"2023-11-28T12:02:54.456359+0000","last_clean":"2023-11-28T12:02:54.456359+0000","last_became_active":"2023-11-28T12:02:53.432226+0000","last_became_peered":"2023-11-28T12:02:53.432226+0000","last_unstale":"2023-11-28T12:02:54.456359+0000","last_undegraded":"2023-11-28T12:02:54.456359+0000","last_fullsized":"2023-11-28T12:02:54.456359+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:28:43.717539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457348+0000","last_change":"2023-11-28T12:02:53.433399+0000","last_active":"2023-11-28T12:02:54.457348+0000","last_peered":"2023-11-28T12:02:54.457348+0000","last_clean":"2023-11-28T12:02:54.457348+0000","last_became_active":"2023-11-28T12:02:53.433145+0000","last_became_peered":"2023-11-28T12:02:53.433145+0000","last_unstale":"2023-11-28T12:02:54.457348+0000","last_undegraded":"2023-11-28T12:02:54.457348+0000","last_fullsized":"2023-11-28T12:02:54.457348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:51.792418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413017+0000","last_change":"2023-11-28T12:02:53.431533+0000","last_active":"2023-11-28T12:02:54.413017+0000","last_peered":"2023-11-28T12:02:54.413017+0000","last_clean":"2023-11-28T12:02:54.413017+0000","last_became_active":"2023-11-28T12:02:53.430385+0000","last_became_peered":"2023-11-28T12:02:53.430385+0000","last_unstale":"2023-11-28T12:02:54.413017+0000","last_undegraded":"2023-11-28T12:02:54.413017+0000","last_fullsized":"2023-11-28T12:02:54.413017+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:53.807130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457162+0000","last_change":"2023-11-28T12:02:53.430266+0000","last_active":"2023-11-28T12:02:54.457162+0000","last_peered":"2023-11-28T12:02:54.457162+0000","last_clean":"2023-11-28T12:02:54.457162+0000","last_became_active":"2023-11-28T12:02:53.429884+0000","last_became_peered":"2023-11-28T12:02:53.429884+0000","last_unstale":"2023-11-28T12:02:54.457162+0000","last_undegraded":"2023-11-28T12:02:54.457162+0000","last_fullsized":"2023-11-28T12:02:54.457162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:31.438055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456702+0000","last_change":"2023-11-28T12:02:53.430543+0000","last_active":"2023-11-28T12:02:54.456702+0000","last_peered":"2023-11-28T12:02:54.456702+0000","last_clean":"2023-11-28T12:02:54.456702+0000","last_became_active":"2023-11-28T12:02:53.430163+0000","last_became_peered":"2023-11-28T12:02:53.430163+0000","last_unstale":"2023-11-28T12:02:54.456702+0000","last_undegraded":"2023-11-28T12:02:54.456702+0000","last_fullsized":"2023-11-28T12:02:54.456702+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:25:36.902715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456604+0000","last_change":"2023-11-28T12:02:53.431815+0000","last_active":"2023-11-28T12:02:54.456604+0000","last_peered":"2023-11-28T12:02:54.456604+0000","last_clean":"2023-11-28T12:02:54.456604+0000","last_became_active":"2023-11-28T12:02:53.431487+0000","last_became_peered":"2023-11-28T12:02:53.431487+0000","last_unstale":"2023-11-28T12:02:54.456604+0000","last_undegraded":"2023-11-28T12:02:54.456604+0000","last_fullsized":"2023-11-28T12:02:54.456604+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:58:27.198338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456794+0000","last_change":"2023-11-28T12:02:53.434210+0000","last_active":"2023-11-28T12:02:54.456794+0000","last_peered":"2023-11-28T12:02:54.456794+0000","last_clean":"2023-11-28T12:02:54.456794+0000","last_became_active":"2023-11-28T12:02:53.433887+0000","last_became_peered":"2023-11-28T12:02:53.433887+0000","last_unstale":"2023-11-28T12:02:54.456794+0000","last_undegraded":"2023-11-28T12:02:54.456794+0000","last_fullsized":"2023-11-28T12:02:54.456794+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:51.792418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456303+0000","last_change":"2023-11-28T12:02:53.429928+0000","last_active":"2023-11-28T12:02:54.456303+0000","last_peered":"2023-11-28T12:02:54.456303+0000","last_clean":"2023-11-28T12:02:54.456303+0000","last_became_active":"2023-11-28T12:02:53.427946+0000","last_became_peered":"2023-11-28T12:02:53.427946+0000","last_unstale":"2023-11-28T12:02:54.456303+0000","last_undegraded":"2023-11-28T12:02:54.456303+0000","last_fullsized":"2023-11-28T12:02:54.456303+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:57:29.151797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413769+0000","last_change":"2023-11-28T12:02:53.424388+0000","last_active":"2023-11-28T12:02:54.413769+0000","last_peered":"2023-11-28T12:02:54.413769+0000","last_clean":"2023-11-28T12:02:54.413769+0000","last_became_active":"2023-11-28T12:02:53.423710+0000","last_became_peered":"2023-11-28T12:02:53.423710+0000","last_unstale":"2023-11-28T12:02:54.413769+0000","last_undegraded":"2023-11-28T12:02:54.413769+0000","last_fullsized":"2023-11-28T12:02:54.413769+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:52.828071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457260+0000","last_change":"2023-11-28T12:02:53.429927+0000","last_active":"2023-11-28T12:02:54.457260+0000","last_peered":"2023-11-28T12:02:54.457260+0000","last_clean":"2023-11-28T12:02:54.457260+0000","last_became_active":"2023-11-28T12:02:53.429643+0000","last_became_peered":"2023-11-28T12:02:53.429643+0000","last_unstale":"2023-11-28T12:02:54.457260+0000","last_undegraded":"2023-11-28T12:02:54.457260+0000","last_fullsized":"2023-11-28T12:02:54.457260+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:29.971331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.455991+0000","last_change":"2023-11-28T12:02:53.431690+0000","last_active":"2023-11-28T12:02:54.455991+0000","last_peered":"2023-11-28T12:02:54.455991+0000","last_clean":"2023-11-28T12:02:54.455991+0000","last_became_active":"2023-11-28T12:02:53.431250+0000","last_became_peered":"2023-11-28T12:02:53.431250+0000","last_unstale":"2023-11-28T12:02:54.455991+0000","last_undegraded":"2023-11-28T12:02:54.455991+0000","last_fullsized":"2023-11-28T12:02:54.455991+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:57:29.151797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457284+0000","last_change":"2023-11-28T12:02:53.430870+0000","last_active":"2023-11-28T12:02:54.457284+0000","last_peered":"2023-11-28T12:02:54.457284+0000","last_clean":"2023-11-28T12:02:54.457284+0000","last_became_active":"2023-11-28T12:02:53.430603+0000","last_became_peered":"2023-11-28T12:02:53.430603+0000","last_unstale":"2023-11-28T12:02:54.457284+0000","last_undegraded":"2023-11-28T12:02:54.457284+0000","last_fullsized":"2023-11-28T12:02:54.457284+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:25:36.902715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413213+0000","last_change":"2023-11-28T12:02:53.424074+0000","last_active":"2023-11-28T12:02:54.413213+0000","last_peered":"2023-11-28T12:02:54.413213+0000","last_clean":"2023-11-28T12:02:54.413213+0000","last_became_active":"2023-11-28T12:02:53.423755+0000","last_became_peered":"2023-11-28T12:02:53.423755+0000","last_unstale":"2023-11-28T12:02:54.413213+0000","last_undegraded":"2023-11-28T12:02:54.413213+0000","last_fullsized":"2023-11-28T12:02:54.413213+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:57:32.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456421+0000","last_change":"2023-11-28T12:02:53.423211+0000","last_active":"2023-11-28T12:02:54.456421+0000","last_peered":"2023-11-28T12:02:54.456421+0000","last_clean":"2023-11-28T12:02:54.456421+0000","last_became_active":"2023-11-28T12:02:53.422900+0000","last_became_peered":"2023-11-28T12:02:53.422900+0000","last_unstale":"2023-11-28T12:02:54.456421+0000","last_undegraded":"2023-11-28T12:02:54.456421+0000","last_fullsized":"2023-11-28T12:02:54.456421+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:59:07.837003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.412905+0000","last_change":"2023-11-28T12:02:53.429830+0000","last_active":"2023-11-28T12:02:54.412905+0000","last_peered":"2023-11-28T12:02:54.412905+0000","last_clean":"2023-11-28T12:02:54.412905+0000","last_became_active":"2023-11-28T12:02:53.429418+0000","last_became_peered":"2023-11-28T12:02:53.429418+0000","last_unstale":"2023-11-28T12:02:54.412905+0000","last_undegraded":"2023-11-28T12:02:54.412905+0000","last_fullsized":"2023-11-28T12:02:54.412905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:33.639319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456538+0000","last_change":"2023-11-28T12:02:53.428372+0000","last_active":"2023-11-28T12:02:54.456538+0000","last_peered":"2023-11-28T12:02:54.456538+0000","last_clean":"2023-11-28T12:02:54.456538+0000","last_became_active":"2023-11-28T12:02:53.428091+0000","last_became_peered":"2023-11-28T12:02:53.428091+0000","last_unstale":"2023-11-28T12:02:54.456538+0000","last_undegraded":"2023-11-28T12:02:54.456538+0000","last_fullsized":"2023-11-28T12:02:54.456538+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:29:16.559693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.412920+0000","last_change":"2023-11-28T12:02:53.425945+0000","last_active":"2023-11-28T12:02:54.412920+0000","last_peered":"2023-11-28T12:02:54.412920+0000","last_clean":"2023-11-28T12:02:54.412920+0000","last_became_active":"2023-11-28T12:02:53.424911+0000","last_became_peered":"2023-11-28T12:02:53.424911+0000","last_unstale":"2023-11-28T12:02:54.412920+0000","last_undegraded":"2023-11-28T12:02:54.412920+0000","last_fullsized":"2023-11-28T12:02:54.412920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:04.964774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456151+0000","last_change":"2023-11-28T12:02:53.428719+0000","last_active":"2023-11-28T12:02:54.456151+0000","last_peered":"2023-11-28T12:02:54.456151+0000","last_clean":"2023-11-28T12:02:54.456151+0000","last_became_active":"2023-11-28T12:02:53.428464+0000","last_became_peered":"2023-11-28T12:02:53.428464+0000","last_unstale":"2023-11-28T12:02:54.456151+0000","last_undegraded":"2023-11-28T12:02:54.456151+0000","last_fullsized":"2023-11-28T12:02:54.456151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:38.587636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.455912+0000","last_change":"2023-11-28T12:02:53.422609+0000","last_active":"2023-11-28T12:02:54.455912+0000","last_peered":"2023-11-28T12:02:54.455912+0000","last_clean":"2023-11-28T12:02:54.455912+0000","last_became_active":"2023-11-28T12:02:53.422326+0000","last_became_peered":"2023-11-28T12:02:53.422326+0000","last_unstale":"2023-11-28T12:02:54.455912+0000","last_undegraded":"2023-11-28T12:02:54.455912+0000","last_fullsized":"2023-11-28T12:02:54.455912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:58.499744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413394+0000","last_change":"2023-11-28T12:02:53.423455+0000","last_active":"2023-11-28T12:02:54.413394+0000","last_peered":"2023-11-28T12:02:54.413394+0000","last_clean":"2023-11-28T12:02:54.413394+0000","last_became_active":"2023-11-28T12:02:53.423172+0000","last_became_peered":"2023-11-28T12:02:53.423172+0000","last_unstale":"2023-11-28T12:02:54.413394+0000","last_undegraded":"2023-11-28T12:02:54.413394+0000","last_fullsized":"2023-11-28T12:02:54.413394+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:54.855708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456586+0000","last_change":"2023-11-28T12:02:53.432363+0000","last_active":"2023-11-28T12:02:54.456586+0000","last_peered":"2023-11-28T12:02:54.456586+0000","last_clean":"2023-11-28T12:02:54.456586+0000","last_became_active":"2023-11-28T12:02:53.432047+0000","last_became_peered":"2023-11-28T12:02:53.432047+0000","last_unstale":"2023-11-28T12:02:54.456586+0000","last_undegraded":"2023-11-28T12:02:54.456586+0000","last_fullsized":"2023-11-28T12:02:54.456586+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:40.682921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413244+0000","last_change":"2023-11-28T12:02:53.423976+0000","last_active":"2023-11-28T12:02:54.413244+0000","last_peered":"2023-11-28T12:02:54.413244+0000","last_clean":"2023-11-28T12:02:54.413244+0000","last_became_active":"2023-11-28T12:02:53.423299+0000","last_became_peered":"2023-11-28T12:02:53.423299+0000","last_unstale":"2023-11-28T12:02:54.413244+0000","last_undegraded":"2023-11-28T12:02:54.413244+0000","last_fullsized":"2023-11-28T12:02:54.413244+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:27.463305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456349+0000","last_change":"2023-11-28T12:02:53.433435+0000","last_active":"2023-11-28T12:02:54.456349+0000","last_peered":"2023-11-28T12:02:54.456349+0000","last_clean":"2023-11-28T12:02:54.456349+0000","last_became_active":"2023-11-28T12:02:53.433130+0000","last_became_peered":"2023-11-28T12:02:53.433130+0000","last_unstale":"2023-11-28T12:02:54.456349+0000","last_undegraded":"2023-11-28T12:02:54.456349+0000","last_fullsized":"2023-11-28T12:02:54.456349+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:33.606174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.412858+0000","last_change":"2023-11-28T12:02:53.424327+0000","last_active":"2023-11-28T12:02:54.412858+0000","last_peered":"2023-11-28T12:02:54.412858+0000","last_clean":"2023-11-28T12:02:54.412858+0000","last_became_active":"2023-11-28T12:02:53.422570+0000","last_became_peered":"2023-11-28T12:02:53.422570+0000","last_unstale":"2023-11-28T12:02:54.412858+0000","last_undegraded":"2023-11-28T12:02:54.412858+0000","last_fullsized":"2023-11-28T12:02:54.412858+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:09:07.273671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.412877+0000","last_change":"2023-11-28T12:02:53.434373+0000","last_active":"2023-11-28T12:02:54.412877+0000","last_peered":"2023-11-28T12:02:54.412877+0000","last_clean":"2023-11-28T12:02:54.412877+0000","last_became_active":"2023-11-28T12:02:53.434064+0000","last_became_peered":"2023-11-28T12:02:53.434064+0000","last_unstale":"2023-11-28T12:02:54.412877+0000","last_undegraded":"2023-11-28T12:02:54.412877+0000","last_fullsized":"2023-11-28T12:02:54.412877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:09:11.895327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456078+0000","last_change":"2023-11-28T12:02:53.420458+0000","last_active":"2023-11-28T12:02:54.456078+0000","last_peered":"2023-11-28T12:02:54.456078+0000","last_clean":"2023-11-28T12:02:54.456078+0000","last_became_active":"2023-11-28T12:02:53.420107+0000","last_became_peered":"2023-11-28T12:02:53.420107+0000","last_unstale":"2023-11-28T12:02:54.456078+0000","last_undegraded":"2023-11-28T12:02:54.456078+0000","last_fullsized":"2023-11-28T12:02:54.456078+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:04.866139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456033+0000","last_change":"2023-11-28T12:02:53.431783+0000","last_active":"2023-11-28T12:02:54.456033+0000","last_peered":"2023-11-28T12:02:54.456033+0000","last_clean":"2023-11-28T12:02:54.456033+0000","last_became_active":"2023-11-28T12:02:53.431523+0000","last_became_peered":"2023-11-28T12:02:53.431523+0000","last_unstale":"2023-11-28T12:02:54.456033+0000","last_undegraded":"2023-11-28T12:02:54.456033+0000","last_fullsized":"2023-11-28T12:02:54.456033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:04.866139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456658+0000","last_change":"2023-11-28T12:02:53.431006+0000","last_active":"2023-11-28T12:02:54.456658+0000","last_peered":"2023-11-28T12:02:54.456658+0000","last_clean":"2023-11-28T12:02:54.456658+0000","last_became_active":"2023-11-28T12:02:53.430747+0000","last_became_peered":"2023-11-28T12:02:53.430747+0000","last_unstale":"2023-11-28T12:02:54.456658+0000","last_undegraded":"2023-11-28T12:02:54.456658+0000","last_fullsized":"2023-11-28T12:02:54.456658+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:33.606174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"34'4","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413445+0000","last_change":"2023-11-28T12:02:52.970699+0000","last_active":"2023-11-28T12:02:54.413445+0000","last_peered":"2023-11-28T12:02:54.413445+0000","last_clean":"2023-11-28T12:02:54.413445+0000","last_became_active":"2023-11-28T12:02:52.970418+0000","last_became_peered":"2023-11-28T12:02:52.970418+0000","last_unstale":"2023-11-28T12:02:54.413445+0000","last_undegraded":"2023-11-28T12:02:54.413445+0000","last_fullsized":"2023-11-28T12:02:54.413445+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:59:06.822931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456775+0000","last_change":"2023-11-28T12:02:53.430375+0000","last_active":"2023-11-28T12:02:54.456775+0000","last_peered":"2023-11-28T12:02:54.456775+0000","last_clean":"2023-11-28T12:02:54.456775+0000","last_became_active":"2023-11-28T12:02:53.430109+0000","last_became_peered":"2023-11-28T12:02:53.430109+0000","last_unstale":"2023-11-28T12:02:54.456775+0000","last_undegraded":"2023-11-28T12:02:54.456775+0000","last_fullsized":"2023-11-28T12:02:54.456775+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:02.067084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456902+0000","last_change":"2023-11-28T12:02:52.971272+0000","last_active":"2023-11-28T12:02:54.456902+0000","last_peered":"2023-11-28T12:02:54.456902+0000","last_clean":"2023-11-28T12:02:54.456902+0000","last_became_active":"2023-11-28T12:02:52.970916+0000","last_became_peered":"2023-11-28T12:02:52.970916+0000","last_unstale":"2023-11-28T12:02:54.456902+0000","last_undegraded":"2023-11-28T12:02:54.456902+0000","last_fullsized":"2023-11-28T12:02:54.456902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:00:43.937765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413926+0000","last_change":"2023-11-28T12:02:53.423719+0000","last_active":"2023-11-28T12:02:54.413926+0000","last_peered":"2023-11-28T12:02:54.413926+0000","last_clean":"2023-11-28T12:02:54.413926+0000","last_became_active":"2023-11-28T12:02:53.423453+0000","last_became_peered":"2023-11-28T12:02:53.423453+0000","last_unstale":"2023-11-28T12:02:54.413926+0000","last_undegraded":"2023-11-28T12:02:54.413926+0000","last_fullsized":"2023-11-28T12:02:54.413926+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:30.218802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456924+0000","last_change":"2023-11-28T12:02:52.973621+0000","last_active":"2023-11-28T12:02:54.456924+0000","last_peered":"2023-11-28T12:02:54.456924+0000","last_clean":"2023-11-28T12:02:54.456924+0000","last_became_active":"2023-11-28T12:02:52.973239+0000","last_became_peered":"2023-11-28T12:02:52.973239+0000","last_unstale":"2023-11-28T12:02:54.456924+0000","last_undegraded":"2023-11-28T12:02:54.456924+0000","last_fullsized":"2023-11-28T12:02:54.456924+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:32.625247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413467+0000","last_change":"2023-11-28T12:02:53.423815+0000","last_active":"2023-11-28T12:02:54.413467+0000","last_peered":"2023-11-28T12:02:54.413467+0000","last_clean":"2023-11-28T12:02:54.413467+0000","last_became_active":"2023-11-28T12:02:53.423526+0000","last_became_peered":"2023-11-28T12:02:53.423526+0000","last_unstale":"2023-11-28T12:02:54.413467+0000","last_undegraded":"2023-11-28T12:02:54.413467+0000","last_fullsized":"2023-11-28T12:02:54.413467+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:04.964774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413820+0000","last_change":"2023-11-28T12:02:52.973240+0000","last_active":"2023-11-28T12:02:54.413820+0000","last_peered":"2023-11-28T12:02:54.413820+0000","last_clean":"2023-11-28T12:02:54.413820+0000","last_became_active":"2023-11-28T12:02:52.972819+0000","last_became_peered":"2023-11-28T12:02:52.972819+0000","last_unstale":"2023-11-28T12:02:54.413820+0000","last_undegraded":"2023-11-28T12:02:54.413820+0000","last_fullsized":"2023-11-28T12:02:54.413820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:48.088028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.455937+0000","last_change":"2023-11-28T12:02:53.422051+0000","last_active":"2023-11-28T12:02:54.455937+0000","last_peered":"2023-11-28T12:02:54.455937+0000","last_clean":"2023-11-28T12:02:54.455937+0000","last_became_active":"2023-11-28T12:02:53.421718+0000","last_became_peered":"2023-11-28T12:02:53.421718+0000","last_unstale":"2023-11-28T12:02:54.455937+0000","last_undegraded":"2023-11-28T12:02:54.455937+0000","last_fullsized":"2023-11-28T12:02:54.455937+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:57:29.151797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456414+0000","last_change":"2023-11-28T12:02:52.980130+0000","last_active":"2023-11-28T12:02:54.456414+0000","last_peered":"2023-11-28T12:02:54.456414+0000","last_clean":"2023-11-28T12:02:54.456414+0000","last_became_active":"2023-11-28T12:02:52.979862+0000","last_became_peered":"2023-11-28T12:02:52.979862+0000","last_unstale":"2023-11-28T12:02:54.456414+0000","last_undegraded":"2023-11-28T12:02:54.456414+0000","last_fullsized":"2023-11-28T12:02:54.456414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:57:31.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457267+0000","last_change":"2023-11-28T12:02:53.429333+0000","last_active":"2023-11-28T12:02:54.457267+0000","last_peered":"2023-11-28T12:02:54.457267+0000","last_clean":"2023-11-28T12:02:54.457267+0000","last_became_active":"2023-11-28T12:02:53.428978+0000","last_became_peered":"2023-11-28T12:02:53.428978+0000","last_unstale":"2023-11-28T12:02:54.457267+0000","last_undegraded":"2023-11-28T12:02:54.457267+0000","last_fullsized":"2023-11-28T12:02:54.457267+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:49.938647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456413+0000","last_change":"2023-11-28T12:02:52.976849+0000","last_active":"2023-11-28T12:02:54.456413+0000","last_peered":"2023-11-28T12:02:54.456413+0000","last_clean":"2023-11-28T12:02:54.456413+0000","last_became_active":"2023-11-28T12:02:52.976552+0000","last_became_peered":"2023-11-28T12:02:52.976552+0000","last_unstale":"2023-11-28T12:02:54.456413+0000","last_undegraded":"2023-11-28T12:02:54.456413+0000","last_fullsized":"2023-11-28T12:02:54.456413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:26.679201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413624+0000","last_change":"2023-11-28T12:02:53.431299+0000","last_active":"2023-11-28T12:02:54.413624+0000","last_peered":"2023-11-28T12:02:54.413624+0000","last_clean":"2023-11-28T12:02:54.413624+0000","last_became_active":"2023-11-28T12:02:53.431025+0000","last_became_peered":"2023-11-28T12:02:53.431025+0000","last_unstale":"2023-11-28T12:02:54.413624+0000","last_undegraded":"2023-11-28T12:02:54.413624+0000","last_fullsized":"2023-11-28T12:02:54.413624+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:33.520090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456423+0000","last_change":"2023-11-28T12:02:52.977272+0000","last_active":"2023-11-28T12:02:54.456423+0000","last_peered":"2023-11-28T12:02:54.456423+0000","last_clean":"2023-11-28T12:02:54.456423+0000","last_became_active":"2023-11-28T12:02:52.976980+0000","last_became_peered":"2023-11-28T12:02:52.976980+0000","last_unstale":"2023-11-28T12:02:54.456423+0000","last_undegraded":"2023-11-28T12:02:54.456423+0000","last_fullsized":"2023-11-28T12:02:54.456423+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:04.556861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413309+0000","last_change":"2023-11-28T12:02:53.420853+0000","last_active":"2023-11-28T12:02:54.413309+0000","last_peered":"2023-11-28T12:02:54.413309+0000","last_clean":"2023-11-28T12:02:54.413309+0000","last_became_active":"2023-11-28T12:02:53.420410+0000","last_became_peered":"2023-11-28T12:02:53.420410+0000","last_unstale":"2023-11-28T12:02:54.413309+0000","last_undegraded":"2023-11-28T12:02:54.413309+0000","last_fullsized":"2023-11-28T12:02:54.413309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:22:19.291125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.412617+0000","last_change":"2023-11-28T12:02:52.966640+0000","last_active":"2023-11-28T12:02:54.412617+0000","last_peered":"2023-11-28T12:02:54.412617+0000","last_clean":"2023-11-28T12:02:54.412617+0000","last_became_active":"2023-11-28T12:02:52.966367+0000","last_became_peered":"2023-11-28T12:02:52.966367+0000","last_unstale":"2023-11-28T12:02:54.412617+0000","last_undegraded":"2023-11-28T12:02:54.412617+0000","last_fullsized":"2023-11-28T12:02:54.412617+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:48.924576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456642+0000","last_change":"2023-11-28T12:02:53.429341+0000","last_active":"2023-11-28T12:02:54.456642+0000","last_peered":"2023-11-28T12:02:54.456642+0000","last_clean":"2023-11-28T12:02:54.456642+0000","last_became_active":"2023-11-28T12:02:53.425238+0000","last_became_peered":"2023-11-28T12:02:53.425238+0000","last_unstale":"2023-11-28T12:02:54.456642+0000","last_undegraded":"2023-11-28T12:02:54.456642+0000","last_fullsized":"2023-11-28T12:02:54.456642+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:58.447498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413577+0000","last_change":"2023-11-28T12:02:52.975369+0000","last_active":"2023-11-28T12:02:54.413577+0000","last_peered":"2023-11-28T12:02:54.413577+0000","last_clean":"2023-11-28T12:02:54.413577+0000","last_became_active":"2023-11-28T12:02:52.975069+0000","last_became_peered":"2023-11-28T12:02:52.975069+0000","last_unstale":"2023-11-28T12:02:54.413577+0000","last_undegraded":"2023-11-28T12:02:54.413577+0000","last_fullsized":"2023-11-28T12:02:54.413577+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:37.573564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456722+0000","last_change":"2023-11-28T12:02:53.421660+0000","last_active":"2023-11-28T12:02:54.456722+0000","last_peered":"2023-11-28T12:02:54.456722+0000","last_clean":"2023-11-28T12:02:54.456722+0000","last_became_active":"2023-11-28T12:02:53.421331+0000","last_became_peered":"2023-11-28T12:02:53.421331+0000","last_unstale":"2023-11-28T12:02:54.456722+0000","last_undegraded":"2023-11-28T12:02:54.456722+0000","last_fullsized":"2023-11-28T12:02:54.456722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:29.971331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456922+0000","last_change":"2023-11-28T12:02:52.978378+0000","last_active":"2023-11-28T12:02:54.456922+0000","last_peered":"2023-11-28T12:02:54.456922+0000","last_clean":"2023-11-28T12:02:54.456922+0000","last_became_active":"2023-11-28T12:02:52.978129+0000","last_became_peered":"2023-11-28T12:02:52.978129+0000","last_unstale":"2023-11-28T12:02:54.456922+0000","last_undegraded":"2023-11-28T12:02:54.456922+0000","last_fullsized":"2023-11-28T12:02:54.456922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:04.556861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456843+0000","last_change":"2023-11-28T12:02:53.421981+0000","last_active":"2023-11-28T12:02:54.456843+0000","last_peered":"2023-11-28T12:02:54.456843+0000","last_clean":"2023-11-28T12:02:54.456843+0000","last_became_active":"2023-11-28T12:02:53.421724+0000","last_became_peered":"2023-11-28T12:02:53.421724+0000","last_unstale":"2023-11-28T12:02:54.456843+0000","last_undegraded":"2023-11-28T12:02:54.456843+0000","last_fullsized":"2023-11-28T12:02:54.456843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:34.704261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456806+0000","last_change":"2023-11-28T12:02:52.977588+0000","last_active":"2023-11-28T12:02:54.456806+0000","last_peered":"2023-11-28T12:02:54.456806+0000","last_clean":"2023-11-28T12:02:54.456806+0000","last_became_active":"2023-11-28T12:02:52.977272+0000","last_became_peered":"2023-11-28T12:02:52.977272+0000","last_unstale":"2023-11-28T12:02:54.456806+0000","last_undegraded":"2023-11-28T12:02:54.456806+0000","last_fullsized":"2023-11-28T12:02:54.456806+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:50.678023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413276+0000","last_change":"2023-11-28T12:02:53.431964+0000","last_active":"2023-11-28T12:02:54.413276+0000","last_peered":"2023-11-28T12:02:54.413276+0000","last_clean":"2023-11-28T12:02:54.413276+0000","last_became_active":"2023-11-28T12:02:53.431359+0000","last_became_peered":"2023-11-28T12:02:53.431359+0000","last_unstale":"2023-11-28T12:02:54.413276+0000","last_undegraded":"2023-11-28T12:02:54.413276+0000","last_fullsized":"2023-11-28T12:02:54.413276+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:09.337505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"34'6","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413542+0000","last_change":"2023-11-28T12:02:52.971689+0000","last_active":"2023-11-28T12:02:54.413542+0000","last_peered":"2023-11-28T12:02:54.413542+0000","last_clean":"2023-11-28T12:02:54.413542+0000","last_became_active":"2023-11-28T12:02:52.971397+0000","last_became_peered":"2023-11-28T12:02:52.971397+0000","last_unstale":"2023-11-28T12:02:54.413542+0000","last_undegraded":"2023-11-28T12:02:54.413542+0000","last_fullsized":"2023-11-28T12:02:54.413542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:03.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456207+0000","last_change":"2023-11-28T12:02:53.428756+0000","last_active":"2023-11-28T12:02:54.456207+0000","last_peered":"2023-11-28T12:02:54.456207+0000","last_clean":"2023-11-28T12:02:54.456207+0000","last_became_active":"2023-11-28T12:02:53.428439+0000","last_became_peered":"2023-11-28T12:02:53.428439+0000","last_unstale":"2023-11-28T12:02:54.456207+0000","last_undegraded":"2023-11-28T12:02:54.456207+0000","last_fullsized":"2023-11-28T12:02:54.456207+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:58:27.198338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413304+0000","last_change":"2023-11-28T12:02:52.972607+0000","last_active":"2023-11-28T12:02:54.413304+0000","last_peered":"2023-11-28T12:02:54.413304+0000","last_clean":"2023-11-28T12:02:54.413304+0000","last_became_active":"2023-11-28T12:02:52.972323+0000","last_became_peered":"2023-11-28T12:02:52.972323+0000","last_unstale":"2023-11-28T12:02:54.413304+0000","last_undegraded":"2023-11-28T12:02:54.413304+0000","last_fullsized":"2023-11-28T12:02:54.413304+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:48.088028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413176+0000","last_change":"2023-11-28T12:02:53.423855+0000","last_active":"2023-11-28T12:02:54.413176+0000","last_peered":"2023-11-28T12:02:54.413176+0000","last_clean":"2023-11-28T12:02:54.413176+0000","last_became_active":"2023-11-28T12:02:53.423561+0000","last_became_peered":"2023-11-28T12:02:53.423561+0000","last_unstale":"2023-11-28T12:02:54.413176+0000","last_undegraded":"2023-11-28T12:02:54.413176+0000","last_fullsized":"2023-11-28T12:02:54.413176+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:52.468097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456441+0000","last_change":"2023-11-28T12:02:52.975551+0000","last_active":"2023-11-28T12:02:54.456441+0000","last_peered":"2023-11-28T12:02:54.456441+0000","last_clean":"2023-11-28T12:02:54.456441+0000","last_became_active":"2023-11-28T12:02:52.975144+0000","last_became_peered":"2023-11-28T12:02:52.975144+0000","last_unstale":"2023-11-28T12:02:54.456441+0000","last_undegraded":"2023-11-28T12:02:54.456441+0000","last_fullsized":"2023-11-28T12:02:54.456441+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:50.678023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456133+0000","last_change":"2023-11-28T12:02:53.425607+0000","last_active":"2023-11-28T12:02:54.456133+0000","last_peered":"2023-11-28T12:02:54.456133+0000","last_clean":"2023-11-28T12:02:54.456133+0000","last_became_active":"2023-11-28T12:02:53.423207+0000","last_became_peered":"2023-11-28T12:02:53.423207+0000","last_unstale":"2023-11-28T12:02:54.456133+0000","last_undegraded":"2023-11-28T12:02:54.456133+0000","last_fullsized":"2023-11-28T12:02:54.456133+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:49.938647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413548+0000","last_change":"2023-11-28T12:02:52.969795+0000","last_active":"2023-11-28T12:02:54.413548+0000","last_peered":"2023-11-28T12:02:54.413548+0000","last_clean":"2023-11-28T12:02:54.413548+0000","last_became_active":"2023-11-28T12:02:52.969515+0000","last_became_peered":"2023-11-28T12:02:52.969515+0000","last_unstale":"2023-11-28T12:02:54.413548+0000","last_undegraded":"2023-11-28T12:02:54.413548+0000","last_fullsized":"2023-11-28T12:02:54.413548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:57.485672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413178+0000","last_change":"2023-11-28T12:02:53.427776+0000","last_active":"2023-11-28T12:02:54.413178+0000","last_peered":"2023-11-28T12:02:54.413178+0000","last_clean":"2023-11-28T12:02:54.413178+0000","last_became_active":"2023-11-28T12:02:53.427494+0000","last_became_peered":"2023-11-28T12:02:53.427494+0000","last_unstale":"2023-11-28T12:02:54.413178+0000","last_undegraded":"2023-11-28T12:02:54.413178+0000","last_fullsized":"2023-11-28T12:02:54.413178+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:33.520090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"34'8","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456102+0000","last_change":"2023-11-28T12:02:52.976780+0000","last_active":"2023-11-28T12:02:54.456102+0000","last_peered":"2023-11-28T12:02:54.456102+0000","last_clean":"2023-11-28T12:02:54.456102+0000","last_became_active":"2023-11-28T12:02:52.976507+0000","last_became_peered":"2023-11-28T12:02:52.976507+0000","last_unstale":"2023-11-28T12:02:54.456102+0000","last_undegraded":"2023-11-28T12:02:54.456102+0000","last_fullsized":"2023-11-28T12:02:54.456102+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:32.625247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413434+0000","last_change":"2023-11-28T12:02:53.419256+0000","last_active":"2023-11-28T12:02:54.413434+0000","last_peered":"2023-11-28T12:02:54.413434+0000","last_clean":"2023-11-28T12:02:54.413434+0000","last_became_active":"2023-11-28T12:02:53.418953+0000","last_became_peered":"2023-11-28T12:02:53.418953+0000","last_unstale":"2023-11-28T12:02:54.413434+0000","last_undegraded":"2023-11-28T12:02:54.413434+0000","last_fullsized":"2023-11-28T12:02:54.413434+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:52.828071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456714+0000","last_change":"2023-11-28T12:02:52.977172+0000","last_active":"2023-11-28T12:02:54.456714+0000","last_peered":"2023-11-28T12:02:54.456714+0000","last_clean":"2023-11-28T12:02:54.456714+0000","last_became_active":"2023-11-28T12:02:52.976724+0000","last_became_peered":"2023-11-28T12:02:52.976724+0000","last_unstale":"2023-11-28T12:02:54.456714+0000","last_undegraded":"2023-11-28T12:02:54.456714+0000","last_fullsized":"2023-11-28T12:02:54.456714+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:26.679201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456482+0000","last_change":"2023-11-28T12:02:53.428368+0000","last_active":"2023-11-28T12:02:54.456482+0000","last_peered":"2023-11-28T12:02:54.456482+0000","last_clean":"2023-11-28T12:02:54.456482+0000","last_became_active":"2023-11-28T12:02:53.428058+0000","last_became_peered":"2023-11-28T12:02:53.428058+0000","last_unstale":"2023-11-28T12:02:54.456482+0000","last_undegraded":"2023-11-28T12:02:54.456482+0000","last_fullsized":"2023-11-28T12:02:54.456482+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:49.102100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.414034+0000","last_change":"2023-11-28T12:02:52.975455+0000","last_active":"2023-11-28T12:02:54.414034+0000","last_peered":"2023-11-28T12:02:54.414034+0000","last_clean":"2023-11-28T12:02:54.414034+0000","last_became_active":"2023-11-28T12:02:52.975190+0000","last_became_peered":"2023-11-28T12:02:52.975190+0000","last_unstale":"2023-11-28T12:02:54.414034+0000","last_undegraded":"2023-11-28T12:02:54.414034+0000","last_fullsized":"2023-11-28T12:02:54.414034+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:29:15.545621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413981+0000","last_change":"2023-11-28T12:02:53.420843+0000","last_active":"2023-11-28T12:02:54.413981+0000","last_peered":"2023-11-28T12:02:54.413981+0000","last_clean":"2023-11-28T12:02:54.413981+0000","last_became_active":"2023-11-28T12:02:53.420573+0000","last_became_peered":"2023-11-28T12:02:53.420573+0000","last_unstale":"2023-11-28T12:02:54.413981+0000","last_undegraded":"2023-11-28T12:02:54.413981+0000","last_fullsized":"2023-11-28T12:02:54.413981+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:30.182850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456491+0000","last_change":"2023-11-28T12:02:52.976363+0000","last_active":"2023-11-28T12:02:54.456491+0000","last_peered":"2023-11-28T12:02:54.456491+0000","last_clean":"2023-11-28T12:02:54.456491+0000","last_became_active":"2023-11-28T12:02:52.975991+0000","last_became_peered":"2023-11-28T12:02:52.975991+0000","last_unstale":"2023-11-28T12:02:54.456491+0000","last_undegraded":"2023-11-28T12:02:54.456491+0000","last_fullsized":"2023-11-28T12:02:54.456491+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:49.406926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456847+0000","last_change":"2023-11-28T12:02:53.428127+0000","last_active":"2023-11-28T12:02:54.456847+0000","last_peered":"2023-11-28T12:02:54.456847+0000","last_clean":"2023-11-28T12:02:54.456847+0000","last_became_active":"2023-11-28T12:02:53.427562+0000","last_became_peered":"2023-11-28T12:02:53.427562+0000","last_unstale":"2023-11-28T12:02:54.456847+0000","last_undegraded":"2023-11-28T12:02:54.456847+0000","last_fullsized":"2023-11-28T12:02:54.456847+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:55.967764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457303+0000","last_change":"2023-11-28T12:02:52.971336+0000","last_active":"2023-11-28T12:02:54.457303+0000","last_peered":"2023-11-28T12:02:54.457303+0000","last_clean":"2023-11-28T12:02:54.457303+0000","last_became_active":"2023-11-28T12:02:52.971068+0000","last_became_peered":"2023-11-28T12:02:52.971068+0000","last_unstale":"2023-11-28T12:02:54.457303+0000","last_undegraded":"2023-11-28T12:02:54.457303+0000","last_fullsized":"2023-11-28T12:02:54.457303+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:26.679201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456066+0000","last_change":"2023-11-28T12:02:53.431509+0000","last_active":"2023-11-28T12:02:54.456066+0000","last_peered":"2023-11-28T12:02:54.456066+0000","last_clean":"2023-11-28T12:02:54.456066+0000","last_became_active":"2023-11-28T12:02:53.430231+0000","last_became_peered":"2023-11-28T12:02:53.430231+0000","last_unstale":"2023-11-28T12:02:54.456066+0000","last_undegraded":"2023-11-28T12:02:54.456066+0000","last_fullsized":"2023-11-28T12:02:54.456066+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:34.704261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456132+0000","last_change":"2023-11-28T12:02:52.973243+0000","last_active":"2023-11-28T12:02:54.456132+0000","last_peered":"2023-11-28T12:02:54.456132+0000","last_clean":"2023-11-28T12:02:54.456132+0000","last_became_active":"2023-11-28T12:02:52.972875+0000","last_became_peered":"2023-11-28T12:02:52.972875+0000","last_unstale":"2023-11-28T12:02:54.456132+0000","last_undegraded":"2023-11-28T12:02:54.456132+0000","last_fullsized":"2023-11-28T12:02:54.456132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:39:56.165145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456017+0000","last_change":"2023-11-28T12:02:53.427161+0000","last_active":"2023-11-28T12:02:54.456017+0000","last_peered":"2023-11-28T12:02:54.456017+0000","last_clean":"2023-11-28T12:02:54.456017+0000","last_became_active":"2023-11-28T12:02:53.426793+0000","last_became_peered":"2023-11-28T12:02:53.426793+0000","last_unstale":"2023-11-28T12:02:54.456017+0000","last_undegraded":"2023-11-28T12:02:54.456017+0000","last_fullsized":"2023-11-28T12:02:54.456017+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:33.606174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456302+0000","last_change":"2023-11-28T12:02:52.978335+0000","last_active":"2023-11-28T12:02:54.456302+0000","last_peered":"2023-11-28T12:02:54.456302+0000","last_clean":"2023-11-28T12:02:54.456302+0000","last_became_active":"2023-11-28T12:02:52.978042+0000","last_became_peered":"2023-11-28T12:02:52.978042+0000","last_unstale":"2023-11-28T12:02:54.456302+0000","last_undegraded":"2023-11-28T12:02:54.456302+0000","last_fullsized":"2023-11-28T12:02:54.456302+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:21:46.803306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457146+0000","last_change":"2023-11-28T12:02:53.428424+0000","last_active":"2023-11-28T12:02:54.457146+0000","last_peered":"2023-11-28T12:02:54.457146+0000","last_clean":"2023-11-28T12:02:54.457146+0000","last_became_active":"2023-11-28T12:02:53.428118+0000","last_became_peered":"2023-11-28T12:02:53.428118+0000","last_unstale":"2023-11-28T12:02:54.457146+0000","last_undegraded":"2023-11-28T12:02:54.457146+0000","last_fullsized":"2023-11-28T12:02:54.457146+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:02.067084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456196+0000","last_change":"2023-11-28T12:02:52.975477+0000","last_active":"2023-11-28T12:02:54.456196+0000","last_peered":"2023-11-28T12:02:54.456196+0000","last_clean":"2023-11-28T12:02:54.456196+0000","last_became_active":"2023-11-28T12:02:52.975205+0000","last_became_peered":"2023-11-28T12:02:52.975205+0000","last_unstale":"2023-11-28T12:02:54.456196+0000","last_undegraded":"2023-11-28T12:02:54.456196+0000","last_fullsized":"2023-11-28T12:02:54.456196+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:50.678023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413190+0000","last_change":"2023-11-28T12:02:53.412062+0000","last_active":"2023-11-28T12:02:54.413190+0000","last_peered":"2023-11-28T12:02:54.413190+0000","last_clean":"2023-11-28T12:02:54.413190+0000","last_became_active":"2023-11-28T12:02:53.411795+0000","last_became_peered":"2023-11-28T12:02:53.411795+0000","last_unstale":"2023-11-28T12:02:54.413190+0000","last_undegraded":"2023-11-28T12:02:54.413190+0000","last_fullsized":"2023-11-28T12:02:54.413190+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:09.337505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457063+0000","last_change":"2023-11-28T12:02:52.978001+0000","last_active":"2023-11-28T12:02:54.457063+0000","last_peered":"2023-11-28T12:02:54.457063+0000","last_clean":"2023-11-28T12:02:54.457063+0000","last_became_active":"2023-11-28T12:02:52.977718+0000","last_became_peered":"2023-11-28T12:02:52.977718+0000","last_unstale":"2023-11-28T12:02:54.457063+0000","last_undegraded":"2023-11-28T12:02:54.457063+0000","last_fullsized":"2023-11-28T12:02:54.457063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:00.920928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413740+0000","last_change":"2023-11-28T12:02:53.412545+0000","last_active":"2023-11-28T12:02:54.413740+0000","last_peered":"2023-11-28T12:02:54.413740+0000","last_clean":"2023-11-28T12:02:54.413740+0000","last_became_active":"2023-11-28T12:02:53.412237+0000","last_became_peered":"2023-11-28T12:02:53.412237+0000","last_unstale":"2023-11-28T12:02:54.413740+0000","last_undegraded":"2023-11-28T12:02:54.413740+0000","last_fullsized":"2023-11-28T12:02:54.413740+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:31.438055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413218+0000","last_change":"2023-11-28T12:02:52.964204+0000","last_active":"2023-11-28T12:02:54.413218+0000","last_peered":"2023-11-28T12:02:54.413218+0000","last_clean":"2023-11-28T12:02:54.413218+0000","last_became_active":"2023-11-28T12:02:52.963932+0000","last_became_peered":"2023-11-28T12:02:52.963932+0000","last_unstale":"2023-11-28T12:02:54.413218+0000","last_undegraded":"2023-11-28T12:02:54.413218+0000","last_fullsized":"2023-11-28T12:02:54.413218+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:03.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.409022+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:02:53.409022+0000","last_peered":"2023-11-28T12:02:53.409022+0000","last_clean":"2023-11-28T12:02:53.409022+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:02:53.409022+0000","last_undegraded":"2023-11-28T12:02:53.409022+0000","last_fullsized":"2023-11-28T12:02:53.409022+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456473+0000","last_change":"2023-11-28T12:02:53.422244+0000","last_active":"2023-11-28T12:02:54.456473+0000","last_peered":"2023-11-28T12:02:54.456473+0000","last_clean":"2023-11-28T12:02:54.456473+0000","last_became_active":"2023-11-28T12:02:53.421984+0000","last_became_peered":"2023-11-28T12:02:53.421984+0000","last_unstale":"2023-11-28T12:02:54.456473+0000","last_undegraded":"2023-11-28T12:02:54.456473+0000","last_fullsized":"2023-11-28T12:02:54.456473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:49.102100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413070+0000","last_change":"2023-11-28T12:02:52.970359+0000","last_active":"2023-11-28T12:02:54.413070+0000","last_peered":"2023-11-28T12:02:54.413070+0000","last_clean":"2023-11-28T12:02:54.413070+0000","last_became_active":"2023-11-28T12:02:52.970057+0000","last_became_peered":"2023-11-28T12:02:52.970057+0000","last_unstale":"2023-11-28T12:02:54.413070+0000","last_undegraded":"2023-11-28T12:02:54.413070+0000","last_fullsized":"2023-11-28T12:02:54.413070+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:01.053013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":29,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.408303+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:02:53.408303+0000","last_peered":"2023-11-28T12:02:53.408303+0000","last_clean":"2023-11-28T12:02:53.408303+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:02:53.408303+0000","last_undegraded":"2023-11-28T12:02:53.408303+0000","last_fullsized":"2023-11-28T12:02:53.408303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457187+0000","last_change":"2023-11-28T12:02:53.430056+0000","last_active":"2023-11-28T12:02:54.457187+0000","last_peered":"2023-11-28T12:02:54.457187+0000","last_clean":"2023-11-28T12:02:54.457187+0000","last_became_active":"2023-11-28T12:02:53.429788+0000","last_became_peered":"2023-11-28T12:02:53.429788+0000","last_unstale":"2023-11-28T12:02:54.457187+0000","last_undegraded":"2023-11-28T12:02:54.457187+0000","last_fullsized":"2023-11-28T12:02:54.457187+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:38.587636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456866+0000","last_change":"2023-11-28T12:02:52.977390+0000","last_active":"2023-11-28T12:02:54.456866+0000","last_peered":"2023-11-28T12:02:54.456866+0000","last_clean":"2023-11-28T12:02:54.456866+0000","last_became_active":"2023-11-28T12:02:52.977100+0000","last_became_peered":"2023-11-28T12:02:52.977100+0000","last_unstale":"2023-11-28T12:02:54.456866+0000","last_undegraded":"2023-11-28T12:02:54.456866+0000","last_fullsized":"2023-11-28T12:02:54.456866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:32.625247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.414654+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:02:53.414654+0000","last_peered":"2023-11-28T12:02:53.414654+0000","last_clean":"2023-11-28T12:02:53.414654+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:02:53.414654+0000","last_undegraded":"2023-11-28T12:02:53.414654+0000","last_fullsized":"2023-11-28T12:02:53.414654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":174,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.409457+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:02:53.409457+0000","last_peered":"2023-11-28T12:02:53.409457+0000","last_clean":"2023-11-28T12:02:53.409457+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:02:53.409457+0000","last_undegraded":"2023-11-28T12:02:53.409457+0000","last_fullsized":"2023-11-28T12:02:53.409457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456821+0000","last_change":"2023-11-28T12:02:53.422423+0000","last_active":"2023-11-28T12:02:54.456821+0000","last_peered":"2023-11-28T12:02:54.456821+0000","last_clean":"2023-11-28T12:02:54.456821+0000","last_became_active":"2023-11-28T12:02:53.420944+0000","last_became_peered":"2023-11-28T12:02:53.420944+0000","last_unstale":"2023-11-28T12:02:54.456821+0000","last_undegraded":"2023-11-28T12:02:54.456821+0000","last_fullsized":"2023-11-28T12:02:54.456821+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:29:16.559693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:55.080492+0000","last_change":"2023-11-28T12:02:52.976180+0000","last_active":"2023-11-28T12:02:55.080492+0000","last_peered":"2023-11-28T12:02:55.080492+0000","last_clean":"2023-11-28T12:02:55.080492+0000","last_became_active":"2023-11-28T12:02:52.975850+0000","last_became_peered":"2023-11-28T12:02:52.975850+0000","last_unstale":"2023-11-28T12:02:55.080492+0000","last_undegraded":"2023-11-28T12:02:55.080492+0000","last_fullsized":"2023-11-28T12:02:55.080492+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:16.116447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.408998+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:02:53.408998+0000","last_peered":"2023-11-28T12:02:53.408998+0000","last_clean":"2023-11-28T12:02:53.408998+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:02:53.408998+0000","last_undegraded":"2023-11-28T12:02:53.408998+0000","last_fullsized":"2023-11-28T12:02:53.408998+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456705+0000","last_change":"2023-11-28T12:02:53.433298+0000","last_active":"2023-11-28T12:02:54.456705+0000","last_peered":"2023-11-28T12:02:54.456705+0000","last_clean":"2023-11-28T12:02:54.456705+0000","last_became_active":"2023-11-28T12:02:53.432985+0000","last_became_peered":"2023-11-28T12:02:53.432985+0000","last_unstale":"2023-11-28T12:02:54.456705+0000","last_undegraded":"2023-11-28T12:02:54.456705+0000","last_fullsized":"2023-11-28T12:02:54.456705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:34.704261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456757+0000","last_change":"2023-11-28T12:02:52.977173+0000","last_active":"2023-11-28T12:02:54.456757+0000","last_peered":"2023-11-28T12:02:54.456757+0000","last_clean":"2023-11-28T12:02:54.456757+0000","last_became_active":"2023-11-28T12:02:52.976856+0000","last_became_peered":"2023-11-28T12:02:52.976856+0000","last_unstale":"2023-11-28T12:02:54.456757+0000","last_undegraded":"2023-11-28T12:02:54.456757+0000","last_fullsized":"2023-11-28T12:02:54.456757+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:57:31.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.409355+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:02:53.409355+0000","last_peered":"2023-11-28T12:02:53.409355+0000","last_clean":"2023-11-28T12:02:53.409355+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:02:53.409355+0000","last_undegraded":"2023-11-28T12:02:53.409355+0000","last_fullsized":"2023-11-28T12:02:53.409355+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.413412+0000","last_change":"2023-11-28T12:02:53.411661+0000","last_active":"2023-11-28T12:02:54.413412+0000","last_peered":"2023-11-28T12:02:54.413412+0000","last_clean":"2023-11-28T12:02:54.413412+0000","last_became_active":"2023-11-28T12:02:53.411367+0000","last_became_peered":"2023-11-28T12:02:53.411367+0000","last_unstale":"2023-11-28T12:02:54.413412+0000","last_undegraded":"2023-11-28T12:02:54.413412+0000","last_fullsized":"2023-11-28T12:02:54.413412+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:34.704261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456996+0000","last_change":"2023-11-28T12:02:52.972309+0000","last_active":"2023-11-28T12:02:54.456996+0000","last_peered":"2023-11-28T12:02:54.456996+0000","last_clean":"2023-11-28T12:02:54.456996+0000","last_became_active":"2023-11-28T12:02:52.971984+0000","last_became_peered":"2023-11-28T12:02:52.971984+0000","last_unstale":"2023-11-28T12:02:54.456996+0000","last_undegraded":"2023-11-28T12:02:54.456996+0000","last_fullsized":"2023-11-28T12:02:54.456996+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:54:35.436932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.422184+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:02:53.422184+0000","last_peered":"2023-11-28T12:02:53.422184+0000","last_clean":"2023-11-28T12:02:53.422184+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:02:53.422184+0000","last_undegraded":"2023-11-28T12:02:53.422184+0000","last_fullsized":"2023-11-28T12:02:53.422184+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456336+0000","last_change":"2023-11-28T12:02:53.419953+0000","last_active":"2023-11-28T12:02:54.456336+0000","last_peered":"2023-11-28T12:02:54.456336+0000","last_clean":"2023-11-28T12:02:54.456336+0000","last_became_active":"2023-11-28T12:02:53.419617+0000","last_became_peered":"2023-11-28T12:02:53.419617+0000","last_unstale":"2023-11-28T12:02:54.456336+0000","last_undegraded":"2023-11-28T12:02:54.456336+0000","last_fullsized":"2023-11-28T12:02:54.456336+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:17:04.593407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.412912+0000","last_change":"2023-11-28T12:02:52.970894+0000","last_active":"2023-11-28T12:02:54.412912+0000","last_peered":"2023-11-28T12:02:54.412912+0000","last_clean":"2023-11-28T12:02:54.412912+0000","last_became_active":"2023-11-28T12:02:52.970072+0000","last_became_peered":"2023-11-28T12:02:52.970072+0000","last_unstale":"2023-11-28T12:02:54.412912+0000","last_undegraded":"2023-11-28T12:02:54.412912+0000","last_fullsized":"2023-11-28T12:02:54.412912+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:55.568072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.415975+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:02:53.415975+0000","last_peered":"2023-11-28T12:02:53.415975+0000","last_clean":"2023-11-28T12:02:53.415975+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:02:53.415975+0000","last_undegraded":"2023-11-28T12:02:53.415975+0000","last_fullsized":"2023-11-28T12:02:53.415975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.456101+0000","last_change":"2023-11-28T12:02:53.435138+0000","last_active":"2023-11-28T12:02:54.456101+0000","last_peered":"2023-11-28T12:02:54.456101+0000","last_clean":"2023-11-28T12:02:54.456101+0000","last_became_active":"2023-11-28T12:02:53.434289+0000","last_became_peered":"2023-11-28T12:02:53.434289+0000","last_unstale":"2023-11-28T12:02:54.456101+0000","last_undegraded":"2023-11-28T12:02:54.456101+0000","last_fullsized":"2023-11-28T12:02:54.456101+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:52.947300+0000","last_clean_scrub_stamp":"2023-11-28T12:02:52.947300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:25:36.902715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T12:02:54.457292+0000","last_change":"2023-11-28T12:02:52.968838+0000","last_active":"2023-11-28T12:02:54.457292+0000","last_peered":"2023-11-28T12:02:54.457292+0000","last_clean":"2023-11-28T12:02:54.457292+0000","last_became_active":"2023-11-28T12:02:52.968475+0000","last_became_peered":"2023-11-28T12:02:52.968475+0000","last_unstale":"2023-11-28T12:02:54.457292+0000","last_undegraded":"2023-11-28T12:02:54.457292+0000","last_fullsized":"2023-11-28T12:02:54.457292+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:02:51.933228+0000","last_clean_scrub_stamp":"2023-11-28T12:02:51.933228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:57.068408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T12:02:53.416348+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:02:53.416348+0000","last_peered":"2023-11-28T12:02:53.416348+0000","last_clean":"2023-11-28T12:02:53.416348+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:02:53.416348+0000","last_undegraded":"2023-11-28T12:02:53.416348+0000","last_fullsized":"2023-11-28T12:02:53.416348+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9568,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362272,"statfs":{"total":96636764160,"available":96626966528,"internally_reserved":0,"allocated":1007616,"data_stored":757235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.67100000000000004}]},{"osd":1,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.7460000000000004,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.69599999999999995}]},{"osd":2,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":7.7789999999999999,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68300000000000005}]},{"osd":3,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.8199999999999998,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.65500000000000003}]},{"osd":4,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.439}]},{"osd":5,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47099999999999997}]},{"osd":6,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.49199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":401408,"data_stored":165815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.66500000000000004}]},{"osd":1,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.60299999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.69299999999999995}]},{"osd":3,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.70799999999999996}]},{"osd":4,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.49099999999999999}]},{"osd":5,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53000000000000003}]},{"osd":7,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.55100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9492,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362348,"statfs":{"total":96636764160,"available":96627044352,"internally_reserved":0,"allocated":995328,"data_stored":755707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.33600000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.45100000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.29799999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.66000000000000003}]},{"osd":5,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.67700000000000005}]},{"osd":6,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.60099999999999998}]},{"osd":7,"last update":"Tue Nov 28 12:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.55300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8400,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363440,"statfs":{"total":96636764160,"available":96628162560,"internally_reserved":0,"allocated":401408,"data_stored":165283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":4.4950000000000001,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.46500000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":4.5369999999999999,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.48599999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.5999999999999996,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.67100000000000004}]},{"osd":5,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.752},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.70499999999999996}]},{"osd":6,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.5709999999999997,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.623},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.72799999999999998}]},{"osd":7,"last update":"Tue Nov 28 12:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.60799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":401408,"data_stored":165319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.34399999999999997}]},{"osd":1,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.30599999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.38400000000000001}]},{"osd":4,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47399999999999998}]},{"osd":5,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.502},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57199999999999995}]},{"osd":6,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54700000000000004}]},{"osd":7,"last update":"Tue Nov 28 12:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.52300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8540,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363300,"statfs":{"total":96636764160,"available":96628019200,"internally_reserved":0,"allocated":413696,"data_stored":166831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.371}]},{"osd":1,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.49199999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.53500000000000003}]},{"osd":4,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59799999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8532,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363308,"statfs":{"total":96636764160,"available":96628027392,"internally_reserved":0,"allocated":405504,"data_stored":165837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.56899999999999995}]},{"osd":2,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.60099999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.61799999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69699999999999995}]},{"osd":7,"last update":"Tue Nov 28 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8468,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363372,"statfs":{"total":96636764160,"available":96628092928,"internally_reserved":0,"allocated":405504,"data_stored":165361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":9.0630000000000006,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.41899999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.50900000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55400000000000005}]},{"osd":4,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.47199999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57699999999999996}]},{"osd":7,"last update":"Tue Nov 28 12:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.624},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:02:58.136 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T12:02:58.136 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T12:02:58.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.317+0000 7f4b1a281700 1 Processor -- start 2023-11-28T12:02:58.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.317+0000 7f4b1a281700 1 -- start start 2023-11-28T12:02:58.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b1a281700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b14159710 0x7f4b140b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:58.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b1a281700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4b140cdb90 0x7f4b140ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:58.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b1a281700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b14159ae0 con 0x7f4b14071630 2023-11-28T12:02:58.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b1a281700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b140ce550 con 0x7f4b140cdb90 2023-11-28T12:02:58.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b1a281700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b14071e10 con 0x7f4b14159710 2023-11-28T12:02:58.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b1881e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4b140cdb90 0x7f4b140ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:58.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b137fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4b14071630 0x7f4b140cd090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47386/0 (socket says 172.21.15.57:47386) 2023-11-28T12:02:58.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b137fe700 1 -- 172.21.15.57:0/1496814526 learned_addr learned my addr 172.21.15.57:0/1496814526 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:02:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b13fff700 1 --2- 172.21.15.57:0/1496814526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b14159710 0x7f4b140b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b1881e700 1 -- 172.21.15.57:0/1496814526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b14159710 msgr2=0x7f4b140b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b1881e700 1 --2- 172.21.15.57:0/1496814526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b14159710 0x7f4b140b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b1881e700 1 -- 172.21.15.57:0/1496814526 >> v1:172.21.15.57:6789/0 conn(0x7f4b14071630 legacy=0x7f4b140cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b1881e700 1 -- 172.21.15.57:0/1496814526 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b140cc2d0 con 0x7f4b140cdb90 2023-11-28T12:02:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b12ffd700 1 -- 172.21.15.57:0/1496814526 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914666948 0 0) 0x7f4b14159ae0 con 0x7f4b14071630 2023-11-28T12:02:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.318+0000 7f4b13fff700 1 --2- 172.21.15.57:0/1496814526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b14159710 0x7f4b140b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:02:58.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.319+0000 7f4b1881e700 1 --2- 172.21.15.57:0/1496814526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b140cdb90 0x7f4b140ce010 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009d90 tx=0x7f4b0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc395a21c1b64217 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:58.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.319+0000 7f4b12ffd700 1 -- 172.21.15.57:0/1496814526 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c00e070 con 0x7f4b140cdb90 2023-11-28T12:02:58.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.319+0000 7f4b12ffd700 1 -- 172.21.15.57:0/1496814526 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c00bbd0 con 0x7f4b140cdb90 2023-11-28T12:02:58.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.319+0000 7f4b12ffd700 1 -- 172.21.15.57:0/1496814526 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c012430 con 0x7f4b140cdb90 2023-11-28T12:02:58.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.319+0000 7f4b1a281700 1 -- 172.21.15.57:0/1496814526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b140cdb90 msgr2=0x7f4b140ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:58.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.319+0000 7f4b1a281700 1 --2- 172.21.15.57:0/1496814526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b140cdb90 0x7f4b140ce010 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009d90 tx=0x7f4b0c00b330 comp rx=0 tx=0).stop 2023-11-28T12:02:58.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.319+0000 7f4b1a281700 1 -- 172.21.15.57:0/1496814526 shutdown_connections 2023-11-28T12:02:58.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.319+0000 7f4b1a281700 1 --2- 172.21.15.57:0/1496814526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b140cdb90 0x7f4b140ce010 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.319+0000 7f4b1a281700 1 --2- 172.21.15.57:0/1496814526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b14159710 0x7f4b140b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.319+0000 7f4b1a281700 1 -- 172.21.15.57:0/1496814526 >> 172.21.15.57:0/1496814526 conn(0x7f4b1406bfd0 msgr2=0x7f4b140b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:58.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.319+0000 7f4b1a281700 1 -- 172.21.15.57:0/1496814526 shutdown_connections 2023-11-28T12:02:58.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.319+0000 7f4b1a281700 1 -- 172.21.15.57:0/1496814526 wait complete. 2023-11-28T12:02:58.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.321+0000 7f4b1a281700 1 Processor -- start 2023-11-28T12:02:58.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.321+0000 7f4b1a281700 1 -- start start 2023-11-28T12:02:58.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b1a281700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b14071630 0x7f4b141539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:58.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b1a281700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b140cdb90 0x7f4b140c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:58.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b1a281700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b14159710 0x7f4b140c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:58.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b1a281700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b140ae0b0 con 0x7f4b14071630 2023-11-28T12:02:58.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b1a281700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b140c47f0 con 0x7f4b14159710 2023-11-28T12:02:58.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b1a281700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b140c8240 con 0x7f4b140cdb90 2023-11-28T12:02:58.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b13fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b14071630 0x7f4b141539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:58.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b13fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b14071630 0x7f4b141539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55924/0 (socket says 172.21.15.57:55924) 2023-11-28T12:02:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b13fff700 1 -- 172.21.15.57:0/1397590971 learned_addr learned my addr 172.21.15.57:0/1397590971 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b137fe700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b140cdb90 0x7f4b140c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b1881e700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b14159710 0x7f4b140c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b13fff700 1 -- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b140cdb90 msgr2=0x7f4b140c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b13fff700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b140cdb90 0x7f4b140c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b13fff700 1 -- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b14159710 msgr2=0x7f4b140c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b13fff700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b14159710 0x7f4b140c3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b13fff700 1 -- 172.21.15.57:0/1397590971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c009a70 con 0x7f4b14071630 2023-11-28T12:02:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b1881e700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b14159710 0x7f4b140c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:58.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b13fff700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b14071630 0x7f4b141539f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400c7f0 tx=0x7f4b0400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:58.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.322+0000 7f4b10ff9700 1 -- 172.21.15.57:0/1397590971 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04011800 con 0x7f4b14071630 2023-11-28T12:02:58.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.323+0000 7f4b10ff9700 1 -- 172.21.15.57:0/1397590971 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b04011960 con 0x7f4b14071630 2023-11-28T12:02:58.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.323+0000 7f4b1a281700 1 -- 172.21.15.57:0/1397590971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b140cc890 con 0x7f4b14071630 2023-11-28T12:02:58.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.323+0000 7f4b10ff9700 1 -- 172.21.15.57:0/1397590971 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04011cf0 con 0x7f4b14071630 2023-11-28T12:02:58.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.323+0000 7f4b1a281700 1 -- 172.21.15.57:0/1397590971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b140cc5b0 con 0x7f4b14071630 2023-11-28T12:02:58.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.324+0000 7f4b1a281700 1 -- 172.21.15.57:0/1397590971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b140ccb70 con 0x7f4b14071630 2023-11-28T12:02:58.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.324+0000 7f4b10ff9700 1 -- 172.21.15.57:0/1397590971 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4b04021070 con 0x7f4b14071630 2023-11-28T12:02:58.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.325+0000 7f4b10ff9700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4af40682f0 0x7f4af406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:58.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.325+0000 7f4b10ff9700 1 -- 172.21.15.57:0/1397590971 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4b04085b30 con 0x7f4b14071630 2023-11-28T12:02:58.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.325+0000 7f4b137fe700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4af40682f0 0x7f4af406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:58.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.325+0000 7f4b137fe700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4af40682f0 0x7f4af406a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4b140b5860 tx=0x7f4afc007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:58.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.329+0000 7f4b10ff9700 1 -- 172.21.15.57:0/1397590971 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4b04052310 con 0x7f4b14071630 2023-11-28T12:02:58.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.611+0000 7f4b1a281700 1 -- 172.21.15.57:0/1397590971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f4b140678c0 con 0x7f4b14071630 2023-11-28T12:02:58.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.615+0000 7f4b10ff9700 1 -- 172.21.15.57:0/1397590971 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4b0401f070 con 0x7f4b14071630 2023-11-28T12:02:58.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.617+0000 7f4b1a281700 1 -- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4af40682f0 msgr2=0x7f4af406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:58.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.617+0000 7f4b1a281700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4af40682f0 0x7f4af406a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4b140b5860 tx=0x7f4afc007de0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.617+0000 7f4b1a281700 1 -- 172.21.15.57:0/1397590971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b14071630 msgr2=0x7f4b141539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:58.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.617+0000 7f4b1a281700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b14071630 0x7f4b141539f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400c7f0 tx=0x7f4b0400cb00 comp rx=0 tx=0).stop 2023-11-28T12:02:58.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.617+0000 7f4b1a281700 1 -- 172.21.15.57:0/1397590971 shutdown_connections 2023-11-28T12:02:58.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.617+0000 7f4b1a281700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4af40682f0 0x7f4af406a7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.617+0000 7f4b1a281700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b14159710 0x7f4b140c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.617+0000 7f4b1a281700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b14071630 0x7f4b141539f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.617+0000 7f4b1a281700 1 --2- 172.21.15.57:0/1397590971 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b140cdb90 0x7f4b140c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.617+0000 7f4b1a281700 1 -- 172.21.15.57:0/1397590971 >> 172.21.15.57:0/1397590971 conn(0x7f4b1406bfd0 msgr2=0x7f4b140cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:58.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.617+0000 7f4b1a281700 1 -- 172.21.15.57:0/1397590971 shutdown_connections 2023-11-28T12:02:58.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.617+0000 7f4b1a281700 1 -- 172.21.15.57:0/1397590971 wait complete. 2023-11-28T12:02:58.620 INFO:teuthology.orchestra.run.smithi057.stderr:updated caps for client.0 2023-11-28T12:02:58.632 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:02:58.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.766+0000 7f21eaf33700 1 Processor -- start 2023-11-28T12:02:58.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.766+0000 7f21eaf33700 1 -- start start 2023-11-28T12:02:58.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21eaf33700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21e4142db0 0x7f21e413ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:58.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21eaf33700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f21e40b1770 0x7f21e40b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:58.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21eaf33700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e415e010 con 0x7f21e40ce040 2023-11-28T12:02:58.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21eaf33700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e413fc90 con 0x7f21e40b1770 2023-11-28T12:02:58.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21eaf33700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e40b20d0 con 0x7f21e4142db0 2023-11-28T12:02:58.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21e94d0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f21e40b1770 0x7f21e40b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:58.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21e94d0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f21e40b1770 0x7f21e40b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55936/0 (socket says 172.21.15.57:55936) 2023-11-28T12:02:58.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21e94d0700 1 -- 172.21.15.57:0/1566074813 learned_addr learned my addr 172.21.15.57:0/1566074813 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:58.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21e3fff700 1 --2- 172.21.15.57:0/1566074813 >> v2:172.21.15.181:3300/0 conn(0x7f21e4142db0 0x7f21e413ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:58.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21e94d0700 1 -- 172.21.15.57:0/1566074813 >> v2:172.21.15.181:3300/0 conn(0x7f21e4142db0 msgr2=0x7f21e413ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:58.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21e94d0700 1 --2- 172.21.15.57:0/1566074813 >> v2:172.21.15.181:3300/0 conn(0x7f21e4142db0 0x7f21e413ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21e94d0700 1 -- 172.21.15.57:0/1566074813 >> v1:172.21.15.181:6789/0 conn(0x7f21e40ce040 legacy=0x7f21e415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:58.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21e37fe700 1 -- 172.21.15.57:0/1566074813 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639797434 0 0) 0x7f21d8001110 con 0x7f21e40ce040 2023-11-28T12:02:58.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.767+0000 7f21e94d0700 1 -- 172.21.15.57:0/1566074813 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e413f6d0 con 0x7f21e40b1770 2023-11-28T12:02:58.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.768+0000 7f21e94d0700 1 --2- 172.21.15.57:0/1566074813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21e40b1770 0x7f21e40b1b90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f21e40cfc20 tx=0x7f21dc00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37b77a58fed0ba2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:58.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.768+0000 7f21e37fe700 1 -- 172.21.15.57:0/1566074813 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc010070 con 0x7f21e40b1770 2023-11-28T12:02:58.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.768+0000 7f21e37fe700 1 -- 172.21.15.57:0/1566074813 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc00dce0 con 0x7f21e40b1770 2023-11-28T12:02:58.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.768+0000 7f21e37fe700 1 -- 172.21.15.57:0/1566074813 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc00b310 con 0x7f21e40b1770 2023-11-28T12:02:58.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.768+0000 7f21eaf33700 1 -- 172.21.15.57:0/1566074813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21e40b1770 msgr2=0x7f21e40b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:58.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.768+0000 7f21eaf33700 1 --2- 172.21.15.57:0/1566074813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21e40b1770 0x7f21e40b1b90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f21e40cfc20 tx=0x7f21dc00d430 comp rx=0 tx=0).stop 2023-11-28T12:02:58.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.768+0000 7f21eaf33700 1 -- 172.21.15.57:0/1566074813 shutdown_connections 2023-11-28T12:02:58.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.768+0000 7f21eaf33700 1 --2- 172.21.15.57:0/1566074813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21e40b1770 0x7f21e40b1b90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.768+0000 7f21eaf33700 1 --2- 172.21.15.57:0/1566074813 >> v2:172.21.15.181:3300/0 conn(0x7f21e4142db0 0x7f21e413ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.768+0000 7f21eaf33700 1 -- 172.21.15.57:0/1566074813 >> 172.21.15.57:0/1566074813 conn(0x7f21e406c010 msgr2=0x7f21e40cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:58.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.768+0000 7f21eaf33700 1 -- 172.21.15.57:0/1566074813 shutdown_connections 2023-11-28T12:02:58.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.769+0000 7f21eaf33700 1 -- 172.21.15.57:0/1566074813 wait complete. 2023-11-28T12:02:58.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.770+0000 7f21eaf33700 1 Processor -- start 2023-11-28T12:02:58.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.770+0000 7f21eaf33700 1 -- start start 2023-11-28T12:02:58.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21eaf33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e40b1770 0x7f21e4156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:58.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21eaf33700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21e40ce040 0x7f21e41e42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:58.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21eaf33700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21e4142db0 0x7f21e415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:58.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21eaf33700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41559b0 con 0x7f21e4142db0 2023-11-28T12:02:58.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21eaf33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4155c90 con 0x7f21e40b1770 2023-11-28T12:02:58.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21eaf33700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4155f70 con 0x7f21e40ce040 2023-11-28T12:02:58.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21e94d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21e4142db0 0x7f21e415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:58.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21e94d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21e4142db0 0x7f21e415aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55948/0 (socket says 172.21.15.57:55948) 2023-11-28T12:02:58.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21e94d0700 1 -- 172.21.15.57:0/1087026368 learned_addr learned my addr 172.21.15.57:0/1087026368 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:58.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21e8ccf700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e40b1770 0x7f21e4156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:58.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21e3fff700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21e40ce040 0x7f21e41e42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:58.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21e94d0700 1 -- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21e40ce040 msgr2=0x7f21e41e42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:58.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21e94d0700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21e40ce040 0x7f21e41e42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21e94d0700 1 -- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e40b1770 msgr2=0x7f21e4156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:58.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21e94d0700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e40b1770 0x7f21e4156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:58.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21e94d0700 1 -- 172.21.15.57:0/1087026368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc00bd40 con 0x7f21e4142db0 2023-11-28T12:02:58.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.771+0000 7f21e94d0700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21e4142db0 0x7f21e415aba0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00d780 tx=0x7f21dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:58.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.772+0000 7f21e17fa700 1 -- 172.21.15.57:0/1087026368 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc009a10 con 0x7f21e4142db0 2023-11-28T12:02:58.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.772+0000 7f21e17fa700 1 -- 172.21.15.57:0/1087026368 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc009b70 con 0x7f21e4142db0 2023-11-28T12:02:58.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.772+0000 7f21eaf33700 1 -- 172.21.15.57:0/1087026368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e415d100 con 0x7f21e4142db0 2023-11-28T12:02:58.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.772+0000 7f21e17fa700 1 -- 172.21.15.57:0/1087026368 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc00b4c0 con 0x7f21e4142db0 2023-11-28T12:02:58.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.772+0000 7f21eaf33700 1 -- 172.21.15.57:0/1087026368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e413f9b0 con 0x7f21e4142db0 2023-11-28T12:02:58.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.773+0000 7f21eaf33700 1 -- 172.21.15.57:0/1087026368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e415d3e0 con 0x7f21e4142db0 2023-11-28T12:02:58.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.773+0000 7f21e17fa700 1 -- 172.21.15.57:0/1087026368 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f21dc004040 con 0x7f21e4142db0 2023-11-28T12:02:58.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.774+0000 7f21e17fa700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f21c80682f0 0x7f21c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:58.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.774+0000 7f21e17fa700 1 -- 172.21.15.57:0/1087026368 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f21dc086fc0 con 0x7f21e4142db0 2023-11-28T12:02:58.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.774+0000 7f21e8ccf700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f21c80682f0 0x7f21c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:58.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.774+0000 7f21e8ccf700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f21c80682f0 0x7f21c806a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f21d8000f60 tx=0x7f21d8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:58.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:58.778+0000 7f21e17fa700 1 -- 172.21.15.57:0/1087026368 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f21dc053820 con 0x7f21e4142db0 2023-11-28T12:02:59.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.026+0000 7f21eaf33700 1 -- 172.21.15.57:0/1087026368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21e4155af0 con 0x7f21e4142db0 2023-11-28T12:02:59.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.027+0000 7f21e17fa700 1 -- 172.21.15.57:0/1087026368 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f21e4155af0 con 0x7f21e4142db0 2023-11-28T12:02:59.028 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:02:59.028 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/340292109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":340292109},{"type":"v1","addr":"172.21.15.181:6836","nonce":340292109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2442321825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2442321825},{"type":"v1","addr":"172.21.15.181:6837","nonce":2442321825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/3452242211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3452242211},{"type":"v1","addr":"172.21.15.57:6833","nonce":3452242211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:02:53.402391+0000","modified":"2023-11-28T12:02:56.540229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.57:6835/968299041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":968299041},{"type":"v1","addr":"172.21.15.57:6835","nonce":968299041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T12:02:59.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.029+0000 7f21eaf33700 1 -- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f21c80682f0 msgr2=0x7f21c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.029+0000 7f21eaf33700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f21c80682f0 0x7f21c806a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f21d8000f60 tx=0x7f21d8009040 comp rx=0 tx=0).stop 2023-11-28T12:02:59.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.029+0000 7f21eaf33700 1 -- 172.21.15.57:0/1087026368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21e4142db0 msgr2=0x7f21e415aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.029+0000 7f21eaf33700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21e4142db0 0x7f21e415aba0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00d780 tx=0x7f21dc004ad0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.029+0000 7f21eaf33700 1 -- 172.21.15.57:0/1087026368 shutdown_connections 2023-11-28T12:02:59.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.029+0000 7f21eaf33700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f21c80682f0 0x7f21c806a7a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.029+0000 7f21eaf33700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21e4142db0 0x7f21e415aba0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.029+0000 7f21eaf33700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e40b1770 0x7f21e4156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.029+0000 7f21eaf33700 1 --2- 172.21.15.57:0/1087026368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21e40ce040 0x7f21e41e42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.029+0000 7f21eaf33700 1 -- 172.21.15.57:0/1087026368 >> 172.21.15.57:0/1087026368 conn(0x7f21e406c010 msgr2=0x7f21e4070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:59.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.029+0000 7f21eaf33700 1 -- 172.21.15.57:0/1087026368 shutdown_connections 2023-11-28T12:02:59.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.029+0000 7f21eaf33700 1 -- 172.21.15.57:0/1087026368 wait complete. 2023-11-28T12:02:59.031 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 16 2023-11-28T12:02:59.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T12:02:53.402391+0000', 'modified': '2023-11-28T12:02:56.540229+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5238}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5238': {'gid': 5238, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.57:6835/968299041', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6834', 'nonce': 968299041}, {'type': 'v1', 'addr': '172.21.15.57:6835', 'nonce': 968299041}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T12:02:59.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T12:02:59.446 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T12:02:59.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.592+0000 7fb2fa9b8700 1 Processor -- start 2023-11-28T12:02:59.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.592+0000 7fb2fa9b8700 1 -- start start 2023-11-28T12:02:59.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.593+0000 7fb2fa9b8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb2f40b1650 0x7fb2f40b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.593+0000 7fb2fa9b8700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb2f41599b0 0x7fb2f415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.593+0000 7fb2fa9b8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f4159ec0 con 0x7fb2f40af0a0 2023-11-28T12:02:59.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.593+0000 7fb2fa9b8700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f415aa60 con 0x7fb2f41599b0 2023-11-28T12:02:59.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.593+0000 7fb2fa9b8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f40b1f60 con 0x7fb2f40b1650 2023-11-28T12:02:59.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.593+0000 7fb2f37fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb2f41599b0 0x7fb2f415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.593+0000 7fb2f37fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb2f41599b0 0x7fb2f415d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55964/0 (socket says 172.21.15.57:55964) 2023-11-28T12:02:59.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.593+0000 7fb2f37fe700 1 -- 172.21.15.57:0/3443343721 learned_addr learned my addr 172.21.15.57:0/3443343721 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:59.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.593+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/3443343721 >> v2:172.21.15.181:3300/0 conn(0x7fb2f40b1650 0x7fb2f40b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.594+0000 7fb2f37fe700 1 -- 172.21.15.57:0/3443343721 >> v2:172.21.15.181:3300/0 conn(0x7fb2f40b1650 msgr2=0x7fb2f40b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:59.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.594+0000 7fb2f37fe700 1 --2- 172.21.15.57:0/3443343721 >> v2:172.21.15.181:3300/0 conn(0x7fb2f40b1650 0x7fb2f40b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.594+0000 7fb2f37fe700 1 -- 172.21.15.57:0/3443343721 >> v1:172.21.15.181:6789/0 conn(0x7fb2f40af0a0 legacy=0x7fb2f415a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.594+0000 7fb2f37fe700 1 -- 172.21.15.57:0/3443343721 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f415d8a0 con 0x7fb2f41599b0 2023-11-28T12:02:59.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.594+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/3443343721 >> v2:172.21.15.181:3300/0 conn(0x7fb2f40b1650 0x7fb2f40b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:59.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.594+0000 7fb2f37fe700 1 --2- 172.21.15.57:0/3443343721 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f41599b0 0x7fb2f415d050 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4067310 tx=0x7fb2e800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76aaec59c45c21fc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:59.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.594+0000 7fb2f2ffd700 1 -- 172.21.15.57:0/3443343721 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e800dd30 con 0x7fb2f41599b0 2023-11-28T12:02:59.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.594+0000 7fb2f2ffd700 1 -- 172.21.15.57:0/3443343721 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e800de90 con 0x7fb2f41599b0 2023-11-28T12:02:59.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.594+0000 7fb2f2ffd700 1 -- 172.21.15.57:0/3443343721 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8010690 con 0x7fb2f41599b0 2023-11-28T12:02:59.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.595+0000 7fb2fa9b8700 1 -- 172.21.15.57:0/3443343721 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f41599b0 msgr2=0x7fb2f415d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.595+0000 7fb2fa9b8700 1 --2- 172.21.15.57:0/3443343721 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f41599b0 0x7fb2f415d050 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4067310 tx=0x7fb2e800be70 comp rx=0 tx=0).stop 2023-11-28T12:02:59.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.595+0000 7fb2fa9b8700 1 -- 172.21.15.57:0/3443343721 shutdown_connections 2023-11-28T12:02:59.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.595+0000 7fb2fa9b8700 1 --2- 172.21.15.57:0/3443343721 >> v2:172.21.15.181:3300/0 conn(0x7fb2f40b1650 0x7fb2f40b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.595+0000 7fb2fa9b8700 1 --2- 172.21.15.57:0/3443343721 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f41599b0 0x7fb2f415d050 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.595+0000 7fb2fa9b8700 1 -- 172.21.15.57:0/3443343721 >> 172.21.15.57:0/3443343721 conn(0x7fb2f406c010 msgr2=0x7fb2f40cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:59.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.595+0000 7fb2fa9b8700 1 -- 172.21.15.57:0/3443343721 shutdown_connections 2023-11-28T12:02:59.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.595+0000 7fb2fa9b8700 1 -- 172.21.15.57:0/3443343721 wait complete. 2023-11-28T12:02:59.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.596+0000 7fb2fa9b8700 1 Processor -- start 2023-11-28T12:02:59.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.596+0000 7fb2fa9b8700 1 -- start start 2023-11-28T12:02:59.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2fa9b8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f40af0a0 0x7fb2f415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2fa9b8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2f40b1650 0x7fb2f4145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2fa9b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2f4146000 0x7fb2f41f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2fa9b8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f41f6c40 con 0x7fb2f40af0a0 2023-11-28T12:02:59.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2fa9b8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f41f6960 con 0x7fb2f4146000 2023-11-28T12:02:59.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2fa9b8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f4156950 con 0x7fb2f40b1650 2023-11-28T12:02:59.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f40af0a0 0x7fb2f415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f40af0a0 0x7fb2f415bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55976/0 (socket says 172.21.15.57:55976) 2023-11-28T12:02:59.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2f3fff700 1 -- 172.21.15.57:0/1529069158 learned_addr learned my addr 172.21.15.57:0/1529069158 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:59.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2f37fe700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2f40b1650 0x7fb2f4145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2f8f55700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2f4146000 0x7fb2f41f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2f3fff700 1 -- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2f40b1650 msgr2=0x7fb2f4145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2f40b1650 0x7fb2f4145ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2f3fff700 1 -- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2f4146000 msgr2=0x7fb2f41f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.597+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2f4146000 0x7fb2f41f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.598+0000 7fb2f3fff700 1 -- 172.21.15.57:0/1529069158 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e800b7f0 con 0x7fb2f40af0a0 2023-11-28T12:02:59.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.598+0000 7fb2f37fe700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2f40b1650 0x7fb2f4145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:59.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.598+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f40af0a0 0x7fb2f415bf90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e810 tx=0x7fb2e400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:59.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.598+0000 7fb2f0ff9700 1 -- 172.21.15.57:0/1529069158 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4004230 con 0x7fb2f40af0a0 2023-11-28T12:02:59.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.598+0000 7fb2f0ff9700 1 -- 172.21.15.57:0/1529069158 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e4004390 con 0x7fb2f40af0a0 2023-11-28T12:02:59.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.598+0000 7fb2fa9b8700 1 -- 172.21.15.57:0/1529069158 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f415de60 con 0x7fb2f40af0a0 2023-11-28T12:02:59.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.598+0000 7fb2f0ff9700 1 -- 172.21.15.57:0/1529069158 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40036a0 con 0x7fb2f40af0a0 2023-11-28T12:02:59.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.598+0000 7fb2fa9b8700 1 -- 172.21.15.57:0/1529069158 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f415db80 con 0x7fb2f40af0a0 2023-11-28T12:02:59.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.602+0000 7fb306b57b80 1 Processor -- start 2023-11-28T12:02:59.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.602+0000 7fb306b57b80 1 -- start start 2023-11-28T12:02:59.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.603+0000 7fb306b57b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9eef50 0x55925f9a89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.603+0000 7fb306b57b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 0x55925f9ef320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.603+0000 7fb306b57b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9d1660 0x55925f9a9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.603+0000 7fb306b57b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925f926de0 con 0x55925f9e5ac0 2023-11-28T12:02:59.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.603+0000 7fb306b57b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925f893140 con 0x55925f9d1660 2023-11-28T12:02:59.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.603+0000 7fb306b57b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55925f901db0 con 0x55925f9eef50 2023-11-28T12:02:59.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.603+0000 7fb2f0ff9700 1 -- 172.21.15.57:0/1529069158 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb2e4022070 con 0x7fb2f40af0a0 2023-11-28T12:02:59.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.603+0000 7fb2f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 0x55925f9ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.603+0000 7fb2f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 0x55925f9ef320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55986/0 (socket says 172.21.15.57:55986) 2023-11-28T12:02:59.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.603+0000 7fb2f3fff700 1 -- 172.21.15.57:0/4197095065 learned_addr learned my addr 172.21.15.57:0/4197095065 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:59.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.603+0000 7fb2f8f55700 1 --2- 172.21.15.57:0/4197095065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9eef50 0x55925f9a89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.603+0000 7fb2f37fe700 1 --2- 172.21.15.57:0/4197095065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9d1660 0x55925f9a9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.604+0000 7fb2f0ff9700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb2e00682f0 0x7fb2e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.604+0000 7fb2f3fff700 1 -- 172.21.15.57:0/4197095065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9eef50 msgr2=0x55925f9a89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:59.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.604+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/4197095065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9eef50 0x55925f9a89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.604+0000 7fb2f3fff700 1 -- 172.21.15.57:0/4197095065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9d1660 msgr2=0x55925f9a9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:59.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.604+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/4197095065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9d1660 0x55925f9a9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.604+0000 7fb2f3fff700 1 -- 172.21.15.57:0/4197095065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55925f6ef790 con 0x55925f9e5ac0 2023-11-28T12:02:59.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.604+0000 7fb2f0ff9700 1 -- 172.21.15.57:0/1529069158 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2e4085740 con 0x7fb2f40af0a0 2023-11-28T12:02:59.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.604+0000 7fb2f8f55700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb2e00682f0 0x7fb2e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.604+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/4197095065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 0x55925f9ef320 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4053d70 tx=0x7fb2e4051fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f488376a63e5454 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:59.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.604+0000 7fb2d9ffb700 1 -- 172.21.15.57:0/4197095065 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40094e0 con 0x55925f9e5ac0 2023-11-28T12:02:59.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.604+0000 7fb2d9ffb700 1 -- 172.21.15.57:0/4197095065 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e4055740 con 0x55925f9e5ac0 2023-11-28T12:02:59.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.604+0000 7fb2d9ffb700 1 -- 172.21.15.57:0/4197095065 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4051bf0 con 0x55925f9e5ac0 2023-11-28T12:02:59.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.605+0000 7fb306b57b80 1 -- 172.21.15.57:0/4197095065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 msgr2=0x55925f9ef320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.605+0000 7fb306b57b80 1 --2- 172.21.15.57:0/4197095065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 0x55925f9ef320 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4053d70 tx=0x7fb2e4051fc0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.605+0000 7fb2f8f55700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb2e00682f0 0x7fb2e006a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0012e0 tx=0x7fb2ec009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:59.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.605+0000 7fb306b57b80 1 -- 172.21.15.57:0/4197095065 shutdown_connections 2023-11-28T12:02:59.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.605+0000 7fb306b57b80 1 --2- 172.21.15.57:0/4197095065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9d1660 0x55925f9a9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.605+0000 7fb306b57b80 1 --2- 172.21.15.57:0/4197095065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9eef50 0x55925f9a89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.605+0000 7fb306b57b80 1 --2- 172.21.15.57:0/4197095065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 0x55925f9ef320 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.605+0000 7fb306b57b80 1 -- 172.21.15.57:0/4197095065 >> 172.21.15.57:0/4197095065 conn(0x55925f99ed20 msgr2=0x55925f9a7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:59.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.605+0000 7fb306b57b80 1 -- 172.21.15.57:0/4197095065 shutdown_connections 2023-11-28T12:02:59.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.605+0000 7fb306b57b80 1 -- 172.21.15.57:0/4197095065 wait complete. 2023-11-28T12:02:59.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.605+0000 7fb306b57b80 1 -- start start 2023-11-28T12:02:59.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.606+0000 7fb306b57b80 1 Processor -- start 2023-11-28T12:02:59.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.606+0000 7fb306b57b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9d1660 0x55925f9ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb306b57b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9e5ac0 0x55925f9e9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb306b57b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9eef50 0x55925f9eb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb306b57b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925f8fb120 con 0x55925f9d1660 2023-11-28T12:02:59.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb306b57b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925f6f0390 con 0x55925f9eef50 2023-11-28T12:02:59.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb306b57b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55925f893340 con 0x55925f9e5ac0 2023-11-28T12:02:59.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb306b57b80 10 client.0 _ll_register_callbacks cb 0x55925f899400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:02:59.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9d1660 0x55925f9ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9d1660 0x55925f9ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55996/0 (socket says 172.21.15.57:55996) 2023-11-28T12:02:59.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2f3fff700 1 -- 172.21.15.57:0/3400050031 learned_addr learned my addr 172.21.15.57:0/3400050031 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:59.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2f37fe700 1 --2- 172.21.15.57:0/3400050031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9e5ac0 0x55925f9e9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2f8f55700 1 --2- 172.21.15.57:0/3400050031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9eef50 0x55925f9eb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2f3fff700 1 -- 172.21.15.57:0/3400050031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9e5ac0 msgr2=0x55925f9e9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:59.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/3400050031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9e5ac0 0x55925f9e9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2f3fff700 1 -- 172.21.15.57:0/3400050031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9eef50 msgr2=0x55925f9eb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:59.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/3400050031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9eef50 0x55925f9eb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2f3fff700 1 -- 172.21.15.57:0/3400050031 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f415db80 con 0x55925f9d1660 2023-11-28T12:02:59.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/3400050031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9d1660 0x55925f9ccb40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009fd0 tx=0x7fb2e405a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c6d1cf28db1c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:59.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:02:59.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2d2ffd700 1 -- 172.21.15.57:0/3400050031 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40552f0 con 0x55925f9d1660 2023-11-28T12:02:59.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2d2ffd700 1 -- 172.21.15.57:0/3400050031 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2e405dcf0 con 0x55925f9d1660 2023-11-28T12:02:59.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb306b57b80 1 -- 172.21.15.57:0/3400050031 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55925f8ff030 con 0x55925f9d1660 2023-11-28T12:02:59.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.607+0000 7fb2d2ffd700 1 -- 172.21.15.57:0/3400050031 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2e4083d00 con 0x55925f9d1660 2023-11-28T12:02:59.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.608+0000 7fb306b57b80 10 client.5415 fetch_fsmap learned FSMap version 16 2023-11-28T12:02:59.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.608+0000 7fb306b57b80 1 -- 172.21.15.57:0/3400050031 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55925f7dc0a0 con 0x55925f9d1660 2023-11-28T12:02:59.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.608+0000 7fb2d2ffd700 1 -- 172.21.15.57:0/3400050031 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55925f8ff030 con 0x55925f9d1660 2023-11-28T12:02:59.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.608+0000 7fb2d2ffd700 1 -- 172.21.15.57:0/3400050031 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb2e408b030 con 0x55925f9d1660 2023-11-28T12:02:59.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.608+0000 7fb306b57b80 10 client.5415 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T12:02:59.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.608+0000 7fb306b57b80 10 client.5415 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T12:02:59.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.608+0000 7fb306b57b80 20 client.5415 populate_metadata read hostname 'smithi057' 2023-11-28T12:02:59.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.608+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3400050031 >> [v2:172.21.15.57:6834/968299041,v1:172.21.15.57:6835/968299041] conn(0x55925f9e6180 0x55925f9e6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.608+0000 7fb306b57b80 4 client.5415 mds_command: new command op to 5238 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T12:02:59.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.608+0000 7fb306b57b80 1 -- 172.21.15.57:0/3400050031 --> [v2:172.21.15.57:6834/968299041,v1:172.21.15.57:6835/968299041] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55925f74aef0 con 0x55925f9e6180 2023-11-28T12:02:59.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.608+0000 7fb2f37fe700 1 --2- 172.21.15.57:0/3400050031 >> [v2:172.21.15.57:6834/968299041,v1:172.21.15.57:6835/968299041] conn(0x55925f9e6180 0x55925f9e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.608+0000 7fb2d2ffd700 10 client.5415 ms_handle_connect on v2:172.21.15.57:6834/968299041 2023-11-28T12:02:59.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.608+0000 7fb2f37fe700 1 --2- 172.21.15.57:0/3400050031 >> [v2:172.21.15.57:6834/968299041,v1:172.21.15.57:6835/968299041] conn(0x55925f9e6180 0x55925f9e6550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f62bcfe7941f38 server_cookie=d02524c02c8e457e in_seq=0 out_seq=0 2023-11-28T12:02:59.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb2d2ffd700 1 -- 172.21.15.57:0/3400050031 <== mds.0 v2:172.21.15.57:6834/968299041 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55925f74aef0 con 0x55925f9e6180 2023-11-28T12:02:59.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb2d2ffd700 10 client.5415 handle_command_reply: tid=0 2023-11-28T12:02:59.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb306b57b80 1 client.5415 shutdown 2023-11-28T12:02:59.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb306b57b80 1 -- 172.21.15.57:0/3400050031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9d1660 msgr2=0x55925f9ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3400050031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9d1660 0x55925f9ccb40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009fd0 tx=0x7fb2e405a000 comp rx=0 tx=0).stop 2023-11-28T12:02:59.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb306b57b80 1 -- 172.21.15.57:0/3400050031 shutdown_connections 2023-11-28T12:02:59.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3400050031 >> [v2:172.21.15.57:6834/968299041,v1:172.21.15.57:6835/968299041] conn(0x55925f9e6180 0x55925f9e6550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3400050031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9eef50 0x55925f9eb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3400050031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9d1660 0x55925f9ccb40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb2d2ffd700 0 client.5415 ms_handle_reset on v2:172.21.15.57:6834/968299041 2023-11-28T12:02:59.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3400050031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9e5ac0 0x55925f9e9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb306b57b80 1 -- 172.21.15.57:0/3400050031 >> 172.21.15.57:0/3400050031 conn(0x55925f99ed20 msgr2=0x55925f9a7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:59.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb306b57b80 1 -- 172.21.15.57:0/3400050031 shutdown_connections 2023-11-28T12:02:59.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb306b57b80 1 -- 172.21.15.57:0/3400050031 wait complete. 2023-11-28T12:02:59.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.609+0000 7fb306b57b80 20 client.5415 trim_cache size 0 max 16384 2023-11-28T12:02:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.628+0000 7fb306b57b80 1 Processor -- start 2023-11-28T12:02:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.628+0000 7fb306b57b80 1 -- start start 2023-11-28T12:02:59.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.628+0000 7fb306b57b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f99ed20 0x55925f9a4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.628+0000 7fb306b57b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9d1660 0x55925f9f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.628+0000 7fb306b57b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 0x55925f9eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.628+0000 7fb306b57b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925fa07800 con 0x55925f9e5ac0 2023-11-28T12:02:59.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.628+0000 7fb306b57b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925f9cc570 con 0x55925f9d1660 2023-11-28T12:02:59.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.628+0000 7fb306b57b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55925f9f1480 con 0x55925f99ed20 2023-11-28T12:02:59.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2f8f55700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 0x55925f9eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2f8f55700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 0x55925f9eaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55998/0 (socket says 172.21.15.57:55998) 2023-11-28T12:02:59.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2f8f55700 1 -- 172.21.15.57:0/3954224293 learned_addr learned my addr 172.21.15.57:0/3954224293 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:59.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/3954224293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f99ed20 0x55925f9a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2f37fe700 1 --2- 172.21.15.57:0/3954224293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9d1660 0x55925f9f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2f8f55700 1 -- 172.21.15.57:0/3954224293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f99ed20 msgr2=0x55925f9a4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:59.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2f8f55700 1 --2- 172.21.15.57:0/3954224293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f99ed20 0x55925f9a4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2f8f55700 1 -- 172.21.15.57:0/3954224293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9d1660 msgr2=0x55925f9f0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:59.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2f8f55700 1 --2- 172.21.15.57:0/3954224293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9d1660 0x55925f9f0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2f8f55700 1 -- 172.21.15.57:0/3954224293 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55925f8c22e0 con 0x55925f9e5ac0 2023-11-28T12:02:59.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2f8f55700 1 --2- 172.21.15.57:0/3954224293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 0x55925f9eaf70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec004010 tx=0x7fb2ec013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70269b5f2cac41be server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:59.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2da7fc700 1 -- 172.21.15.57:0/3954224293 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec014a60 con 0x55925f9e5ac0 2023-11-28T12:02:59.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2da7fc700 1 -- 172.21.15.57:0/3954224293 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec014bc0 con 0x55925f9e5ac0 2023-11-28T12:02:59.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.629+0000 7fb2da7fc700 1 -- 172.21.15.57:0/3954224293 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec022410 con 0x55925f9e5ac0 2023-11-28T12:02:59.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.630+0000 7fb306b57b80 1 -- 172.21.15.57:0/3954224293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 msgr2=0x55925f9eaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.630+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3954224293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 0x55925f9eaf70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec004010 tx=0x7fb2ec013000 comp rx=0 tx=0).stop 2023-11-28T12:02:59.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.630+0000 7fb306b57b80 1 -- 172.21.15.57:0/3954224293 shutdown_connections 2023-11-28T12:02:59.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.630+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3954224293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f9e5ac0 0x55925f9eaf70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.630+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3954224293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f99ed20 0x55925f9a4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.630+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3954224293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9d1660 0x55925f9f0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.630+0000 7fb306b57b80 1 -- 172.21.15.57:0/3954224293 >> 172.21.15.57:0/3954224293 conn(0x55925f9cccb0 msgr2=0x55925f9a5710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:59.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.630+0000 7fb306b57b80 1 -- 172.21.15.57:0/3954224293 shutdown_connections 2023-11-28T12:02:59.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.630+0000 7fb306b57b80 1 -- 172.21.15.57:0/3954224293 wait complete. 2023-11-28T12:02:59.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.630+0000 7fb306b57b80 1 -- start start 2023-11-28T12:02:59.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.631+0000 7fb306b57b80 1 Processor -- start 2023-11-28T12:02:59.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb306b57b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f99ed20 0x55925fa54170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb306b57b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9d1660 0x55925f9eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb306b57b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9e5ac0 0x55925f9eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb306b57b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925fa55d80 con 0x55925f99ed20 2023-11-28T12:02:59.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb306b57b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925f9e52e0 con 0x55925f9e5ac0 2023-11-28T12:02:59.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb306b57b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55925f9e5810 con 0x55925f9d1660 2023-11-28T12:02:59.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb306b57b80 10 client.0 _ll_register_callbacks cb 0x55925f899400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:02:59.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f99ed20 0x55925fa54170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f99ed20 0x55925fa54170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56010/0 (socket says 172.21.15.57:56010) 2023-11-28T12:02:59.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2f3fff700 1 -- 172.21.15.57:0/3981450400 learned_addr learned my addr 172.21.15.57:0/3981450400 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:59.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2f37fe700 1 --2- 172.21.15.57:0/3981450400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9d1660 0x55925f9eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2f8f55700 1 --2- 172.21.15.57:0/3981450400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9e5ac0 0x55925f9eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2f3fff700 1 -- 172.21.15.57:0/3981450400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9d1660 msgr2=0x55925f9eaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:59.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/3981450400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9d1660 0x55925f9eaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2f3fff700 1 -- 172.21.15.57:0/3981450400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9e5ac0 msgr2=0x55925f9eb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:59.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/3981450400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9e5ac0 0x55925f9eb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2f3fff700 1 -- 172.21.15.57:0/3981450400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec00f040 con 0x55925f99ed20 2023-11-28T12:02:59.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2f3fff700 1 --2- 172.21.15.57:0/3981450400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f99ed20 0x55925fa54170 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4055b90 tx=0x7fb2e4087cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f65ac607ac5a26 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:59.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:02:59.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2d2ffd700 1 -- 172.21.15.57:0/3981450400 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4083d00 con 0x55925f99ed20 2023-11-28T12:02:59.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb2d2ffd700 1 -- 172.21.15.57:0/3981450400 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2e405dcf0 con 0x55925f99ed20 2023-11-28T12:02:59.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.632+0000 7fb306b57b80 1 -- 172.21.15.57:0/3981450400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2e4055740 con 0x55925f99ed20 2023-11-28T12:02:59.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb2d2ffd700 1 -- 172.21.15.57:0/3981450400 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2e4051d70 con 0x55925f99ed20 2023-11-28T12:02:59.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb306b57b80 10 client.5421 fetch_fsmap learned FSMap version 16 2023-11-28T12:02:59.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb306b57b80 1 -- 172.21.15.57:0/3981450400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55925f7501d0 con 0x55925f99ed20 2023-11-28T12:02:59.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb2d2ffd700 1 -- 172.21.15.57:0/3981450400 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e4055740 con 0x55925f99ed20 2023-11-28T12:02:59.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb2d2ffd700 1 -- 172.21.15.57:0/3981450400 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb2e408b030 con 0x55925f99ed20 2023-11-28T12:02:59.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb306b57b80 10 client.5421 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T12:02:59.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb306b57b80 10 client.5421 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T12:02:59.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb306b57b80 20 client.5421 populate_metadata read hostname 'smithi057' 2023-11-28T12:02:59.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3981450400 >> [v2:172.21.15.57:6834/968299041,v1:172.21.15.57:6835/968299041] conn(0x55925f9eef50 0x55925f9c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb306b57b80 4 client.5421 mds_command: new command op to 5238 tid=0[{"prefix": "status"}] 2023-11-28T12:02:59.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb306b57b80 1 -- 172.21.15.57:0/3981450400 --> [v2:172.21.15.57:6834/968299041,v1:172.21.15.57:6835/968299041] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55925f780040 con 0x55925f9eef50 2023-11-28T12:02:59.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb2f37fe700 1 --2- 172.21.15.57:0/3981450400 >> [v2:172.21.15.57:6834/968299041,v1:172.21.15.57:6835/968299041] conn(0x55925f9eef50 0x55925f9c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb2d2ffd700 10 client.5421 ms_handle_connect on v2:172.21.15.57:6834/968299041 2023-11-28T12:02:59.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.633+0000 7fb2f37fe700 1 --2- 172.21.15.57:0/3981450400 >> [v2:172.21.15.57:6834/968299041,v1:172.21.15.57:6835/968299041] conn(0x55925f9eef50 0x55925f9c9ed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89977816483e5109 server_cookie=2a4a10be1ffabf99 in_seq=0 out_seq=0 2023-11-28T12:02:59.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb2d2ffd700 1 -- 172.21.15.57:0/3981450400 <== mds.0 v2:172.21.15.57:6834/968299041 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55925f780040 con 0x55925f9eef50 2023-11-28T12:02:59.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb2d2ffd700 10 client.5421 handle_command_reply: tid=0 2023-11-28T12:02:59.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb306b57b80 1 client.5421 shutdown 2023-11-28T12:02:59.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb306b57b80 1 -- 172.21.15.57:0/3981450400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f99ed20 msgr2=0x55925fa54170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3981450400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f99ed20 0x55925fa54170 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4055b90 tx=0x7fb2e4087cd0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb306b57b80 1 -- 172.21.15.57:0/3981450400 shutdown_connections 2023-11-28T12:02:59.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3981450400 >> [v2:172.21.15.57:6834/968299041,v1:172.21.15.57:6835/968299041] conn(0x55925f9eef50 0x55925f9c9ed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3981450400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55925f9e5ac0 0x55925f9eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3981450400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55925f99ed20 0x55925fa54170 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb2d2ffd700 0 client.5421 ms_handle_reset on v2:172.21.15.57:6834/968299041 2023-11-28T12:02:59.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb306b57b80 1 --2- 172.21.15.57:0/3981450400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55925f9d1660 0x55925f9eaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb306b57b80 1 -- 172.21.15.57:0/3981450400 >> 172.21.15.57:0/3981450400 conn(0x55925f9cccb0 msgr2=0x55925f9a3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:59.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb306b57b80 1 -- 172.21.15.57:0/3981450400 shutdown_connections 2023-11-28T12:02:59.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb306b57b80 1 -- 172.21.15.57:0/3981450400 wait complete. 2023-11-28T12:02:59.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.634+0000 7fb306b57b80 20 client.5421 trim_cache size 0 max 16384 2023-11-28T12:02:59.643 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:02:59.643 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "0309b83f-6570-4eee-96fa-279822cd7766", 2023-11-28T12:02:59.643 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2023-11-28T12:02:59.643 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 5238, 2023-11-28T12:02:59.643 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-11-28T12:02:59.643 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-11-28T12:02:59.643 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-11-28T12:02:59.643 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 6.2305691689999998, 2023-11-28T12:02:59.644 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 16, 2023-11-28T12:02:59.644 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 35, 2023-11-28T12:02:59.644 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 35, 2023-11-28T12:02:59.644 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 14.559092749 2023-11-28T12:02:59.644 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:02:59.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.635+0000 7fb2fa9b8700 1 -- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb2e00682f0 msgr2=0x7fb2e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.635+0000 7fb2fa9b8700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb2e00682f0 0x7fb2e006a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0012e0 tx=0x7fb2ec009040 comp rx=0 tx=0).stop 2023-11-28T12:02:59.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.635+0000 7fb2fa9b8700 1 -- 172.21.15.57:0/1529069158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f40af0a0 msgr2=0x7fb2f415bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.635+0000 7fb2fa9b8700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f40af0a0 0x7fb2f415bf90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e810 tx=0x7fb2e400ebd0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.635+0000 7fb2fa9b8700 1 -- 172.21.15.57:0/1529069158 shutdown_connections 2023-11-28T12:02:59.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.635+0000 7fb2fa9b8700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb2e00682f0 0x7fb2e006a7a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.635+0000 7fb2fa9b8700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2f4146000 0x7fb2f41f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.635+0000 7fb2fa9b8700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f40af0a0 0x7fb2f415bf90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.635+0000 7fb2fa9b8700 1 --2- 172.21.15.57:0/1529069158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2f40b1650 0x7fb2f4145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.635+0000 7fb2fa9b8700 1 -- 172.21.15.57:0/1529069158 >> 172.21.15.57:0/1529069158 conn(0x7fb2f406c010 msgr2=0x7fb2f40b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:59.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.635+0000 7fb2fa9b8700 1 -- 172.21.15.57:0/1529069158 shutdown_connections 2023-11-28T12:02:59.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.635+0000 7fb2fa9b8700 1 -- 172.21.15.57:0/1529069158 wait complete. 2023-11-28T12:02:59.649 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T12:02:59.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.784+0000 7fb7fb02f700 1 Processor -- start 2023-11-28T12:02:59.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.784+0000 7fb7fb02f700 1 -- start start 2023-11-28T12:02:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7fb02f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb7f406e400 0x7fb7f406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7fb02f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7f40d15f0 0x7fb7f415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7fb02f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f415e010 0x7fb7f40ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7fb02f700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f415a9d0 con 0x7fb7f406e400 2023-11-28T12:02:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7fb02f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f41584e0 con 0x7fb7f40d15f0 2023-11-28T12:02:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7fb02f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f415af30 con 0x7fb7f415e010 2023-11-28T12:02:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7f8dcb700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb7f406e400 0x7fb7f406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7f8dcb700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb7f406e400 0x7fb7f406e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56014/0 (socket says 172.21.15.57:56014) 2023-11-28T12:02:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7f8dcb700 1 -- 172.21.15.57:0/3202696546 learned_addr learned my addr 172.21.15.57:0/3202696546 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7f95cc700 1 --2- 172.21.15.57:0/3202696546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f415e010 0x7fb7f40ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7f3fff700 1 --2- 172.21.15.57:0/3202696546 >> v2:172.21.15.181:3300/0 conn(0x7fb7f40d15f0 0x7fb7f415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7f8dcb700 1 -- 172.21.15.57:0/3202696546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f415e010 msgr2=0x7fb7f40ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:59.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7f8dcb700 1 --2- 172.21.15.57:0/3202696546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f415e010 0x7fb7f40ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7f8dcb700 1 -- 172.21.15.57:0/3202696546 >> v2:172.21.15.181:3300/0 conn(0x7fb7f40d15f0 msgr2=0x7fb7f415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:02:59.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7f8dcb700 1 --2- 172.21.15.57:0/3202696546 >> v2:172.21.15.181:3300/0 conn(0x7fb7f40d15f0 0x7fb7f415b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7f8dcb700 1 -- 172.21.15.57:0/3202696546 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f415ef40 con 0x7fb7f406e400 2023-11-28T12:02:59.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.785+0000 7fb7f95cc700 1 --2- 172.21.15.57:0/3202696546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f415e010 0x7fb7f40ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:02:59.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.786+0000 7fb7f8dcb700 1 --2- 172.21.15.57:0/3202696546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7f406e400 0x7fb7f406e7d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40ce6e0 tx=0x7fb7e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdbbbcd3e9f329a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:59.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.786+0000 7fb7f37fe700 1 -- 172.21.15.57:0/3202696546 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8011830 con 0x7fb7f406e400 2023-11-28T12:02:59.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.786+0000 7fb7f37fe700 1 -- 172.21.15.57:0/3202696546 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e8011990 con 0x7fb7f406e400 2023-11-28T12:02:59.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.786+0000 7fb7f37fe700 1 -- 172.21.15.57:0/3202696546 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8011d20 con 0x7fb7f406e400 2023-11-28T12:02:59.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.786+0000 7fb7fb02f700 1 -- 172.21.15.57:0/3202696546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7f406e400 msgr2=0x7fb7f406e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.786+0000 7fb7fb02f700 1 --2- 172.21.15.57:0/3202696546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7f406e400 0x7fb7f406e7d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40ce6e0 tx=0x7fb7e8009bd0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.786+0000 7fb7fb02f700 1 -- 172.21.15.57:0/3202696546 shutdown_connections 2023-11-28T12:02:59.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.786+0000 7fb7fb02f700 1 --2- 172.21.15.57:0/3202696546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f415e010 0x7fb7f40ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.786+0000 7fb7fb02f700 1 --2- 172.21.15.57:0/3202696546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7f406e400 0x7fb7f406e7d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.786+0000 7fb7fb02f700 1 --2- 172.21.15.57:0/3202696546 >> v2:172.21.15.181:3300/0 conn(0x7fb7f40d15f0 0x7fb7f415b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.786+0000 7fb7fb02f700 1 -- 172.21.15.57:0/3202696546 >> 172.21.15.57:0/3202696546 conn(0x7fb7f406a270 msgr2=0x7fb7f406a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:02:59.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.786+0000 7fb7fb02f700 1 -- 172.21.15.57:0/3202696546 shutdown_connections 2023-11-28T12:02:59.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.786+0000 7fb7fb02f700 1 -- 172.21.15.57:0/3202696546 wait complete. 2023-11-28T12:02:59.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.788+0000 7fb7fb02f700 1 Processor -- start 2023-11-28T12:02:59.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.788+0000 7fb7fb02f700 1 -- start start 2023-11-28T12:02:59.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.789+0000 7fb7fb02f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7f406e400 0x7fb7f40b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.789+0000 7fb7fb02f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7f40d15f0 0x7fb7f41549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.789+0000 7fb7fb02f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f415e010 0x7fb7f4154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.789+0000 7fb7fb02f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f40b13a0 con 0x7fb7f40d15f0 2023-11-28T12:02:59.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.789+0000 7fb7fb02f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f41e3f10 con 0x7fb7f406e400 2023-11-28T12:02:59.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.789+0000 7fb7fb02f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f40b08e0 con 0x7fb7f415e010 2023-11-28T12:02:59.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7f40d15f0 0x7fb7f41549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7f40d15f0 0x7fb7f41549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56028/0 (socket says 172.21.15.57:56028) 2023-11-28T12:02:59.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f3fff700 1 -- 172.21.15.57:0/2146663584 learned_addr learned my addr 172.21.15.57:0/2146663584 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:02:59.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f95cc700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f415e010 0x7fb7f4154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f8dcb700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7f406e400 0x7fb7f40b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f3fff700 1 -- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f415e010 msgr2=0x7fb7f4154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f3fff700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f415e010 0x7fb7f4154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f3fff700 1 -- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7f406e400 msgr2=0x7fb7f40b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:02:59.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f3fff700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7f406e400 0x7fb7f40b0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:02:59.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f3fff700 1 -- 172.21.15.57:0/2146663584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e8009550 con 0x7fb7f40d15f0 2023-11-28T12:02:59.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f3fff700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7f40d15f0 0x7fb7f41549f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000e900 tx=0x7fb7e000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:59.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f17fa700 1 -- 172.21.15.57:0/2146663584 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0013680 con 0x7fb7f40d15f0 2023-11-28T12:02:59.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f17fa700 1 -- 172.21.15.57:0/2146663584 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e00137e0 con 0x7fb7f40d15f0 2023-11-28T12:02:59.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7fb02f700 1 -- 172.21.15.57:0/2146663584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f415e980 con 0x7fb7f40d15f0 2023-11-28T12:02:59.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7f17fa700 1 -- 172.21.15.57:0/2146663584 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0013b70 con 0x7fb7f40d15f0 2023-11-28T12:02:59.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.790+0000 7fb7fb02f700 1 -- 172.21.15.57:0/2146663584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f406ed10 con 0x7fb7f40d15f0 2023-11-28T12:02:59.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.791+0000 7fb7fb02f700 1 -- 172.21.15.57:0/2146663584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f415ec60 con 0x7fb7f40d15f0 2023-11-28T12:02:59.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.792+0000 7fb7f17fa700 1 -- 172.21.15.57:0/2146663584 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb7e0019030 con 0x7fb7f40d15f0 2023-11-28T12:02:59.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.792+0000 7fb7f17fa700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7d80682f0 0x7fb7d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:02:59.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.792+0000 7fb7f17fa700 1 -- 172.21.15.57:0/2146663584 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb7e00859b0 con 0x7fb7f40d15f0 2023-11-28T12:02:59.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.792+0000 7fb7f8dcb700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7d80682f0 0x7fb7d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:02:59.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.793+0000 7fb7f8dcb700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7d80682f0 0x7fb7d806a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009c10 tx=0x7fb7e8008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:02:59.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:02:59.797+0000 7fb7f17fa700 1 -- 172.21.15.57:0/2146663584 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb7e0052210 con 0x7fb7f40d15f0 2023-11-28T12:03:00.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.048+0000 7fb7fb02f700 1 -- 172.21.15.57:0/2146663584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb7f40743f0 con 0x7fb7f40d15f0 2023-11-28T12:03:00.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.048+0000 7fb7f17fa700 1 -- 172.21.15.57:0/2146663584 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb7e0017070 con 0x7fb7f40d15f0 2023-11-28T12:03:00.050 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T12:03:00.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.050+0000 7fb7fb02f700 1 -- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7d80682f0 msgr2=0x7fb7d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:00.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.050+0000 7fb7fb02f700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7d80682f0 0x7fb7d806a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009c10 tx=0x7fb7e8008030 comp rx=0 tx=0).stop 2023-11-28T12:03:00.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.050+0000 7fb7fb02f700 1 -- 172.21.15.57:0/2146663584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7f40d15f0 msgr2=0x7fb7f41549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:00.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.050+0000 7fb7fb02f700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7f40d15f0 0x7fb7f41549f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000e900 tx=0x7fb7e000ec10 comp rx=0 tx=0).stop 2023-11-28T12:03:00.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.050+0000 7fb7fb02f700 1 -- 172.21.15.57:0/2146663584 shutdown_connections 2023-11-28T12:03:00.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.050+0000 7fb7fb02f700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7d80682f0 0x7fb7d806a7a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:00.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.050+0000 7fb7fb02f700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f415e010 0x7fb7f4154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:00.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.050+0000 7fb7fb02f700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7f406e400 0x7fb7f40b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:00.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.050+0000 7fb7fb02f700 1 --2- 172.21.15.57:0/2146663584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7f40d15f0 0x7fb7f41549f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:00.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.050+0000 7fb7fb02f700 1 -- 172.21.15.57:0/2146663584 >> 172.21.15.57:0/2146663584 conn(0x7fb7f406a270 msgr2=0x7fb7f40b55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:00.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.050+0000 7fb7fb02f700 1 -- 172.21.15.57:0/2146663584 shutdown_connections 2023-11-28T12:03:00.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:00.051+0000 7fb7fb02f700 1 -- 172.21.15.57:0/2146663584 wait complete. 2023-11-28T12:03:00.065 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T12:03:00.065 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T12:03:00.065 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2023-11-28T12:03:00.065 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T12:03:00.065 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T12:03:00.065 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T12:03:00.065 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T12:03:00.066 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T12:03:00.066 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T12:03:00.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:00.066 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2023-11-28T12:03:00.086 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T12:03:00.087 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T12:03:00.087 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T12:03:00.087 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:00.087 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-11-28T12:03:00.087 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:00.087 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:03:00.087 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:03:00.087 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f0 2023-11-28T12:03:00.087 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:03:00.087 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:03:00.087 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f1 2023-11-28T12:03:00.088 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T12:03:00.088 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:03:00.088 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f0 2023-11-28T12:03:00.088 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T12:03:00.088 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 41879sec preferred_lft 41879sec 2023-11-28T12:03:00.088 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-11-28T12:03:00.088 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:00.088 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:03:00.088 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:03:00.088 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f1 2023-11-28T12:03:00.088 INFO:teuthology.orchestra.run.smithi057.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T12:03:00.088 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 8e:b7:77:39:69:ff brd ff:ff:ff:ff:ff:ff 2023-11-28T12:03:00.089 INFO:teuthology.orchestra.run.smithi057.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T12:03:00.089 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:00.089 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::3f:caff:fe00:648e/64 scope link 2023-11-28T12:03:00.089 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:00.089 INFO:teuthology.orchestra.run.smithi057.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T12:03:00.089 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether f2:ef:c1:11:c5:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T12:03:00.089 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::f0ef:c1ff:fe11:c5fb/64 scope link 2023-11-28T12:03:00.089 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:00.089 INFO:teuthology.orchestra.run.smithi057.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T12:03:00.089 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 8e:b7:77:39:69:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T12:03:00.089 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::8cb7:77ff:fe39:69ff/64 scope link 2023-11-28T12:03:00.089 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:00.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:00.090 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T12:03:00.157 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-11-28T12:03:00.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:00.160 DEBUG:teuthology.orchestra.run.smithi057:> route 2023-11-28T12:03:00.225 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2023-11-28T12:03:00.225 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T12:03:00.225 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T12:03:00.225 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T12:03:00.225 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T12:03:00.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:00.226 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-11-28T12:03:00.226 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-11-28T12:03:00.226 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T12:03:00.226 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T12:03:00.226 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T12:03:00.226 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-11-28T12:03:00.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:00.348 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-11-28T12:03:00.369 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T12:03:00.369 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T12:03:00.369 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T12:03:00.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:00.369 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2023-11-28T12:03:00.443 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T12:03:00.444 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:03:00.444 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T12:03:00.507 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T12:03:00.508 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:03:00.508 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T12:03:00.571 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-11-28T12:03:00.647 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:03:00.647 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:03:00.721 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:03:00.722 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:03:00.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:00.723 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-11-28T12:03:00.787 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T12:03:00.787 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/) 2023-11-28T12:03:00.829 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-11-28T12:03:00.864 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:03:00.864 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:03:00.935 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:2023-11-28T12:03:00.934+0000 7f2ccd58d5c0 -1 init, newargv = 0x5573e7e9ef40 newargc=15 2023-11-28T12:03:00.935 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[137297]: starting ceph client 2023-11-28T12:03:00.936 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:03:00.937 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:03:00.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:00.937 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-11-28T12:03:00.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[137297]: starting fuse 2023-11-28T12:03:00.961 INFO:teuthology.orchestra.run.smithi057.stdout:47 2023-11-28T12:03:00.961 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T12:03:00.962 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:03:00.962 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2023-11-28T12:03:00.962 DEBUG:teuthology.orchestra.run.smithi057:> import re 2023-11-28T12:03:00.962 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-11-28T12:03:00.962 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2023-11-28T12:03:00.962 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:00.962 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2023-11-28T12:03:00.962 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T12:03:00.962 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2023-11-28T12:03:00.962 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T12:03:00.963 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:00.963 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2023-11-28T12:03:00.963 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2023-11-28T12:03:00.963 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2023-11-28T12:03:00.963 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2023-11-28T12:03:00.963 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:00.963 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2023-11-28T12:03:00.963 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T12:03:00.963 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T12:03:00.963 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T12:03:00.963 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2023-11-28T12:03:00.963 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2023-11-28T12:03:00.963 DEBUG:teuthology.orchestra.run.smithi057:> return f 2023-11-28T12:03:00.964 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T12:03:00.964 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:00.964 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.0")) 2023-11-28T12:03:00.964 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-11-28T12:03:01.100 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.0.137297.asok 2023-11-28T12:03:01.281 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.137297.asok 2023-11-28T12:03:01.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:01.281 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.137297.asok status 2023-11-28T12:03:01.462 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:03:01.462 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2023-11-28T12:03:01.462 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T12:03:01.462 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T12:03:01.462 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "0", 2023-11-28T12:03:01.462 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2023-11-28T12:03:01.462 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T12:03:01.463 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "137297", 2023-11-28T12:03:01.463 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2023-11-28T12:03:01.463 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:01.463 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2023-11-28T12:03:01.463 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2023-11-28T12:03:01.463 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 5433, 2023-11-28T12:03:01.463 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2023-11-28T12:03:01.463 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2023-11-28T12:03:01.463 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2023-11-28T12:03:01.463 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 5433 2023-11-28T12:03:01.463 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:01.463 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-11-28T12:03:01.464 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-11-28T12:03:01.464 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-11-28T12:03:01.464 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 392456301 2023-11-28T12:03:01.464 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:01.464 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:01.464 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-11-28T12:03:01.464 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-11-28T12:03:01.464 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-11-28T12:03:01.464 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 392456301 2023-11-28T12:03:01.464 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:01.464 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.5433 192.168.0.1:0/392456301", 2023-11-28T12:03:01.464 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.1:0/392456301", 2023-11-28T12:03:01.464 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2023-11-28T12:03:01.465 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 16, 2023-11-28T12:03:01.465 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 35, 2023-11-28T12:03:01.465 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2023-11-28T12:03:01.465 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2023-11-28T12:03:01.465 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2023-11-28T12:03:01.465 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:03:01.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:01.473 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2023-11-28T12:03:01.473 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T12:03:01.493 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2023-11-28T12:03:01.493 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T12:03:01.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:01.494 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T12:03:01.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:01.567 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2023-11-28T12:03:01.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:01.638 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2023-11-28T12:03:01.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:01.705 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2023-11-28T12:03:01.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:01.772 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2023-11-28T12:03:01.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:01.841 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2023-11-28T12:03:01.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:01.908 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-28T12:03:01.982 INFO:teuthology.orchestra.run.smithi057.stdout:./ 2023-11-28T12:03:01.983 INFO:teuthology.orchestra.run.smithi057.stdout:./rfile 2023-11-28T12:03:01.983 INFO:teuthology.orchestra.run.smithi057.stdout:./parentdir 2023-11-28T12:03:01.983 INFO:teuthology.orchestra.run.smithi057.stdout:./parentdir/pfile 2023-11-28T12:03:01.983 INFO:teuthology.orchestra.run.smithi057.stdout:./parentdir/childdir 2023-11-28T12:03:01.983 INFO:teuthology.orchestra.run.smithi057.stdout:./parentdir/childdir/cfile 2023-11-28T12:03:01.984 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:03:01.984 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-11-28T12:03:01.984 DEBUG:teuthology.orchestra.run.smithi057:> import stat 2023-11-28T12:03:01.984 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:01.984 DEBUG:teuthology.orchestra.run.smithi057:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-28T12:03:01.984 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-11-28T12:03:02.106 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-11-28T12:03:06.540 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:03:06.540 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-11-28T12:03:06.540 DEBUG:teuthology.orchestra.run.smithi057:> import stat 2023-11-28T12:03:06.540 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:06.540 DEBUG:teuthology.orchestra.run.smithi057:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2023-11-28T12:03:06.540 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-11-28T12:03:06.660 INFO:teuthology.orchestra.run.smithi057.stdout:1099511627778 2023-11-28T12:03:06.849 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:03:06.849 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-11-28T12:03:06.849 DEBUG:teuthology.orchestra.run.smithi057:> import stat 2023-11-28T12:03:06.849 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:06.850 DEBUG:teuthology.orchestra.run.smithi057:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2023-11-28T12:03:06.850 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-11-28T12:03:06.971 INFO:teuthology.orchestra.run.smithi057.stdout:1099511627776 2023-11-28T12:03:07.159 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:03:07.159 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-11-28T12:03:07.159 DEBUG:teuthology.orchestra.run.smithi057:> import stat 2023-11-28T12:03:07.159 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:07.159 DEBUG:teuthology.orchestra.run.smithi057:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2023-11-28T12:03:07.159 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-11-28T12:03:07.284 INFO:teuthology.orchestra.run.smithi057.stdout:1099511628279 2023-11-28T12:03:07.472 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:03:07.472 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-11-28T12:03:07.472 DEBUG:teuthology.orchestra.run.smithi057:> import stat 2023-11-28T12:03:07.472 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:07.472 DEBUG:teuthology.orchestra.run.smithi057:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2023-11-28T12:03:07.472 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-11-28T12:03:07.594 INFO:teuthology.orchestra.run.smithi057.stdout:1099511627777 2023-11-28T12:03:07.783 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:03:07.783 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-11-28T12:03:07.783 DEBUG:teuthology.orchestra.run.smithi057:> import stat 2023-11-28T12:03:07.783 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:07.783 DEBUG:teuthology.orchestra.run.smithi057:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2023-11-28T12:03:07.783 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-11-28T12:03:07.905 INFO:teuthology.orchestra.run.smithi057.stdout:1099511628280 2023-11-28T12:03:08.093 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:03:08.093 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:03:08.157 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:03:08.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.339+0000 7f975e308700 1 Processor -- start 2023-11-28T12:03:08.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.339+0000 7f975e308700 1 -- start start 2023-11-28T12:03:08.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.340+0000 7f975e308700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f975815cc80 0x7f975806de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.340+0000 7f975e308700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97580b1810 con 0x7f975806e3a0 2023-11-28T12:03:08.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.340+0000 7f975e308700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97580b1af0 con 0x7f975806a270 2023-11-28T12:03:08.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.340+0000 7f975e308700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97580b2490 con 0x7f975815cc80 2023-11-28T12:03:08.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.340+0000 7f975c8a5700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f975806e3a0 0x7f9758070d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46970/0 (socket says 172.21.15.57:46970) 2023-11-28T12:03:08.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.340+0000 7f975c8a5700 1 -- 172.21.15.57:0/1113366770 learned_addr learned my addr 172.21.15.57:0/1113366770 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:08.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.340+0000 7f97577fe700 1 --2- 172.21.15.57:0/1113366770 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f975815cc80 0x7f975806de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720532508 0 0) 0x7f97580b1810 con 0x7f975806e3a0 2023-11-28T12:03:08.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97400035b0 con 0x7f975806e3a0 2023-11-28T12:03:08.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417726614 0 0) 0x7f97580b1af0 con 0x7f975806a270 2023-11-28T12:03:08.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97580b1810 con 0x7f975806a270 2023-11-28T12:03:08.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 524215019 0 0) 0x7f97400035b0 con 0x7f975806e3a0 2023-11-28T12:03:08.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f975815cc80 msgr2=0x7f975806de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:08.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 --2- 172.21.15.57:0/1113366770 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f975815cc80 0x7f975806de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 >> v1:172.21.15.181:6789/0 conn(0x7f975806a270 legacy=0x7f975806a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97580b1dd0 con 0x7f975806e3a0 2023-11-28T12:03:08.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f97500019f0 con 0x7f975806e3a0 2023-11-28T12:03:08.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 >> v1:172.21.15.57:6789/0 conn(0x7f975806e3a0 legacy=0x7f9758070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 --2- 172.21.15.57:0/1113366770 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f975815cc80 0x7f9740003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 --2- 172.21.15.57:0/1113366770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9740004120 0x7f9740004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 --2- 172.21.15.57:0/1113366770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9740004d90 0x7f9740005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97580b2490 con 0x7f9740004120 2023-11-28T12:03:08.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974000bc00 con 0x7f9740004d90 2023-11-28T12:03:08.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974000bdc0 con 0x7f975815cc80 2023-11-28T12:03:08.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f97577fe700 1 --2- 172.21.15.57:0/1113366770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9740004120 0x7f9740004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f9757fff700 1 --2- 172.21.15.57:0/1113366770 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f975815cc80 0x7f9740003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.341+0000 7f975c8a5700 1 --2- 172.21.15.57:0/1113366770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9740004d90 0x7f9740005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f97577fe700 1 -- 172.21.15.57:0/1113366770 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f975815cc80 msgr2=0x7f9740003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:08.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f97577fe700 1 --2- 172.21.15.57:0/1113366770 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f975815cc80 0x7f9740003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f97577fe700 1 -- 172.21.15.57:0/1113366770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9740004d90 msgr2=0x7f9740005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:08.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f97577fe700 1 --2- 172.21.15.57:0/1113366770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9740004d90 0x7f9740005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f97577fe700 1 -- 172.21.15.57:0/1113366770 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97400035b0 con 0x7f9740004120 2023-11-28T12:03:08.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f9757fff700 1 --2- 172.21.15.57:0/1113366770 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f975815cc80 0x7f9740003bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:03:08.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f97577fe700 1 --2- 172.21.15.57:0/1113366770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9740004120 0x7f9740004850 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f974c009b30 tx=0x7f9740005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8172cf882b64e5a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:08.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c00e6c0 con 0x7f9740004120 2023-11-28T12:03:08.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c00e820 con 0x7f9740004120 2023-11-28T12:03:08.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f9756ffd700 1 -- 172.21.15.57:0/1113366770 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c00eb90 con 0x7f9740004120 2023-11-28T12:03:08.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f975e308700 1 -- 172.21.15.57:0/1113366770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9740004120 msgr2=0x7f9740004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f975e308700 1 --2- 172.21.15.57:0/1113366770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9740004120 0x7f9740004850 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f974c009b30 tx=0x7f9740005880 comp rx=0 tx=0).stop 2023-11-28T12:03:08.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f975c8a5700 1 -- 172.21.15.57:0/1113366770 reap_dead start 2023-11-28T12:03:08.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f975e308700 1 -- 172.21.15.57:0/1113366770 shutdown_connections 2023-11-28T12:03:08.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f975e308700 1 -- 172.21.15.57:0/1113366770 >> 172.21.15.57:0/1113366770 conn(0x7f975806c010 msgr2=0x7f97580cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:08.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.342+0000 7f975e308700 1 -- 172.21.15.57:0/1113366770 shutdown_connections 2023-11-28T12:03:08.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.343+0000 7f975e308700 1 -- 172.21.15.57:0/1113366770 wait complete. 2023-11-28T12:03:08.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.344+0000 7f975e308700 1 Processor -- start 2023-11-28T12:03:08.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.344+0000 7f975e308700 1 -- start start 2023-11-28T12:03:08.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f975e308700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9740004120 0x7f975815dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f975e308700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9758150f90 0x7f97581506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f975e308700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97580ad510 0x7f97580ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f975e308700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97580adef0 con 0x7f9740004120 2023-11-28T12:03:08.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f975e308700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97580b3c20 con 0x7f97580ad510 2023-11-28T12:03:08.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f975e308700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97580b3ec0 con 0x7f9758150f90 2023-11-28T12:03:08.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f9757fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9740004120 0x7f975815dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f9757fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9740004120 0x7f975815dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40574/0 (socket says 172.21.15.57:40574) 2023-11-28T12:03:08.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f9757fff700 1 -- 172.21.15.57:0/2606777855 learned_addr learned my addr 172.21.15.57:0/2606777855 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:08.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f97577fe700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9758150f90 0x7f97581506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f975c8a5700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97580ad510 0x7f97580ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f9757fff700 1 -- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9758150f90 msgr2=0x7f97581506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f9757fff700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9758150f90 0x7f97581506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f9757fff700 1 -- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97580ad510 msgr2=0x7f97580ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f9757fff700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97580ad510 0x7f97580ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f9757fff700 1 -- 172.21.15.57:0/2606777855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975806e7e0 con 0x7f9740004120 2023-11-28T12:03:08.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f97577fe700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9758150f90 0x7f97581506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:08.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.345+0000 7f9757fff700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9740004120 0x7f975815dde0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f974800fae0 tx=0x7f974800fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:08.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.346+0000 7f9754ff9700 1 -- 172.21.15.57:0/2606777855 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97480052e0 con 0x7f9740004120 2023-11-28T12:03:08.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.346+0000 7f9754ff9700 1 -- 172.21.15.57:0/2606777855 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9748005440 con 0x7f9740004120 2023-11-28T12:03:08.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.346+0000 7f975e308700 1 -- 172.21.15.57:0/2606777855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97580b20b0 con 0x7f9740004120 2023-11-28T12:03:08.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.346+0000 7f9754ff9700 1 -- 172.21.15.57:0/2606777855 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97480038a0 con 0x7f9740004120 2023-11-28T12:03:08.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.346+0000 7f975e308700 1 -- 172.21.15.57:0/2606777855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975815daa0 con 0x7f9740004120 2023-11-28T12:03:08.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.347+0000 7f975e308700 1 -- 172.21.15.57:0/2606777855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97580743f0 con 0x7f9740004120 2023-11-28T12:03:08.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.347+0000 7f9754ff9700 1 -- 172.21.15.57:0/2606777855 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9748003a00 con 0x7f9740004120 2023-11-28T12:03:08.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.348+0000 7f9754ff9700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f97440682f0 0x7f974406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.348+0000 7f9754ff9700 1 -- 172.21.15.57:0/2606777855 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9748085e50 con 0x7f9740004120 2023-11-28T12:03:08.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.348+0000 7f97577fe700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f97440682f0 0x7f974406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.349+0000 7f97577fe700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f97440682f0 0x7f974406a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f97581f5e20 tx=0x7f974c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:08.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.352+0000 7f9754ff9700 1 -- 172.21.15.57:0/2606777855 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f97480526b0 con 0x7f9740004120 2023-11-28T12:03:08.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.556+0000 7f975e308700 1 -- 172.21.15.57:0/2606777855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97580cd5f0 con 0x7f9740004120 2023-11-28T12:03:08.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.557+0000 7f9754ff9700 1 -- 172.21.15.57:0/2606777855 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+17921 (secure 0 0 0) 0x7f9748055cd0 con 0x7f9740004120 2023-11-28T12:03:08.558 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:08.559 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":35,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:02:54.404560+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:02:51.245666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T12:02:52.363863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:03:08.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.559+0000 7f975e308700 1 -- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f97440682f0 msgr2=0x7f974406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.559+0000 7f975e308700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f97440682f0 0x7f974406a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f97581f5e20 tx=0x7f974c008030 comp rx=0 tx=0).stop 2023-11-28T12:03:08.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.559+0000 7f975e308700 1 -- 172.21.15.57:0/2606777855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9740004120 msgr2=0x7f975815dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.559+0000 7f975e308700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9740004120 0x7f975815dde0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f974800fae0 tx=0x7f974800fea0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.559+0000 7f975e308700 1 -- 172.21.15.57:0/2606777855 shutdown_connections 2023-11-28T12:03:08.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.559+0000 7f975e308700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f97440682f0 0x7f974406a7a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.559+0000 7f975e308700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97580ad510 0x7f97580ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.559+0000 7f975e308700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9740004120 0x7f975815dde0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.559+0000 7f975e308700 1 --2- 172.21.15.57:0/2606777855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9758150f90 0x7f97581506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.559+0000 7f975e308700 1 -- 172.21.15.57:0/2606777855 >> 172.21.15.57:0/2606777855 conn(0x7f975806c010 msgr2=0x7f9758153200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:08.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.559+0000 7f975e308700 1 -- 172.21.15.57:0/2606777855 shutdown_connections 2023-11-28T12:03:08.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.559+0000 7f975e308700 1 -- 172.21.15.57:0/2606777855 wait complete. 2023-11-28T12:03:08.575 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:02:51.245666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:02:52.363863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:03:08.575 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:03:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.716+0000 7f493c756700 1 Processor -- start 2023-11-28T12:03:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.716+0000 7f493c756700 1 -- start start 2023-11-28T12:03:08.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.717+0000 7f493c756700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f49340d0ad0 0x7f49340cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.717+0000 7f493c756700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49340cc740 con 0x7f493415a5f0 2023-11-28T12:03:08.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.717+0000 7f493c756700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493415a9c0 con 0x7f49340b2ee0 2023-11-28T12:03:08.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.717+0000 7f493c756700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49340c7b80 con 0x7f49340d0ad0 2023-11-28T12:03:08.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.717+0000 7f493a4f2700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f493415a5f0 0x7f49340b27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46984/0 (socket says 172.21.15.57:46984) 2023-11-28T12:03:08.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.717+0000 7f493a4f2700 1 -- 172.21.15.57:0/3317326883 learned_addr learned my addr 172.21.15.57:0/3317326883 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.717+0000 7f493acf3700 1 --2- 172.21.15.57:0/3317326883 >> v2:172.21.15.181:3300/0 conn(0x7f49340d0ad0 0x7f49340cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.717+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186275570 0 0) 0x7f49340cc740 con 0x7f493415a5f0 2023-11-28T12:03:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.717+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49200035b0 con 0x7f493415a5f0 2023-11-28T12:03:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682688709 0 0) 0x7f493415a9c0 con 0x7f49340b2ee0 2023-11-28T12:03:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49340cc740 con 0x7f49340b2ee0 2023-11-28T12:03:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1698034901 0 0) 0x7f49200035b0 con 0x7f493415a5f0 2023-11-28T12:03:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 >> v2:172.21.15.181:3300/0 conn(0x7f49340d0ad0 msgr2=0x7f49340cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 --2- 172.21.15.57:0/3317326883 >> v2:172.21.15.181:3300/0 conn(0x7f49340d0ad0 0x7f49340cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 >> v1:172.21.15.181:6789/0 conn(0x7f49340b2ee0 legacy=0x7f49340ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340cc990 con 0x7f493415a5f0 2023-11-28T12:03:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f4928001670 con 0x7f493415a5f0 2023-11-28T12:03:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 >> v1:172.21.15.57:6789/0 conn(0x7f493415a5f0 legacy=0x7f49340b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 --2- 172.21.15.57:0/3317326883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4920003b90 0x7f4920003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 --2- 172.21.15.57:0/3317326883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4920004850 0x7f4920004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 --2- 172.21.15.57:0/3317326883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4920005200 0x7f492000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49340c7b80 con 0x7f4920004850 2023-11-28T12:03:08.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492000c060 con 0x7f4920005200 2023-11-28T12:03:08.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492000c220 con 0x7f4920003b90 2023-11-28T12:03:08.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f4939cf1700 1 --2- 172.21.15.57:0/3317326883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4920004850 0x7f4920004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f493a4f2700 1 --2- 172.21.15.57:0/3317326883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4920003b90 0x7f4920003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f493acf3700 1 --2- 172.21.15.57:0/3317326883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4920005200 0x7f492000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f4939cf1700 1 -- 172.21.15.57:0/3317326883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4920003b90 msgr2=0x7f4920003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:08.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f4939cf1700 1 --2- 172.21.15.57:0/3317326883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4920003b90 0x7f4920003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f4939cf1700 1 -- 172.21.15.57:0/3317326883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4920005200 msgr2=0x7f492000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:08.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f4939cf1700 1 --2- 172.21.15.57:0/3317326883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4920005200 0x7f492000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f493acf3700 1 -- 172.21.15.57:0/3317326883 reap_dead start 2023-11-28T12:03:08.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f4939cf1700 1 -- 172.21.15.57:0/3317326883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49200035b0 con 0x7f4920004850 2023-11-28T12:03:08.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.718+0000 7f4939cf1700 1 --2- 172.21.15.57:0/3317326883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4920004850 0x7f4920004cc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f49340b32d0 tx=0x7f4920005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de1317e9f716a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:08.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.719+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930001430 con 0x7f4920004850 2023-11-28T12:03:08.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.719+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49300052f0 con 0x7f4920004850 2023-11-28T12:03:08.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.719+0000 7f49394f0700 1 -- 172.21.15.57:0/3317326883 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930001d40 con 0x7f4920004850 2023-11-28T12:03:08.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.719+0000 7f493c756700 1 -- 172.21.15.57:0/3317326883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4920004850 msgr2=0x7f4920004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.719+0000 7f493c756700 1 --2- 172.21.15.57:0/3317326883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4920004850 0x7f4920004cc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f49340b32d0 tx=0x7f4920005880 comp rx=0 tx=0).stop 2023-11-28T12:03:08.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.719+0000 7f493c756700 1 -- 172.21.15.57:0/3317326883 shutdown_connections 2023-11-28T12:03:08.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.719+0000 7f493c756700 1 --2- 172.21.15.57:0/3317326883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4920004850 0x7f4920004cc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.719+0000 7f493c756700 1 -- 172.21.15.57:0/3317326883 >> 172.21.15.57:0/3317326883 conn(0x7f493406ea80 msgr2=0x7f493406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:08.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.719+0000 7f493c756700 1 -- 172.21.15.57:0/3317326883 shutdown_connections 2023-11-28T12:03:08.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.719+0000 7f493c756700 1 -- 172.21.15.57:0/3317326883 wait complete. 2023-11-28T12:03:08.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.721+0000 7f493c756700 1 Processor -- start 2023-11-28T12:03:08.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.721+0000 7f493c756700 1 -- start start 2023-11-28T12:03:08.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.721+0000 7f493c756700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4920004850 0x7f4934152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.721+0000 7f493c756700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49341512b0 0x7f49341e1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.721+0000 7f493c756700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49341e40d0 0x7f4934158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.721+0000 7f493c756700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4934151720 con 0x7f4920004850 2023-11-28T12:03:08.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.721+0000 7f493c756700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49340b5190 con 0x7f49341512b0 2023-11-28T12:03:08.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.721+0000 7f493c756700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49341e4920 con 0x7f49341e40d0 2023-11-28T12:03:08.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f493a4f2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4920004850 0x7f4934152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f493a4f2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4920004850 0x7f4934152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40594/0 (socket says 172.21.15.57:40594) 2023-11-28T12:03:08.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f493a4f2700 1 -- 172.21.15.57:0/1549108662 learned_addr learned my addr 172.21.15.57:0/1549108662 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:08.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f4939cf1700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49341512b0 0x7f49341e1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f493acf3700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49341e40d0 0x7f4934158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f493a4f2700 1 -- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49341e40d0 msgr2=0x7f4934158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f493a4f2700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49341e40d0 0x7f4934158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f493a4f2700 1 -- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49341512b0 msgr2=0x7f49341e1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f493a4f2700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49341512b0 0x7f49341e1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f493a4f2700 1 -- 172.21.15.57:0/1549108662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340ccf50 con 0x7f4920004850 2023-11-28T12:03:08.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f4939cf1700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49341512b0 0x7f49341e1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:08.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f493a4f2700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4920004850 0x7f4934152810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4928007340 tx=0x7f4928007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:08.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f4926ffd700 1 -- 172.21.15.57:0/1549108662 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800f660 con 0x7f4920004850 2023-11-28T12:03:08.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f4926ffd700 1 -- 172.21.15.57:0/1549108662 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492800f800 con 0x7f4920004850 2023-11-28T12:03:08.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f4926ffd700 1 -- 172.21.15.57:0/1549108662 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800fbb0 con 0x7f4920004850 2023-11-28T12:03:08.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f493c756700 1 -- 172.21.15.57:0/1549108662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49340ccc70 con 0x7f4920004850 2023-11-28T12:03:08.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.722+0000 7f493c756700 1 -- 172.21.15.57:0/1549108662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49340cd230 con 0x7f4920004850 2023-11-28T12:03:08.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.723+0000 7f493c756700 1 -- 172.21.15.57:0/1549108662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49340743f0 con 0x7f4920004850 2023-11-28T12:03:08.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.724+0000 7f4926ffd700 1 -- 172.21.15.57:0/1549108662 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f492800fd10 con 0x7f4920004850 2023-11-28T12:03:08.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.724+0000 7f4926ffd700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f49180682f0 0x7f491806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:08.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.724+0000 7f4926ffd700 1 -- 172.21.15.57:0/1549108662 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4928085dc0 con 0x7f4920004850 2023-11-28T12:03:08.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.724+0000 7f4939cf1700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f49180682f0 0x7f491806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:08.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.725+0000 7f4939cf1700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f49180682f0 0x7f491806a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4930005040 tx=0x7f493000a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:08.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.729+0000 7f4926ffd700 1 -- 172.21.15.57:0/1549108662 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4928052620 con 0x7f4920004850 2023-11-28T12:03:08.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.927+0000 7f493c756700 1 -- 172.21.15.57:0/1549108662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f49341e2130 con 0x7f4920004850 2023-11-28T12:03:08.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.928+0000 7f4926ffd700 1 -- 172.21.15.57:0/1549108662 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4928055c40 con 0x7f4920004850 2023-11-28T12:03:08.929 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:03:08.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.930+0000 7f493c756700 1 -- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f49180682f0 msgr2=0x7f491806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.930+0000 7f493c756700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f49180682f0 0x7f491806a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4930005040 tx=0x7f493000a000 comp rx=0 tx=0).stop 2023-11-28T12:03:08.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.930+0000 7f493c756700 1 -- 172.21.15.57:0/1549108662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4920004850 msgr2=0x7f4934152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:08.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.930+0000 7f493c756700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4920004850 0x7f4934152810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4928007340 tx=0x7f4928007650 comp rx=0 tx=0).stop 2023-11-28T12:03:08.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.930+0000 7f493c756700 1 -- 172.21.15.57:0/1549108662 shutdown_connections 2023-11-28T12:03:08.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.930+0000 7f493c756700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f49180682f0 0x7f491806a7a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.930+0000 7f493c756700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49341e40d0 0x7f4934158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.930+0000 7f493c756700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4920004850 0x7f4934152810 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.930+0000 7f493c756700 1 --2- 172.21.15.57:0/1549108662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49341512b0 0x7f49341e1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:08.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.930+0000 7f493c756700 1 -- 172.21.15.57:0/1549108662 >> 172.21.15.57:0/1549108662 conn(0x7f493406ea80 msgr2=0x7f49340cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:08.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.930+0000 7f493c756700 1 -- 172.21.15.57:0/1549108662 shutdown_connections 2023-11-28T12:03:08.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:08.930+0000 7f493c756700 1 -- 172.21.15.57:0/1549108662 wait complete. 2023-11-28T12:03:08.945 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:03:09.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.083+0000 7fbf6bf0b700 1 Processor -- start 2023-11-28T12:03:09.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.083+0000 7fbf6bf0b700 1 -- start start 2023-11-28T12:03:09.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf6bf0b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbf6406a270 0x7fbf6406a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf6bf0b700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6406abd0 con 0x7fbf640718e0 2023-11-28T12:03:09.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf6bf0b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf64160a50 con 0x7fbf6415c810 2023-11-28T12:03:09.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf6bf0b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6415cbe0 con 0x7fbf6406a270 2023-11-28T12:03:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf694a6700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fbf640718e0 0x7fbf6406c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47000/0 (socket says 172.21.15.57:47000) 2023-11-28T12:03:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf694a6700 1 -- 172.21.15.57:0/2785668619 learned_addr learned my addr 172.21.15.57:0/2785668619 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf6a4a8700 1 --2- 172.21.15.57:0/2785668619 >> v2:172.21.15.181:3300/0 conn(0x7fbf6406a270 0x7fbf6406a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910049062 0 0) 0x7fbf6406abd0 con 0x7fbf640718e0 2023-11-28T12:03:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf500035b0 con 0x7fbf640718e0 2023-11-28T12:03:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3873044271 0 0) 0x7fbf500035b0 con 0x7fbf640718e0 2023-11-28T12:03:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 >> v2:172.21.15.181:3300/0 conn(0x7fbf6406a270 msgr2=0x7fbf6406a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf68ca5700 1 --2- 172.21.15.57:0/2785668619 >> v2:172.21.15.181:3300/0 conn(0x7fbf6406a270 0x7fbf6406a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 >> v1:172.21.15.181:6789/0 conn(0x7fbf6415c810 legacy=0x7fbf64160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf64160d30 con 0x7fbf640718e0 2023-11-28T12:03:09.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fbf58001670 con 0x7fbf640718e0 2023-11-28T12:03:09.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.084+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 >> v1:172.21.15.57:6789/0 conn(0x7fbf640718e0 legacy=0x7fbf6406c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf68ca5700 1 --2- 172.21.15.57:0/2785668619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf50003a00 0x7fbf50003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf68ca5700 1 --2- 172.21.15.57:0/2785668619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf50004850 0x7fbf50004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf68ca5700 1 --2- 172.21.15.57:0/2785668619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf50005200 0x7fbf5000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6415cbe0 con 0x7fbf50005200 2023-11-28T12:03:09.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5000c080 con 0x7fbf50004850 2023-11-28T12:03:09.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5000c240 con 0x7fbf50003a00 2023-11-28T12:03:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf6a4a8700 1 --2- 172.21.15.57:0/2785668619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf50005200 0x7fbf5000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf694a6700 1 --2- 172.21.15.57:0/2785668619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf50004850 0x7fbf50004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf69ca7700 1 --2- 172.21.15.57:0/2785668619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf50003a00 0x7fbf50003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf6a4a8700 1 -- 172.21.15.57:0/2785668619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf50003a00 msgr2=0x7fbf50003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf6a4a8700 1 --2- 172.21.15.57:0/2785668619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf50003a00 0x7fbf50003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf6a4a8700 1 -- 172.21.15.57:0/2785668619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf50004850 msgr2=0x7fbf50004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf6a4a8700 1 --2- 172.21.15.57:0/2785668619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf50004850 0x7fbf50004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf6a4a8700 1 -- 172.21.15.57:0/2785668619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf500035b0 con 0x7fbf50005200 2023-11-28T12:03:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf6a4a8700 1 -- 172.21.15.57:0/2785668619 reap_dead start 2023-11-28T12:03:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf6a4a8700 1 --2- 172.21.15.57:0/2785668619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf50005200 0x7fbf5000ba10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009fc0 tx=0x7fbf50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf07e6c9a391f74 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c003a20 con 0x7fbf50005200 2023-11-28T12:03:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c00f040 con 0x7fbf50005200 2023-11-28T12:03:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.085+0000 7fbf68ca5700 1 -- 172.21.15.57:0/2785668619 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c003de0 con 0x7fbf50005200 2023-11-28T12:03:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.086+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/2785668619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf50005200 msgr2=0x7fbf5000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.086+0000 7fbf6bf0b700 1 --2- 172.21.15.57:0/2785668619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf50005200 0x7fbf5000ba10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009fc0 tx=0x7fbf50005880 comp rx=0 tx=0).stop 2023-11-28T12:03:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.086+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/2785668619 shutdown_connections 2023-11-28T12:03:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.086+0000 7fbf6bf0b700 1 --2- 172.21.15.57:0/2785668619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf50005200 0x7fbf5000ba10 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.086+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/2785668619 >> 172.21.15.57:0/2785668619 conn(0x7fbf6406c010 msgr2=0x7fbf640cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.086+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/2785668619 shutdown_connections 2023-11-28T12:03:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.086+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/2785668619 wait complete. 2023-11-28T12:03:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.087+0000 7fbf6bf0b700 1 Processor -- start 2023-11-28T12:03:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.087+0000 7fbf6bf0b700 1 -- start start 2023-11-28T12:03:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.088+0000 7fbf6bf0b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf50005200 0x7fbf64158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.088+0000 7fbf6bf0b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf6415da00 0x7fbf6415ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.088+0000 7fbf6bf0b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf6414bb00 0x7fbf6414bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.088+0000 7fbf6bf0b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf641587f0 con 0x7fbf50005200 2023-11-28T12:03:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.088+0000 7fbf6bf0b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6414c4c0 con 0x7fbf6415da00 2023-11-28T12:03:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.088+0000 7fbf6bf0b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6415de70 con 0x7fbf6414bb00 2023-11-28T12:03:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.088+0000 7fbf69ca7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf50005200 0x7fbf64158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.088+0000 7fbf69ca7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf50005200 0x7fbf64158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40606/0 (socket says 172.21.15.57:40606) 2023-11-28T12:03:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.088+0000 7fbf69ca7700 1 -- 172.21.15.57:0/3934265290 learned_addr learned my addr 172.21.15.57:0/3934265290 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf6a4a8700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf6414bb00 0x7fbf6414bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf694a6700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf6415da00 0x7fbf6415ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf69ca7700 1 -- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf6414bb00 msgr2=0x7fbf6414bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf69ca7700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf6414bb00 0x7fbf6414bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf69ca7700 1 -- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf6415da00 msgr2=0x7fbf6415ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf69ca7700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf6415da00 0x7fbf6415ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf69ca7700 1 -- 172.21.15.57:0/3934265290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf640ceb20 con 0x7fbf50005200 2023-11-28T12:03:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf694a6700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf6415da00 0x7fbf6415ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf69ca7700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf50005200 0x7fbf64158110 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000f820 tx=0x7fbf6000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf567fc700 1 -- 172.21.15.57:0/3934265290 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf600052e0 con 0x7fbf50005200 2023-11-28T12:03:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf567fc700 1 -- 172.21.15.57:0/3934265290 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf60005440 con 0x7fbf50005200 2023-11-28T12:03:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/3934265290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf64161010 con 0x7fbf50005200 2023-11-28T12:03:09.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf567fc700 1 -- 172.21.15.57:0/3934265290 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf600038a0 con 0x7fbf50005200 2023-11-28T12:03:09.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.089+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/3934265290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf640ce740 con 0x7fbf50005200 2023-11-28T12:03:09.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.090+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/3934265290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf640743f0 con 0x7fbf50005200 2023-11-28T12:03:09.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.091+0000 7fbf567fc700 1 -- 172.21.15.57:0/3934265290 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbf60003a00 con 0x7fbf50005200 2023-11-28T12:03:09.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.091+0000 7fbf567fc700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbf480682f0 0x7fbf4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.091+0000 7fbf567fc700 1 -- 172.21.15.57:0/3934265290 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf60085bf0 con 0x7fbf50005200 2023-11-28T12:03:09.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.091+0000 7fbf694a6700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbf480682f0 0x7fbf4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.092+0000 7fbf694a6700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbf480682f0 0x7fbf4806a7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbf580013b0 tx=0x7fbf58009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:09.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.095+0000 7fbf567fc700 1 -- 172.21.15.57:0/3934265290 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbf600523c0 con 0x7fbf50005200 2023-11-28T12:03:09.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.290+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/3934265290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf640ad510 con 0x7fbf50005200 2023-11-28T12:03:09.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.291+0000 7fbf567fc700 1 -- 172.21.15.57:0/3934265290 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf600559e0 con 0x7fbf50005200 2023-11-28T12:03:09.292 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:03:09.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.293+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbf480682f0 msgr2=0x7fbf4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.293+0000 7fbf6bf0b700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbf480682f0 0x7fbf4806a7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbf580013b0 tx=0x7fbf58009040 comp rx=0 tx=0).stop 2023-11-28T12:03:09.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.293+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/3934265290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf50005200 msgr2=0x7fbf64158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.293+0000 7fbf6bf0b700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf50005200 0x7fbf64158110 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000f820 tx=0x7fbf6000fbe0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.293+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/3934265290 shutdown_connections 2023-11-28T12:03:09.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.293+0000 7fbf6bf0b700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbf480682f0 0x7fbf4806a7a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.293+0000 7fbf6bf0b700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf6414bb00 0x7fbf6414bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.293+0000 7fbf6bf0b700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf50005200 0x7fbf64158110 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.293+0000 7fbf6bf0b700 1 --2- 172.21.15.57:0/3934265290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf6415da00 0x7fbf6415ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.293+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/3934265290 >> 172.21.15.57:0/3934265290 conn(0x7fbf6406c010 msgr2=0x7fbf640b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:09.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.293+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/3934265290 shutdown_connections 2023-11-28T12:03:09.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.293+0000 7fbf6bf0b700 1 -- 172.21.15.57:0/3934265290 wait complete. 2023-11-28T12:03:09.308 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:03:09.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.447+0000 7fdee747f700 1 Processor -- start 2023-11-28T12:03:09.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.447+0000 7fdee747f700 1 -- start start 2023-11-28T12:03:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.448+0000 7fdee747f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdee01599b0 0x7fdee015d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.448+0000 7fdee747f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdee00af0a0 0x7fdee015a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.448+0000 7fdee747f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee0159ec0 con 0x7fdee00b1650 2023-11-28T12:03:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.448+0000 7fdee747f700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee015d8a0 con 0x7fdee01599b0 2023-11-28T12:03:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.448+0000 7fdee747f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee015db80 con 0x7fdee00af0a0 2023-11-28T12:03:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.448+0000 7fdee4a1a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdee01599b0 0x7fdee015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.448+0000 7fdee521b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fdee00b1650 0x7fdee00b1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47008/0 (socket says 172.21.15.57:47008) 2023-11-28T12:03:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.448+0000 7fdee521b700 1 -- 172.21.15.57:0/2583104208 learned_addr learned my addr 172.21.15.57:0/2583104208 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.448+0000 7fdee5a1c700 1 --2- 172.21.15.57:0/2583104208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdee00af0a0 0x7fdee015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fdee4a1a700 1 -- 172.21.15.57:0/2583104208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdee00af0a0 msgr2=0x7fdee015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:09.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fdee4a1a700 1 --2- 172.21.15.57:0/2583104208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdee00af0a0 0x7fdee015a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fdee4a1a700 1 -- 172.21.15.57:0/2583104208 >> v1:172.21.15.57:6789/0 conn(0x7fdee00b1650 legacy=0x7fdee00b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fdee4a1a700 1 -- 172.21.15.57:0/2583104208 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee015de60 con 0x7fdee01599b0 2023-11-28T12:03:09.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fded3fff700 1 -- 172.21.15.57:0/2583104208 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519885863 0 0) 0x7fdee0159ec0 con 0x7fdee00b1650 2023-11-28T12:03:09.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fdee5a1c700 1 --2- 172.21.15.57:0/2583104208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdee00af0a0 0x7fdee015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:09.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fdee4a1a700 1 --2- 172.21.15.57:0/2583104208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee01599b0 0x7fdee015d050 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fded4009840 tx=0x7fded4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7600c628844746 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:09.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fded3fff700 1 -- 172.21.15.57:0/2583104208 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded4011830 con 0x7fdee01599b0 2023-11-28T12:03:09.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fded3fff700 1 -- 172.21.15.57:0/2583104208 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded4011990 con 0x7fdee01599b0 2023-11-28T12:03:09.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fded3fff700 1 -- 172.21.15.57:0/2583104208 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded4011d20 con 0x7fdee01599b0 2023-11-28T12:03:09.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fdee747f700 1 -- 172.21.15.57:0/2583104208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee01599b0 msgr2=0x7fdee015d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fdee747f700 1 --2- 172.21.15.57:0/2583104208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee01599b0 0x7fdee015d050 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fded4009840 tx=0x7fded4009b50 comp rx=0 tx=0).stop 2023-11-28T12:03:09.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fdee747f700 1 -- 172.21.15.57:0/2583104208 shutdown_connections 2023-11-28T12:03:09.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fdee747f700 1 --2- 172.21.15.57:0/2583104208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdee00af0a0 0x7fdee015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fdee747f700 1 --2- 172.21.15.57:0/2583104208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee01599b0 0x7fdee015d050 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.449+0000 7fdee747f700 1 -- 172.21.15.57:0/2583104208 >> 172.21.15.57:0/2583104208 conn(0x7fdee006c010 msgr2=0x7fdee00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:09.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.450+0000 7fdee747f700 1 -- 172.21.15.57:0/2583104208 shutdown_connections 2023-11-28T12:03:09.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.450+0000 7fdee747f700 1 -- 172.21.15.57:0/2583104208 wait complete. 2023-11-28T12:03:09.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.451+0000 7fdee747f700 1 Processor -- start 2023-11-28T12:03:09.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.451+0000 7fdee747f700 1 -- start start 2023-11-28T12:03:09.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee747f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee00af0a0 0x7fdee00b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee747f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdee00b1650 0x7fdee01f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee747f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee0148910 0x7fdee00cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee747f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee0154ca0 con 0x7fdee00af0a0 2023-11-28T12:03:09.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee747f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee01f65c0 con 0x7fdee0148910 2023-11-28T12:03:09.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee747f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00b3230 con 0x7fdee00b1650 2023-11-28T12:03:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee521b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee00af0a0 0x7fdee00b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee521b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee00af0a0 0x7fdee00b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40634/0 (socket says 172.21.15.57:40634) 2023-11-28T12:03:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee521b700 1 -- 172.21.15.57:0/2179924174 learned_addr learned my addr 172.21.15.57:0/2179924174 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee4a1a700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdee00b1650 0x7fdee01f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee5a1c700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee0148910 0x7fdee00cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee521b700 1 -- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdee00b1650 msgr2=0x7fdee01f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee521b700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdee00b1650 0x7fdee01f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee521b700 1 -- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee0148910 msgr2=0x7fdee00cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee521b700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee0148910 0x7fdee00cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee521b700 1 -- 172.21.15.57:0/2179924174 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded40094a0 con 0x7fdee00af0a0 2023-11-28T12:03:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee521b700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee00af0a0 0x7fdee00b2a00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00c580 tx=0x7fdedc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fded1ffb700 1 -- 172.21.15.57:0/2179924174 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc004c80 con 0x7fdee00af0a0 2023-11-28T12:03:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fded1ffb700 1 -- 172.21.15.57:0/2179924174 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdedc00ce90 con 0x7fdee00af0a0 2023-11-28T12:03:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fdee747f700 1 -- 172.21.15.57:0/2179924174 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee015a890 con 0x7fdee00af0a0 2023-11-28T12:03:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.452+0000 7fded1ffb700 1 -- 172.21.15.57:0/2179924174 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc0070d0 con 0x7fdee00af0a0 2023-11-28T12:03:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.453+0000 7fdee747f700 1 -- 172.21.15.57:0/2179924174 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee015d590 con 0x7fdee00af0a0 2023-11-28T12:03:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.454+0000 7fdee747f700 1 -- 172.21.15.57:0/2179924174 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee015aa20 con 0x7fdee00af0a0 2023-11-28T12:03:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.454+0000 7fded1ffb700 1 -- 172.21.15.57:0/2179924174 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdedc01a020 con 0x7fdee00af0a0 2023-11-28T12:03:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.454+0000 7fded1ffb700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdec40682f0 0x7fdec406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.455+0000 7fded1ffb700 1 -- 172.21.15.57:0/2179924174 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdedc085700 con 0x7fdee00af0a0 2023-11-28T12:03:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.455+0000 7fdee4a1a700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdec40682f0 0x7fdec406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.455+0000 7fdee4a1a700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdec40682f0 0x7fdec406a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fded4009840 tx=0x7fded401c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:09.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.459+0000 7fded1ffb700 1 -- 172.21.15.57:0/2179924174 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdedc051f60 con 0x7fdee00af0a0 2023-11-28T12:03:09.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.657+0000 7fdee747f700 1 -- 172.21.15.57:0/2179924174 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdee0066a20 con 0x7fdee00af0a0 2023-11-28T12:03:09.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.657+0000 7fded1ffb700 1 -- 172.21.15.57:0/2179924174 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdedc055580 con 0x7fdee00af0a0 2023-11-28T12:03:09.659 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:03:09.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.659+0000 7fdee747f700 1 -- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdec40682f0 msgr2=0x7fdec406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.659+0000 7fdee747f700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdec40682f0 0x7fdec406a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fded4009840 tx=0x7fded401c040 comp rx=0 tx=0).stop 2023-11-28T12:03:09.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.659+0000 7fdee747f700 1 -- 172.21.15.57:0/2179924174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee00af0a0 msgr2=0x7fdee00b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.659+0000 7fdee747f700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee00af0a0 0x7fdee00b2a00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00c580 tx=0x7fdedc00c890 comp rx=0 tx=0).stop 2023-11-28T12:03:09.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.659+0000 7fdee747f700 1 -- 172.21.15.57:0/2179924174 shutdown_connections 2023-11-28T12:03:09.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.659+0000 7fdee747f700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdec40682f0 0x7fdec406a7a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.659+0000 7fdee747f700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee0148910 0x7fdee00cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.659+0000 7fdee747f700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee00af0a0 0x7fdee00b2a00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.659+0000 7fdee747f700 1 --2- 172.21.15.57:0/2179924174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdee00b1650 0x7fdee01f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.659+0000 7fdee747f700 1 -- 172.21.15.57:0/2179924174 >> 172.21.15.57:0/2179924174 conn(0x7fdee006c010 msgr2=0x7fdee0159060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:09.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.659+0000 7fdee747f700 1 -- 172.21.15.57:0/2179924174 shutdown_connections 2023-11-28T12:03:09.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.659+0000 7fdee747f700 1 -- 172.21.15.57:0/2179924174 wait complete. 2023-11-28T12:03:09.674 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:03:09.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.811+0000 7f4b65618700 1 Processor -- start 2023-11-28T12:03:09.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.811+0000 7f4b65618700 1 -- start start 2023-11-28T12:03:09.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.812+0000 7f4b65618700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4b6015fe50 0x7f4b6006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.812+0000 7f4b65618700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b600b27d0 0x7f4b600b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.812+0000 7f4b65618700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b60161370 con 0x7f4b600b3b80 2023-11-28T12:03:09.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.812+0000 7f4b65618700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b600b3190 con 0x7f4b6015fe50 2023-11-28T12:03:09.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.812+0000 7f4b65618700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b60160220 con 0x7f4b600b27d0 2023-11-28T12:03:09.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.812+0000 7f4b5effd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4b6015fe50 0x7f4b6006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.812+0000 7f4b5effd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4b6015fe50 0x7f4b6006d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40642/0 (socket says 172.21.15.57:40642) 2023-11-28T12:03:09.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.812+0000 7f4b5effd700 1 -- 172.21.15.57:0/2854811705 learned_addr learned my addr 172.21.15.57:0/2854811705 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:09.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.813+0000 7f4b5f7fe700 1 --2- 172.21.15.57:0/2854811705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b600b27d0 0x7f4b600b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.813+0000 7f4b5effd700 1 -- 172.21.15.57:0/2854811705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b600b27d0 msgr2=0x7f4b600b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:09.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.813+0000 7f4b5effd700 1 --2- 172.21.15.57:0/2854811705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b600b27d0 0x7f4b600b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.813+0000 7f4b5effd700 1 -- 172.21.15.57:0/2854811705 >> v1:172.21.15.181:6789/0 conn(0x7f4b600b3b80 legacy=0x7f4b60160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.813+0000 7f4b5effd700 1 -- 172.21.15.57:0/2854811705 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b600cf700 con 0x7f4b6015fe50 2023-11-28T12:03:09.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.813+0000 7f4b5effd700 1 --2- 172.21.15.57:0/2854811705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b6015fe50 0x7f4b6006d170 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4b500098f0 tx=0x7f4b50009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80325548d0675fd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:09.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.813+0000 7f4b5dffb700 1 -- 172.21.15.57:0/2854811705 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50011830 con 0x7f4b6015fe50 2023-11-28T12:03:09.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.813+0000 7f4b5dffb700 1 -- 172.21.15.57:0/2854811705 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b50011990 con 0x7f4b6015fe50 2023-11-28T12:03:09.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.813+0000 7f4b5dffb700 1 -- 172.21.15.57:0/2854811705 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50011d20 con 0x7f4b6015fe50 2023-11-28T12:03:09.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.814+0000 7f4b65618700 1 -- 172.21.15.57:0/2854811705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b6015fe50 msgr2=0x7f4b6006d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.814+0000 7f4b65618700 1 --2- 172.21.15.57:0/2854811705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b6015fe50 0x7f4b6006d170 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4b500098f0 tx=0x7f4b50009c00 comp rx=0 tx=0).stop 2023-11-28T12:03:09.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.814+0000 7f4b65618700 1 -- 172.21.15.57:0/2854811705 shutdown_connections 2023-11-28T12:03:09.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.814+0000 7f4b65618700 1 --2- 172.21.15.57:0/2854811705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b6015fe50 0x7f4b6006d170 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.814+0000 7f4b65618700 1 --2- 172.21.15.57:0/2854811705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b600b27d0 0x7f4b600b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.814+0000 7f4b65618700 1 -- 172.21.15.57:0/2854811705 >> 172.21.15.57:0/2854811705 conn(0x7f4b6006ea80 msgr2=0x7f4b6006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:09.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.814+0000 7f4b65618700 1 -- 172.21.15.57:0/2854811705 shutdown_connections 2023-11-28T12:03:09.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.814+0000 7f4b65618700 1 -- 172.21.15.57:0/2854811705 wait complete. 2023-11-28T12:03:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.815+0000 7f4b65618700 1 Processor -- start 2023-11-28T12:03:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.815+0000 7f4b65618700 1 -- start start 2023-11-28T12:03:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.816+0000 7f4b65618700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b600b27d0 0x7f4b601e6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.816+0000 7f4b65618700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b600b3b80 0x7f4b6015e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.816+0000 7f4b65618700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b6015fe50 0x7f4b601606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.816+0000 7f4b65618700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601e6aa0 con 0x7f4b600b27d0 2023-11-28T12:03:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.816+0000 7f4b65618700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601e6d80 con 0x7f4b600b3b80 2023-11-28T12:03:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.816+0000 7f4b65618700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601e7060 con 0x7f4b6015fe50 2023-11-28T12:03:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.816+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b600b27d0 0x7f4b601e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.816+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b600b27d0 0x7f4b601e6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40648/0 (socket says 172.21.15.57:40648) 2023-11-28T12:03:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.816+0000 7f4b5effd700 1 -- 172.21.15.57:0/3402049272 learned_addr learned my addr 172.21.15.57:0/3402049272 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.816+0000 7f4b5f7fe700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b6015fe50 0x7f4b601606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.816+0000 7f4b5e7fc700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b600b3b80 0x7f4b6015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.817+0000 7f4b5effd700 1 -- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b6015fe50 msgr2=0x7f4b601606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.817+0000 7f4b5effd700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b6015fe50 0x7f4b601606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.817+0000 7f4b5effd700 1 -- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b600b3b80 msgr2=0x7f4b6015e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:09.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.817+0000 7f4b5effd700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b600b3b80 0x7f4b6015e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:09.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.817+0000 7f4b5effd700 1 -- 172.21.15.57:0/3402049272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50009550 con 0x7f4b600b27d0 2023-11-28T12:03:09.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.817+0000 7f4b5f7fe700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b6015fe50 0x7f4b601606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:09.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.817+0000 7f4b5e7fc700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b600b3b80 0x7f4b6015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:09.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.817+0000 7f4b5effd700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b600b27d0 0x7f4b601e6210 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009c40 tx=0x7f4b50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:09.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.817+0000 7f4b477fe700 1 -- 172.21.15.57:0/3402049272 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b500036a0 con 0x7f4b600b27d0 2023-11-28T12:03:09.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.817+0000 7f4b477fe700 1 -- 172.21.15.57:0/3402049272 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5001e070 con 0x7f4b600b27d0 2023-11-28T12:03:09.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.817+0000 7f4b65618700 1 -- 172.21.15.57:0/3402049272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b600cedd0 con 0x7f4b600b27d0 2023-11-28T12:03:09.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.817+0000 7f4b477fe700 1 -- 172.21.15.57:0/3402049272 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50003a60 con 0x7f4b600b27d0 2023-11-28T12:03:09.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.817+0000 7f4b65618700 1 -- 172.21.15.57:0/3402049272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b600b3fd0 con 0x7f4b600b27d0 2023-11-28T12:03:09.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.818+0000 7f4b65618700 1 -- 172.21.15.57:0/3402049272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b600cf0b0 con 0x7f4b600b27d0 2023-11-28T12:03:09.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.818+0000 7f4b477fe700 1 -- 172.21.15.57:0/3402049272 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4b5000f050 con 0x7f4b600b27d0 2023-11-28T12:03:09.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.819+0000 7f4b477fe700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4b4c0682f0 0x7f4b4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:09.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.819+0000 7f4b477fe700 1 -- 172.21.15.57:0/3402049272 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4b50086c20 con 0x7f4b600b27d0 2023-11-28T12:03:09.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.819+0000 7f4b5e7fc700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4b4c0682f0 0x7f4b4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:09.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.820+0000 7f4b5e7fc700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4b4c0682f0 0x7f4b4c06a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b600af390 tx=0x7f4b54009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:09.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:09.823+0000 7f4b477fe700 1 -- 172.21.15.57:0/3402049272 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4b50053480 con 0x7f4b600b27d0 2023-11-28T12:03:10.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.022+0000 7f4b65618700 1 -- 172.21.15.57:0/3402049272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b600743f0 con 0x7f4b600b27d0 2023-11-28T12:03:10.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.022+0000 7f4b477fe700 1 -- 172.21.15.57:0/3402049272 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b500150a0 con 0x7f4b600b27d0 2023-11-28T12:03:10.024 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:03:10.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.024+0000 7f4b65618700 1 -- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4b4c0682f0 msgr2=0x7f4b4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.024+0000 7f4b65618700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4b4c0682f0 0x7f4b4c06a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b600af390 tx=0x7f4b54009040 comp rx=0 tx=0).stop 2023-11-28T12:03:10.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.024+0000 7f4b65618700 1 -- 172.21.15.57:0/3402049272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b600b27d0 msgr2=0x7f4b601e6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.024+0000 7f4b65618700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b600b27d0 0x7f4b601e6210 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009c40 tx=0x7f4b50004750 comp rx=0 tx=0).stop 2023-11-28T12:03:10.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.025+0000 7f4b65618700 1 -- 172.21.15.57:0/3402049272 shutdown_connections 2023-11-28T12:03:10.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.025+0000 7f4b65618700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4b4c0682f0 0x7f4b4c06a7a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.025+0000 7f4b65618700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b6015fe50 0x7f4b601606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.025+0000 7f4b65618700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b600b27d0 0x7f4b601e6210 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.025+0000 7f4b65618700 1 --2- 172.21.15.57:0/3402049272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b600b3b80 0x7f4b6015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.025+0000 7f4b65618700 1 -- 172.21.15.57:0/3402049272 >> 172.21.15.57:0/3402049272 conn(0x7f4b6006ea80 msgr2=0x7f4b6015f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:10.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.025+0000 7f4b65618700 1 -- 172.21.15.57:0/3402049272 shutdown_connections 2023-11-28T12:03:10.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.025+0000 7f4b65618700 1 -- 172.21.15.57:0/3402049272 wait complete. 2023-11-28T12:03:10.039 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T12:03:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.176+0000 7fc40a8af700 1 Processor -- start 2023-11-28T12:03:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.176+0000 7fc40a8af700 1 -- start start 2023-11-28T12:03:10.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.177+0000 7fc40a8af700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc40406a270 0x7fc40406a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.177+0000 7fc40a8af700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc40406abd0 con 0x7fc4040718e0 2023-11-28T12:03:10.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.177+0000 7fc40a8af700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc404160a50 con 0x7fc40415c810 2023-11-28T12:03:10.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.177+0000 7fc40a8af700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc40415cbe0 con 0x7fc40406a270 2023-11-28T12:03:10.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.177+0000 7fc4037fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc4040718e0 0x7fc40406c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47024/0 (socket says 172.21.15.57:47024) 2023-11-28T12:03:10.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.177+0000 7fc4037fe700 1 -- 172.21.15.57:0/208587511 learned_addr learned my addr 172.21.15.57:0/208587511 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:10.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.177+0000 7fc408e4c700 1 --2- 172.21.15.57:0/208587511 >> v2:172.21.15.181:3300/0 conn(0x7fc40406a270 0x7fc40406a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.177+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393200411 0 0) 0x7fc40406abd0 con 0x7fc4040718e0 2023-11-28T12:03:10.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.177+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3f00035b0 con 0x7fc4040718e0 2023-11-28T12:03:10.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2693570361 0 0) 0x7fc3f00035b0 con 0x7fc4040718e0 2023-11-28T12:03:10.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 >> v2:172.21.15.181:3300/0 conn(0x7fc40406a270 msgr2=0x7fc40406a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:10.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc402ffd700 1 --2- 172.21.15.57:0/208587511 >> v2:172.21.15.181:3300/0 conn(0x7fc40406a270 0x7fc40406a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 >> v1:172.21.15.181:6789/0 conn(0x7fc40415c810 legacy=0x7fc404160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc404160d30 con 0x7fc4040718e0 2023-11-28T12:03:10.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fc3ec001670 con 0x7fc4040718e0 2023-11-28T12:03:10.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 >> v1:172.21.15.57:6789/0 conn(0x7fc4040718e0 legacy=0x7fc40406c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc402ffd700 1 --2- 172.21.15.57:0/208587511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0003a00 0x7fc3f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc402ffd700 1 --2- 172.21.15.57:0/208587511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3f00043f0 0x7fc3f00048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc402ffd700 1 --2- 172.21.15.57:0/208587511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3f00050b0 0x7fc3f000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40415cbe0 con 0x7fc3f0003a00 2023-11-28T12:03:10.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f000c000 con 0x7fc3f00043f0 2023-11-28T12:03:10.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f000c1c0 con 0x7fc3f00050b0 2023-11-28T12:03:10.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc408e4c700 1 --2- 172.21.15.57:0/208587511 >> v2:172.21.15.181:3300/0 conn(0x7fc40406a270 0x7fc40406a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:03:10.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc403fff700 1 --2- 172.21.15.57:0/208587511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0003a00 0x7fc3f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc4037fe700 1 --2- 172.21.15.57:0/208587511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3f00043f0 0x7fc3f00048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc408e4c700 1 --2- 172.21.15.57:0/208587511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3f00050b0 0x7fc3f000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc403fff700 1 -- 172.21.15.57:0/208587511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3f00050b0 msgr2=0x7fc3f000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:10.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc403fff700 1 --2- 172.21.15.57:0/208587511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3f00050b0 0x7fc3f000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc403fff700 1 -- 172.21.15.57:0/208587511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3f00043f0 msgr2=0x7fc3f00048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:10.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc403fff700 1 --2- 172.21.15.57:0/208587511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3f00043f0 0x7fc3f00048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc403fff700 1 -- 172.21.15.57:0/208587511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f00035b0 con 0x7fc3f0003a00 2023-11-28T12:03:10.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc408e4c700 1 -- 172.21.15.57:0/208587511 reap_dead start 2023-11-28T12:03:10.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.178+0000 7fc403fff700 1 --2- 172.21.15.57:0/208587511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0003a00 0x7fc3f0003dd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40018d0 tx=0x7fc3f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560c4b832a337d60 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:10.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.179+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f400c690 con 0x7fc3f0003a00 2023-11-28T12:03:10.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.179+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f400c810 con 0x7fc3f0003a00 2023-11-28T12:03:10.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.179+0000 7fc402ffd700 1 -- 172.21.15.57:0/208587511 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f400cbc0 con 0x7fc3f0003a00 2023-11-28T12:03:10.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.179+0000 7fc40a8af700 1 -- 172.21.15.57:0/208587511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0003a00 msgr2=0x7fc3f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.179+0000 7fc40a8af700 1 --2- 172.21.15.57:0/208587511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0003a00 0x7fc3f0003dd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40018d0 tx=0x7fc3f0005880 comp rx=0 tx=0).stop 2023-11-28T12:03:10.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.179+0000 7fc40a8af700 1 -- 172.21.15.57:0/208587511 shutdown_connections 2023-11-28T12:03:10.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.179+0000 7fc40a8af700 1 --2- 172.21.15.57:0/208587511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0003a00 0x7fc3f0003dd0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.179+0000 7fc40a8af700 1 -- 172.21.15.57:0/208587511 >> 172.21.15.57:0/208587511 conn(0x7fc40406c010 msgr2=0x7fc4040cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:10.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.179+0000 7fc40a8af700 1 -- 172.21.15.57:0/208587511 shutdown_connections 2023-11-28T12:03:10.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.179+0000 7fc40a8af700 1 -- 172.21.15.57:0/208587511 wait complete. 2023-11-28T12:03:10.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.180+0000 7fc40a8af700 1 Processor -- start 2023-11-28T12:03:10.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.181+0000 7fc40a8af700 1 -- start start 2023-11-28T12:03:10.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.181+0000 7fc40a8af700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3f0003a00 0x7fc40415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc40a8af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc40415f450 0x7fc40414fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc40a8af700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc40415f8c0 0x7fc404150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc40a8af700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40415d140 con 0x7fc40415f8c0 2023-11-28T12:03:10.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc40a8af700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40415d420 con 0x7fc40415f450 2023-11-28T12:03:10.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc40a8af700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40415d700 con 0x7fc3f0003a00 2023-11-28T12:03:10.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc408e4c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc40415f8c0 0x7fc404150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc408e4c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc40415f8c0 0x7fc404150300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40676/0 (socket says 172.21.15.57:40676) 2023-11-28T12:03:10.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc408e4c700 1 -- 172.21.15.57:0/1405672235 learned_addr learned my addr 172.21.15.57:0/1405672235 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:10.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc403fff700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3f0003a00 0x7fc40415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc4037fe700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc40415f450 0x7fc40414fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc408e4c700 1 -- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3f0003a00 msgr2=0x7fc40415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc408e4c700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3f0003a00 0x7fc40415e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc408e4c700 1 -- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc40415f450 msgr2=0x7fc40414fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc408e4c700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc40415f450 0x7fc40414fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc408e4c700 1 -- 172.21.15.57:0/1405672235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4040ceb20 con 0x7fc40415f8c0 2023-11-28T12:03:10.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc408e4c700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc40415f8c0 0x7fc404150300 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc4040aec60 tx=0x7fc3fc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:10.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc400ff9700 1 -- 172.21.15.57:0/1405672235 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc00e070 con 0x7fc40415f8c0 2023-11-28T12:03:10.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc400ff9700 1 -- 172.21.15.57:0/1405672235 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc003c30 con 0x7fc40415f8c0 2023-11-28T12:03:10.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc40a8af700 1 -- 172.21.15.57:0/1405672235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc404161010 con 0x7fc40415f8c0 2023-11-28T12:03:10.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc400ff9700 1 -- 172.21.15.57:0/1405672235 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc003dc0 con 0x7fc40415f8c0 2023-11-28T12:03:10.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.182+0000 7fc40a8af700 1 -- 172.21.15.57:0/1405672235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4040ce740 con 0x7fc40415f8c0 2023-11-28T12:03:10.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.183+0000 7fc40a8af700 1 -- 172.21.15.57:0/1405672235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4040743f0 con 0x7fc40415f8c0 2023-11-28T12:03:10.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.184+0000 7fc400ff9700 1 -- 172.21.15.57:0/1405672235 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc3fc0119b0 con 0x7fc40415f8c0 2023-11-28T12:03:10.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.184+0000 7fc400ff9700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc3e40682f0 0x7fc3e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.184+0000 7fc400ff9700 1 -- 172.21.15.57:0/1405672235 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3fc0862a0 con 0x7fc40415f8c0 2023-11-28T12:03:10.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.185+0000 7fc403fff700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc3e40682f0 0x7fc3e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.185+0000 7fc403fff700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc3e40682f0 0x7fc3e406a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4000c00 tx=0x7fc3f4005040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:10.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.189+0000 7fc400ff9700 1 -- 172.21.15.57:0/1405672235 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc3fc052a70 con 0x7fc40415f8c0 2023-11-28T12:03:10.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.409+0000 7fc40a8af700 1 -- 172.21.15.57:0/1405672235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc40415be90 con 0x7fc40415f8c0 2023-11-28T12:03:10.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.409+0000 7fc400ff9700 1 -- 172.21.15.57:0/1405672235 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc3fc056090 con 0x7fc40415f8c0 2023-11-28T12:03:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.411+0000 7fc40a8af700 1 -- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc3e40682f0 msgr2=0x7fc3e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.412+0000 7fc40a8af700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc3e40682f0 0x7fc3e406a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4000c00 tx=0x7fc3f4005040 comp rx=0 tx=0).stop 2023-11-28T12:03:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.412+0000 7fc40a8af700 1 -- 172.21.15.57:0/1405672235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc40415f8c0 msgr2=0x7fc404150300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.412+0000 7fc40a8af700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc40415f8c0 0x7fc404150300 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc4040aec60 tx=0x7fc3fc009f90 comp rx=0 tx=0).stop 2023-11-28T12:03:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.412+0000 7fc40a8af700 1 -- 172.21.15.57:0/1405672235 shutdown_connections 2023-11-28T12:03:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.412+0000 7fc40a8af700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc3e40682f0 0x7fc3e406a7a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.412+0000 7fc40a8af700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc40415f8c0 0x7fc404150300 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.412+0000 7fc40a8af700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3f0003a00 0x7fc40415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.412+0000 7fc40a8af700 1 --2- 172.21.15.57:0/1405672235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc40415f450 0x7fc40414fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.412+0000 7fc40a8af700 1 -- 172.21.15.57:0/1405672235 >> 172.21.15.57:0/1405672235 conn(0x7fc40406c010 msgr2=0x7fc4040cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:10.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.412+0000 7fc40a8af700 1 -- 172.21.15.57:0/1405672235 shutdown_connections 2023-11-28T12:03:10.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.412+0000 7fc40a8af700 1 -- 172.21.15.57:0/1405672235 wait complete. 2023-11-28T12:03:10.414 INFO:teuthology.orchestra.run.smithi057.stderr:listed 0 entries 2023-11-28T12:03:10.426 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:03:10.426 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:03:10.447 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:03:10.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.629+0000 7fd6fa6d8700 1 Processor -- start 2023-11-28T12:03:10.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.629+0000 7fd6fa6d8700 1 -- start start 2023-11-28T12:03:10.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.629+0000 7fd6fa6d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f406a270 0x7fd6f406a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.629+0000 7fd6fa6d8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6f4070d90 0x7fd6f40711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6fa6d8700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f4151a50 con 0x7fd6f406da40 2023-11-28T12:03:10.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6fa6d8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f415f0c0 con 0x7fd6f4070d90 2023-11-28T12:03:10.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6fa6d8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f415b440 con 0x7fd6f406a270 2023-11-28T12:03:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f8c75700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd6f406da40 0x7fd6f406dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47028/0 (socket says 172.21.15.57:47028) 2023-11-28T12:03:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f8c75700 1 -- 172.21.15.57:0/2056141392 learned_addr learned my addr 172.21.15.57:0/2056141392 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f3fff700 1 --2- 172.21.15.57:0/2056141392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f406a270 0x7fd6f406a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f37fe700 1 --2- 172.21.15.57:0/2056141392 >> v2:172.21.15.181:3300/0 conn(0x7fd6f4070d90 0x7fd6f40711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996589211 0 0) 0x7fd6f4151a50 con 0x7fd6f406da40 2023-11-28T12:03:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6dc0035b0 con 0x7fd6f406da40 2023-11-28T12:03:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3091099081 0 0) 0x7fd6dc0035b0 con 0x7fd6f406da40 2023-11-28T12:03:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f406a270 msgr2=0x7fd6f406a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 --2- 172.21.15.57:0/2056141392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f406a270 0x7fd6f406a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 >> v2:172.21.15.181:3300/0 conn(0x7fd6f4070d90 msgr2=0x7fd6f40711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:10.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 --2- 172.21.15.57:0/2056141392 >> v2:172.21.15.181:3300/0 conn(0x7fd6f4070d90 0x7fd6f40711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f406ab80 con 0x7fd6f406da40 2023-11-28T12:03:10.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fd6ec0019f0 con 0x7fd6f406da40 2023-11-28T12:03:10.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 >> v1:172.21.15.57:6789/0 conn(0x7fd6f406da40 legacy=0x7fd6f406dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 --2- 172.21.15.57:0/2056141392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f406a270 0x7fd6dc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 --2- 172.21.15.57:0/2056141392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6dc003f80 0x7fd6dc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 --2- 172.21.15.57:0/2056141392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6dc004d90 0x7fd6dc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f415f0c0 con 0x7fd6dc004d90 2023-11-28T12:03:10.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f415b440 con 0x7fd6dc003f80 2023-11-28T12:03:10.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.630+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc00ba10 con 0x7fd6f406a270 2023-11-28T12:03:10.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.631+0000 7fd6f8c75700 1 --2- 172.21.15.57:0/2056141392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6dc004d90 0x7fd6dc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.631+0000 7fd6f3fff700 1 --2- 172.21.15.57:0/2056141392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f406a270 0x7fd6dc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.631+0000 7fd6f37fe700 1 --2- 172.21.15.57:0/2056141392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6dc003f80 0x7fd6dc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.631+0000 7fd6f8c75700 1 -- 172.21.15.57:0/2056141392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f406a270 msgr2=0x7fd6dc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:10.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.631+0000 7fd6f8c75700 1 --2- 172.21.15.57:0/2056141392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f406a270 0x7fd6dc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.631+0000 7fd6f8c75700 1 -- 172.21.15.57:0/2056141392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6dc003f80 msgr2=0x7fd6dc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:10.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.631+0000 7fd6f8c75700 1 --2- 172.21.15.57:0/2056141392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6dc003f80 0x7fd6dc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.631+0000 7fd6f8c75700 1 -- 172.21.15.57:0/2056141392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc0035b0 con 0x7fd6dc004d90 2023-11-28T12:03:10.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.631+0000 7fd6f8c75700 1 --2- 172.21.15.57:0/2056141392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6dc004d90 0x7fd6dc005200 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00a8d0 tx=0x7fd6dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f306c897389f4db7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:10.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.631+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00a340 con 0x7fd6dc004d90 2023-11-28T12:03:10.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.631+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ec012600 con 0x7fd6dc004d90 2023-11-28T12:03:10.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.631+0000 7fd6f2ffd700 1 -- 172.21.15.57:0/2056141392 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec0129b0 con 0x7fd6dc004d90 2023-11-28T12:03:10.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.632+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/2056141392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6dc004d90 msgr2=0x7fd6dc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.632+0000 7fd6fa6d8700 1 --2- 172.21.15.57:0/2056141392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6dc004d90 0x7fd6dc005200 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00a8d0 tx=0x7fd6dc005880 comp rx=0 tx=0).stop 2023-11-28T12:03:10.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.632+0000 7fd6f8c75700 1 -- 172.21.15.57:0/2056141392 reap_dead start 2023-11-28T12:03:10.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.632+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/2056141392 shutdown_connections 2023-11-28T12:03:10.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.632+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/2056141392 >> 172.21.15.57:0/2056141392 conn(0x7fd6f406c010 msgr2=0x7fd6f40cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:10.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.632+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/2056141392 shutdown_connections 2023-11-28T12:03:10.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.632+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/2056141392 wait complete. 2023-11-28T12:03:10.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.633+0000 7fd6fa6d8700 1 Processor -- start 2023-11-28T12:03:10.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.633+0000 7fd6fa6d8700 1 -- start start 2023-11-28T12:03:10.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.634+0000 7fd6fa6d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f41461f0 0x7fd6f40aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.634+0000 7fd6fa6d8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f40cece0 0x7fd6f40cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.634+0000 7fd6fa6d8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6f40b00e0 0x7fd6f40b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.634+0000 7fd6fa6d8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f40aeb30 con 0x7fd6f40b00e0 2023-11-28T12:03:10.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.634+0000 7fd6fa6d8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f40b0a90 con 0x7fd6f40cece0 2023-11-28T12:03:10.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.634+0000 7fd6fa6d8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f40ce8d0 con 0x7fd6f41461f0 2023-11-28T12:03:10.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.634+0000 7fd6f8c75700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6f40b00e0 0x7fd6f40b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.634+0000 7fd6f8c75700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6f40b00e0 0x7fd6f40b0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40702/0 (socket says 172.21.15.57:40702) 2023-11-28T12:03:10.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.634+0000 7fd6f8c75700 1 -- 172.21.15.57:0/3320762184 learned_addr learned my addr 172.21.15.57:0/3320762184 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:10.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.634+0000 7fd6f37fe700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f40cece0 0x7fd6f40cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.634+0000 7fd6f3fff700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f41461f0 0x7fd6f40aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.635+0000 7fd6f8c75700 1 -- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f41461f0 msgr2=0x7fd6f40aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.635+0000 7fd6f8c75700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f41461f0 0x7fd6f40aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.635+0000 7fd6f8c75700 1 -- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f40cece0 msgr2=0x7fd6f40cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.635+0000 7fd6f8c75700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f40cece0 0x7fd6f40cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.635+0000 7fd6f8c75700 1 -- 172.21.15.57:0/3320762184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f415adf0 con 0x7fd6f40b00e0 2023-11-28T12:03:10.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.635+0000 7fd6f8c75700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6f40b00e0 0x7fd6f40b0550 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec015040 tx=0x7fd6ec001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:10.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.635+0000 7fd6f0ff9700 1 -- 172.21.15.57:0/3320762184 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00a340 con 0x7fd6f40b00e0 2023-11-28T12:03:10.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.635+0000 7fd6f0ff9700 1 -- 172.21.15.57:0/3320762184 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ec0019c0 con 0x7fd6f40b00e0 2023-11-28T12:03:10.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.635+0000 7fd6f0ff9700 1 -- 172.21.15.57:0/3320762184 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec027410 con 0x7fd6f40b00e0 2023-11-28T12:03:10.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.635+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/3320762184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f4071ac0 con 0x7fd6f40b00e0 2023-11-28T12:03:10.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.635+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/3320762184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f415b0d0 con 0x7fd6f40b00e0 2023-11-28T12:03:10.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.636+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/3320762184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f40743f0 con 0x7fd6f40b00e0 2023-11-28T12:03:10.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.636+0000 7fd6f0ff9700 1 -- 172.21.15.57:0/3320762184 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd6ec02e070 con 0x7fd6f40b00e0 2023-11-28T12:03:10.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.637+0000 7fd6f0ff9700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd6e00682f0 0x7fd6e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:10.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.637+0000 7fd6f0ff9700 1 -- 172.21.15.57:0/3320762184 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd6ec087120 con 0x7fd6f40b00e0 2023-11-28T12:03:10.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.637+0000 7fd6f3fff700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd6e00682f0 0x7fd6e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:10.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.638+0000 7fd6f3fff700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd6e00682f0 0x7fd6e006a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009c60 tx=0x7fd6e4009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:10.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.642+0000 7fd6f0ff9700 1 -- 172.21.15.57:0/3320762184 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd6ec0539c0 con 0x7fd6f40b00e0 2023-11-28T12:03:10.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.844+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/3320762184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6f40cd520 con 0x7fd6f40b00e0 2023-11-28T12:03:10.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.845+0000 7fd6f0ff9700 1 -- 172.21.15.57:0/3320762184 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+17921 (secure 0 0 0) 0x7fd6ec022090 con 0x7fd6f40b00e0 2023-11-28T12:03:10.847 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:10.847 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":35,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:02:54.404560+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:02:51.245666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T12:02:52.363863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:03:10.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.847+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd6e00682f0 msgr2=0x7fd6e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.847+0000 7fd6fa6d8700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd6e00682f0 0x7fd6e006a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009c60 tx=0x7fd6e4009210 comp rx=0 tx=0).stop 2023-11-28T12:03:10.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.847+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/3320762184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6f40b00e0 msgr2=0x7fd6f40b0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:10.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.847+0000 7fd6fa6d8700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6f40b00e0 0x7fd6f40b0550 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec015040 tx=0x7fd6ec001360 comp rx=0 tx=0).stop 2023-11-28T12:03:10.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.848+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/3320762184 shutdown_connections 2023-11-28T12:03:10.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.848+0000 7fd6fa6d8700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd6e00682f0 0x7fd6e006a7a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.848+0000 7fd6fa6d8700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6f40b00e0 0x7fd6f40b0550 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.848+0000 7fd6fa6d8700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f41461f0 0x7fd6f40aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.848+0000 7fd6fa6d8700 1 --2- 172.21.15.57:0/3320762184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f40cece0 0x7fd6f40cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:10.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.848+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/3320762184 >> 172.21.15.57:0/3320762184 conn(0x7fd6f406c010 msgr2=0x7fd6f406de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:10.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.848+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/3320762184 shutdown_connections 2023-11-28T12:03:10.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:10.848+0000 7fd6fa6d8700 1 -- 172.21.15.57:0/3320762184 wait complete. 2023-11-28T12:03:10.862 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:02:51.245666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:02:52.363863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:03:10.863 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:03:11.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.000+0000 7fbf1da53700 1 Processor -- start 2023-11-28T12:03:11.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.000+0000 7fbf1da53700 1 -- start start 2023-11-28T12:03:11.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.001+0000 7fbf1da53700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf18160cb0 0x7fbf1806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.001+0000 7fbf1da53700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbf1806b050 0x7fbf180d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.001+0000 7fbf1da53700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf180d2d30 con 0x7fbf1815c8c0 2023-11-28T12:03:11.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.001+0000 7fbf1da53700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1806b4d0 con 0x7fbf1806b050 2023-11-28T12:03:11.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.001+0000 7fbf1da53700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180d0b40 con 0x7fbf18160cb0 2023-11-28T12:03:11.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.001+0000 7fbf16ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fbf1815c8c0 0x7fbf180b5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47032/0 (socket says 172.21.15.57:47032) 2023-11-28T12:03:11.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.001+0000 7fbf16ffd700 1 -- 172.21.15.57:0/188227456 learned_addr learned my addr 172.21.15.57:0/188227456 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:11.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.001+0000 7fbf177fe700 1 --2- 172.21.15.57:0/188227456 >> v2:172.21.15.57:3300/0 conn(0x7fbf1806b050 0x7fbf180d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.001+0000 7fbf167fc700 1 --2- 172.21.15.57:0/188227456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf18160cb0 0x7fbf1806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.001+0000 7fbf15ffb700 1 -- 172.21.15.57:0/188227456 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842217949 0 0) 0x7fbf08001110 con 0x7fbf1815c8c0 2023-11-28T12:03:11.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.001+0000 7fbf15ffb700 1 -- 172.21.15.57:0/188227456 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf040035b0 con 0x7fbf1815c8c0 2023-11-28T12:03:11.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.001+0000 7fbf177fe700 1 -- 172.21.15.57:0/188227456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf18160cb0 msgr2=0x7fbf1806ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:11.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.001+0000 7fbf177fe700 1 --2- 172.21.15.57:0/188227456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf18160cb0 0x7fbf1806ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.002+0000 7fbf177fe700 1 -- 172.21.15.57:0/188227456 >> v1:172.21.15.57:6789/0 conn(0x7fbf1815c8c0 legacy=0x7fbf180b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.002+0000 7fbf177fe700 1 -- 172.21.15.57:0/188227456 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf180d2a50 con 0x7fbf1806b050 2023-11-28T12:03:11.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.002+0000 7fbf177fe700 1 --2- 172.21.15.57:0/188227456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf1806b050 0x7fbf180d0fd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009d90 tx=0x7fbf1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4652f186c592f31c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:11.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.002+0000 7fbf15ffb700 1 -- 172.21.15.57:0/188227456 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1000e070 con 0x7fbf1806b050 2023-11-28T12:03:11.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.002+0000 7fbf15ffb700 1 -- 172.21.15.57:0/188227456 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1000bb20 con 0x7fbf1806b050 2023-11-28T12:03:11.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.002+0000 7fbf15ffb700 1 -- 172.21.15.57:0/188227456 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10012430 con 0x7fbf1806b050 2023-11-28T12:03:11.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.002+0000 7fbf1da53700 1 -- 172.21.15.57:0/188227456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf1806b050 msgr2=0x7fbf180d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.002+0000 7fbf1da53700 1 --2- 172.21.15.57:0/188227456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf1806b050 0x7fbf180d0fd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009d90 tx=0x7fbf1000b330 comp rx=0 tx=0).stop 2023-11-28T12:03:11.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.003+0000 7fbf1da53700 1 -- 172.21.15.57:0/188227456 shutdown_connections 2023-11-28T12:03:11.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.003+0000 7fbf1da53700 1 --2- 172.21.15.57:0/188227456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf1806b050 0x7fbf180d0fd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.003+0000 7fbf1da53700 1 --2- 172.21.15.57:0/188227456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf18160cb0 0x7fbf1806ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.003+0000 7fbf1da53700 1 -- 172.21.15.57:0/188227456 >> 172.21.15.57:0/188227456 conn(0x7fbf1806c520 msgr2=0x7fbf1806c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:11.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.003+0000 7fbf1da53700 1 -- 172.21.15.57:0/188227456 shutdown_connections 2023-11-28T12:03:11.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.003+0000 7fbf1da53700 1 -- 172.21.15.57:0/188227456 wait complete. 2023-11-28T12:03:11.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.004+0000 7fbf1da53700 1 Processor -- start 2023-11-28T12:03:11.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.004+0000 7fbf1da53700 1 -- start start 2023-11-28T12:03:11.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf1da53700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf1806b050 0x7fbf18154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf1da53700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf1815c8c0 0x7fbf180c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf1da53700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf18160cb0 0x7fbf180d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf1da53700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf181f2f20 con 0x7fbf1815c8c0 2023-11-28T12:03:11.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf1da53700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180ca1e0 con 0x7fbf18160cb0 2023-11-28T12:03:11.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf1da53700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180ca4c0 con 0x7fbf1806b050 2023-11-28T12:03:11.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf167fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf1815c8c0 0x7fbf180c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf167fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf1815c8c0 0x7fbf180c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40708/0 (socket says 172.21.15.57:40708) 2023-11-28T12:03:11.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf167fc700 1 -- 172.21.15.57:0/2058055844 learned_addr learned my addr 172.21.15.57:0/2058055844 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:11.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf16ffd700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf1806b050 0x7fbf18154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf177fe700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf18160cb0 0x7fbf180d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf167fc700 1 -- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf1806b050 msgr2=0x7fbf18154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf167fc700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf1806b050 0x7fbf18154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf167fc700 1 -- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf18160cb0 msgr2=0x7fbf180d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf167fc700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf18160cb0 0x7fbf180d2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.005+0000 7fbf167fc700 1 -- 172.21.15.57:0/2058055844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10009a70 con 0x7fbf1815c8c0 2023-11-28T12:03:11.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.006+0000 7fbf167fc700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf1815c8c0 0x7fbf180c84e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000e870 tx=0x7fbf0000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:11.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.006+0000 7fbf0ffff700 1 -- 172.21.15.57:0/2058055844 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00009590 con 0x7fbf1815c8c0 2023-11-28T12:03:11.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.006+0000 7fbf0ffff700 1 -- 172.21.15.57:0/2058055844 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf00013680 con 0x7fbf1815c8c0 2023-11-28T12:03:11.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.006+0000 7fbf1da53700 1 -- 172.21.15.57:0/2058055844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf180cda40 con 0x7fbf1815c8c0 2023-11-28T12:03:11.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.006+0000 7fbf0ffff700 1 -- 172.21.15.57:0/2058055844 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf000139f0 con 0x7fbf1815c8c0 2023-11-28T12:03:11.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.006+0000 7fbf1da53700 1 -- 172.21.15.57:0/2058055844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf180cd760 con 0x7fbf1815c8c0 2023-11-28T12:03:11.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.007+0000 7fbf1da53700 1 -- 172.21.15.57:0/2058055844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf18161120 con 0x7fbf1815c8c0 2023-11-28T12:03:11.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.007+0000 7fbf0ffff700 1 -- 172.21.15.57:0/2058055844 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbf00019030 con 0x7fbf1815c8c0 2023-11-28T12:03:11.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.008+0000 7fbf0ffff700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbef80682f0 0x7fbef806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.008+0000 7fbf0ffff700 1 -- 172.21.15.57:0/2058055844 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf00085950 con 0x7fbf1815c8c0 2023-11-28T12:03:11.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.008+0000 7fbf16ffd700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbef80682f0 0x7fbef806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.009+0000 7fbf16ffd700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbef80682f0 0x7fbef806a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbf08007fd0 tx=0x7fbf08007c40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:11.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.012+0000 7fbf0ffff700 1 -- 172.21.15.57:0/2058055844 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbf000521b0 con 0x7fbf1815c8c0 2023-11-28T12:03:11.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.210+0000 7fbf1da53700 1 -- 172.21.15.57:0/2058055844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbf18067d70 con 0x7fbf1815c8c0 2023-11-28T12:03:11.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.211+0000 7fbf0ffff700 1 -- 172.21.15.57:0/2058055844 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbf000557d0 con 0x7fbf1815c8c0 2023-11-28T12:03:11.212 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:03:11.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.213+0000 7fbf1da53700 1 -- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbef80682f0 msgr2=0x7fbef806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.213+0000 7fbf1da53700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbef80682f0 0x7fbef806a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbf08007fd0 tx=0x7fbf08007c40 comp rx=0 tx=0).stop 2023-11-28T12:03:11.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.213+0000 7fbf1da53700 1 -- 172.21.15.57:0/2058055844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf1815c8c0 msgr2=0x7fbf180c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.213+0000 7fbf1da53700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf1815c8c0 0x7fbf180c84e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000e870 tx=0x7fbf0000eb80 comp rx=0 tx=0).stop 2023-11-28T12:03:11.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.213+0000 7fbf1da53700 1 -- 172.21.15.57:0/2058055844 shutdown_connections 2023-11-28T12:03:11.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.213+0000 7fbf1da53700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbef80682f0 0x7fbef806a7a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.213+0000 7fbf1da53700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf18160cb0 0x7fbf180d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.213+0000 7fbf1da53700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf1806b050 0x7fbf18154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.213+0000 7fbf1da53700 1 --2- 172.21.15.57:0/2058055844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf1815c8c0 0x7fbf180c84e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.213+0000 7fbf1da53700 1 -- 172.21.15.57:0/2058055844 >> 172.21.15.57:0/2058055844 conn(0x7fbf1806c520 msgr2=0x7fbf180b8370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:11.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.213+0000 7fbf1da53700 1 -- 172.21.15.57:0/2058055844 shutdown_connections 2023-11-28T12:03:11.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.213+0000 7fbf1da53700 1 -- 172.21.15.57:0/2058055844 wait complete. 2023-11-28T12:03:11.227 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:03:11.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.364+0000 7fb961575700 1 Processor -- start 2023-11-28T12:03:11.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.364+0000 7fb961575700 1 -- start start 2023-11-28T12:03:11.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.365+0000 7fb961575700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb95c1599b0 0x7fb95c15d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.365+0000 7fb961575700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb95c0af0a0 0x7fb95c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.365+0000 7fb961575700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95c159ec0 con 0x7fb95c0b1650 2023-11-28T12:03:11.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.365+0000 7fb961575700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95c15d8a0 con 0x7fb95c0af0a0 2023-11-28T12:03:11.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.365+0000 7fb961575700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c15db80 con 0x7fb95c1599b0 2023-11-28T12:03:11.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.365+0000 7fb95affd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fb95c0b1650 0x7fb95c0b1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47046/0 (socket says 172.21.15.57:47046) 2023-11-28T12:03:11.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.365+0000 7fb95affd700 1 -- 172.21.15.57:0/3400074610 learned_addr learned my addr 172.21.15.57:0/3400074610 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:11.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb95b7fe700 1 --2- 172.21.15.57:0/3400074610 >> v2:172.21.15.181:3300/0 conn(0x7fb95c0af0a0 0x7fb95c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb95a7fc700 1 --2- 172.21.15.57:0/3400074610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb95c1599b0 0x7fb95c15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb959ffb700 1 -- 172.21.15.57:0/3400074610 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639626447 0 0) 0x7fb95c159ec0 con 0x7fb95c0b1650 2023-11-28T12:03:11.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb959ffb700 1 -- 172.21.15.57:0/3400074610 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9440035b0 con 0x7fb95c0b1650 2023-11-28T12:03:11.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb959ffb700 1 -- 172.21.15.57:0/3400074610 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 61403139 0 0) 0x7fb9440035b0 con 0x7fb95c0b1650 2023-11-28T12:03:11.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb95b7fe700 1 -- 172.21.15.57:0/3400074610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb95c1599b0 msgr2=0x7fb95c15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:11.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb95b7fe700 1 --2- 172.21.15.57:0/3400074610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb95c1599b0 0x7fb95c15d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb95b7fe700 1 -- 172.21.15.57:0/3400074610 >> v1:172.21.15.57:6789/0 conn(0x7fb95c0b1650 legacy=0x7fb95c0b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb95b7fe700 1 -- 172.21.15.57:0/3400074610 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c15de60 con 0x7fb95c0af0a0 2023-11-28T12:03:11.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb95a7fc700 1 --2- 172.21.15.57:0/3400074610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb95c1599b0 0x7fb95c15d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:03:11.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb95b7fe700 1 --2- 172.21.15.57:0/3400074610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb95c0af0a0 0x7fb95c15a350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb954009d90 tx=0x7fb95400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4b4d31b444fe598 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:11.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb959ffb700 1 -- 172.21.15.57:0/3400074610 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95400e070 con 0x7fb95c0af0a0 2023-11-28T12:03:11.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb959ffb700 1 -- 172.21.15.57:0/3400074610 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb954003a20 con 0x7fb95c0af0a0 2023-11-28T12:03:11.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.366+0000 7fb959ffb700 1 -- 172.21.15.57:0/3400074610 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c001670 con 0x7fb95c0af0a0 2023-11-28T12:03:11.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.367+0000 7fb961575700 1 -- 172.21.15.57:0/3400074610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb95c0af0a0 msgr2=0x7fb95c15a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.367+0000 7fb961575700 1 --2- 172.21.15.57:0/3400074610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb95c0af0a0 0x7fb95c15a350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb954009d90 tx=0x7fb95400b330 comp rx=0 tx=0).stop 2023-11-28T12:03:11.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.367+0000 7fb961575700 1 -- 172.21.15.57:0/3400074610 shutdown_connections 2023-11-28T12:03:11.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.367+0000 7fb961575700 1 --2- 172.21.15.57:0/3400074610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb95c0af0a0 0x7fb95c15a350 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.367+0000 7fb961575700 1 --2- 172.21.15.57:0/3400074610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb95c1599b0 0x7fb95c15d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.367+0000 7fb961575700 1 -- 172.21.15.57:0/3400074610 >> 172.21.15.57:0/3400074610 conn(0x7fb95c06c010 msgr2=0x7fb95c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:11.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.367+0000 7fb961575700 1 -- 172.21.15.57:0/3400074610 shutdown_connections 2023-11-28T12:03:11.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.367+0000 7fb961575700 1 -- 172.21.15.57:0/3400074610 wait complete. 2023-11-28T12:03:11.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.369+0000 7fb961575700 1 Processor -- start 2023-11-28T12:03:11.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.369+0000 7fb961575700 1 -- start start 2023-11-28T12:03:11.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.369+0000 7fb961575700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb95c0af0a0 0x7fb95c150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.369+0000 7fb961575700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb95c0b1650 0x7fb95c14f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.369+0000 7fb961575700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb95c0cb0b0 0x7fb95c0cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.369+0000 7fb961575700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0cba90 con 0x7fb95c0b1650 2023-11-28T12:03:11.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.369+0000 7fb961575700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c1673f0 con 0x7fb95c0af0a0 2023-11-28T12:03:11.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.369+0000 7fb961575700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c151120 con 0x7fb95c0cb0b0 2023-11-28T12:03:11.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb95a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb95c0b1650 0x7fb95c14f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb95a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb95c0b1650 0x7fb95c14f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40710/0 (socket says 172.21.15.57:40710) 2023-11-28T12:03:11.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb95a7fc700 1 -- 172.21.15.57:0/1856592827 learned_addr learned my addr 172.21.15.57:0/1856592827 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb95b7fe700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb95c0cb0b0 0x7fb95c0cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb95affd700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb95c0af0a0 0x7fb95c150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb95a7fc700 1 -- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb95c0cb0b0 msgr2=0x7fb95c0cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb95a7fc700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb95c0cb0b0 0x7fb95c0cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb95a7fc700 1 -- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb95c0af0a0 msgr2=0x7fb95c150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb95a7fc700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb95c0af0a0 0x7fb95c150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb95a7fc700 1 -- 172.21.15.57:0/1856592827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9500094a0 con 0x7fb95c0b1650 2023-11-28T12:03:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb95a7fc700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb95c0b1650 0x7fb95c14f180 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb95000ebb0 tx=0x7fb95000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb9437fe700 1 -- 172.21.15.57:0/1856592827 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95000c7f0 con 0x7fb95c0b1650 2023-11-28T12:03:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb9437fe700 1 -- 172.21.15.57:0/1856592827 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95000c950 con 0x7fb95c0b1650 2023-11-28T12:03:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb961575700 1 -- 172.21.15.57:0/1856592827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb954009a70 con 0x7fb95c0b1650 2023-11-28T12:03:11.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb9437fe700 1 -- 172.21.15.57:0/1856592827 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95000cce0 con 0x7fb95c0b1650 2023-11-28T12:03:11.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.370+0000 7fb961575700 1 -- 172.21.15.57:0/1856592827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95c15a890 con 0x7fb95c0b1650 2023-11-28T12:03:11.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.371+0000 7fb961575700 1 -- 172.21.15.57:0/1856592827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95c15aa20 con 0x7fb95c0b1650 2023-11-28T12:03:11.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.372+0000 7fb9437fe700 1 -- 172.21.15.57:0/1856592827 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb950019030 con 0x7fb95c0b1650 2023-11-28T12:03:11.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.372+0000 7fb9437fe700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9480682f0 0x7fb94806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.373+0000 7fb9437fe700 1 -- 172.21.15.57:0/1856592827 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb950085af0 con 0x7fb95c0b1650 2023-11-28T12:03:11.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.373+0000 7fb95affd700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9480682f0 0x7fb94806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.373+0000 7fb95affd700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9480682f0 0x7fb94806a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb95c158ed0 tx=0x7fb94c001da0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:11.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.377+0000 7fb9437fe700 1 -- 172.21.15.57:0/1856592827 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb950052350 con 0x7fb95c0b1650 2023-11-28T12:03:11.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.577+0000 7fb961575700 1 -- 172.21.15.57:0/1856592827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb95c0743f0 con 0x7fb95c0b1650 2023-11-28T12:03:11.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.578+0000 7fb9437fe700 1 -- 172.21.15.57:0/1856592827 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb950055970 con 0x7fb95c0b1650 2023-11-28T12:03:11.579 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:03:11.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.580+0000 7fb961575700 1 -- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9480682f0 msgr2=0x7fb94806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.580+0000 7fb961575700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9480682f0 0x7fb94806a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb95c158ed0 tx=0x7fb94c001da0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.580+0000 7fb961575700 1 -- 172.21.15.57:0/1856592827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb95c0b1650 msgr2=0x7fb95c14f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.580+0000 7fb961575700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb95c0b1650 0x7fb95c14f180 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb95000ebb0 tx=0x7fb95000eec0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.580+0000 7fb961575700 1 -- 172.21.15.57:0/1856592827 shutdown_connections 2023-11-28T12:03:11.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.580+0000 7fb961575700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9480682f0 0x7fb94806a7a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.580+0000 7fb961575700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb95c0cb0b0 0x7fb95c0cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.580+0000 7fb961575700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb95c0af0a0 0x7fb95c150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.580+0000 7fb961575700 1 --2- 172.21.15.57:0/1856592827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb95c0b1650 0x7fb95c14f180 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.580+0000 7fb961575700 1 -- 172.21.15.57:0/1856592827 >> 172.21.15.57:0/1856592827 conn(0x7fb95c06c010 msgr2=0x7fb95c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:11.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.580+0000 7fb961575700 1 -- 172.21.15.57:0/1856592827 shutdown_connections 2023-11-28T12:03:11.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.580+0000 7fb961575700 1 -- 172.21.15.57:0/1856592827 wait complete. 2023-11-28T12:03:11.594 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:03:11.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.732+0000 7efe995ab700 1 Processor -- start 2023-11-28T12:03:11.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.732+0000 7efe995ab700 1 -- start start 2023-11-28T12:03:11.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe995ab700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7efe9415c8c0 0x7efe94160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe995ab700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe9406b220 0x7efe940d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe995ab700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe940d2d30 con 0x7efe940b5dc0 2023-11-28T12:03:11.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe995ab700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe940d0b40 con 0x7efe9415c8c0 2023-11-28T12:03:11.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe995ab700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe941611f0 con 0x7efe9406b220 2023-11-28T12:03:11.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe92ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7efe9415c8c0 0x7efe94160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe92ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7efe9415c8c0 0x7efe94160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40724/0 (socket says 172.21.15.57:40724) 2023-11-28T12:03:11.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe92ffd700 1 -- 172.21.15.57:0/1735715814 learned_addr learned my addr 172.21.15.57:0/1735715814 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:11.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe937fe700 1 --2- 172.21.15.57:0/1735715814 >> v2:172.21.15.181:3300/0 conn(0x7efe9406b220 0x7efe940d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe92ffd700 1 -- 172.21.15.57:0/1735715814 >> v2:172.21.15.181:3300/0 conn(0x7efe9406b220 msgr2=0x7efe940d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:11.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe92ffd700 1 --2- 172.21.15.57:0/1735715814 >> v2:172.21.15.181:3300/0 conn(0x7efe9406b220 0x7efe940d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe92ffd700 1 -- 172.21.15.57:0/1735715814 >> v1:172.21.15.181:6789/0 conn(0x7efe940b5dc0 legacy=0x7efe9406ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe92ffd700 1 -- 172.21.15.57:0/1735715814 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe940d2a50 con 0x7efe9415c8c0 2023-11-28T12:03:11.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe92ffd700 1 --2- 172.21.15.57:0/1735715814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe9415c8c0 0x7efe94160cb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efe9408c3d0 tx=0x7efe84009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e3ab6a548a782c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:11.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.734+0000 7efe91ffb700 1 -- 172.21.15.57:0/1735715814 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84011830 con 0x7efe9415c8c0 2023-11-28T12:03:11.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.735+0000 7efe91ffb700 1 -- 172.21.15.57:0/1735715814 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe84011990 con 0x7efe9415c8c0 2023-11-28T12:03:11.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.735+0000 7efe91ffb700 1 -- 172.21.15.57:0/1735715814 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84011d20 con 0x7efe9415c8c0 2023-11-28T12:03:11.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.735+0000 7efe995ab700 1 -- 172.21.15.57:0/1735715814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe9415c8c0 msgr2=0x7efe94160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.735+0000 7efe995ab700 1 --2- 172.21.15.57:0/1735715814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe9415c8c0 0x7efe94160cb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efe9408c3d0 tx=0x7efe84009bd0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.735+0000 7efe995ab700 1 -- 172.21.15.57:0/1735715814 shutdown_connections 2023-11-28T12:03:11.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.735+0000 7efe995ab700 1 --2- 172.21.15.57:0/1735715814 >> v2:172.21.15.181:3300/0 conn(0x7efe9406b220 0x7efe940d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.735+0000 7efe995ab700 1 --2- 172.21.15.57:0/1735715814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe9415c8c0 0x7efe94160cb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.735+0000 7efe995ab700 1 -- 172.21.15.57:0/1735715814 >> 172.21.15.57:0/1735715814 conn(0x7efe9406c520 msgr2=0x7efe9406c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:11.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.735+0000 7efe995ab700 1 -- 172.21.15.57:0/1735715814 shutdown_connections 2023-11-28T12:03:11.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.735+0000 7efe995ab700 1 -- 172.21.15.57:0/1735715814 wait complete. 2023-11-28T12:03:11.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.737+0000 7efe995ab700 1 Processor -- start 2023-11-28T12:03:11.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.737+0000 7efe995ab700 1 -- start start 2023-11-28T12:03:11.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.737+0000 7efe995ab700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe9406b220 0x7efe941e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.737+0000 7efe995ab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe940b5dc0 0x7efe9415ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.737+0000 7efe995ab700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe9415c8c0 0x7efe94154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.737+0000 7efe995ab700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940b83e0 con 0x7efe9406b220 2023-11-28T12:03:11.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.737+0000 7efe995ab700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe941e8530 con 0x7efe940b5dc0 2023-11-28T12:03:11.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.737+0000 7efe995ab700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9415d200 con 0x7efe9415c8c0 2023-11-28T12:03:11.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe92ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe9406b220 0x7efe941e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe92ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe9406b220 0x7efe941e7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40736/0 (socket says 172.21.15.57:40736) 2023-11-28T12:03:11.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe92ffd700 1 -- 172.21.15.57:0/3987014337 learned_addr learned my addr 172.21.15.57:0/3987014337 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:11.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe927fc700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe940b5dc0 0x7efe9415ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe937fe700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe9415c8c0 0x7efe94154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe92ffd700 1 -- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe9415c8c0 msgr2=0x7efe94154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe92ffd700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe9415c8c0 0x7efe94154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe92ffd700 1 -- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe940b5dc0 msgr2=0x7efe9415ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe92ffd700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe940b5dc0 0x7efe9415ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe92ffd700 1 -- 172.21.15.57:0/3987014337 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe84009550 con 0x7efe9406b220 2023-11-28T12:03:11.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe937fe700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe9415c8c0 0x7efe94154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:11.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe92ffd700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe9406b220 0x7efe941e7ff0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efe84009c10 tx=0x7efe84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:11.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe7b7fe700 1 -- 172.21.15.57:0/3987014337 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84004590 con 0x7efe9406b220 2023-11-28T12:03:11.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe7b7fe700 1 -- 172.21.15.57:0/3987014337 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8401e070 con 0x7efe9406b220 2023-11-28T12:03:11.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe995ab700 1 -- 172.21.15.57:0/3987014337 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe940cda40 con 0x7efe9406b220 2023-11-28T12:03:11.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe7b7fe700 1 -- 172.21.15.57:0/3987014337 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840038e0 con 0x7efe9406b220 2023-11-28T12:03:11.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.738+0000 7efe995ab700 1 -- 172.21.15.57:0/3987014337 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe940cd760 con 0x7efe9406b220 2023-11-28T12:03:11.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.739+0000 7efe995ab700 1 -- 172.21.15.57:0/3987014337 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe940d2700 con 0x7efe9406b220 2023-11-28T12:03:11.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.740+0000 7efe7b7fe700 1 -- 172.21.15.57:0/3987014337 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efe8400f050 con 0x7efe9406b220 2023-11-28T12:03:11.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.740+0000 7efe7b7fe700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efe800682f0 0x7efe8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:11.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.740+0000 7efe7b7fe700 1 -- 172.21.15.57:0/3987014337 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efe84086b20 con 0x7efe9406b220 2023-11-28T12:03:11.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.740+0000 7efe927fc700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efe800682f0 0x7efe8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:11.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.741+0000 7efe927fc700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efe800682f0 0x7efe8006a7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efe941e7360 tx=0x7efe88009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:11.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.745+0000 7efe7b7fe700 1 -- 172.21.15.57:0/3987014337 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efe84053380 con 0x7efe9406b220 2023-11-28T12:03:11.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.943+0000 7efe995ab700 1 -- 172.21.15.57:0/3987014337 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe94067d70 con 0x7efe9406b220 2023-11-28T12:03:11.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.944+0000 7efe7b7fe700 1 -- 172.21.15.57:0/3987014337 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe84020020 con 0x7efe9406b220 2023-11-28T12:03:11.945 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:03:11.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.946+0000 7efe995ab700 1 -- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efe800682f0 msgr2=0x7efe8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.946+0000 7efe995ab700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efe800682f0 0x7efe8006a7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efe941e7360 tx=0x7efe88009040 comp rx=0 tx=0).stop 2023-11-28T12:03:11.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.946+0000 7efe995ab700 1 -- 172.21.15.57:0/3987014337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe9406b220 msgr2=0x7efe941e7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:11.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.946+0000 7efe995ab700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe9406b220 0x7efe941e7ff0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efe84009c10 tx=0x7efe84004750 comp rx=0 tx=0).stop 2023-11-28T12:03:11.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.946+0000 7efe995ab700 1 -- 172.21.15.57:0/3987014337 shutdown_connections 2023-11-28T12:03:11.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.946+0000 7efe995ab700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efe800682f0 0x7efe8006a7a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.946+0000 7efe995ab700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe9415c8c0 0x7efe94154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.946+0000 7efe995ab700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe9406b220 0x7efe941e7ff0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.946+0000 7efe995ab700 1 --2- 172.21.15.57:0/3987014337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe940b5dc0 0x7efe9415ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:11.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.946+0000 7efe995ab700 1 -- 172.21.15.57:0/3987014337 >> 172.21.15.57:0/3987014337 conn(0x7efe9406c520 msgr2=0x7efe940cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:11.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.946+0000 7efe995ab700 1 -- 172.21.15.57:0/3987014337 shutdown_connections 2023-11-28T12:03:11.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:11.946+0000 7efe995ab700 1 -- 172.21.15.57:0/3987014337 wait complete. 2023-11-28T12:03:11.961 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:03:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.098+0000 7fafa3bf4700 1 Processor -- start 2023-11-28T12:03:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.098+0000 7fafa3bf4700 1 -- start start 2023-11-28T12:03:12.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.099+0000 7fafa3bf4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faf9c0af0a0 0x7faf9c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.099+0000 7fafa3bf4700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c159ec0 con 0x7faf9c0b1650 2023-11-28T12:03:12.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.099+0000 7fafa3bf4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c15d8a0 con 0x7faf9c1599b0 2023-11-28T12:03:12.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.099+0000 7fafa3bf4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c15db80 con 0x7faf9c0af0a0 2023-11-28T12:03:12.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.099+0000 7fafa1990700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7faf9c0b1650 0x7faf9c0b1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47058/0 (socket says 172.21.15.57:47058) 2023-11-28T12:03:12.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.099+0000 7fafa1990700 1 -- 172.21.15.57:0/155186654 learned_addr learned my addr 172.21.15.57:0/155186654 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:12.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.099+0000 7fafa2191700 1 --2- 172.21.15.57:0/155186654 >> v2:172.21.15.181:3300/0 conn(0x7faf9c0af0a0 0x7faf9c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.099+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632415717 0 0) 0x7faf9c159ec0 con 0x7faf9c0b1650 2023-11-28T12:03:12.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.099+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf880035b0 con 0x7faf9c0b1650 2023-11-28T12:03:12.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119998287 0 0) 0x7faf9c15d8a0 con 0x7faf9c1599b0 2023-11-28T12:03:12.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf9c159ec0 con 0x7faf9c1599b0 2023-11-28T12:03:12.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1045661112 0 0) 0x7faf880035b0 con 0x7faf9c0b1650 2023-11-28T12:03:12.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 >> v2:172.21.15.181:3300/0 conn(0x7faf9c0af0a0 msgr2=0x7faf9c15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:12.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 --2- 172.21.15.57:0/155186654 >> v2:172.21.15.181:3300/0 conn(0x7faf9c0af0a0 0x7faf9c15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 >> v1:172.21.15.181:6789/0 conn(0x7faf9c1599b0 legacy=0x7faf9c15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:12.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c15de60 con 0x7faf9c0b1650 2023-11-28T12:03:12.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7faf98001670 con 0x7faf9c0b1650 2023-11-28T12:03:12.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa2191700 1 --2- 172.21.15.57:0/155186654 >> v2:172.21.15.181:3300/0 conn(0x7faf9c0af0a0 0x7faf9c15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:03:12.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 >> v1:172.21.15.57:6789/0 conn(0x7faf9c0b1650 legacy=0x7faf9c0b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:12.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 --2- 172.21.15.57:0/155186654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88003be0 0x7faf88003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 --2- 172.21.15.57:0/155186654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf88004850 0x7faf88004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 --2- 172.21.15.57:0/155186654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf88005200 0x7faf8800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c15db80 con 0x7faf88005200 2023-11-28T12:03:12.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8800c0e0 con 0x7faf88003be0 2023-11-28T12:03:12.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8800c270 con 0x7faf88004850 2023-11-28T12:03:12.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa2191700 1 --2- 172.21.15.57:0/155186654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf88005200 0x7faf8800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa1990700 1 --2- 172.21.15.57:0/155186654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88003be0 0x7faf88003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa118f700 1 --2- 172.21.15.57:0/155186654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf88004850 0x7faf88004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa2191700 1 -- 172.21.15.57:0/155186654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf88004850 msgr2=0x7faf88004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:12.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa2191700 1 --2- 172.21.15.57:0/155186654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf88004850 0x7faf88004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa2191700 1 -- 172.21.15.57:0/155186654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88003be0 msgr2=0x7faf88003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:12.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa2191700 1 --2- 172.21.15.57:0/155186654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88003be0 0x7faf88003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa2191700 1 -- 172.21.15.57:0/155186654 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf880035b0 con 0x7faf88005200 2023-11-28T12:03:12.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa2191700 1 -- 172.21.15.57:0/155186654 reap_dead start 2023-11-28T12:03:12.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.100+0000 7fafa2191700 1 --2- 172.21.15.57:0/155186654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf88005200 0x7faf8800ba10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e320 tx=0x7faf88005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f152d6a1280863 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:12.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.101+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9400edc0 con 0x7faf88005200 2023-11-28T12:03:12.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.101+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9400f040 con 0x7faf88005200 2023-11-28T12:03:12.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.101+0000 7fafa098e700 1 -- 172.21.15.57:0/155186654 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf940146c0 con 0x7faf88005200 2023-11-28T12:03:12.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.101+0000 7fafa3bf4700 1 -- 172.21.15.57:0/155186654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf88005200 msgr2=0x7faf8800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:12.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.101+0000 7fafa3bf4700 1 --2- 172.21.15.57:0/155186654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf88005200 0x7faf8800ba10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e320 tx=0x7faf88005880 comp rx=0 tx=0).stop 2023-11-28T12:03:12.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.101+0000 7fafa3bf4700 1 -- 172.21.15.57:0/155186654 shutdown_connections 2023-11-28T12:03:12.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.101+0000 7fafa3bf4700 1 --2- 172.21.15.57:0/155186654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf88005200 0x7faf8800ba10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.101+0000 7fafa3bf4700 1 -- 172.21.15.57:0/155186654 >> 172.21.15.57:0/155186654 conn(0x7faf9c06c010 msgr2=0x7faf9c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:12.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.101+0000 7fafa3bf4700 1 -- 172.21.15.57:0/155186654 shutdown_connections 2023-11-28T12:03:12.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.101+0000 7fafa3bf4700 1 -- 172.21.15.57:0/155186654 wait complete. 2023-11-28T12:03:12.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.103+0000 7fafa3bf4700 1 Processor -- start 2023-11-28T12:03:12.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.103+0000 7fafa3bf4700 1 -- start start 2023-11-28T12:03:12.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.104+0000 7fafa3bf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88005200 0x7faf9c14be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.104+0000 7fafa3bf4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf9c0c5df0 0x7faf9c0c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.104+0000 7fafa3bf4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf9c1e49d0 0x7faf9c1e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.104+0000 7fafa3bf4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0c3740 con 0x7faf9c1e49d0 2023-11-28T12:03:12.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.104+0000 7fafa3bf4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0c67a0 con 0x7faf88005200 2023-11-28T12:03:12.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.104+0000 7fafa3bf4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c1ef880 con 0x7faf9c0c5df0 2023-11-28T12:03:12.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.104+0000 7fafa2191700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf9c1e49d0 0x7faf9c1e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.104+0000 7fafa2191700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf9c1e49d0 0x7faf9c1e4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40752/0 (socket says 172.21.15.57:40752) 2023-11-28T12:03:12.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.104+0000 7fafa2191700 1 -- 172.21.15.57:0/348766827 learned_addr learned my addr 172.21.15.57:0/348766827 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:12.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.104+0000 7fafa1990700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88005200 0x7faf9c14be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.104+0000 7fafa118f700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf9c0c5df0 0x7faf9c0c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.105+0000 7fafa2191700 1 -- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf9c0c5df0 msgr2=0x7faf9c0c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:12.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.105+0000 7fafa2191700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf9c0c5df0 0x7faf9c0c6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.105+0000 7fafa2191700 1 -- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88005200 msgr2=0x7faf9c14be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:12.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.105+0000 7fafa2191700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88005200 0x7faf9c14be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.105+0000 7fafa2191700 1 -- 172.21.15.57:0/348766827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c0b1bb0 con 0x7faf9c1e49d0 2023-11-28T12:03:12.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.105+0000 7fafa118f700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf9c0c5df0 0x7faf9c0c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:12.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.105+0000 7fafa1990700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88005200 0x7faf9c14be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:12.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.105+0000 7fafa2191700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf9c1e49d0 0x7faf9c1e4e40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7faf9c067450 tx=0x7faf94009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:12.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.105+0000 7faf927fc700 1 -- 172.21.15.57:0/348766827 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94016aa0 con 0x7faf9c1e49d0 2023-11-28T12:03:12.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.105+0000 7faf927fc700 1 -- 172.21.15.57:0/348766827 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf94016c00 con 0x7faf9c1e49d0 2023-11-28T12:03:12.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.105+0000 7fafa3bf4700 1 -- 172.21.15.57:0/348766827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c0b1a20 con 0x7faf9c1e49d0 2023-11-28T12:03:12.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.105+0000 7faf927fc700 1 -- 172.21.15.57:0/348766827 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94004040 con 0x7faf9c1e49d0 2023-11-28T12:03:12.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.105+0000 7fafa3bf4700 1 -- 172.21.15.57:0/348766827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c0b1d40 con 0x7faf9c1e49d0 2023-11-28T12:03:12.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.106+0000 7fafa3bf4700 1 -- 172.21.15.57:0/348766827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9c0743f0 con 0x7faf9c1e49d0 2023-11-28T12:03:12.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.106+0000 7faf927fc700 1 -- 172.21.15.57:0/348766827 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faf940041a0 con 0x7faf9c1e49d0 2023-11-28T12:03:12.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.107+0000 7faf927fc700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faf800682f0 0x7faf8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.107+0000 7faf927fc700 1 -- 172.21.15.57:0/348766827 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf94089f00 con 0x7faf9c1e49d0 2023-11-28T12:03:12.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.107+0000 7fafa1990700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faf800682f0 0x7faf8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.108+0000 7fafa1990700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faf800682f0 0x7faf8006a7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faf98001eb0 tx=0x7faf98009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:12.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.112+0000 7faf927fc700 1 -- 172.21.15.57:0/348766827 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faf94056760 con 0x7faf9c1e49d0 2023-11-28T12:03:12.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.313+0000 7fafa3bf4700 1 -- 172.21.15.57:0/348766827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faf9c0ce800 con 0x7faf9c1e49d0 2023-11-28T12:03:12.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.313+0000 7faf927fc700 1 -- 172.21.15.57:0/348766827 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7faf940180a0 con 0x7faf9c1e49d0 2023-11-28T12:03:12.314 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:03:12.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.315+0000 7fafa3bf4700 1 -- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faf800682f0 msgr2=0x7faf8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:12.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.315+0000 7fafa3bf4700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faf800682f0 0x7faf8006a7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faf98001eb0 tx=0x7faf98009040 comp rx=0 tx=0).stop 2023-11-28T12:03:12.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.315+0000 7fafa3bf4700 1 -- 172.21.15.57:0/348766827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf9c1e49d0 msgr2=0x7faf9c1e4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:12.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.315+0000 7fafa3bf4700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf9c1e49d0 0x7faf9c1e4e40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7faf9c067450 tx=0x7faf94009750 comp rx=0 tx=0).stop 2023-11-28T12:03:12.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.315+0000 7fafa3bf4700 1 -- 172.21.15.57:0/348766827 shutdown_connections 2023-11-28T12:03:12.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.315+0000 7fafa3bf4700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faf800682f0 0x7faf8006a7a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.315+0000 7fafa3bf4700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faf9c1e49d0 0x7faf9c1e4e40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.315+0000 7fafa3bf4700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf88005200 0x7faf9c14be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.315+0000 7fafa3bf4700 1 --2- 172.21.15.57:0/348766827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf9c0c5df0 0x7faf9c0c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.315+0000 7fafa3bf4700 1 -- 172.21.15.57:0/348766827 >> 172.21.15.57:0/348766827 conn(0x7faf9c06c010 msgr2=0x7faf9c15c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:12.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.316+0000 7fafa3bf4700 1 -- 172.21.15.57:0/348766827 shutdown_connections 2023-11-28T12:03:12.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.316+0000 7fafa3bf4700 1 -- 172.21.15.57:0/348766827 wait complete. 2023-11-28T12:03:12.330 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T12:03:12.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.465+0000 7fd9eea99700 1 Processor -- start 2023-11-28T12:03:12.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.465+0000 7fd9eea99700 1 -- start start 2023-11-28T12:03:12.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9eea99700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9e806a980 0x7fd9e80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9eea99700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd9e8071890 0x7fd9e8071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9eea99700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e815cc10 con 0x7fd9e80cf710 2023-11-28T12:03:12.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9eea99700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e806c9b0 con 0x7fd9e8071890 2023-11-28T12:03:12.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9eea99700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e806cc90 con 0x7fd9e806a980 2023-11-28T12:03:12.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9ec835700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd9e80cf710 0x7fd9e806a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47064/0 (socket says 172.21.15.57:47064) 2023-11-28T12:03:12.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9ec835700 1 -- 172.21.15.57:0/749854787 learned_addr learned my addr 172.21.15.57:0/749854787 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:12.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9ed036700 1 --2- 172.21.15.57:0/749854787 >> v2:172.21.15.181:3300/0 conn(0x7fd9e8071890 0x7fd9e8071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9e7fff700 1 --2- 172.21.15.57:0/749854787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9e806a980 0x7fd9e80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9e77fe700 1 -- 172.21.15.57:0/749854787 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630493363 0 0) 0x7fd9e815cc10 con 0x7fd9e80cf710 2023-11-28T12:03:12.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9e77fe700 1 -- 172.21.15.57:0/749854787 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9d40035b0 con 0x7fd9e80cf710 2023-11-28T12:03:12.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9e7fff700 1 -- 172.21.15.57:0/749854787 >> v2:172.21.15.181:3300/0 conn(0x7fd9e8071890 msgr2=0x7fd9e8071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:12.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9e7fff700 1 --2- 172.21.15.57:0/749854787 >> v2:172.21.15.181:3300/0 conn(0x7fd9e8071890 0x7fd9e8071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9e7fff700 1 -- 172.21.15.57:0/749854787 >> v1:172.21.15.57:6789/0 conn(0x7fd9e80cf710 legacy=0x7fd9e806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:12.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9e7fff700 1 -- 172.21.15.57:0/749854787 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e806cf70 con 0x7fd9e806a980 2023-11-28T12:03:12.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.466+0000 7fd9ed036700 1 --2- 172.21.15.57:0/749854787 >> v2:172.21.15.181:3300/0 conn(0x7fd9e8071890 0x7fd9e8071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:03:12.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.467+0000 7fd9e7fff700 1 --2- 172.21.15.57:0/749854787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9e806a980 0x7fd9e80ce740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009840 tx=0x7fd9d0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ba8ee8e90e9975e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:12.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.467+0000 7fd9e77fe700 1 -- 172.21.15.57:0/749854787 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0011830 con 0x7fd9e806a980 2023-11-28T12:03:12.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.467+0000 7fd9e77fe700 1 -- 172.21.15.57:0/749854787 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d0011990 con 0x7fd9e806a980 2023-11-28T12:03:12.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.467+0000 7fd9e77fe700 1 -- 172.21.15.57:0/749854787 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0011d20 con 0x7fd9e806a980 2023-11-28T12:03:12.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.467+0000 7fd9eea99700 1 -- 172.21.15.57:0/749854787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9e806a980 msgr2=0x7fd9e80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:12.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.467+0000 7fd9eea99700 1 --2- 172.21.15.57:0/749854787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9e806a980 0x7fd9e80ce740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009840 tx=0x7fd9d0009b50 comp rx=0 tx=0).stop 2023-11-28T12:03:12.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.467+0000 7fd9eea99700 1 -- 172.21.15.57:0/749854787 shutdown_connections 2023-11-28T12:03:12.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.467+0000 7fd9eea99700 1 --2- 172.21.15.57:0/749854787 >> v2:172.21.15.181:3300/0 conn(0x7fd9e8071890 0x7fd9e8071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.467+0000 7fd9eea99700 1 --2- 172.21.15.57:0/749854787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9e806a980 0x7fd9e80ce740 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.467+0000 7fd9eea99700 1 -- 172.21.15.57:0/749854787 >> 172.21.15.57:0/749854787 conn(0x7fd9e806c010 msgr2=0x7fd9e80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:12.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.467+0000 7fd9eea99700 1 -- 172.21.15.57:0/749854787 shutdown_connections 2023-11-28T12:03:12.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.467+0000 7fd9eea99700 1 -- 172.21.15.57:0/749854787 wait complete. 2023-11-28T12:03:12.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.469+0000 7fd9eea99700 1 Processor -- start 2023-11-28T12:03:12.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.469+0000 7fd9eea99700 1 -- start start 2023-11-28T12:03:12.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9eea99700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9e806a980 0x7fd9e815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9eea99700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9e8071890 0x7fd9e80c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9eea99700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9e80cf710 0x7fd9e80c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9eea99700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e815c080 con 0x7fd9e80cf710 2023-11-28T12:03:12.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9eea99700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e80c6ca0 con 0x7fd9e8071890 2023-11-28T12:03:12.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9eea99700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e80c6f80 con 0x7fd9e806a980 2023-11-28T12:03:12.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9ed036700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9e80cf710 0x7fd9e80c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9ed036700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9e80cf710 0x7fd9e80c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40756/0 (socket says 172.21.15.57:40756) 2023-11-28T12:03:12.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9ed036700 1 -- 172.21.15.57:0/3524849076 learned_addr learned my addr 172.21.15.57:0/3524849076 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:12.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9ec835700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9e806a980 0x7fd9e815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9e7fff700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9e8071890 0x7fd9e80c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9ed036700 1 -- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9e806a980 msgr2=0x7fd9e815a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:12.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9ed036700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9e806a980 0x7fd9e815a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9ed036700 1 -- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9e8071890 msgr2=0x7fd9e80c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:12.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9ed036700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9e8071890 0x7fd9e80c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9ed036700 1 -- 172.21.15.57:0/3524849076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e0009a70 con 0x7fd9e80cf710 2023-11-28T12:03:12.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9e7fff700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9e8071890 0x7fd9e80c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:12.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9ed036700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9e80cf710 0x7fd9e80c8cb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009f00 tx=0x7fd9e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:12.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9e57fa700 1 -- 172.21.15.57:0/3524849076 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0009d70 con 0x7fd9e80cf710 2023-11-28T12:03:12.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.470+0000 7fd9e57fa700 1 -- 172.21.15.57:0/3524849076 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e000f040 con 0x7fd9e80cf710 2023-11-28T12:03:12.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.471+0000 7fd9eea99700 1 -- 172.21.15.57:0/3524849076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d00094a0 con 0x7fd9e80cf710 2023-11-28T12:03:12.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.471+0000 7fd9e57fa700 1 -- 172.21.15.57:0/3524849076 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0014680 con 0x7fd9e80cf710 2023-11-28T12:03:12.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.471+0000 7fd9eea99700 1 -- 172.21.15.57:0/3524849076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e806d530 con 0x7fd9e80cf710 2023-11-28T12:03:12.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.472+0000 7fd9eea99700 1 -- 172.21.15.57:0/3524849076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e806d250 con 0x7fd9e80cf710 2023-11-28T12:03:12.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.472+0000 7fd9e57fa700 1 -- 172.21.15.57:0/3524849076 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd9e00147e0 con 0x7fd9e80cf710 2023-11-28T12:03:12.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.472+0000 7fd9e57fa700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9c80682f0 0x7fd9c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:12.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.473+0000 7fd9e57fa700 1 -- 172.21.15.57:0/3524849076 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd9e0086bc0 con 0x7fd9e80cf710 2023-11-28T12:03:12.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.473+0000 7fd9ec835700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9c80682f0 0x7fd9c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:12.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.473+0000 7fd9ec835700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9c80682f0 0x7fd9c806a7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80011e0 tx=0x7fd9d8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:12.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.477+0000 7fd9e57fa700 1 -- 172.21.15.57:0/3524849076 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd9e0053420 con 0x7fd9e80cf710 2023-11-28T12:03:12.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.696+0000 7fd9eea99700 1 -- 172.21.15.57:0/3524849076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd9e8160480 con 0x7fd9e80cf710 2023-11-28T12:03:12.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.696+0000 7fd9e57fa700 1 -- 172.21.15.57:0/3524849076 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd9e0018020 con 0x7fd9e80cf710 2023-11-28T12:03:12.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.698+0000 7fd9eea99700 1 -- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9c80682f0 msgr2=0x7fd9c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:12.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.698+0000 7fd9eea99700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9c80682f0 0x7fd9c806a7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80011e0 tx=0x7fd9d8009040 comp rx=0 tx=0).stop 2023-11-28T12:03:12.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.698+0000 7fd9eea99700 1 -- 172.21.15.57:0/3524849076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9e80cf710 msgr2=0x7fd9e80c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:12.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.698+0000 7fd9eea99700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9e80cf710 0x7fd9e80c8cb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009f00 tx=0x7fd9e000c3a0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.699+0000 7fd9eea99700 1 -- 172.21.15.57:0/3524849076 shutdown_connections 2023-11-28T12:03:12.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.699+0000 7fd9eea99700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9c80682f0 0x7fd9c806a7a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.699+0000 7fd9eea99700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9e80cf710 0x7fd9e80c8cb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.699+0000 7fd9eea99700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9e806a980 0x7fd9e815a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.699+0000 7fd9eea99700 1 --2- 172.21.15.57:0/3524849076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9e8071890 0x7fd9e80c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:12.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.699+0000 7fd9eea99700 1 -- 172.21.15.57:0/3524849076 >> 172.21.15.57:0/3524849076 conn(0x7fd9e806c010 msgr2=0x7fd9e80b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:12.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.699+0000 7fd9eea99700 1 -- 172.21.15.57:0/3524849076 shutdown_connections 2023-11-28T12:03:12.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:12.699+0000 7fd9eea99700 1 -- 172.21.15.57:0/3524849076 wait complete. 2023-11-28T12:03:12.701 INFO:teuthology.orchestra.run.smithi057.stderr:listed 0 entries 2023-11-28T12:03:12.713 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2023-11-28T12:03:12.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:12.713 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T12:03:12.756 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[137297]: fuse finished with error 0 and tester_r 0 2023-11-28T12:03:12.757 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:03:16.322 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:03:16.325 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:03:18.859 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:03:18.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:18.860 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T12:03:18.884 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:03:19.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.067+0000 7fad37ee4700 1 Processor -- start 2023-11-28T12:03:19.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.068+0000 7fad37ee4700 1 -- start start 2023-11-28T12:03:19.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.068+0000 7fad37ee4700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fad3006a270 0x7fad3006a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:19.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad37ee4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fad3006e570 0x7fad30070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:19.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad37ee4700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3006ab80 con 0x7fad3015cc80 2023-11-28T12:03:19.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad37ee4700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad300b2490 con 0x7fad3006a270 2023-11-28T12:03:19.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad37ee4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad3006e990 con 0x7fad3006e570 2023-11-28T12:03:19.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad35c80700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fad3006a270 0x7fad3006a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:19.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad3547f700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fad3015cc80 0x7fad3006de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:53182/0 (socket says 172.21.15.57:53182) 2023-11-28T12:03:19.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad3547f700 1 -- 172.21.15.57:0/3947257389 learned_addr learned my addr 172.21.15.57:0/3947257389 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:19.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad35c80700 1 --2- 172.21.15.57:0/3947257389 >> v2:172.21.15.57:3300/0 conn(0x7fad3006a270 0x7fad3006a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42382/0 (socket says 172.21.15.57:42382) 2023-11-28T12:03:19.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad36481700 1 --2- 172.21.15.57:0/3947257389 >> v2:172.21.15.181:3300/0 conn(0x7fad3006e570 0x7fad30070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:19.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad35c80700 1 -- 172.21.15.57:0/3947257389 >> v2:172.21.15.181:3300/0 conn(0x7fad3006e570 msgr2=0x7fad30070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:19.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad35c80700 1 --2- 172.21.15.57:0/3947257389 >> v2:172.21.15.181:3300/0 conn(0x7fad3006e570 0x7fad30070d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad35c80700 1 -- 172.21.15.57:0/3947257389 >> v1:172.21.15.57:6789/0 conn(0x7fad3015cc80 legacy=0x7fad3006de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:19.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad35c80700 1 -- 172.21.15.57:0/3947257389 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad300b1810 con 0x7fad3006a270 2023-11-28T12:03:19.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad34c7e700 1 -- 172.21.15.57:0/3947257389 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414791250 0 0) 0x7fad3006ab80 con 0x7fad3015cc80 2023-11-28T12:03:19.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad35c80700 1 --2- 172.21.15.57:0/3947257389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad3006a270 0x7fad3006a640 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fad3006ee30 tx=0x7fad2c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6336e7f44692968b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:19.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad34c7e700 1 -- 172.21.15.57:0/3947257389 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c011830 con 0x7fad3006a270 2023-11-28T12:03:19.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad34c7e700 1 -- 172.21.15.57:0/3947257389 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c011990 con 0x7fad3006a270 2023-11-28T12:03:19.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.069+0000 7fad34c7e700 1 -- 172.21.15.57:0/3947257389 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c011d20 con 0x7fad3006a270 2023-11-28T12:03:19.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.070+0000 7fad37ee4700 1 -- 172.21.15.57:0/3947257389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad3006a270 msgr2=0x7fad3006a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:19.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.070+0000 7fad37ee4700 1 --2- 172.21.15.57:0/3947257389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad3006a270 0x7fad3006a640 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fad3006ee30 tx=0x7fad2c009b50 comp rx=0 tx=0).stop 2023-11-28T12:03:19.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.070+0000 7fad37ee4700 1 -- 172.21.15.57:0/3947257389 shutdown_connections 2023-11-28T12:03:19.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.070+0000 7fad37ee4700 1 --2- 172.21.15.57:0/3947257389 >> v2:172.21.15.181:3300/0 conn(0x7fad3006e570 0x7fad30070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.070+0000 7fad37ee4700 1 --2- 172.21.15.57:0/3947257389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad3006a270 0x7fad3006a640 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.070+0000 7fad37ee4700 1 -- 172.21.15.57:0/3947257389 >> 172.21.15.57:0/3947257389 conn(0x7fad3006c010 msgr2=0x7fad300cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:19.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.070+0000 7fad37ee4700 1 -- 172.21.15.57:0/3947257389 shutdown_connections 2023-11-28T12:03:19.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.070+0000 7fad37ee4700 1 -- 172.21.15.57:0/3947257389 wait complete. 2023-11-28T12:03:19.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.071+0000 7fad37ee4700 1 Processor -- start 2023-11-28T12:03:19.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.071+0000 7fad37ee4700 1 -- start start 2023-11-28T12:03:19.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.072+0000 7fad37ee4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad3006a270 0x7fad300b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:19.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.072+0000 7fad37ee4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad3006e570 0x7fad30152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:19.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.072+0000 7fad37ee4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad3015cc80 0x7fad3015b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:19.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.072+0000 7fad37ee4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad301e5500 con 0x7fad3015cc80 2023-11-28T12:03:19.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.072+0000 7fad37ee4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad301f6b00 con 0x7fad3006a270 2023-11-28T12:03:19.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.072+0000 7fad37ee4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad301f6570 con 0x7fad3006e570 2023-11-28T12:03:19.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.072+0000 7fad36481700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad3015cc80 0x7fad3015b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:19.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.072+0000 7fad36481700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad3015cc80 0x7fad3015b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42392/0 (socket says 172.21.15.57:42392) 2023-11-28T12:03:19.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.072+0000 7fad36481700 1 -- 172.21.15.57:0/1912145354 learned_addr learned my addr 172.21.15.57:0/1912145354 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:19.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad35c80700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad3006a270 0x7fad300b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:19.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad3547f700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad3006e570 0x7fad30152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:19.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad36481700 1 -- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad3006e570 msgr2=0x7fad30152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:19.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad36481700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad3006e570 0x7fad30152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad36481700 1 -- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad3006a270 msgr2=0x7fad300b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:19.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad36481700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad3006a270 0x7fad300b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad36481700 1 -- 172.21.15.57:0/1912145354 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c0094a0 con 0x7fad3015cc80 2023-11-28T12:03:19.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad35c80700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad3006a270 0x7fad300b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:19.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad3547f700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad3006e570 0x7fad30152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:19.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad36481700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad3015cc80 0x7fad3015b880 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fad28007e60 tx=0x7fad2800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:19.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad267fc700 1 -- 172.21.15.57:0/1912145354 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2800ede0 con 0x7fad3015cc80 2023-11-28T12:03:19.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad267fc700 1 -- 172.21.15.57:0/1912145354 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad28014420 con 0x7fad3015cc80 2023-11-28T12:03:19.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad267fc700 1 -- 172.21.15.57:0/1912145354 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad280147d0 con 0x7fad3015cc80 2023-11-28T12:03:19.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad37ee4700 1 -- 172.21.15.57:0/1912145354 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad300b1dd0 con 0x7fad3015cc80 2023-11-28T12:03:19.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.073+0000 7fad37ee4700 1 -- 172.21.15.57:0/1912145354 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad300b1af0 con 0x7fad3015cc80 2023-11-28T12:03:19.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.074+0000 7fad37ee4700 1 -- 172.21.15.57:0/1912145354 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad300b20b0 con 0x7fad3015cc80 2023-11-28T12:03:19.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.075+0000 7fad267fc700 1 -- 172.21.15.57:0/1912145354 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fad28012050 con 0x7fad3015cc80 2023-11-28T12:03:19.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.075+0000 7fad267fc700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fad14068320 0x7fad1406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:19.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.075+0000 7fad267fc700 1 -- 172.21.15.57:0/1912145354 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fad280861e0 con 0x7fad3015cc80 2023-11-28T12:03:19.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.075+0000 7fad35c80700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fad14068320 0x7fad1406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:19.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.076+0000 7fad35c80700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fad14068320 0x7fad1406a7d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009b90 tx=0x7fad2c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:19.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.080+0000 7fad267fc700 1 -- 172.21.15.57:0/1912145354 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fad28052a40 con 0x7fad3015cc80 2023-11-28T12:03:19.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.332+0000 7fad37ee4700 1 -- 172.21.15.57:0/1912145354 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad300b4ca0 con 0x7fad3015cc80 2023-11-28T12:03:19.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.333+0000 7fad267fc700 1 -- 172.21.15.57:0/1912145354 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4648 (secure 0 0 0) 0x7fad300b4ca0 con 0x7fad3015cc80 2023-11-28T12:03:19.334 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:19.334 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/340292109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":340292109},{"type":"v1","addr":"172.21.15.181:6836","nonce":340292109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2442321825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2442321825},{"type":"v1","addr":"172.21.15.181:6837","nonce":2442321825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/3452242211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3452242211},{"type":"v1","addr":"172.21.15.57:6833","nonce":3452242211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:02:53.402391+0000","modified":"2023-11-28T12:02:56.540229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.57:6835/968299041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":968299041},{"type":"v1","addr":"172.21.15.57:6835","nonce":968299041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T12:03:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.335+0000 7fad37ee4700 1 -- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fad14068320 msgr2=0x7fad1406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.335+0000 7fad37ee4700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fad14068320 0x7fad1406a7d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009b90 tx=0x7fad2c008030 comp rx=0 tx=0).stop 2023-11-28T12:03:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.335+0000 7fad37ee4700 1 -- 172.21.15.57:0/1912145354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad3015cc80 msgr2=0x7fad3015b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.335+0000 7fad37ee4700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad3015cc80 0x7fad3015b880 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fad28007e60 tx=0x7fad2800e3c0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.335+0000 7fad37ee4700 1 -- 172.21.15.57:0/1912145354 shutdown_connections 2023-11-28T12:03:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.335+0000 7fad37ee4700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fad14068320 0x7fad1406a7d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.335+0000 7fad37ee4700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad3015cc80 0x7fad3015b880 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.335+0000 7fad37ee4700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad3006a270 0x7fad300b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.335+0000 7fad37ee4700 1 --2- 172.21.15.57:0/1912145354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad3006e570 0x7fad30152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.335+0000 7fad37ee4700 1 -- 172.21.15.57:0/1912145354 >> 172.21.15.57:0/1912145354 conn(0x7fad3006c010 msgr2=0x7fad300ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:19.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.335+0000 7fad37ee4700 1 -- 172.21.15.57:0/1912145354 shutdown_connections 2023-11-28T12:03:19.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.335+0000 7fad37ee4700 1 -- 172.21.15.57:0/1912145354 wait complete. 2023-11-28T12:03:19.338 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 16 2023-11-28T12:03:19.350 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-28T12:03:19.515 INFO:teuthology.orchestra.run.smithi057.stdout:{"message":"","return_code":0} 2023-11-28T12:03:19.515 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T12:03:19.515 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:03:19.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.649+0000 7f16f7779700 1 Processor -- start 2023-11-28T12:03:19.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.649+0000 7f16f7779700 1 -- start start 2023-11-28T12:03:19.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f7779700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f16f006e400 0x7f16f006e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:19.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f7779700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16f00d15f0 0x7f16f015b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:19.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f7779700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f015af30 con 0x7f16f015e010 2023-11-28T12:03:19.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f7779700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f015a9d0 con 0x7f16f006e400 2023-11-28T12:03:19.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f7779700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f01584e0 con 0x7f16f00d15f0 2023-11-28T12:03:19.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f5515700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f16f006e400 0x7f16f006e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:19.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f5515700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f16f006e400 0x7f16f006e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42400/0 (socket says 172.21.15.57:42400) 2023-11-28T12:03:19.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f5515700 1 -- 172.21.15.57:0/4170556989 learned_addr learned my addr 172.21.15.57:0/4170556989 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:19.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f4d14700 1 --2- 172.21.15.57:0/4170556989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16f00d15f0 0x7f16f015b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:19.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f5515700 1 -- 172.21.15.57:0/4170556989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16f00d15f0 msgr2=0x7f16f015b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:19.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f5515700 1 --2- 172.21.15.57:0/4170556989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16f00d15f0 0x7f16f015b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f5515700 1 -- 172.21.15.57:0/4170556989 >> v1:172.21.15.181:6789/0 conn(0x7f16f015e010 legacy=0x7f16f00ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:19.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f5515700 1 -- 172.21.15.57:0/4170556989 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f015ef40 con 0x7f16f006e400 2023-11-28T12:03:19.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.650+0000 7f16f5515700 1 --2- 172.21.15.57:0/4170556989 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16f006e400 0x7f16f006e7d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0098f0 tx=0x7f16ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8c7ee5af9036b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:19.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.651+0000 7f16e7fff700 1 -- 172.21.15.57:0/4170556989 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec004230 con 0x7f16f006e400 2023-11-28T12:03:19.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.651+0000 7f16e7fff700 1 -- 172.21.15.57:0/4170556989 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec004390 con 0x7f16f006e400 2023-11-28T12:03:19.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.651+0000 7f16e7fff700 1 -- 172.21.15.57:0/4170556989 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0036a0 con 0x7f16f006e400 2023-11-28T12:03:19.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.651+0000 7f16f7779700 1 -- 172.21.15.57:0/4170556989 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16f006e400 msgr2=0x7f16f006e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:19.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.651+0000 7f16f7779700 1 --2- 172.21.15.57:0/4170556989 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16f006e400 0x7f16f006e7d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0098f0 tx=0x7f16ec009c00 comp rx=0 tx=0).stop 2023-11-28T12:03:19.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.651+0000 7f16f7779700 1 -- 172.21.15.57:0/4170556989 shutdown_connections 2023-11-28T12:03:19.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.651+0000 7f16f7779700 1 --2- 172.21.15.57:0/4170556989 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16f006e400 0x7f16f006e7d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.651+0000 7f16f7779700 1 --2- 172.21.15.57:0/4170556989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16f00d15f0 0x7f16f015b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.651+0000 7f16f7779700 1 -- 172.21.15.57:0/4170556989 >> 172.21.15.57:0/4170556989 conn(0x7f16f006a270 msgr2=0x7f16f006a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:19.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.651+0000 7f16f7779700 1 -- 172.21.15.57:0/4170556989 shutdown_connections 2023-11-28T12:03:19.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.651+0000 7f16f7779700 1 -- 172.21.15.57:0/4170556989 wait complete. 2023-11-28T12:03:19.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.653+0000 7f16f7779700 1 Processor -- start 2023-11-28T12:03:19.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.653+0000 7f16f7779700 1 -- start start 2023-11-28T12:03:19.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.653+0000 7f16f7779700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16f006e400 0x7f16f015b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:19.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.653+0000 7f16f7779700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f00d15f0 0x7f16f01e4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:19.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.653+0000 7f16f7779700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16f015e010 0x7f16f01e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:19.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.653+0000 7f16f7779700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f00b07f0 con 0x7f16f006e400 2023-11-28T12:03:19.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.653+0000 7f16f7779700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f0156d80 con 0x7f16f00d15f0 2023-11-28T12:03:19.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.653+0000 7f16f7779700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f0157060 con 0x7f16f015e010 2023-11-28T12:03:19.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16f5515700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16f006e400 0x7f16f015b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:19.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16f5515700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16f006e400 0x7f16f015b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42412/0 (socket says 172.21.15.57:42412) 2023-11-28T12:03:19.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16f5515700 1 -- 172.21.15.57:0/3238482783 learned_addr learned my addr 172.21.15.57:0/3238482783 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:19.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16f4d14700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f00d15f0 0x7f16f01e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:19.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16f5d16700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16f015e010 0x7f16f01e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:19.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16f5515700 1 -- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16f015e010 msgr2=0x7f16f01e49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:19.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16f5515700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16f015e010 0x7f16f01e49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16f5515700 1 -- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f00d15f0 msgr2=0x7f16f01e4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:19.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16f5515700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f00d15f0 0x7f16f01e4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16f5515700 1 -- 172.21.15.57:0/3238482783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec009550 con 0x7f16f006e400 2023-11-28T12:03:19.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16f5515700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16f006e400 0x7f16f015b720 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f16f00b6eb0 tx=0x7f16ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:19.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16e5ffb700 1 -- 172.21.15.57:0/3238482783 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec003cc0 con 0x7f16f006e400 2023-11-28T12:03:19.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16e5ffb700 1 -- 172.21.15.57:0/3238482783 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec01f070 con 0x7f16f006e400 2023-11-28T12:03:19.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16f7779700 1 -- 172.21.15.57:0/3238482783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f015e980 con 0x7f16f006e400 2023-11-28T12:03:19.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.654+0000 7f16e5ffb700 1 -- 172.21.15.57:0/3238482783 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec00b8a0 con 0x7f16f006e400 2023-11-28T12:03:19.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.655+0000 7f16f7779700 1 -- 172.21.15.57:0/3238482783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f006ed10 con 0x7f16f006e400 2023-11-28T12:03:19.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.655+0000 7f16f7779700 1 -- 172.21.15.57:0/3238482783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f015ec60 con 0x7f16f006e400 2023-11-28T12:03:19.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.656+0000 7f16e5ffb700 1 -- 172.21.15.57:0/3238482783 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f16ec02a020 con 0x7f16f006e400 2023-11-28T12:03:19.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.656+0000 7f16e5ffb700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f16d40682f0 0x7f16d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:19.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.656+0000 7f16e5ffb700 1 -- 172.21.15.57:0/3238482783 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f16ec087430 con 0x7f16f006e400 2023-11-28T12:03:19.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.657+0000 7f16f4d14700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f16d40682f0 0x7f16d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:19.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.657+0000 7f16f4d14700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f16d40682f0 0x7f16d406a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f16f00cff90 tx=0x7f16e0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:19.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.661+0000 7f16e5ffb700 1 -- 172.21.15.57:0/3238482783 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f16ec053c90 con 0x7f16f006e400 2023-11-28T12:03:19.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.904+0000 7f16f7779700 1 -- 172.21.15.57:0/3238482783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16f015ec60 con 0x7f16f006e400 2023-11-28T12:03:19.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.905+0000 7f16e5ffb700 1 -- 172.21.15.57:0/3238482783 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f16ec010380 con 0x7f16f006e400 2023-11-28T12:03:19.906 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:19.906 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/340292109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":340292109},{"type":"v1","addr":"172.21.15.181:6836","nonce":340292109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2442321825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2442321825},{"type":"v1","addr":"172.21.15.181:6837","nonce":2442321825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/3452242211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3452242211},{"type":"v1","addr":"172.21.15.57:6833","nonce":3452242211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:02:53.402391+0000","modified":"2023-11-28T12:02:56.540229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.57:6835/968299041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":968299041},{"type":"v1","addr":"172.21.15.57:6835","nonce":968299041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T12:03:19.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.907+0000 7f16f7779700 1 -- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f16d40682f0 msgr2=0x7f16d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:19.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.907+0000 7f16f7779700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f16d40682f0 0x7f16d406a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f16f00cff90 tx=0x7f16e0008040 comp rx=0 tx=0).stop 2023-11-28T12:03:19.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.907+0000 7f16f7779700 1 -- 172.21.15.57:0/3238482783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16f006e400 msgr2=0x7f16f015b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:19.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.907+0000 7f16f7779700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16f006e400 0x7f16f015b720 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f16f00b6eb0 tx=0x7f16ec004750 comp rx=0 tx=0).stop 2023-11-28T12:03:19.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.907+0000 7f16f7779700 1 -- 172.21.15.57:0/3238482783 shutdown_connections 2023-11-28T12:03:19.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.907+0000 7f16f7779700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f16d40682f0 0x7f16d406a7a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.907+0000 7f16f7779700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16f015e010 0x7f16f01e49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.907+0000 7f16f7779700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f16f006e400 0x7f16f015b720 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.907+0000 7f16f7779700 1 --2- 172.21.15.57:0/3238482783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f00d15f0 0x7f16f01e4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:19.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.907+0000 7f16f7779700 1 -- 172.21.15.57:0/3238482783 >> 172.21.15.57:0/3238482783 conn(0x7f16f006a270 msgr2=0x7f16f00b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:19.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.907+0000 7f16f7779700 1 -- 172.21.15.57:0/3238482783 shutdown_connections 2023-11-28T12:03:19.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:19.907+0000 7f16f7779700 1 -- 172.21.15.57:0/3238482783 wait complete. 2023-11-28T12:03:19.910 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 16 2023-11-28T12:03:19.922 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2023-11-28T12:03:20.080 INFO:teuthology.orchestra.run.smithi057.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2023-11-28T12:03:20.080 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2023-11-28T12:03:30.082 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2023-11-28T12:03:30.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.139+0000 7fbbb414f0c0 1 Processor -- start 2023-11-28T12:03:30.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.139+0000 7fbbb414f0c0 1 -- start start 2023-11-28T12:03:30.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbb414f0c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557370db2860 0x557370db1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbb414f0c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557370db2c30 0x557370df7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbb414f0c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557370df8c70 0x557370df7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbb414f0c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557370cedfe0 con 0x557370df8c70 2023-11-28T12:03:30.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbb414f0c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557370dfd110 con 0x557370db2860 2023-11-28T12:03:30.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbb414f0c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x557370de3c20 con 0x557370db2c30 2023-11-28T12:03:30.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbaaa3c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557370df8c70 0x557370df7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbaaa3c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557370df8c70 0x557370df7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35782/0 (socket says 172.21.15.57:35782) 2023-11-28T12:03:30.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbaaa3c700 1 -- 172.21.15.57:0/3722051947 learned_addr learned my addr 172.21.15.57:0/3722051947 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:30.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbaa23b700 1 --2- 172.21.15.57:0/3722051947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557370db2860 0x557370db1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbba9a3a700 1 --2- 172.21.15.57:0/3722051947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557370db2c30 0x557370df7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbaaa3c700 1 -- 172.21.15.57:0/3722051947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557370db2c30 msgr2=0x557370df7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:30.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbaaa3c700 1 --2- 172.21.15.57:0/3722051947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557370db2c30 0x557370df7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbaaa3c700 1 -- 172.21.15.57:0/3722051947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557370db2860 msgr2=0x557370db1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbaaa3c700 1 --2- 172.21.15.57:0/3722051947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557370db2860 0x557370db1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbaaa3c700 1 -- 172.21.15.57:0/3722051947 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba400a1c0 con 0x557370df8c70 2023-11-28T12:03:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.140+0000 7fbbaaa3c700 1 --2- 172.21.15.57:0/3722051947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557370df8c70 0x557370df7ed0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbba400a4b0 tx=0x7fbba400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93076e3a4d902ec5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.141+0000 7fbba8a38700 1 -- 172.21.15.57:0/3722051947 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400e7d0 con 0x557370df8c70 2023-11-28T12:03:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.141+0000 7fbba8a38700 1 -- 172.21.15.57:0/3722051947 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba400eaa0 con 0x557370df8c70 2023-11-28T12:03:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.141+0000 7fbba8a38700 1 -- 172.21.15.57:0/3722051947 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400ee30 con 0x557370df8c70 2023-11-28T12:03:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.141+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/3722051947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557370df8c70 msgr2=0x557370df7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.141+0000 7fbbb414f0c0 1 --2- 172.21.15.57:0/3722051947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557370df8c70 0x557370df7ed0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbba400a4b0 tx=0x7fbba400e000 comp rx=0 tx=0).stop 2023-11-28T12:03:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.141+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/3722051947 shutdown_connections 2023-11-28T12:03:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.141+0000 7fbbb414f0c0 1 --2- 172.21.15.57:0/3722051947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557370df8c70 0x557370df7ed0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.141+0000 7fbbb414f0c0 1 --2- 172.21.15.57:0/3722051947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557370db2860 0x557370db1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.141+0000 7fbbb414f0c0 1 --2- 172.21.15.57:0/3722051947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557370db2c30 0x557370df7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.141+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/3722051947 >> 172.21.15.57:0/3722051947 conn(0x557370dfe620 msgr2=0x557370db5500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:30.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.141+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/3722051947 shutdown_connections 2023-11-28T12:03:30.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.141+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/3722051947 wait complete. 2023-11-28T12:03:30.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.142+0000 7fbbb414f0c0 1 Processor -- start 2023-11-28T12:03:30.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.142+0000 7fbbb414f0c0 1 -- start start 2023-11-28T12:03:30.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbb414f0c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557370db2860 0x557370f24870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbb414f0c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557370db2c30 0x557370f24db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbb414f0c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557370df8c70 0x557370f252f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbb414f0c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557370e89440 con 0x557370db2860 2023-11-28T12:03:30.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbb414f0c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557370de2ff0 con 0x557370db2c30 2023-11-28T12:03:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbb414f0c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x557370e7fff0 con 0x557370df8c70 2023-11-28T12:03:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbaa23b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557370db2860 0x557370f24870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbaa23b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557370db2860 0x557370f24870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35784/0 (socket says 172.21.15.57:35784) 2023-11-28T12:03:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbaa23b700 1 -- 172.21.15.57:0/1180902517 learned_addr learned my addr 172.21.15.57:0/1180902517 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbba9a3a700 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557370db2c30 0x557370f24db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbaaa3c700 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557370df8c70 0x557370f252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbaa23b700 1 -- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557370df8c70 msgr2=0x557370f252f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbaa23b700 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557370df8c70 0x557370f252f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbaa23b700 1 -- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557370db2c30 msgr2=0x557370f24db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbaa23b700 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557370db2c30 0x557370f24db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbaa23b700 1 -- 172.21.15.57:0/1180902517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557370d204f0 con 0x557370db2860 2023-11-28T12:03:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.143+0000 7fbbaa23b700 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557370db2860 0x557370f24870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c001400 tx=0x7fbb9c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.144+0000 7fbb9b7fe700 1 -- 172.21.15.57:0/1180902517 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c016030 con 0x557370db2860 2023-11-28T12:03:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.144+0000 7fbb9b7fe700 1 -- 172.21.15.57:0/1180902517 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c005070 con 0x557370db2860 2023-11-28T12:03:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.144+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/1180902517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557370d06120 con 0x557370db2860 2023-11-28T12:03:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.144+0000 7fbb9b7fe700 1 -- 172.21.15.57:0/1180902517 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0116a0 con 0x557370db2860 2023-11-28T12:03:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.144+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/1180902517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557370cf9830 con 0x557370db2860 2023-11-28T12:03:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.145+0000 7fbb9b7fe700 1 -- 172.21.15.57:0/1180902517 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbb9c011800 con 0x557370db2860 2023-11-28T12:03:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.146+0000 7fbb9b7fe700 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb8c066c10 0x7fbb8c0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.146+0000 7fbb9b7fe700 1 -- 172.21.15.57:0/1180902517 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbb9c017070 con 0x557370db2860 2023-11-28T12:03:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.146+0000 7fbba9a3a700 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb8c066c10 0x7fbb8c0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.146+0000 7fbbb414f0c0 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x557370e802f0 0x557370f1a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.146+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/1180902517 --> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x557370f18e40 con 0x557370e802f0 2023-11-28T12:03:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.146+0000 7fbbaaa3c700 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x557370e802f0 0x557370f1a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.146+0000 7fbba9a3a700 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb8c066c10 0x7fbb8c0690c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x557370dddf00 tx=0x7fbba0005b60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.147+0000 7fbbaaa3c700 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x557370e802f0 0x557370f1a850 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.147+0000 7fbbaaa3c700 1 -- 172.21.15.57:0/1180902517 <== osd.7 v2:172.21.15.181:6808/120959 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fbba400ec10 con 0x557370e802f0 2023-11-28T12:03:30.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.147+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x557370e802f0 msgr2=0x557370f1a850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.147+0000 7fbbb414f0c0 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x557370e802f0 0x557370f1a850 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.147+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb8c066c10 msgr2=0x7fbb8c0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.147+0000 7fbbb414f0c0 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb8c066c10 0x7fbb8c0690c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x557370dddf00 tx=0x7fbba0005b60 comp rx=0 tx=0).stop 2023-11-28T12:03:30.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.147+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/1180902517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557370db2860 msgr2=0x557370f24870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.147+0000 7fbbb414f0c0 1 --2- 172.21.15.57:0/1180902517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557370db2860 0x557370f24870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c001400 tx=0x7fbb9c00c860 comp rx=0 tx=0).stop 2023-11-28T12:03:30.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.147+0000 7fbbaaa3c700 1 -- 172.21.15.57:0/1180902517 reap_dead start 2023-11-28T12:03:30.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.147+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/1180902517 shutdown_connections 2023-11-28T12:03:30.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.147+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/1180902517 >> 172.21.15.57:0/1180902517 conn(0x557370dfe620 msgr2=0x557370db5500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:30.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.147+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/1180902517 shutdown_connections 2023-11-28T12:03:30.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.148+0000 7fbbb414f0c0 1 -- 172.21.15.57:0/1180902517 wait complete. 2023-11-28T12:03:30.153 INFO:teuthology.orchestra.run.smithi057.stdout:mytag 2023-11-28T12:03:30.154 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2023-11-28T12:03:30.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.205+0000 7f680f03b0c0 1 Processor -- start 2023-11-28T12:03:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.205+0000 7f680f03b0c0 1 -- start start 2023-11-28T12:03:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f680f03b0c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556b01cc4690 0x556b01d90e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f680f03b0c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556b01d0c310 0x556b01d07980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f680f03b0c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b01d954a0 0x556b01d99420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f680f03b0c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b01c6b000 con 0x556b01d0c310 2023-11-28T12:03:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f680f03b0c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b01bfdfe0 con 0x556b01d954a0 2023-11-28T12:03:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f680f03b0c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b01d9b3a0 con 0x556b01cc4690 2023-11-28T12:03:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f6804926700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556b01d0c310 0x556b01d07980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f6804926700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556b01d0c310 0x556b01d07980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35794/0 (socket says 172.21.15.57:35794) 2023-11-28T12:03:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f6804926700 1 -- 172.21.15.57:0/3947089075 learned_addr learned my addr 172.21.15.57:0/3947089075 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f6805928700 1 --2- 172.21.15.57:0/3947089075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b01d954a0 0x556b01d99420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f6805127700 1 --2- 172.21.15.57:0/3947089075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556b01cc4690 0x556b01d90e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f6804926700 1 -- 172.21.15.57:0/3947089075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556b01cc4690 msgr2=0x556b01d90e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:30.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f6804926700 1 --2- 172.21.15.57:0/3947089075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556b01cc4690 0x556b01d90e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f6804926700 1 -- 172.21.15.57:0/3947089075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b01d954a0 msgr2=0x556b01d99420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:30.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f6804926700 1 --2- 172.21.15.57:0/3947089075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b01d954a0 0x556b01d99420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f6804926700 1 -- 172.21.15.57:0/3947089075 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc005d20 con 0x556b01d0c310 2023-11-28T12:03:30.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.206+0000 7f6805928700 1 --2- 172.21.15.57:0/3947089075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b01d954a0 0x556b01d99420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:30.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.207+0000 7f6804926700 1 --2- 172.21.15.57:0/3947089075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556b01d0c310 0x556b01d07980 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f67f8001cf0 tx=0x7f67f800b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4181b9af51858360 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.207+0000 7f67f77fe700 1 -- 172.21.15.57:0/3947089075 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800be20 con 0x556b01d0c310 2023-11-28T12:03:30.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.207+0000 7f67f77fe700 1 -- 172.21.15.57:0/3947089075 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f800e470 con 0x556b01d0c310 2023-11-28T12:03:30.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.207+0000 7f67f77fe700 1 -- 172.21.15.57:0/3947089075 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800e820 con 0x556b01d0c310 2023-11-28T12:03:30.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.207+0000 7f680f03b0c0 1 -- 172.21.15.57:0/3947089075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556b01d0c310 msgr2=0x556b01d07980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.207+0000 7f680f03b0c0 1 --2- 172.21.15.57:0/3947089075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556b01d0c310 0x556b01d07980 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f67f8001cf0 tx=0x7f67f800b5d0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.207+0000 7f680f03b0c0 1 -- 172.21.15.57:0/3947089075 shutdown_connections 2023-11-28T12:03:30.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.207+0000 7f680f03b0c0 1 --2- 172.21.15.57:0/3947089075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b01d954a0 0x556b01d99420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.207+0000 7f680f03b0c0 1 --2- 172.21.15.57:0/3947089075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556b01cc4690 0x556b01d90e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.207+0000 7f680f03b0c0 1 --2- 172.21.15.57:0/3947089075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556b01d0c310 0x556b01d07980 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.207+0000 7f680f03b0c0 1 -- 172.21.15.57:0/3947089075 >> 172.21.15.57:0/3947089075 conn(0x556b01d0b070 msgr2=0x556b01cf1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:30.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.207+0000 7f680f03b0c0 1 -- 172.21.15.57:0/3947089075 shutdown_connections 2023-11-28T12:03:30.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.207+0000 7f680f03b0c0 1 -- 172.21.15.57:0/3947089075 wait complete. 2023-11-28T12:03:30.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.208+0000 7f680f03b0c0 1 Processor -- start 2023-11-28T12:03:30.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.208+0000 7f680f03b0c0 1 -- start start 2023-11-28T12:03:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f680f03b0c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556b01cc4690 0x556b01d8f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f680f03b0c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b01d0c310 0x556b01d09bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f680f03b0c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556b01d954a0 0x556b01d0a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f680f03b0c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b01d8fa10 con 0x556b01cc4690 2023-11-28T12:03:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f680f03b0c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b01d8f540 con 0x556b01d0c310 2023-11-28T12:03:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f680f03b0c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b01d90550 con 0x556b01d954a0 2023-11-28T12:03:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f6805127700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556b01cc4690 0x556b01d8f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f6805127700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556b01cc4690 0x556b01d8f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35808/0 (socket says 172.21.15.57:35808) 2023-11-28T12:03:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f6805127700 1 -- 172.21.15.57:0/2106798935 learned_addr learned my addr 172.21.15.57:0/2106798935 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f6805928700 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556b01d954a0 0x556b01d0a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f6804926700 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b01d0c310 0x556b01d09bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f6805127700 1 -- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556b01d954a0 msgr2=0x556b01d0a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f6805127700 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556b01d954a0 0x556b01d0a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f6805127700 1 -- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b01d0c310 msgr2=0x556b01d09bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f6805127700 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b01d0c310 0x556b01d09bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f6805127700 1 -- 172.21.15.57:0/2106798935 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f0001e10 con 0x556b01cc4690 2023-11-28T12:03:30.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.209+0000 7f6804926700 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556b01d0c310 0x556b01d09bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:30.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.210+0000 7f6805127700 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556b01cc4690 0x556b01d8f000 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f67fc011000 tx=0x7f67fc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.210+0000 7f67f67fc700 1 -- 172.21.15.57:0/2106798935 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc008690 con 0x556b01cc4690 2023-11-28T12:03:30.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.210+0000 7f67f67fc700 1 -- 172.21.15.57:0/2106798935 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc00a340 con 0x556b01cc4690 2023-11-28T12:03:30.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.210+0000 7f680f03b0c0 1 -- 172.21.15.57:0/2106798935 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556b01c09830 con 0x556b01cc4690 2023-11-28T12:03:30.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.210+0000 7f67f67fc700 1 -- 172.21.15.57:0/2106798935 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc01d660 con 0x556b01cc4690 2023-11-28T12:03:30.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.210+0000 7f680f03b0c0 1 -- 172.21.15.57:0/2106798935 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b01c16360 con 0x556b01cc4690 2023-11-28T12:03:30.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.211+0000 7f67f67fc700 1 -- 172.21.15.57:0/2106798935 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f67fc02d070 con 0x556b01cc4690 2023-11-28T12:03:30.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.212+0000 7f67f67fc700 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f67e8066c10 0x7f67e80690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.212+0000 7f67f67fc700 1 -- 172.21.15.57:0/2106798935 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f67fc086160 con 0x556b01cc4690 2023-11-28T12:03:30.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.212+0000 7f680f03b0c0 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x556b01e25af0 0x556b01d9c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.212+0000 7f6804926700 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f67e8066c10 0x7f67e80690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.212+0000 7f680f03b0c0 1 -- 172.21.15.57:0/2106798935 --> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x556b01d98d80 con 0x556b01e25af0 2023-11-28T12:03:30.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.212+0000 7f6805928700 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x556b01e25af0 0x556b01d9c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f6804926700 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f67e8066c10 0x7f67e80690c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f67f8013000 tx=0x7f67f8009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f6805928700 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x556b01e25af0 0x556b01d9c970 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f6805928700 1 -- 172.21.15.57:0/2106798935 <== osd.5 v2:172.21.15.181:6824/120977 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f6800003020 con 0x556b01e25af0 2023-11-28T12:03:30.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f680f03b0c0 1 -- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x556b01e25af0 msgr2=0x556b01d9c970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f680f03b0c0 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x556b01e25af0 0x556b01d9c970 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f680f03b0c0 1 -- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f67e8066c10 msgr2=0x7f67e80690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f680f03b0c0 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f67e8066c10 0x7f67e80690c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f67f8013000 tx=0x7f67f8009000 comp rx=0 tx=0).stop 2023-11-28T12:03:30.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f680f03b0c0 1 -- 172.21.15.57:0/2106798935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556b01cc4690 msgr2=0x556b01d8f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f680f03b0c0 1 --2- 172.21.15.57:0/2106798935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556b01cc4690 0x556b01d8f000 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f67fc011000 tx=0x7f67fc002fb0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f6805928700 1 -- 172.21.15.57:0/2106798935 reap_dead start 2023-11-28T12:03:30.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f680f03b0c0 1 -- 172.21.15.57:0/2106798935 shutdown_connections 2023-11-28T12:03:30.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f680f03b0c0 1 -- 172.21.15.57:0/2106798935 >> 172.21.15.57:0/2106798935 conn(0x556b01d0b070 msgr2=0x556b01d99420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:30.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f680f03b0c0 1 -- 172.21.15.57:0/2106798935 shutdown_connections 2023-11-28T12:03:30.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.213+0000 7f680f03b0c0 1 -- 172.21.15.57:0/2106798935 wait complete. 2023-11-28T12:03:30.220 INFO:teuthology.orchestra.run.smithi057.stdout:mytag 2023-11-28T12:03:30.220 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2023-11-28T12:03:30.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.271+0000 7f8aea5990c0 1 Processor -- start 2023-11-28T12:03:30.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.271+0000 7f8aea5990c0 1 -- start start 2023-11-28T12:03:30.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.271+0000 7f8aea5990c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c98d06fa10 0x55c98d145b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.271+0000 7f8aea5990c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c98d072c80 0x55c98d147c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.271+0000 7f8aea5990c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c98d1475a0 0x55c98d0bacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.271+0000 7f8aea5990c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c98cfabfe0 con 0x55c98d072c80 2023-11-28T12:03:30.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.271+0000 7f8aea5990c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c98d0b99e0 con 0x55c98d1475a0 2023-11-28T12:03:30.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.271+0000 7f8aea5990c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c98d145820 con 0x55c98d06fa10 2023-11-28T12:03:30.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.271+0000 7f8adfe84700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c98d072c80 0x55c98d147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8adfe84700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c98d072c80 0x55c98d147c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35826/0 (socket says 172.21.15.57:35826) 2023-11-28T12:03:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8adfe84700 1 -- 172.21.15.57:0/1180067882 learned_addr learned my addr 172.21.15.57:0/1180067882 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8ae0685700 1 --2- 172.21.15.57:0/1180067882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c98d06fa10 0x55c98d145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8ae0e86700 1 --2- 172.21.15.57:0/1180067882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c98d1475a0 0x55c98d0bacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8adfe84700 1 -- 172.21.15.57:0/1180067882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c98d06fa10 msgr2=0x55c98d145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8adfe84700 1 --2- 172.21.15.57:0/1180067882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c98d06fa10 0x55c98d145b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8adfe84700 1 -- 172.21.15.57:0/1180067882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c98d1475a0 msgr2=0x55c98d0bacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8adfe84700 1 --2- 172.21.15.57:0/1180067882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c98d1475a0 0x55c98d0bacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8adfe84700 1 -- 172.21.15.57:0/1180067882 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad0005de0 con 0x55c98d072c80 2023-11-28T12:03:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8ae0685700 1 --2- 172.21.15.57:0/1180067882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c98d06fa10 0x55c98d145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8adfe84700 1 --2- 172.21.15.57:0/1180067882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c98d072c80 0x55c98d147c00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0015040 tx=0x7f8ad000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb7c9a64f534b5f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8adee82700 1 -- 172.21.15.57:0/1180067882 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0008690 con 0x55c98d072c80 2023-11-28T12:03:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8adee82700 1 -- 172.21.15.57:0/1180067882 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad0008b60 con 0x55c98d072c80 2023-11-28T12:03:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.272+0000 7f8adee82700 1 -- 172.21.15.57:0/1180067882 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad001e450 con 0x55c98d072c80 2023-11-28T12:03:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.273+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1180067882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c98d072c80 msgr2=0x55c98d147c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.273+0000 7f8aea5990c0 1 --2- 172.21.15.57:0/1180067882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c98d072c80 0x55c98d147c00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0015040 tx=0x7f8ad000d000 comp rx=0 tx=0).stop 2023-11-28T12:03:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.273+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1180067882 shutdown_connections 2023-11-28T12:03:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.273+0000 7f8aea5990c0 1 --2- 172.21.15.57:0/1180067882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c98d1475a0 0x55c98d0bacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.273+0000 7f8aea5990c0 1 --2- 172.21.15.57:0/1180067882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c98d06fa10 0x55c98d145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.273+0000 7f8aea5990c0 1 --2- 172.21.15.57:0/1180067882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c98d072c80 0x55c98d147c00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.273+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1180067882 >> 172.21.15.57:0/1180067882 conn(0x55c98d0bbda0 msgr2=0x55c98d09fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.273+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1180067882 shutdown_connections 2023-11-28T12:03:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.273+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1180067882 wait complete. 2023-11-28T12:03:30.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.273+0000 7f8aea5990c0 1 Processor -- start 2023-11-28T12:03:30.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.273+0000 7f8aea5990c0 1 -- start start 2023-11-28T12:03:30.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.274+0000 7f8aea5990c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c98d06fa10 0x55c98d1ded80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.274+0000 7f8aea5990c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c98d072c80 0x55c98d0bb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.274+0000 7f8aea5990c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c98d1475a0 0x55c98d1389c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.274+0000 7f8aea5990c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c98d09d820 con 0x55c98d1475a0 2023-11-28T12:03:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.274+0000 7f8aea5990c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c98d09d260 con 0x55c98d072c80 2023-11-28T12:03:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.274+0000 7f8aea5990c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c98d09d540 con 0x55c98d06fa10 2023-11-28T12:03:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.274+0000 7f8ae0e86700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c98d1475a0 0x55c98d1389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.274+0000 7f8ae0e86700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c98d1475a0 0x55c98d1389c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35836/0 (socket says 172.21.15.57:35836) 2023-11-28T12:03:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.274+0000 7f8ae0e86700 1 -- 172.21.15.57:0/1362722751 learned_addr learned my addr 172.21.15.57:0/1362722751 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.274+0000 7f8ae0685700 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c98d06fa10 0x55c98d1ded80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.274+0000 7f8adfe84700 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c98d072c80 0x55c98d0bb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.275+0000 7f8ae0e86700 1 -- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c98d06fa10 msgr2=0x55c98d1ded80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.275+0000 7f8ae0e86700 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c98d06fa10 0x55c98d1ded80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.275+0000 7f8ae0e86700 1 -- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c98d072c80 msgr2=0x55c98d0bb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.275+0000 7f8ae0e86700 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c98d072c80 0x55c98d0bb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.275+0000 7f8ae0e86700 1 -- 172.21.15.57:0/1362722751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c98cfde4f0 con 0x55c98d1475a0 2023-11-28T12:03:30.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.275+0000 7f8ae0e86700 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c98d1475a0 0x55c98d1389c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800f620 tx=0x7f8ad800f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.275+0000 7f8adde80700 1 -- 172.21.15.57:0/1362722751 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8013640 con 0x55c98d1475a0 2023-11-28T12:03:30.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.275+0000 7f8adde80700 1 -- 172.21.15.57:0/1362722751 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad80137a0 con 0x55c98d1475a0 2023-11-28T12:03:30.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.275+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1362722751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c98cfc4120 con 0x55c98d1475a0 2023-11-28T12:03:30.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.275+0000 7f8adde80700 1 -- 172.21.15.57:0/1362722751 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8013b50 con 0x55c98d1475a0 2023-11-28T12:03:30.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.275+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1362722751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c98cfb7830 con 0x55c98d1475a0 2023-11-28T12:03:30.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.277+0000 7f8adde80700 1 -- 172.21.15.57:0/1362722751 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ad8019030 con 0x55c98d1475a0 2023-11-28T12:03:30.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.277+0000 7f8adde80700 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8ac0066c10 0x7f8ac00690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.277+0000 7f8adde80700 1 -- 172.21.15.57:0/1362722751 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8ad8084a00 con 0x55c98d1475a0 2023-11-28T12:03:30.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.277+0000 7f8ae0685700 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8ac0066c10 0x7f8ac00690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.277+0000 7f8aea5990c0 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x55c98d073a80 0x55c98d141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.277+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1362722751 --> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c98d142430 con 0x55c98d073a80 2023-11-28T12:03:30.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.278+0000 7f8adfe84700 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x55c98d073a80 0x55c98d141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.278+0000 7f8ae0685700 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8ac0066c10 0x7f8ac00690c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80104f0 tx=0x7f8ac80037d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.278+0000 7f8adfe84700 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x55c98d073a80 0x55c98d141ef0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.278+0000 7f8adfe84700 1 -- 172.21.15.57:0/1362722751 <== osd.5 v2:172.21.15.181:6824/120977 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f8ad0019040 con 0x55c98d073a80 2023-11-28T12:03:30.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.278+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x55c98d073a80 msgr2=0x55c98d141ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.278+0000 7f8aea5990c0 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x55c98d073a80 0x55c98d141ef0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.278+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8ac0066c10 msgr2=0x7f8ac00690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.278+0000 7f8aea5990c0 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8ac0066c10 0x7f8ac00690c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80104f0 tx=0x7f8ac80037d0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.278+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1362722751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c98d1475a0 msgr2=0x55c98d1389c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.278+0000 7f8aea5990c0 1 --2- 172.21.15.57:0/1362722751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c98d1475a0 0x55c98d1389c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800f620 tx=0x7f8ad800f930 comp rx=0 tx=0).stop 2023-11-28T12:03:30.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.279+0000 7f8ae0e86700 1 -- 172.21.15.57:0/1362722751 reap_dead start 2023-11-28T12:03:30.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.279+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1362722751 shutdown_connections 2023-11-28T12:03:30.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.279+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1362722751 >> 172.21.15.57:0/1362722751 conn(0x55c98d0bbda0 msgr2=0x55c98d146710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:30.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.279+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1362722751 shutdown_connections 2023-11-28T12:03:30.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.279+0000 7f8aea5990c0 1 -- 172.21.15.57:0/1362722751 wait complete. 2023-11-28T12:03:30.287 INFO:teuthology.orchestra.run.smithi057.stdout:mytag 2023-11-28T12:03:30.287 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2023-11-28T12:03:30.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.337+0000 7f977da080c0 1 Processor -- start 2023-11-28T12:03:30.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.337+0000 7f977da080c0 1 -- start start 2023-11-28T12:03:30.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f977da080c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c67d4d2f0 0x555c67d7ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f977da080c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c67d97310 0x555c67e24420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f977da080c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555c67e204a0 0x555c67e21c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f977da080c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c67d7a7f0 con 0x555c67d4d2f0 2023-11-28T12:03:30.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f977da080c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c67e23ab0 con 0x555c67d97310 2023-11-28T12:03:30.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f977da080c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c67e1c720 con 0x555c67e204a0 2023-11-28T12:03:30.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f9773af4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c67d4d2f0 0x555c67d7ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f9773af4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c67d4d2f0 0x555c67d7ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35840/0 (socket says 172.21.15.57:35840) 2023-11-28T12:03:30.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f9773af4700 1 -- 172.21.15.57:0/2066828039 learned_addr learned my addr 172.21.15.57:0/2066828039 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:30.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f97742f5700 1 --2- 172.21.15.57:0/2066828039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555c67e204a0 0x555c67e21c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f97732f3700 1 --2- 172.21.15.57:0/2066828039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c67d97310 0x555c67e24420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f9773af4700 1 -- 172.21.15.57:0/2066828039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555c67e204a0 msgr2=0x555c67e21c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:30.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f9773af4700 1 --2- 172.21.15.57:0/2066828039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555c67e204a0 0x555c67e21c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f9773af4700 1 -- 172.21.15.57:0/2066828039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c67d97310 msgr2=0x555c67e24420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:30.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f9773af4700 1 --2- 172.21.15.57:0/2066828039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c67d97310 0x555c67e24420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f9773af4700 1 -- 172.21.15.57:0/2066828039 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9764005de0 con 0x555c67d4d2f0 2023-11-28T12:03:30.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f97732f3700 1 --2- 172.21.15.57:0/2066828039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c67d97310 0x555c67e24420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:30.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f97742f5700 1 --2- 172.21.15.57:0/2066828039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555c67e204a0 0x555c67e21c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:30.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f9773af4700 1 --2- 172.21.15.57:0/2066828039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c67d4d2f0 0x555c67d7ccc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f97680097e0 tx=0x7f9768005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab624598d4082637 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f97722f1700 1 -- 172.21.15.57:0/2066828039 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976800f040 con 0x555c67d4d2f0 2023-11-28T12:03:30.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.338+0000 7f97722f1700 1 -- 172.21.15.57:0/2066828039 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9768008950 con 0x555c67d4d2f0 2023-11-28T12:03:30.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.339+0000 7f97722f1700 1 -- 172.21.15.57:0/2066828039 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768008d00 con 0x555c67d4d2f0 2023-11-28T12:03:30.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.339+0000 7f977da080c0 1 -- 172.21.15.57:0/2066828039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c67d4d2f0 msgr2=0x555c67d7ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.339+0000 7f977da080c0 1 --2- 172.21.15.57:0/2066828039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c67d4d2f0 0x555c67d7ccc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f97680097e0 tx=0x7f9768005a00 comp rx=0 tx=0).stop 2023-11-28T12:03:30.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.339+0000 7f977da080c0 1 -- 172.21.15.57:0/2066828039 shutdown_connections 2023-11-28T12:03:30.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.339+0000 7f977da080c0 1 --2- 172.21.15.57:0/2066828039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555c67e204a0 0x555c67e21c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.339+0000 7f977da080c0 1 --2- 172.21.15.57:0/2066828039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c67d4d2f0 0x555c67d7ccc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.339+0000 7f977da080c0 1 --2- 172.21.15.57:0/2066828039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c67d97310 0x555c67e24420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.339+0000 7f977da080c0 1 -- 172.21.15.57:0/2066828039 >> 172.21.15.57:0/2066828039 conn(0x555c67d96070 msgr2=0x555c67e25df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:30.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.339+0000 7f977da080c0 1 -- 172.21.15.57:0/2066828039 shutdown_connections 2023-11-28T12:03:30.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.339+0000 7f977da080c0 1 -- 172.21.15.57:0/2066828039 wait complete. 2023-11-28T12:03:30.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.340+0000 7f977da080c0 1 Processor -- start 2023-11-28T12:03:30.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.340+0000 7f977da080c0 1 -- start start 2023-11-28T12:03:30.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.340+0000 7f977da080c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c67d4d2f0 0x555c67d7e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f977da080c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c67d97310 0x555c67e24420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f977da080c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555c67e204a0 0x555c67e24960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f977da080c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c67d7ed70 con 0x555c67d97310 2023-11-28T12:03:30.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f977da080c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c67d51990 con 0x555c67d4d2f0 2023-11-28T12:03:30.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f977da080c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c67d996b0 con 0x555c67e204a0 2023-11-28T12:03:30.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f97732f3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c67d97310 0x555c67e24420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f97732f3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c67d97310 0x555c67e24420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35848/0 (socket says 172.21.15.57:35848) 2023-11-28T12:03:30.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f97732f3700 1 -- 172.21.15.57:0/136381825 learned_addr learned my addr 172.21.15.57:0/136381825 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:30.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f9773af4700 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c67d4d2f0 0x555c67d7e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f97742f5700 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555c67e204a0 0x555c67e24960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f97732f3700 1 -- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555c67e204a0 msgr2=0x555c67e24960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f97732f3700 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555c67e204a0 0x555c67e24960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f97732f3700 1 -- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c67d4d2f0 msgr2=0x555c67d7e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f97732f3700 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c67d4d2f0 0x555c67d7e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f97732f3700 1 -- 172.21.15.57:0/136381825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9768009410 con 0x555c67d97310 2023-11-28T12:03:30.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f9773af4700 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555c67d4d2f0 0x555c67d7e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:30.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f97732f3700 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c67d97310 0x555c67e24420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9764011000 tx=0x7f9764002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f97712ef700 1 -- 172.21.15.57:0/136381825 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764005310 con 0x555c67d97310 2023-11-28T12:03:30.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f97712ef700 1 -- 172.21.15.57:0/136381825 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976401d630 con 0x555c67d97310 2023-11-28T12:03:30.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f977da080c0 1 -- 172.21.15.57:0/136381825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555c67cbb4f0 con 0x555c67d97310 2023-11-28T12:03:30.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f97712ef700 1 -- 172.21.15.57:0/136381825 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976401da20 con 0x555c67d97310 2023-11-28T12:03:30.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.341+0000 7f977da080c0 1 -- 172.21.15.57:0/136381825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c67ca1120 con 0x555c67d97310 2023-11-28T12:03:30.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.343+0000 7f97712ef700 1 -- 172.21.15.57:0/136381825 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f976402d070 con 0x555c67d97310 2023-11-28T12:03:30.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.343+0000 7f97712ef700 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9760066c10 0x7f97600690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.343+0000 7f97712ef700 1 -- 172.21.15.57:0/136381825 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9764085f70 con 0x555c67d97310 2023-11-28T12:03:30.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.343+0000 7f9773af4700 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9760066c10 0x7f97600690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.343+0000 7f977da080c0 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x555c67d93640 0x555c67d93a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.343+0000 7f977da080c0 1 -- 172.21.15.57:0/136381825 --> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x555c67d93f70 con 0x555c67d93640 2023-11-28T12:03:30.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.344+0000 7f97742f5700 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x555c67d93640 0x555c67d93a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.344+0000 7f9773af4700 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9760066c10 0x7f97600690c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f97680097e0 tx=0x7f9768012000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.344+0000 7f97742f5700 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x555c67d93640 0x555c67d93a30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.344+0000 7f97742f5700 1 -- 172.21.15.57:0/136381825 <== osd.5 v2:172.21.15.181:6824/120977 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f976c00d770 con 0x555c67d93640 2023-11-28T12:03:30.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.344+0000 7f977da080c0 1 -- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x555c67d93640 msgr2=0x555c67d93a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.344+0000 7f977da080c0 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x555c67d93640 0x555c67d93a30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.345+0000 7f977da080c0 1 -- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9760066c10 msgr2=0x7f97600690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.345+0000 7f977da080c0 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9760066c10 0x7f97600690c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f97680097e0 tx=0x7f9768012000 comp rx=0 tx=0).stop 2023-11-28T12:03:30.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.345+0000 7f977da080c0 1 -- 172.21.15.57:0/136381825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c67d97310 msgr2=0x555c67e24420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.345+0000 7f977da080c0 1 --2- 172.21.15.57:0/136381825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c67d97310 0x555c67e24420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9764011000 tx=0x7f9764002fb0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.345+0000 7f97742f5700 1 -- 172.21.15.57:0/136381825 reap_dead start 2023-11-28T12:03:30.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.345+0000 7f977da080c0 1 -- 172.21.15.57:0/136381825 shutdown_connections 2023-11-28T12:03:30.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.345+0000 7f977da080c0 1 -- 172.21.15.57:0/136381825 >> 172.21.15.57:0/136381825 conn(0x555c67d96070 msgr2=0x555c67e25980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:30.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.345+0000 7f977da080c0 1 -- 172.21.15.57:0/136381825 shutdown_connections 2023-11-28T12:03:30.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.345+0000 7f977da080c0 1 -- 172.21.15.57:0/136381825 wait complete. 2023-11-28T12:03:30.353 INFO:teuthology.orchestra.run.smithi057.stdout:mytag 2023-11-28T12:03:30.354 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2023-11-28T12:03:30.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.405+0000 7fbde04a50c0 1 Processor -- start 2023-11-28T12:03:30.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.405+0000 7fbde04a50c0 1 -- start start 2023-11-28T12:03:30.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.405+0000 7fbde04a50c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622f539db90 0x5622f535c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.405+0000 7fbde04a50c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5622f53a3e70 0x5622f5422250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.405+0000 7fbde04a50c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5622f5435630 0x5622f5422790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.405+0000 7fbde04a50c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f5384de0 con 0x5622f539db90 2023-11-28T12:03:30.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.405+0000 7fbde04a50c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f53850c0 con 0x5622f53a3e70 2023-11-28T12:03:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.405+0000 7fbde04a50c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622f53853a0 con 0x5622f5435630 2023-11-28T12:03:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.405+0000 7fbdd6591700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622f539db90 0x5622f535c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd6591700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622f539db90 0x5622f535c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35870/0 (socket says 172.21.15.57:35870) 2023-11-28T12:03:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd6591700 1 -- 172.21.15.57:0/1672807791 learned_addr learned my addr 172.21.15.57:0/1672807791 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd6d92700 1 --2- 172.21.15.57:0/1672807791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5622f5435630 0x5622f5422790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd5d90700 1 --2- 172.21.15.57:0/1672807791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5622f53a3e70 0x5622f5422250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd6591700 1 -- 172.21.15.57:0/1672807791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5622f5435630 msgr2=0x5622f5422790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd6591700 1 --2- 172.21.15.57:0/1672807791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5622f5435630 0x5622f5422790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd6591700 1 -- 172.21.15.57:0/1672807791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5622f53a3e70 msgr2=0x5622f5422250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd6591700 1 --2- 172.21.15.57:0/1672807791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5622f53a3e70 0x5622f5422250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd6591700 1 -- 172.21.15.57:0/1672807791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc005de0 con 0x5622f539db90 2023-11-28T12:03:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd5d90700 1 --2- 172.21.15.57:0/1672807791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5622f53a3e70 0x5622f5422250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd6d92700 1 --2- 172.21.15.57:0/1672807791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5622f5435630 0x5622f5422790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd6591700 1 --2- 172.21.15.57:0/1672807791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622f539db90 0x5622f535c020 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80097e0 tx=0x7fbdc8005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e38f297c09c03b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd4d8e700 1 -- 172.21.15.57:0/1672807791 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc800f040 con 0x5622f539db90 2023-11-28T12:03:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd4d8e700 1 -- 172.21.15.57:0/1672807791 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc80088f0 con 0x5622f539db90 2023-11-28T12:03:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.406+0000 7fbdd4d8e700 1 -- 172.21.15.57:0/1672807791 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8008ca0 con 0x5622f539db90 2023-11-28T12:03:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.407+0000 7fbde04a50c0 1 -- 172.21.15.57:0/1672807791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622f539db90 msgr2=0x5622f535c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.407+0000 7fbde04a50c0 1 --2- 172.21.15.57:0/1672807791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622f539db90 0x5622f535c020 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80097e0 tx=0x7fbdc8005a00 comp rx=0 tx=0).stop 2023-11-28T12:03:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.407+0000 7fbde04a50c0 1 -- 172.21.15.57:0/1672807791 shutdown_connections 2023-11-28T12:03:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.407+0000 7fbde04a50c0 1 --2- 172.21.15.57:0/1672807791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5622f5435630 0x5622f5422790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.407+0000 7fbde04a50c0 1 --2- 172.21.15.57:0/1672807791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622f539db90 0x5622f535c020 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.407+0000 7fbde04a50c0 1 --2- 172.21.15.57:0/1672807791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5622f53a3e70 0x5622f5422250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.407+0000 7fbde04a50c0 1 -- 172.21.15.57:0/1672807791 >> 172.21.15.57:0/1672807791 conn(0x5622f53a4620 msgr2=0x5622f5384410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:30.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.407+0000 7fbde04a50c0 1 -- 172.21.15.57:0/1672807791 shutdown_connections 2023-11-28T12:03:30.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.407+0000 7fbde04a50c0 1 -- 172.21.15.57:0/1672807791 wait complete. 2023-11-28T12:03:30.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.408+0000 7fbde04a50c0 1 Processor -- start 2023-11-28T12:03:30.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.408+0000 7fbde04a50c0 1 -- start start 2023-11-28T12:03:30.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:03:30.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbde04a50c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5622f539db90 0x5622f5429fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbde04a50c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5622f53a3e70 0x5622f5388dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbde04a50c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622f5435630 0x5622f5389300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbde04a50c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f5385fb0 con 0x5622f5435630 2023-11-28T12:03:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbde04a50c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622f5431fc0 con 0x5622f539db90 2023-11-28T12:03:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbde04a50c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622f542ae30 con 0x5622f53a3e70 2023-11-28T12:03:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbdd6d92700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622f5435630 0x5622f5389300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbdd6d92700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622f5435630 0x5622f5389300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35886/0 (socket says 172.21.15.57:35886) 2023-11-28T12:03:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbdd6d92700 1 -- 172.21.15.57:0/721597310 learned_addr learned my addr 172.21.15.57:0/721597310 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:30.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbdd5d90700 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5622f53a3e70 0x5622f5388dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbdd6591700 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5622f539db90 0x5622f5429fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbdd6d92700 1 -- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5622f53a3e70 msgr2=0x5622f5388dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbdd6d92700 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5622f53a3e70 0x5622f5388dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbdd6d92700 1 -- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5622f539db90 msgr2=0x5622f5429fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbdd6d92700 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5622f539db90 0x5622f5429fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbdd6d92700 1 -- 172.21.15.57:0/721597310 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc8009410 con 0x5622f5435630 2023-11-28T12:03:30.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.409+0000 7fbdd6d92700 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622f5435630 0x5622f5389300 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000f840 tx=0x7fbdd000fb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.410+0000 7fbdc77fe700 1 -- 172.21.15.57:0/721597310 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0013640 con 0x5622f5435630 2023-11-28T12:03:30.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.410+0000 7fbdc77fe700 1 -- 172.21.15.57:0/721597310 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd00137a0 con 0x5622f5435630 2023-11-28T12:03:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.410+0000 7fbde04a50c0 1 -- 172.21.15.57:0/721597310 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5622f52c64f0 con 0x5622f5435630 2023-11-28T12:03:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.410+0000 7fbdc77fe700 1 -- 172.21.15.57:0/721597310 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0013b50 con 0x5622f5435630 2023-11-28T12:03:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.410+0000 7fbde04a50c0 1 -- 172.21.15.57:0/721597310 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622f52ac120 con 0x5622f5435630 2023-11-28T12:03:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.411+0000 7fbdc77fe700 1 -- 172.21.15.57:0/721597310 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbdd0019030 con 0x5622f5435630 2023-11-28T12:03:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.411+0000 7fbdc77fe700 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbdb8066c10 0x7fbdb80690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.412+0000 7fbdc77fe700 1 -- 172.21.15.57:0/721597310 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbdd0084b00 con 0x5622f5435630 2023-11-28T12:03:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.412+0000 7fbdd6591700 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbdb8066c10 0x7fbdb80690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.412+0000 7fbde04a50c0 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x5622f542a4e0 0x5622f542d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.412+0000 7fbde04a50c0 1 -- 172.21.15.57:0/721597310 --> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5622f5432650 con 0x5622f542a4e0 2023-11-28T12:03:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.412+0000 7fbdd5d90700 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x5622f542a4e0 0x5622f542d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:30.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.412+0000 7fbdd6591700 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbdb8066c10 0x7fbdb80690c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80097e0 tx=0x7fbdc8012000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.413+0000 7fbdd5d90700 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x5622f542a4e0 0x5622f542d0e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:30.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.413+0000 7fbdd5d90700 1 -- 172.21.15.57:0/721597310 <== osd.7 v2:172.21.15.181:6808/120959 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fbdcc019040 con 0x5622f542a4e0 2023-11-28T12:03:30.422 INFO:teuthology.orchestra.run.smithi057.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2023-11-28T12:03:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.413+0000 7fbde04a50c0 1 -- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x5622f542a4e0 msgr2=0x5622f542d0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.413+0000 7fbde04a50c0 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x5622f542a4e0 0x5622f542d0e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.413+0000 7fbde04a50c0 1 -- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbdb8066c10 msgr2=0x7fbdb80690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.413+0000 7fbde04a50c0 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbdb8066c10 0x7fbdb80690c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80097e0 tx=0x7fbdc8012000 comp rx=0 tx=0).stop 2023-11-28T12:03:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.413+0000 7fbde04a50c0 1 -- 172.21.15.57:0/721597310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622f5435630 msgr2=0x5622f5389300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.413+0000 7fbde04a50c0 1 --2- 172.21.15.57:0/721597310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622f5435630 0x5622f5389300 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000f840 tx=0x7fbdd000fb50 comp rx=0 tx=0).stop 2023-11-28T12:03:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.413+0000 7fbdd6d92700 1 -- 172.21.15.57:0/721597310 reap_dead start 2023-11-28T12:03:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.413+0000 7fbde04a50c0 1 -- 172.21.15.57:0/721597310 shutdown_connections 2023-11-28T12:03:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.413+0000 7fbde04a50c0 1 -- 172.21.15.57:0/721597310 >> 172.21.15.57:0/721597310 conn(0x5622f53a4620 msgr2=0x5622f54294c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.413+0000 7fbde04a50c0 1 -- 172.21.15.57:0/721597310 shutdown_connections 2023-11-28T12:03:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:30.413+0000 7fbde04a50c0 1 -- 172.21.15.57:0/721597310 wait complete. 2023-11-28T12:03:30.424 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T12:03:30.425 DEBUG:teuthology.orchestra.run.smithi057:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T12:03:30.471 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T12:03:30.522 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T12:03:30.523 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:03:30.523 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:03:30.546 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16254224k,nr_inodes=4063556,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15749 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3254388k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T12:03:30.546 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:03:30.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:30.547 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T12:03:30.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:03:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-28T12:03:30.611 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:03:30.611 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:03:30.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251064k,nr_inodes=4062766,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14954 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253752k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T12:03:30.633 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:03:30.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:30.634 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T12:03:30.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:03:30.694 INFO:teuthology.orchestra.run.smithi181.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T12:03:30.695 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:03:30.695 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:03:30.718 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16254224k,nr_inodes=4063556,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15749 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3254388k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T12:03:30.718 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:03:30.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:30.719 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T12:03:30.781 INFO:teuthology.orchestra.run.smithi057.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T12:03:30.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:03:30.783 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:03:30.783 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:03:30.804 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251064k,nr_inodes=4062766,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14954 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253752k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T12:03:30.805 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:03:30.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:30.805 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T12:03:30.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:03:30.865 INFO:teuthology.orchestra.run.smithi181.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T12:03:30.866 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:03:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.007+0000 7f49377ad700 1 Processor -- start 2023-11-28T12:03:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.007+0000 7f49377ad700 1 -- start start 2023-11-28T12:03:31.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.009+0000 7f49377ad700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f49300ced70 0x7f49300b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.009+0000 7f49377ad700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49300b4fc0 con 0x7f49300ce1e0 2023-11-28T12:03:31.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.009+0000 7f49377ad700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49300b05e0 con 0x7f493006feb0 2023-11-28T12:03:31.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.009+0000 7f49377ad700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4930159c30 con 0x7f49300ced70 2023-11-28T12:03:31.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.009+0000 7f4935549700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f49300ced70 0x7f49300b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.009+0000 7f4935d4a700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f49300ce1e0 0x7f493006a980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47344/0 (socket says 172.21.15.57:47344) 2023-11-28T12:03:31.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.009+0000 7f4935d4a700 1 -- 172.21.15.57:0/534464221 learned_addr learned my addr 172.21.15.57:0/534464221 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:31.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.009+0000 7f4935549700 1 -- 172.21.15.57:0/534464221 >> v1:172.21.15.181:6789/0 conn(0x7f493006feb0 legacy=0x7f493006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.009+0000 7f4935549700 1 -- 172.21.15.57:0/534464221 >> v1:172.21.15.57:6789/0 conn(0x7f49300ce1e0 legacy=0x7f493006a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.009+0000 7f4935549700 1 -- 172.21.15.57:0/534464221 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49300b08c0 con 0x7f49300ced70 2023-11-28T12:03:31.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.009+0000 7f4927fff700 1 -- 172.21.15.57:0/534464221 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325483397 0 0) 0x7f49300b4fc0 con 0x7f49300ce1e0 2023-11-28T12:03:31.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.009+0000 7f4935549700 1 --2- 172.21.15.57:0/534464221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49300ced70 0x7f49300b43f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4920009840 tx=0x7f4920009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988097a2e2c6a617 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:31.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.009+0000 7f4927fff700 1 -- 172.21.15.57:0/534464221 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920011830 con 0x7f49300ced70 2023-11-28T12:03:31.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.010+0000 7f4927fff700 1 -- 172.21.15.57:0/534464221 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4920011990 con 0x7f49300ced70 2023-11-28T12:03:31.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.010+0000 7f4927fff700 1 -- 172.21.15.57:0/534464221 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920011d20 con 0x7f49300ced70 2023-11-28T12:03:31.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.010+0000 7f49377ad700 1 -- 172.21.15.57:0/534464221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49300ced70 msgr2=0x7f49300b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.010+0000 7f49377ad700 1 --2- 172.21.15.57:0/534464221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49300ced70 0x7f49300b43f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4920009840 tx=0x7f4920009b50 comp rx=0 tx=0).stop 2023-11-28T12:03:31.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.010+0000 7f49377ad700 1 -- 172.21.15.57:0/534464221 shutdown_connections 2023-11-28T12:03:31.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.010+0000 7f49377ad700 1 --2- 172.21.15.57:0/534464221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49300ced70 0x7f49300b43f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.010+0000 7f49377ad700 1 -- 172.21.15.57:0/534464221 >> 172.21.15.57:0/534464221 conn(0x7f49300b2fc0 msgr2=0x7f4930071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:31.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.010+0000 7f49377ad700 1 -- 172.21.15.57:0/534464221 shutdown_connections 2023-11-28T12:03:31.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.010+0000 7f49377ad700 1 -- 172.21.15.57:0/534464221 wait complete. 2023-11-28T12:03:31.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.011+0000 7f49377ad700 1 Processor -- start 2023-11-28T12:03:31.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.012+0000 7f49377ad700 1 -- start start 2023-11-28T12:03:31.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f49377ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f493006feb0 0x7f49300980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f49377ad700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49300ce1e0 0x7f49301520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f49377ad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49300ced70 0x7f49300cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f49377ad700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4930152ad0 con 0x7f49300ce1e0 2023-11-28T12:03:31.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f49377ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4930152db0 con 0x7f493006feb0 2023-11-28T12:03:31.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f49377ad700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4930153090 con 0x7f49300ced70 2023-11-28T12:03:31.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4934d48700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49300ce1e0 0x7f49301520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4934d48700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49300ce1e0 0x7f49301520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35898/0 (socket says 172.21.15.57:35898) 2023-11-28T12:03:31.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4934d48700 1 -- 172.21.15.57:0/2860336416 learned_addr learned my addr 172.21.15.57:0/2860336416 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:31.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4935549700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f493006feb0 0x7f49300980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4935d4a700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49300ced70 0x7f49300cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4934d48700 1 -- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49300ced70 msgr2=0x7f49300cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4934d48700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49300ced70 0x7f49300cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4934d48700 1 -- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f493006feb0 msgr2=0x7f49300980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4934d48700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f493006feb0 0x7f49300980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4934d48700 1 -- 172.21.15.57:0/2860336416 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49200094a0 con 0x7f49300ce1e0 2023-11-28T12:03:31.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4934d48700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49300ce1e0 0x7f49301520a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f492c00c480 tx=0x7f492c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:31.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4925ffb700 1 -- 172.21.15.57:0/2860336416 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c0052d0 con 0x7f49300ce1e0 2023-11-28T12:03:31.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4925ffb700 1 -- 172.21.15.57:0/2860336416 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c00ce90 con 0x7f49300ce1e0 2023-11-28T12:03:31.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f4925ffb700 1 -- 172.21.15.57:0/2860336416 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c003680 con 0x7f49300ce1e0 2023-11-28T12:03:31.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.013+0000 7f49377ad700 1 -- 172.21.15.57:0/2860336416 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49300b5260 con 0x7f49300ce1e0 2023-11-28T12:03:31.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.014+0000 7f49377ad700 1 -- 172.21.15.57:0/2860336416 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49300b0ba0 con 0x7f49300ce1e0 2023-11-28T12:03:31.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.015+0000 7f49377ad700 1 -- 172.21.15.57:0/2860336416 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4930072420 con 0x7f49300ce1e0 2023-11-28T12:03:31.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.015+0000 7f4925ffb700 1 -- 172.21.15.57:0/2860336416 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f492c019030 con 0x7f49300ce1e0 2023-11-28T12:03:31.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.015+0000 7f4925ffb700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f49140682f0 0x7f491406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.015+0000 7f4925ffb700 1 -- 172.21.15.57:0/2860336416 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f492c084eb0 con 0x7f49300ce1e0 2023-11-28T12:03:31.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.016+0000 7f4935549700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f49140682f0 0x7f491406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.016+0000 7f4935549700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f49140682f0 0x7f491406a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f49200090b0 tx=0x7f492001c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:31.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.020+0000 7f4925ffb700 1 -- 172.21.15.57:0/2860336416 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f492c051710 con 0x7f49300ce1e0 2023-11-28T12:03:31.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.273+0000 7f49377ad700 1 -- 172.21.15.57:0/2860336416 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4930067370 con 0x7f49300ce1e0 2023-11-28T12:03:31.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.274+0000 7f4925ffb700 1 -- 172.21.15.57:0/2860336416 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f492c017090 con 0x7f49300ce1e0 2023-11-28T12:03:31.275 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:31.275 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/340292109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":340292109},{"type":"v1","addr":"172.21.15.181:6836","nonce":340292109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2442321825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2442321825},{"type":"v1","addr":"172.21.15.181:6837","nonce":2442321825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/3452242211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3452242211},{"type":"v1","addr":"172.21.15.57:6833","nonce":3452242211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:02:53.402391+0000","modified":"2023-11-28T12:02:56.540229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.57:6835/968299041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":968299041},{"type":"v1","addr":"172.21.15.57:6835","nonce":968299041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T12:03:31.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.276+0000 7f49377ad700 1 -- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f49140682f0 msgr2=0x7f491406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.277+0000 7f49377ad700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f49140682f0 0x7f491406a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f49200090b0 tx=0x7f492001c040 comp rx=0 tx=0).stop 2023-11-28T12:03:31.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.277+0000 7f49377ad700 1 -- 172.21.15.57:0/2860336416 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49300ce1e0 msgr2=0x7f49301520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.277+0000 7f49377ad700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49300ce1e0 0x7f49301520a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f492c00c480 tx=0x7f492c00c790 comp rx=0 tx=0).stop 2023-11-28T12:03:31.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.277+0000 7f49377ad700 1 -- 172.21.15.57:0/2860336416 shutdown_connections 2023-11-28T12:03:31.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.277+0000 7f49377ad700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f49140682f0 0x7f491406a7a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.277+0000 7f49377ad700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49300ced70 0x7f49300cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.277+0000 7f49377ad700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f493006feb0 0x7f49300980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.277+0000 7f49377ad700 1 --2- 172.21.15.57:0/2860336416 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49300ce1e0 0x7f49301520a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.277+0000 7f49377ad700 1 -- 172.21.15.57:0/2860336416 >> 172.21.15.57:0/2860336416 conn(0x7f49300b2fc0 msgr2=0x7f49301584a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:31.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.277+0000 7f49377ad700 1 -- 172.21.15.57:0/2860336416 shutdown_connections 2023-11-28T12:03:31.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.277+0000 7f49377ad700 1 -- 172.21.15.57:0/2860336416 wait complete. 2023-11-28T12:03:31.279 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 16 2023-11-28T12:03:31.291 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:03:31.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.425+0000 7f45b1f00700 1 Processor -- start 2023-11-28T12:03:31.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.426+0000 7f45b1f00700 1 -- start start 2023-11-28T12:03:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.426+0000 7f45b1f00700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac0b4660 0x7f45ac071f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.426+0000 7f45b1f00700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f45ac0b4a30 0x7f45ac0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.426+0000 7f45b1f00700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac0cfb00 con 0x7f45ac0cff90 2023-11-28T12:03:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.426+0000 7f45b1f00700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45ac15aa20 con 0x7f45ac0b4a30 2023-11-28T12:03:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.426+0000 7f45b1f00700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac0cf4d0 con 0x7f45ac0b4660 2023-11-28T12:03:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45abfff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f45ac0b4a30 0x7f45ac0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45aaffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f45ac0cff90 0x7f45ac06b590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47356/0 (socket says 172.21.15.57:47356) 2023-11-28T12:03:31.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45aaffd700 1 -- 172.21.15.57:0/1855542185 learned_addr learned my addr 172.21.15.57:0/1855542185 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:31.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45ab7fe700 1 --2- 172.21.15.57:0/1855542185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac0b4660 0x7f45ac071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45abfff700 1 -- 172.21.15.57:0/1855542185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac0b4660 msgr2=0x7f45ac071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:31.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45abfff700 1 --2- 172.21.15.57:0/1855542185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac0b4660 0x7f45ac071f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45abfff700 1 -- 172.21.15.57:0/1855542185 >> v1:172.21.15.57:6789/0 conn(0x7f45ac0cff90 legacy=0x7f45ac06b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45abfff700 1 -- 172.21.15.57:0/1855542185 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac072550 con 0x7f45ac0b4a30 2023-11-28T12:03:31.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45aa7fc700 1 -- 172.21.15.57:0/1855542185 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843739780 0 0) 0x7f45ac0cfb00 con 0x7f45ac0cff90 2023-11-28T12:03:31.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45ab7fe700 1 --2- 172.21.15.57:0/1855542185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac0b4660 0x7f45ac071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:31.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45abfff700 1 --2- 172.21.15.57:0/1855542185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45ac0b4a30 0x7f45ac0b4eb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009d90 tx=0x7f45a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8656d7b3457d4815 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:31.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45aa7fc700 1 -- 172.21.15.57:0/1855542185 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a400e070 con 0x7f45ac0b4a30 2023-11-28T12:03:31.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45aa7fc700 1 -- 172.21.15.57:0/1855542185 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a400bbd0 con 0x7f45ac0b4a30 2023-11-28T12:03:31.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.427+0000 7f45aa7fc700 1 -- 172.21.15.57:0/1855542185 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4012430 con 0x7f45ac0b4a30 2023-11-28T12:03:31.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.428+0000 7f45b1f00700 1 -- 172.21.15.57:0/1855542185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45ac0b4a30 msgr2=0x7f45ac0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.428+0000 7f45b1f00700 1 --2- 172.21.15.57:0/1855542185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45ac0b4a30 0x7f45ac0b4eb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009d90 tx=0x7f45a400b330 comp rx=0 tx=0).stop 2023-11-28T12:03:31.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.428+0000 7f45b1f00700 1 -- 172.21.15.57:0/1855542185 shutdown_connections 2023-11-28T12:03:31.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.428+0000 7f45b1f00700 1 --2- 172.21.15.57:0/1855542185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45ac0b4a30 0x7f45ac0b4eb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.428+0000 7f45b1f00700 1 --2- 172.21.15.57:0/1855542185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac0b4660 0x7f45ac071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.428+0000 7f45b1f00700 1 -- 172.21.15.57:0/1855542185 >> 172.21.15.57:0/1855542185 conn(0x7f45ac0b2fc0 msgr2=0x7f45ac071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:31.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.428+0000 7f45b1f00700 1 -- 172.21.15.57:0/1855542185 shutdown_connections 2023-11-28T12:03:31.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.428+0000 7f45b1f00700 1 -- 172.21.15.57:0/1855542185 wait complete. 2023-11-28T12:03:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.430+0000 7f45b1f00700 1 Processor -- start 2023-11-28T12:03:31.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.430+0000 7f45b1f00700 1 -- start start 2023-11-28T12:03:31.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45b1f00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45ac0b4660 0x7f45ac154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45b1f00700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac0b4a30 0x7f45ac1efaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45b1f00700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac0cff90 0x7f45ac0c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45b1f00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac1f2e10 con 0x7f45ac0b4660 2023-11-28T12:03:31.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45b1f00700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac1f0030 con 0x7f45ac0cff90 2023-11-28T12:03:31.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45b1f00700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac0caa30 con 0x7f45ac0b4a30 2023-11-28T12:03:31.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45ab7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45ac0b4660 0x7f45ac154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45ab7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45ac0b4660 0x7f45ac154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35904/0 (socket says 172.21.15.57:35904) 2023-11-28T12:03:31.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45ab7fe700 1 -- 172.21.15.57:0/4219647412 learned_addr learned my addr 172.21.15.57:0/4219647412 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:31.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45aaffd700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac0b4a30 0x7f45ac1efaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45abfff700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac0cff90 0x7f45ac0c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45ab7fe700 1 -- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac0b4a30 msgr2=0x7f45ac1efaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45ab7fe700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac0b4a30 0x7f45ac1efaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45ab7fe700 1 -- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac0cff90 msgr2=0x7f45ac0c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45ab7fe700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac0cff90 0x7f45ac0c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45ab7fe700 1 -- 172.21.15.57:0/4219647412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a4009a70 con 0x7f45ac0b4660 2023-11-28T12:03:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45abfff700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac0cff90 0x7f45ac0c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.431+0000 7f45ab7fe700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45ac0b4660 0x7f45ac154e80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f459c00e810 tx=0x7f459c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.432+0000 7f4593fff700 1 -- 172.21.15.57:0/4219647412 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c004230 con 0x7f45ac0b4660 2023-11-28T12:03:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.432+0000 7f4593fff700 1 -- 172.21.15.57:0/4219647412 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c0043b0 con 0x7f45ac0b4660 2023-11-28T12:03:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.432+0000 7f45b1f00700 1 -- 172.21.15.57:0/4219647412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ac15f170 con 0x7f45ac0b4660 2023-11-28T12:03:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.432+0000 7f4593fff700 1 -- 172.21.15.57:0/4219647412 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c0038e0 con 0x7f45ac0b4660 2023-11-28T12:03:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.432+0000 7f45b1f00700 1 -- 172.21.15.57:0/4219647412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac0cf790 con 0x7f45ac0b4660 2023-11-28T12:03:31.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.433+0000 7f45b1f00700 1 -- 172.21.15.57:0/4219647412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45ac15f300 con 0x7f45ac0b4660 2023-11-28T12:03:31.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.433+0000 7f4593fff700 1 -- 172.21.15.57:0/4219647412 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f459c022070 con 0x7f45ac0b4660 2023-11-28T12:03:31.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.434+0000 7f4593fff700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f45980682f0 0x7f459806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.434+0000 7f4593fff700 1 -- 172.21.15.57:0/4219647412 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f459c0856b0 con 0x7f45ac0b4660 2023-11-28T12:03:31.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.434+0000 7f45aaffd700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f45980682f0 0x7f459806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.435+0000 7f45aaffd700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f45980682f0 0x7f459806a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f45ac155d60 tx=0x7f45a0007e70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:31.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.438+0000 7f4593fff700 1 -- 172.21.15.57:0/4219647412 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f459c051f10 con 0x7f45ac0b4660 2023-11-28T12:03:31.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.641+0000 7f45b1f00700 1 -- 172.21.15.57:0/4219647412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45ac0743f0 con 0x7f45ac0b4660 2023-11-28T12:03:31.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.642+0000 7f4593fff700 1 -- 172.21.15.57:0/4219647412 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+17921 (secure 0 0 0) 0x7f459c00f040 con 0x7f45ac0b4660 2023-11-28T12:03:31.644 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:31.644 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":35,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:02:54.404560+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:02:51.245666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T12:02:52.363863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:03:31.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.644+0000 7f45b1f00700 1 -- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f45980682f0 msgr2=0x7f459806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.644+0000 7f45b1f00700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f45980682f0 0x7f459806a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f45ac155d60 tx=0x7f45a0007e70 comp rx=0 tx=0).stop 2023-11-28T12:03:31.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.644+0000 7f45b1f00700 1 -- 172.21.15.57:0/4219647412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45ac0b4660 msgr2=0x7f45ac154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.644+0000 7f45b1f00700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45ac0b4660 0x7f45ac154e80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f459c00e810 tx=0x7f459c00eb20 comp rx=0 tx=0).stop 2023-11-28T12:03:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.644+0000 7f45b1f00700 1 -- 172.21.15.57:0/4219647412 shutdown_connections 2023-11-28T12:03:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.644+0000 7f45b1f00700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f45980682f0 0x7f459806a7a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.644+0000 7f45b1f00700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac0cff90 0x7f45ac0c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.644+0000 7f45b1f00700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45ac0b4660 0x7f45ac154e80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.644+0000 7f45b1f00700 1 --2- 172.21.15.57:0/4219647412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac0b4a30 0x7f45ac1efaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.644+0000 7f45b1f00700 1 -- 172.21.15.57:0/4219647412 >> 172.21.15.57:0/4219647412 conn(0x7f45ac0b2fc0 msgr2=0x7f45ac0adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.645+0000 7f45b1f00700 1 -- 172.21.15.57:0/4219647412 shutdown_connections 2023-11-28T12:03:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.645+0000 7f45b1f00700 1 -- 172.21.15.57:0/4219647412 wait complete. 2023-11-28T12:03:31.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:02:51.245666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:02:52.363863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:03:31.660 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:03:31.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.798+0000 7fce0152e700 1 Processor -- start 2023-11-28T12:03:31.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.798+0000 7fce0152e700 1 -- start start 2023-11-28T12:03:31.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.799+0000 7fce0152e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdfc160cb0 0x7fcdfc06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.799+0000 7fce0152e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdfc0d2d30 con 0x7fcdfc06b050 2023-11-28T12:03:31.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.799+0000 7fce0152e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdfc0d0b40 con 0x7fcdfc15c8c0 2023-11-28T12:03:31.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.799+0000 7fce0152e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc06b4d0 con 0x7fcdfc160cb0 2023-11-28T12:03:31.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.799+0000 7fcdfb7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcdfc06b050 0x7fcdfc0d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47366/0 (socket says 172.21.15.57:47366) 2023-11-28T12:03:31.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.799+0000 7fcdfb7fe700 1 -- 172.21.15.57:0/656460205 learned_addr learned my addr 172.21.15.57:0/656460205 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:31.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.799+0000 7fcdfa7fc700 1 --2- 172.21.15.57:0/656460205 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdfc160cb0 0x7fcdfc06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.799+0000 7fcdf9ffb700 1 -- 172.21.15.57:0/656460205 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296366546 0 0) 0x7fcdf4001490 con 0x7fcdfc06b050 2023-11-28T12:03:31.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.800+0000 7fcdf9ffb700 1 -- 172.21.15.57:0/656460205 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde80035b0 con 0x7fcdfc06b050 2023-11-28T12:03:31.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.800+0000 7fcdf9ffb700 1 -- 172.21.15.57:0/656460205 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037882014 0 0) 0x7fcdfc0d0b40 con 0x7fcdfc15c8c0 2023-11-28T12:03:31.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.800+0000 7fcdf9ffb700 1 -- 172.21.15.57:0/656460205 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdf4001490 con 0x7fcdfc15c8c0 2023-11-28T12:03:31.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.800+0000 7fcdf9ffb700 1 -- 172.21.15.57:0/656460205 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 63154239 0 0) 0x7fcde80035b0 con 0x7fcdfc06b050 2023-11-28T12:03:31.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.800+0000 7fcdfa7fc700 1 -- 172.21.15.57:0/656460205 >> v1:172.21.15.181:6789/0 conn(0x7fcdfc15c8c0 legacy=0x7fcdfc0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.800+0000 7fcdfa7fc700 1 -- 172.21.15.57:0/656460205 >> v1:172.21.15.57:6789/0 conn(0x7fcdfc06b050 legacy=0x7fcdfc0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.800+0000 7fcdfa7fc700 1 -- 172.21.15.57:0/656460205 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc0d2a50 con 0x7fcdfc160cb0 2023-11-28T12:03:31.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.800+0000 7fcdfa7fc700 1 --2- 172.21.15.57:0/656460205 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdfc160cb0 0x7fcdfc06ab10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009870 tx=0x7fcdec009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67c14ddde8baced8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:31.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.800+0000 7fcdf9ffb700 1 -- 172.21.15.57:0/656460205 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4001b80 con 0x7fcdfc160cb0 2023-11-28T12:03:31.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.800+0000 7fcdf9ffb700 1 -- 172.21.15.57:0/656460205 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec004030 con 0x7fcdfc160cb0 2023-11-28T12:03:31.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.800+0000 7fcdf9ffb700 1 -- 172.21.15.57:0/656460205 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec0043c0 con 0x7fcdfc160cb0 2023-11-28T12:03:31.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.801+0000 7fce0152e700 1 -- 172.21.15.57:0/656460205 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdfc160cb0 msgr2=0x7fcdfc06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.801+0000 7fce0152e700 1 --2- 172.21.15.57:0/656460205 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdfc160cb0 0x7fcdfc06ab10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009870 tx=0x7fcdec009b80 comp rx=0 tx=0).stop 2023-11-28T12:03:31.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.801+0000 7fce0152e700 1 -- 172.21.15.57:0/656460205 shutdown_connections 2023-11-28T12:03:31.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.801+0000 7fce0152e700 1 --2- 172.21.15.57:0/656460205 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdfc160cb0 0x7fcdfc06ab10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.801+0000 7fce0152e700 1 -- 172.21.15.57:0/656460205 >> 172.21.15.57:0/656460205 conn(0x7fcdfc06c520 msgr2=0x7fcdfc06c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:31.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.801+0000 7fce0152e700 1 -- 172.21.15.57:0/656460205 shutdown_connections 2023-11-28T12:03:31.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.801+0000 7fce0152e700 1 -- 172.21.15.57:0/656460205 wait complete. 2023-11-28T12:03:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.802+0000 7fce0152e700 1 Processor -- start 2023-11-28T12:03:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.802+0000 7fce0152e700 1 -- start start 2023-11-28T12:03:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.803+0000 7fce0152e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdfc06b050 0x7fcdfc0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.803+0000 7fce0152e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdfc15c8c0 0x7fcdfc0c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.803+0000 7fce0152e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdfc160cb0 0x7fcdfc0c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.803+0000 7fce0152e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc0c77b0 con 0x7fcdfc15c8c0 2023-11-28T12:03:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.803+0000 7fce0152e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc0c7a90 con 0x7fcdfc160cb0 2023-11-28T12:03:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.803+0000 7fce0152e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc0c7d70 con 0x7fcdfc06b050 2023-11-28T12:03:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.803+0000 7fcdfa7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdfc15c8c0 0x7fcdfc0c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.803+0000 7fcdfa7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdfc15c8c0 0x7fcdfc0c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35912/0 (socket says 172.21.15.57:35912) 2023-11-28T12:03:31.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.803+0000 7fcdfa7fc700 1 -- 172.21.15.57:0/420975347 learned_addr learned my addr 172.21.15.57:0/420975347 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:31.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.803+0000 7fcdfaffd700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdfc06b050 0x7fcdfc0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.803+0000 7fcdfb7fe700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdfc160cb0 0x7fcdfc0c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.804+0000 7fcdfa7fc700 1 -- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdfc06b050 msgr2=0x7fcdfc0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.804+0000 7fcdfa7fc700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdfc06b050 0x7fcdfc0c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.804+0000 7fcdfa7fc700 1 -- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdfc160cb0 msgr2=0x7fcdfc0c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:31.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.804+0000 7fcdfa7fc700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdfc160cb0 0x7fcdfc0c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:31.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.804+0000 7fcdfa7fc700 1 -- 172.21.15.57:0/420975347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec0094a0 con 0x7fcdfc15c8c0 2023-11-28T12:03:31.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.804+0000 7fcdfaffd700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdfc06b050 0x7fcdfc0c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:31.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.804+0000 7fcdfa7fc700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdfc15c8c0 0x7fcdfc0c93d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcdec000c00 tx=0x7fcdec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:31.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.804+0000 7fcdf37fe700 1 -- 172.21.15.57:0/420975347 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec003db0 con 0x7fcdfc15c8c0 2023-11-28T12:03:31.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.804+0000 7fcdf37fe700 1 -- 172.21.15.57:0/420975347 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec01f070 con 0x7fcdfc15c8c0 2023-11-28T12:03:31.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.804+0000 7fcdf37fe700 1 -- 172.21.15.57:0/420975347 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec011870 con 0x7fcdfc15c8c0 2023-11-28T12:03:31.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.804+0000 7fce0152e700 1 -- 172.21.15.57:0/420975347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdfc0cda40 con 0x7fcdfc15c8c0 2023-11-28T12:03:31.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.804+0000 7fce0152e700 1 -- 172.21.15.57:0/420975347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc0cd760 con 0x7fcdfc15c8c0 2023-11-28T12:03:31.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.805+0000 7fce0152e700 1 -- 172.21.15.57:0/420975347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdfc161120 con 0x7fcdfc15c8c0 2023-11-28T12:03:31.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.806+0000 7fcdf37fe700 1 -- 172.21.15.57:0/420975347 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcdec02a020 con 0x7fcdfc15c8c0 2023-11-28T12:03:31.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.806+0000 7fcdf37fe700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcddc0682f0 0x7fcddc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:31.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.806+0000 7fcdf37fe700 1 -- 172.21.15.57:0/420975347 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcdec087180 con 0x7fcdfc15c8c0 2023-11-28T12:03:31.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.806+0000 7fcdfaffd700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcddc0682f0 0x7fcddc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:31.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.807+0000 7fcdfaffd700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcddc0682f0 0x7fcddc06a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcde40011e0 tx=0x7fcde4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:31.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:31.811+0000 7fcdf37fe700 1 -- 172.21.15.57:0/420975347 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcdec0539e0 con 0x7fcdfc15c8c0 2023-11-28T12:03:32.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.009+0000 7fce0152e700 1 -- 172.21.15.57:0/420975347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcdfc067d70 con 0x7fcdfc15c8c0 2023-11-28T12:03:32.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.010+0000 7fcdf37fe700 1 -- 172.21.15.57:0/420975347 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcdec022380 con 0x7fcdfc15c8c0 2023-11-28T12:03:32.011 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:03:32.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.012+0000 7fce0152e700 1 -- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcddc0682f0 msgr2=0x7fcddc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.012+0000 7fce0152e700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcddc0682f0 0x7fcddc06a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcde40011e0 tx=0x7fcde4009040 comp rx=0 tx=0).stop 2023-11-28T12:03:32.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.012+0000 7fce0152e700 1 -- 172.21.15.57:0/420975347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdfc15c8c0 msgr2=0x7fcdfc0c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.012+0000 7fce0152e700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdfc15c8c0 0x7fcdfc0c93d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcdec000c00 tx=0x7fcdec004750 comp rx=0 tx=0).stop 2023-11-28T12:03:32.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.012+0000 7fce0152e700 1 -- 172.21.15.57:0/420975347 shutdown_connections 2023-11-28T12:03:32.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.012+0000 7fce0152e700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcddc0682f0 0x7fcddc06a7a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.012+0000 7fce0152e700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdfc160cb0 0x7fcdfc0c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.012+0000 7fce0152e700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdfc06b050 0x7fcdfc0c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.012+0000 7fce0152e700 1 --2- 172.21.15.57:0/420975347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdfc15c8c0 0x7fcdfc0c93d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.012+0000 7fce0152e700 1 -- 172.21.15.57:0/420975347 >> 172.21.15.57:0/420975347 conn(0x7fcdfc06c520 msgr2=0x7fcdfc15b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:32.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.012+0000 7fce0152e700 1 -- 172.21.15.57:0/420975347 shutdown_connections 2023-11-28T12:03:32.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.012+0000 7fce0152e700 1 -- 172.21.15.57:0/420975347 wait complete. 2023-11-28T12:03:32.027 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:03:32.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.163+0000 7f898cae2700 1 Processor -- start 2023-11-28T12:03:32.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.163+0000 7f898cae2700 1 -- start start 2023-11-28T12:03:32.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.164+0000 7f898cae2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f898806a980 0x7f89880cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.164+0000 7f898cae2700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898815e0d0 con 0x7f89880cdd50 2023-11-28T12:03:32.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.164+0000 7f898cae2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89880ce560 con 0x7f898815a4d0 2023-11-28T12:03:32.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.164+0000 7f898cae2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898815e320 con 0x7f898806a980 2023-11-28T12:03:32.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.165+0000 7f8986d9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f89880cdd50 0x7f89880cb010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47370/0 (socket says 172.21.15.57:47370) 2023-11-28T12:03:32.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.165+0000 7f8986d9d700 1 -- 172.21.15.57:0/3495828677 learned_addr learned my addr 172.21.15.57:0/3495828677 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:32.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.165+0000 7f8985d9b700 1 --2- 172.21.15.57:0/3495828677 >> v2:172.21.15.181:3300/0 conn(0x7f898806a980 0x7f89880cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.165+0000 7f898559a700 1 -- 172.21.15.57:0/3495828677 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353667672 0 0) 0x7f898815e0d0 con 0x7f89880cdd50 2023-11-28T12:03:32.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.165+0000 7f898559a700 1 -- 172.21.15.57:0/3495828677 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89740035b0 con 0x7f89880cdd50 2023-11-28T12:03:32.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.165+0000 7f8985d9b700 1 -- 172.21.15.57:0/3495828677 >> v1:172.21.15.181:6789/0 conn(0x7f898815a4d0 legacy=0x7f898806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.165+0000 7f8985d9b700 1 -- 172.21.15.57:0/3495828677 >> v1:172.21.15.57:6789/0 conn(0x7f89880cdd50 legacy=0x7f89880cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.165+0000 7f8985d9b700 1 -- 172.21.15.57:0/3495828677 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898815e600 con 0x7f898806a980 2023-11-28T12:03:32.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.165+0000 7f8985d9b700 1 --2- 172.21.15.57:0/3495828677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f898806a980 0x7f89880cd810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8978009840 tx=0x7f8978009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4fe7475e72bd66 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:32.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.166+0000 7f898559a700 1 -- 172.21.15.57:0/3495828677 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978011830 con 0x7f898806a980 2023-11-28T12:03:32.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.166+0000 7f898559a700 1 -- 172.21.15.57:0/3495828677 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8978011990 con 0x7f898806a980 2023-11-28T12:03:32.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.166+0000 7f898559a700 1 -- 172.21.15.57:0/3495828677 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978011d00 con 0x7f898806a980 2023-11-28T12:03:32.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.166+0000 7f898cae2700 1 -- 172.21.15.57:0/3495828677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f898806a980 msgr2=0x7f89880cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.166+0000 7f898cae2700 1 --2- 172.21.15.57:0/3495828677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f898806a980 0x7f89880cd810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8978009840 tx=0x7f8978009b50 comp rx=0 tx=0).stop 2023-11-28T12:03:32.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.166+0000 7f898cae2700 1 -- 172.21.15.57:0/3495828677 shutdown_connections 2023-11-28T12:03:32.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.166+0000 7f898cae2700 1 --2- 172.21.15.57:0/3495828677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f898806a980 0x7f89880cd810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.166+0000 7f898cae2700 1 -- 172.21.15.57:0/3495828677 >> 172.21.15.57:0/3495828677 conn(0x7f898806c010 msgr2=0x7f89880cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:32.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.166+0000 7f898cae2700 1 -- 172.21.15.57:0/3495828677 shutdown_connections 2023-11-28T12:03:32.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.166+0000 7f898cae2700 1 -- 172.21.15.57:0/3495828677 wait complete. 2023-11-28T12:03:32.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.168+0000 7f898cae2700 1 Processor -- start 2023-11-28T12:03:32.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.168+0000 7f898cae2700 1 -- start start 2023-11-28T12:03:32.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898cae2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898806a980 0x7f8988152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898cae2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89880cdd50 0x7f89880ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898cae2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f898815a4d0 0x7f89880b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898cae2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898815c440 con 0x7f898806a980 2023-11-28T12:03:32.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898cae2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89880ad280 con 0x7f898815a4d0 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898cae2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89880ad560 con 0x7f89880cdd50 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898806a980 0x7f8988152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898806a980 0x7f8988152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35914/0 (socket says 172.21.15.57:35914) 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898659c700 1 -- 172.21.15.57:0/3246057261 learned_addr learned my addr 172.21.15.57:0/3246057261 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f8985d9b700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89880cdd50 0x7f89880ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f8986d9d700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f898815a4d0 0x7f89880b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898659c700 1 -- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89880cdd50 msgr2=0x7f89880ac9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898659c700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89880cdd50 0x7f89880ac9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898659c700 1 -- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f898815a4d0 msgr2=0x7f89880b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898659c700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f898815a4d0 0x7f89880b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898659c700 1 -- 172.21.15.57:0/3246057261 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89780094a0 con 0x7f898806a980 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f898659c700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898806a980 0x7f8988152120 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f897000c480 tx=0x7f897000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.169+0000 7f897effd700 1 -- 172.21.15.57:0/3246057261 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89700052d0 con 0x7f898806a980 2023-11-28T12:03:32.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.170+0000 7f897effd700 1 -- 172.21.15.57:0/3246057261 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8970005430 con 0x7f898806a980 2023-11-28T12:03:32.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.170+0000 7f897effd700 1 -- 172.21.15.57:0/3246057261 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89700038c0 con 0x7f898806a980 2023-11-28T12:03:32.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.170+0000 7f898cae2700 1 -- 172.21.15.57:0/3246057261 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89880ce1b0 con 0x7f898806a980 2023-11-28T12:03:32.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.170+0000 7f898cae2700 1 -- 172.21.15.57:0/3246057261 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898815e8e0 con 0x7f898806a980 2023-11-28T12:03:32.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.171+0000 7f898cae2700 1 -- 172.21.15.57:0/3246057261 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898815a8a0 con 0x7f898806a980 2023-11-28T12:03:32.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.171+0000 7f897effd700 1 -- 172.21.15.57:0/3246057261 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8970019030 con 0x7f898806a980 2023-11-28T12:03:32.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.172+0000 7f897effd700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89680682f0 0x7f896806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.172+0000 7f897effd700 1 -- 172.21.15.57:0/3246057261 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8970085060 con 0x7f898806a980 2023-11-28T12:03:32.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.172+0000 7f8985d9b700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89680682f0 0x7f896806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.172+0000 7f8985d9b700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89680682f0 0x7f896806a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f89880cd350 tx=0x7f8978008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:32.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.176+0000 7f897effd700 1 -- 172.21.15.57:0/3246057261 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f89700517a0 con 0x7f898806a980 2023-11-28T12:03:32.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.374+0000 7f898cae2700 1 -- 172.21.15.57:0/3246057261 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89880743f0 con 0x7f898806a980 2023-11-28T12:03:32.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.374+0000 7f897effd700 1 -- 172.21.15.57:0/3246057261 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8970017020 con 0x7f898806a980 2023-11-28T12:03:32.376 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:03:32.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.376+0000 7f898cae2700 1 -- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89680682f0 msgr2=0x7f896806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.376+0000 7f898cae2700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89680682f0 0x7f896806a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f89880cd350 tx=0x7f8978008030 comp rx=0 tx=0).stop 2023-11-28T12:03:32.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.376+0000 7f898cae2700 1 -- 172.21.15.57:0/3246057261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898806a980 msgr2=0x7f8988152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.376+0000 7f898cae2700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898806a980 0x7f8988152120 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f897000c480 tx=0x7f897000c790 comp rx=0 tx=0).stop 2023-11-28T12:03:32.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.376+0000 7f898cae2700 1 -- 172.21.15.57:0/3246057261 shutdown_connections 2023-11-28T12:03:32.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.376+0000 7f898cae2700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f89680682f0 0x7f896806a7a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.376+0000 7f898cae2700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f898815a4d0 0x7f89880b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.376+0000 7f898cae2700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f898806a980 0x7f8988152120 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.376+0000 7f898cae2700 1 --2- 172.21.15.57:0/3246057261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89880cdd50 0x7f89880ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.376+0000 7f898cae2700 1 -- 172.21.15.57:0/3246057261 >> 172.21.15.57:0/3246057261 conn(0x7f898806c010 msgr2=0x7f89880cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:32.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.376+0000 7f898cae2700 1 -- 172.21.15.57:0/3246057261 shutdown_connections 2023-11-28T12:03:32.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.376+0000 7f898cae2700 1 -- 172.21.15.57:0/3246057261 wait complete. 2023-11-28T12:03:32.392 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:03:32.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.529+0000 7f440cd7c700 1 Processor -- start 2023-11-28T12:03:32.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.529+0000 7f440cd7c700 1 -- start start 2023-11-28T12:03:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f440cd7c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4408160300 0x7f440815d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f440cd7c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f440815d880 0x7f44080cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f440cd7c700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440815ceb0 con 0x7f44080cec00 2023-11-28T12:03:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f440cd7c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44081608c0 con 0x7f440815d880 2023-11-28T12:03:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f440cd7c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4408160ba0 con 0x7f4408160300 2023-11-28T12:03:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f440659c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f44080cec00 0x7f4408065fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47378/0 (socket says 172.21.15.57:47378) 2023-11-28T12:03:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f440659c700 1 -- 172.21.15.57:0/1326931521 learned_addr learned my addr 172.21.15.57:0/1326931521 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f4405d9b700 1 --2- 172.21.15.57:0/1326931521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4408160300 0x7f440815d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f4406d9d700 1 --2- 172.21.15.57:0/1326931521 >> v2:172.21.15.181:3300/0 conn(0x7f440815d880 0x7f44080cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f440559a700 1 -- 172.21.15.57:0/1326931521 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354893374 0 0) 0x7f440815ceb0 con 0x7f44080cec00 2023-11-28T12:03:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f440559a700 1 -- 172.21.15.57:0/1326931521 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f00035b0 con 0x7f44080cec00 2023-11-28T12:03:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f4405d9b700 1 -- 172.21.15.57:0/1326931521 >> v2:172.21.15.181:3300/0 conn(0x7f440815d880 msgr2=0x7f44080cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f4405d9b700 1 --2- 172.21.15.57:0/1326931521 >> v2:172.21.15.181:3300/0 conn(0x7f440815d880 0x7f44080cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f440559a700 1 -- 172.21.15.57:0/1326931521 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 961031641 0 0) 0x7f43f00035b0 con 0x7f44080cec00 2023-11-28T12:03:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f4405d9b700 1 -- 172.21.15.57:0/1326931521 >> v1:172.21.15.57:6789/0 conn(0x7f44080cec00 legacy=0x7f4408065fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.530+0000 7f4405d9b700 1 -- 172.21.15.57:0/1326931521 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4408160e80 con 0x7f4408160300 2023-11-28T12:03:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.531+0000 7f4405d9b700 1 --2- 172.21.15.57:0/1326931521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4408160300 0x7f440815d340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009840 tx=0x7f43fc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d5a7b114921e752 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.531+0000 7f440559a700 1 -- 172.21.15.57:0/1326931521 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f8001670 con 0x7f4408160300 2023-11-28T12:03:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.531+0000 7f440559a700 1 -- 172.21.15.57:0/1326931521 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc004030 con 0x7f4408160300 2023-11-28T12:03:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.531+0000 7f440559a700 1 -- 172.21.15.57:0/1326931521 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0043c0 con 0x7f4408160300 2023-11-28T12:03:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.531+0000 7f440cd7c700 1 -- 172.21.15.57:0/1326931521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4408160300 msgr2=0x7f440815d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.531+0000 7f440cd7c700 1 --2- 172.21.15.57:0/1326931521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4408160300 0x7f440815d340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009840 tx=0x7f43fc009b50 comp rx=0 tx=0).stop 2023-11-28T12:03:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.532+0000 7f440cd7c700 1 -- 172.21.15.57:0/1326931521 shutdown_connections 2023-11-28T12:03:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.532+0000 7f440cd7c700 1 --2- 172.21.15.57:0/1326931521 >> v2:172.21.15.181:3300/0 conn(0x7f440815d880 0x7f44080cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.532+0000 7f440cd7c700 1 --2- 172.21.15.57:0/1326931521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4408160300 0x7f440815d340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.532+0000 7f440cd7c700 1 -- 172.21.15.57:0/1326931521 >> 172.21.15.57:0/1326931521 conn(0x7f44080cf400 msgr2=0x7f44080cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.532+0000 7f440cd7c700 1 -- 172.21.15.57:0/1326931521 shutdown_connections 2023-11-28T12:03:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.532+0000 7f440cd7c700 1 -- 172.21.15.57:0/1326931521 wait complete. 2023-11-28T12:03:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.533+0000 7f440cd7c700 1 Processor -- start 2023-11-28T12:03:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.533+0000 7f440cd7c700 1 -- start start 2023-11-28T12:03:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f440cd7c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44080cec00 0x7f440815f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f440cd7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f440815d880 0x7f44080c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f440cd7c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4408160300 0x7f44080c3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f440cd7c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440815ee10 con 0x7f4408160300 2023-11-28T12:03:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f440cd7c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4408145360 con 0x7f440815d880 2023-11-28T12:03:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f440cd7c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440815e200 con 0x7f44080cec00 2023-11-28T12:03:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f4406d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4408160300 0x7f44080c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f4406d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4408160300 0x7f44080c3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35918/0 (socket says 172.21.15.57:35918) 2023-11-28T12:03:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f4406d9d700 1 -- 172.21.15.57:0/4079998343 learned_addr learned my addr 172.21.15.57:0/4079998343 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f440659c700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44080cec00 0x7f440815f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f4405d9b700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f440815d880 0x7f44080c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f4406d9d700 1 -- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44080cec00 msgr2=0x7f440815f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f4406d9d700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44080cec00 0x7f440815f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f4406d9d700 1 -- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f440815d880 msgr2=0x7f44080c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f4406d9d700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f440815d880 0x7f44080c2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.534+0000 7f4406d9d700 1 -- 172.21.15.57:0/4079998343 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc0094a0 con 0x7f4408160300 2023-11-28T12:03:32.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.535+0000 7f4406d9d700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4408160300 0x7f44080c3470 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f440000efc0 tx=0x7f440000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:32.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.535+0000 7f43f6ffd700 1 -- 172.21.15.57:0/4079998343 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400009ea0 con 0x7f4408160300 2023-11-28T12:03:32.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.535+0000 7f43f6ffd700 1 -- 172.21.15.57:0/4079998343 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440000f040 con 0x7f4408160300 2023-11-28T12:03:32.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.535+0000 7f440cd7c700 1 -- 172.21.15.57:0/4079998343 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44080cefd0 con 0x7f4408160300 2023-11-28T12:03:32.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.535+0000 7f43f6ffd700 1 -- 172.21.15.57:0/4079998343 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440000ced0 con 0x7f4408160300 2023-11-28T12:03:32.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.535+0000 7f440cd7c700 1 -- 172.21.15.57:0/4079998343 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4408161160 con 0x7f4408160300 2023-11-28T12:03:32.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.536+0000 7f440cd7c700 1 -- 172.21.15.57:0/4079998343 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44080cf160 con 0x7f4408160300 2023-11-28T12:03:32.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.536+0000 7f43f6ffd700 1 -- 172.21.15.57:0/4079998343 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4400014480 con 0x7f4408160300 2023-11-28T12:03:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.537+0000 7f43f6ffd700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f43e80682f0 0x7f43e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.537+0000 7f43f6ffd700 1 -- 172.21.15.57:0/4079998343 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4400086660 con 0x7f4408160300 2023-11-28T12:03:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.537+0000 7f440659c700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f43e80682f0 0x7f43e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.537+0000 7f440659c700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f43e80682f0 0x7f43e806a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f43f8000f60 tx=0x7f43f8001e00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:32.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.541+0000 7f43f6ffd700 1 -- 172.21.15.57:0/4079998343 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4400052e60 con 0x7f4408160300 2023-11-28T12:03:32.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.741+0000 7f440cd7c700 1 -- 172.21.15.57:0/4079998343 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f44080aeb40 con 0x7f4408160300 2023-11-28T12:03:32.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.742+0000 7f43f6ffd700 1 -- 172.21.15.57:0/4079998343 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4400056480 con 0x7f4408160300 2023-11-28T12:03:32.743 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:03:32.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.744+0000 7f440cd7c700 1 -- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f43e80682f0 msgr2=0x7f43e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.744+0000 7f440cd7c700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f43e80682f0 0x7f43e806a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f43f8000f60 tx=0x7f43f8001e00 comp rx=0 tx=0).stop 2023-11-28T12:03:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.744+0000 7f440cd7c700 1 -- 172.21.15.57:0/4079998343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4408160300 msgr2=0x7f44080c3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.744+0000 7f440cd7c700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4408160300 0x7f44080c3470 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f440000efc0 tx=0x7f440000c3a0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.744+0000 7f440cd7c700 1 -- 172.21.15.57:0/4079998343 shutdown_connections 2023-11-28T12:03:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.744+0000 7f440cd7c700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f43e80682f0 0x7f43e806a7a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.744+0000 7f440cd7c700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4408160300 0x7f44080c3470 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.744+0000 7f440cd7c700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44080cec00 0x7f440815f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.744+0000 7f440cd7c700 1 --2- 172.21.15.57:0/4079998343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f440815d880 0x7f44080c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.744+0000 7f440cd7c700 1 -- 172.21.15.57:0/4079998343 >> 172.21.15.57:0/4079998343 conn(0x7f44080cf400 msgr2=0x7f44080cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.744+0000 7f440cd7c700 1 -- 172.21.15.57:0/4079998343 shutdown_connections 2023-11-28T12:03:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.744+0000 7f440cd7c700 1 -- 172.21.15.57:0/4079998343 wait complete. 2023-11-28T12:03:32.759 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:03:32.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.899+0000 7f3cd9763700 1 Processor -- start 2023-11-28T12:03:32.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.899+0000 7f3cd9763700 1 -- start start 2023-11-28T12:03:32.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.899+0000 7f3cd9763700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3cd415c8c0 0x7f3cd4160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.899+0000 7f3cd9763700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd40d2d30 con 0x7f3cd40b5dc0 2023-11-28T12:03:32.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.899+0000 7f3cd9763700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd40d0b40 con 0x7f3cd406b220 2023-11-28T12:03:32.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.899+0000 7f3cd9763700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd41611f0 con 0x7f3cd415c8c0 2023-11-28T12:03:32.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.900+0000 7f3cd2ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3cd415c8c0 0x7f3cd4160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.900+0000 7f3cd27fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f3cd40b5dc0 0x7f3cd406ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47392/0 (socket says 172.21.15.57:47392) 2023-11-28T12:03:32.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.900+0000 7f3cd27fc700 1 -- 172.21.15.57:0/4050493397 learned_addr learned my addr 172.21.15.57:0/4050493397 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:32.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.900+0000 7f3cd2ffd700 1 -- 172.21.15.57:0/4050493397 >> v1:172.21.15.181:6789/0 conn(0x7f3cd406b220 legacy=0x7f3cd40d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.900+0000 7f3cd2ffd700 1 -- 172.21.15.57:0/4050493397 >> v1:172.21.15.57:6789/0 conn(0x7f3cd40b5dc0 legacy=0x7f3cd406ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.900+0000 7f3cd2ffd700 1 -- 172.21.15.57:0/4050493397 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd40d2a50 con 0x7f3cd415c8c0 2023-11-28T12:03:32.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.900+0000 7f3cd2ffd700 1 --2- 172.21.15.57:0/4050493397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cd415c8c0 0x7f3cd4160cb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009840 tx=0x7f3cc4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40333ef74809e0d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:32.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.900+0000 7f3cd1ffb700 1 -- 172.21.15.57:0/4050493397 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4011830 con 0x7f3cd415c8c0 2023-11-28T12:03:32.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.900+0000 7f3cd1ffb700 1 -- 172.21.15.57:0/4050493397 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc4011990 con 0x7f3cd415c8c0 2023-11-28T12:03:32.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.900+0000 7f3cd1ffb700 1 -- 172.21.15.57:0/4050493397 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4011d20 con 0x7f3cd415c8c0 2023-11-28T12:03:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.901+0000 7f3cd9763700 1 -- 172.21.15.57:0/4050493397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cd415c8c0 msgr2=0x7f3cd4160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.901+0000 7f3cd9763700 1 --2- 172.21.15.57:0/4050493397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cd415c8c0 0x7f3cd4160cb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009840 tx=0x7f3cc4009b50 comp rx=0 tx=0).stop 2023-11-28T12:03:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.901+0000 7f3cd9763700 1 -- 172.21.15.57:0/4050493397 shutdown_connections 2023-11-28T12:03:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.901+0000 7f3cd9763700 1 --2- 172.21.15.57:0/4050493397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cd415c8c0 0x7f3cd4160cb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.901+0000 7f3cd9763700 1 -- 172.21.15.57:0/4050493397 >> 172.21.15.57:0/4050493397 conn(0x7f3cd406c520 msgr2=0x7f3cd406c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.901+0000 7f3cd9763700 1 -- 172.21.15.57:0/4050493397 shutdown_connections 2023-11-28T12:03:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.901+0000 7f3cd9763700 1 -- 172.21.15.57:0/4050493397 wait complete. 2023-11-28T12:03:32.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.902+0000 7f3cd9763700 1 Processor -- start 2023-11-28T12:03:32.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.903+0000 7f3cd9763700 1 -- start start 2023-11-28T12:03:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.903+0000 7f3cd9763700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cd406b220 0x7f3cd40c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.903+0000 7f3cd9763700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cd40b5dc0 0x7f3cd414bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.903+0000 7f3cd9763700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cd415c8c0 0x7f3cd40cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.903+0000 7f3cd9763700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd414c870 con 0x7f3cd415c8c0 2023-11-28T12:03:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.903+0000 7f3cd9763700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd415d170 con 0x7f3cd40b5dc0 2023-11-28T12:03:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.903+0000 7f3cd9763700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd415d450 con 0x7f3cd406b220 2023-11-28T12:03:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.903+0000 7f3cd37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cd415c8c0 0x7f3cd40cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.903+0000 7f3cd37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cd415c8c0 0x7f3cd40cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35926/0 (socket says 172.21.15.57:35926) 2023-11-28T12:03:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.903+0000 7f3cd37fe700 1 -- 172.21.15.57:0/3817858690 learned_addr learned my addr 172.21.15.57:0/3817858690 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3cd2ffd700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cd406b220 0x7f3cd40c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3cd27fc700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cd40b5dc0 0x7f3cd414bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3cd37fe700 1 -- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cd406b220 msgr2=0x7f3cd40c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3cd37fe700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cd406b220 0x7f3cd40c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3cd37fe700 1 -- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cd40b5dc0 msgr2=0x7f3cd414bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3cd37fe700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cd40b5dc0 0x7f3cd414bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3cd37fe700 1 -- 172.21.15.57:0/3817858690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc40094a0 con 0x7f3cd415c8c0 2023-11-28T12:03:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3cd2ffd700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cd406b220 0x7f3cd40c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3cd27fc700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cd40b5dc0 0x7f3cd414bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3cd37fe700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cd415c8c0 0x7f3cd40cb7d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00cc50 tx=0x7f3ccc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3ccb7fe700 1 -- 172.21.15.57:0/3817858690 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc00ad80 con 0x7f3cd415c8c0 2023-11-28T12:03:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3ccb7fe700 1 -- 172.21.15.57:0/3817858690 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc00ce50 con 0x7f3cd415c8c0 2023-11-28T12:03:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3ccb7fe700 1 -- 172.21.15.57:0/3817858690 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc003c40 con 0x7f3cd415c8c0 2023-11-28T12:03:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3cd9763700 1 -- 172.21.15.57:0/3817858690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd40cda40 con 0x7f3cd415c8c0 2023-11-28T12:03:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.904+0000 7f3cd9763700 1 -- 172.21.15.57:0/3817858690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd40cd760 con 0x7f3cd415c8c0 2023-11-28T12:03:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.905+0000 7f3cd9763700 1 -- 172.21.15.57:0/3817858690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd40d2700 con 0x7f3cd415c8c0 2023-11-28T12:03:32.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.906+0000 7f3ccb7fe700 1 -- 172.21.15.57:0/3817858690 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3ccc010050 con 0x7f3cd415c8c0 2023-11-28T12:03:32.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.906+0000 7f3ccb7fe700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3cb40682f0 0x7f3cb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:32.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.906+0000 7f3ccb7fe700 1 -- 172.21.15.57:0/3817858690 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f3ccc085c30 con 0x7f3cd415c8c0 2023-11-28T12:03:32.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.907+0000 7f3cd2ffd700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3cb40682f0 0x7f3cb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:32.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.907+0000 7f3cd2ffd700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3cb40682f0 0x7f3cb406a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40090b0 tx=0x7f3cc4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:32.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:32.911+0000 7f3ccb7fe700 1 -- 172.21.15.57:0/3817858690 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3ccc052470 con 0x7f3cd415c8c0 2023-11-28T12:03:33.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.113+0000 7f3cd9763700 1 -- 172.21.15.57:0/3817858690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3cd4067d70 con 0x7f3cd415c8c0 2023-11-28T12:03:33.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.113+0000 7f3ccb7fe700 1 -- 172.21.15.57:0/3817858690 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ccc055a90 con 0x7f3cd415c8c0 2023-11-28T12:03:33.114 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:03:33.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.115+0000 7f3cd9763700 1 -- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3cb40682f0 msgr2=0x7f3cb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.115+0000 7f3cd9763700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3cb40682f0 0x7f3cb406a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40090b0 tx=0x7f3cc4008030 comp rx=0 tx=0).stop 2023-11-28T12:03:33.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.115+0000 7f3cd9763700 1 -- 172.21.15.57:0/3817858690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cd415c8c0 msgr2=0x7f3cd40cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.115+0000 7f3cd9763700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cd415c8c0 0x7f3cd40cb7d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00cc50 tx=0x7f3ccc00a340 comp rx=0 tx=0).stop 2023-11-28T12:03:33.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.115+0000 7f3cd9763700 1 -- 172.21.15.57:0/3817858690 shutdown_connections 2023-11-28T12:03:33.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.115+0000 7f3cd9763700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3cb40682f0 0x7f3cb406a7a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.115+0000 7f3cd9763700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3cd415c8c0 0x7f3cd40cb7d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.115+0000 7f3cd9763700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cd406b220 0x7f3cd40c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.115+0000 7f3cd9763700 1 --2- 172.21.15.57:0/3817858690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cd40b5dc0 0x7f3cd414bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.115+0000 7f3cd9763700 1 -- 172.21.15.57:0/3817858690 >> 172.21.15.57:0/3817858690 conn(0x7f3cd406c520 msgr2=0x7f3cd40b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:33.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.116+0000 7f3cd9763700 1 -- 172.21.15.57:0/3817858690 shutdown_connections 2023-11-28T12:03:33.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.116+0000 7f3cd9763700 1 -- 172.21.15.57:0/3817858690 wait complete. 2023-11-28T12:03:33.130 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:03:33.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.264+0000 7fc36970f700 1 Processor -- start 2023-11-28T12:03:33.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.264+0000 7fc36970f700 1 -- start start 2023-11-28T12:03:33.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.265+0000 7fc36970f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc36406ab10 0x7fc36406aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:33.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.265+0000 7fc36970f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc3640d0450 0x7fc364160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:33.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.265+0000 7fc36970f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3640d1ee0 con 0x7fc3640cd5a0 2023-11-28T12:03:33.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.265+0000 7fc36970f700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36406b420 con 0x7fc3640d0450 2023-11-28T12:03:33.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.265+0000 7fc36970f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3640cdb10 con 0x7fc36406ab10 2023-11-28T12:03:33.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.265+0000 7fc3637fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc3640d0450 0x7fc364160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:33.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.265+0000 7fc3637fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc3640d0450 0x7fc364160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35938/0 (socket says 172.21.15.57:35938) 2023-11-28T12:03:33.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.265+0000 7fc3637fe700 1 -- 172.21.15.57:0/4222095076 learned_addr learned my addr 172.21.15.57:0/4222095076 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:33.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.265+0000 7fc362ffd700 1 --2- 172.21.15.57:0/4222095076 >> v2:172.21.15.181:3300/0 conn(0x7fc36406ab10 0x7fc36406aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:33.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.266+0000 7fc3637fe700 1 -- 172.21.15.57:0/4222095076 >> v2:172.21.15.181:3300/0 conn(0x7fc36406ab10 msgr2=0x7fc36406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:33.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.266+0000 7fc3637fe700 1 --2- 172.21.15.57:0/4222095076 >> v2:172.21.15.181:3300/0 conn(0x7fc36406ab10 0x7fc36406aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.266+0000 7fc3637fe700 1 -- 172.21.15.57:0/4222095076 >> v1:172.21.15.181:6789/0 conn(0x7fc3640cd5a0 legacy=0x7fc364160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.266+0000 7fc3637fe700 1 -- 172.21.15.57:0/4222095076 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3640d1890 con 0x7fc3640d0450 2023-11-28T12:03:33.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.266+0000 7fc3637fe700 1 --2- 172.21.15.57:0/4222095076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3640d0450 0x7fc364160a10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc3640d0cb0 tx=0x7fc35c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311b36221e91c17f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:33.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.266+0000 7fc361ffb700 1 -- 172.21.15.57:0/4222095076 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00e070 con 0x7fc3640d0450 2023-11-28T12:03:33.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.266+0000 7fc361ffb700 1 -- 172.21.15.57:0/4222095076 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c00b910 con 0x7fc3640d0450 2023-11-28T12:03:33.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.266+0000 7fc361ffb700 1 -- 172.21.15.57:0/4222095076 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00bcc0 con 0x7fc3640d0450 2023-11-28T12:03:33.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.267+0000 7fc36970f700 1 -- 172.21.15.57:0/4222095076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3640d0450 msgr2=0x7fc364160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.267+0000 7fc36970f700 1 --2- 172.21.15.57:0/4222095076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3640d0450 0x7fc364160a10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc3640d0cb0 tx=0x7fc35c00b330 comp rx=0 tx=0).stop 2023-11-28T12:03:33.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.267+0000 7fc36970f700 1 -- 172.21.15.57:0/4222095076 shutdown_connections 2023-11-28T12:03:33.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.267+0000 7fc36970f700 1 --2- 172.21.15.57:0/4222095076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3640d0450 0x7fc364160a10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.267+0000 7fc36970f700 1 --2- 172.21.15.57:0/4222095076 >> v2:172.21.15.181:3300/0 conn(0x7fc36406ab10 0x7fc36406aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.267+0000 7fc36970f700 1 -- 172.21.15.57:0/4222095076 >> 172.21.15.57:0/4222095076 conn(0x7fc36406c520 msgr2=0x7fc36406c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:33.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.267+0000 7fc36970f700 1 -- 172.21.15.57:0/4222095076 shutdown_connections 2023-11-28T12:03:33.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.267+0000 7fc36970f700 1 -- 172.21.15.57:0/4222095076 wait complete. 2023-11-28T12:03:33.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.269+0000 7fc36970f700 1 Processor -- start 2023-11-28T12:03:33.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.269+0000 7fc36970f700 1 -- start start 2023-11-28T12:03:33.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc36970f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36406ab10 0x7fc3640c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:33.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc36970f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3640cd5a0 0x7fc364158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:33.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc36970f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3640d0450 0x7fc36415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:33.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc36970f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640ca540 con 0x7fc36406ab10 2023-11-28T12:03:33.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc36970f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3641545d0 con 0x7fc3640d0450 2023-11-28T12:03:33.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc36970f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36415b7c0 con 0x7fc3640cd5a0 2023-11-28T12:03:33.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc362ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36406ab10 0x7fc3640c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:33.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc362ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36406ab10 0x7fc3640c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35948/0 (socket says 172.21.15.57:35948) 2023-11-28T12:03:33.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc362ffd700 1 -- 172.21.15.57:0/2612675881 learned_addr learned my addr 172.21.15.57:0/2612675881 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:33.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc3637fe700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3640d0450 0x7fc36415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:33.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc3627fc700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3640cd5a0 0x7fc364158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:33.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc362ffd700 1 -- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3640cd5a0 msgr2=0x7fc364158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc362ffd700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3640cd5a0 0x7fc364158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc362ffd700 1 -- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3640d0450 msgr2=0x7fc36415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc362ffd700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3640d0450 0x7fc36415a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc362ffd700 1 -- 172.21.15.57:0/2612675881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c009b30 con 0x7fc36406ab10 2023-11-28T12:03:33.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc3637fe700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3640d0450 0x7fc36415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:33.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc362ffd700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36406ab10 0x7fc3640c6170 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00e870 tx=0x7fc34c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:33.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc35b7fe700 1 -- 172.21.15.57:0/2612675881 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c009590 con 0x7fc36406ab10 2023-11-28T12:03:33.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc35b7fe700 1 -- 172.21.15.57:0/2612675881 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c00ee70 con 0x7fc36406ab10 2023-11-28T12:03:33.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc36970f700 1 -- 172.21.15.57:0/2612675881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36415cad0 con 0x7fc36406ab10 2023-11-28T12:03:33.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.270+0000 7fc35b7fe700 1 -- 172.21.15.57:0/2612675881 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c00ca50 con 0x7fc36406ab10 2023-11-28T12:03:33.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.271+0000 7fc36970f700 1 -- 172.21.15.57:0/2612675881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3640d1b70 con 0x7fc36406ab10 2023-11-28T12:03:33.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.272+0000 7fc36970f700 1 -- 172.21.15.57:0/2612675881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3640b1ec0 con 0x7fc36406ab10 2023-11-28T12:03:33.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.272+0000 7fc35b7fe700 1 -- 172.21.15.57:0/2612675881 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc34c019030 con 0x7fc36406ab10 2023-11-28T12:03:33.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.272+0000 7fc35b7fe700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc3440682f0 0x7fc34406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:33.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.273+0000 7fc35b7fe700 1 -- 172.21.15.57:0/2612675881 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fc34c085ab0 con 0x7fc36406ab10 2023-11-28T12:03:33.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.273+0000 7fc3627fc700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc3440682f0 0x7fc34406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:33.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.273+0000 7fc3627fc700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc3440682f0 0x7fc34406a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc3640c9780 tx=0x7fc354009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:33.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.277+0000 7fc35b7fe700 1 -- 172.21.15.57:0/2612675881 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc34c0522f0 con 0x7fc36406ab10 2023-11-28T12:03:33.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.525+0000 7fc36970f700 1 -- 172.21.15.57:0/2612675881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3640ca840 con 0x7fc36406ab10 2023-11-28T12:03:33.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.525+0000 7fc35b7fe700 1 -- 172.21.15.57:0/2612675881 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4648 (secure 0 0 0) 0x7fc34c055910 con 0x7fc36406ab10 2023-11-28T12:03:33.527 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:33.527 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/340292109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":340292109},{"type":"v1","addr":"172.21.15.181:6836","nonce":340292109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2442321825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2442321825},{"type":"v1","addr":"172.21.15.181:6837","nonce":2442321825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/3452242211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3452242211},{"type":"v1","addr":"172.21.15.57:6833","nonce":3452242211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:02:53.402391+0000","modified":"2023-11-28T12:02:56.540229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.57:6835/968299041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":968299041},{"type":"v1","addr":"172.21.15.57:6835","nonce":968299041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T12:03:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.528+0000 7fc36970f700 1 -- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc3440682f0 msgr2=0x7fc34406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.528+0000 7fc36970f700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc3440682f0 0x7fc34406a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc3640c9780 tx=0x7fc354009040 comp rx=0 tx=0).stop 2023-11-28T12:03:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.528+0000 7fc36970f700 1 -- 172.21.15.57:0/2612675881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36406ab10 msgr2=0x7fc3640c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.528+0000 7fc36970f700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36406ab10 0x7fc3640c6170 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00e870 tx=0x7fc34c00ec30 comp rx=0 tx=0).stop 2023-11-28T12:03:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.528+0000 7fc36970f700 1 -- 172.21.15.57:0/2612675881 shutdown_connections 2023-11-28T12:03:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.528+0000 7fc36970f700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc3440682f0 0x7fc34406a7a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.528+0000 7fc36970f700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3640d0450 0x7fc36415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.528+0000 7fc36970f700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc36406ab10 0x7fc3640c6170 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.528+0000 7fc36970f700 1 --2- 172.21.15.57:0/2612675881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3640cd5a0 0x7fc364158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.528+0000 7fc36970f700 1 -- 172.21.15.57:0/2612675881 >> 172.21.15.57:0/2612675881 conn(0x7fc36406c520 msgr2=0x7fc3640b8530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.528+0000 7fc36970f700 1 -- 172.21.15.57:0/2612675881 shutdown_connections 2023-11-28T12:03:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.528+0000 7fc36970f700 1 -- 172.21.15.57:0/2612675881 wait complete. 2023-11-28T12:03:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 16 2023-11-28T12:03:33.543 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:03:33.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.677+0000 7f5fe95c3700 1 Processor -- start 2023-11-28T12:03:33.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.677+0000 7f5fe95c3700 1 -- start start 2023-11-28T12:03:33.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.678+0000 7f5fe95c3700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5fe40ad900 0x7f5fe4072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:33.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.678+0000 7f5fe95c3700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe40726c0 con 0x7f5fe406b7b0 2023-11-28T12:03:33.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.678+0000 7f5fe95c3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe40b43f0 con 0x7f5fe40b0880 2023-11-28T12:03:33.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.678+0000 7f5fe95c3700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe406bc50 con 0x7f5fe40ad900 2023-11-28T12:03:33.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.678+0000 7f5fe37fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5fe40ad900 0x7f5fe4072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:33.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.678+0000 7f5fe27fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f5fe406b7b0 0x7f5fe40ad1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47396/0 (socket says 172.21.15.57:47396) 2023-11-28T12:03:33.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.678+0000 7f5fe37fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5fe40ad900 0x7f5fe4072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35950/0 (socket says 172.21.15.57:35950) 2023-11-28T12:03:33.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.678+0000 7f5fe27fc700 1 -- 172.21.15.57:0/2333320414 learned_addr learned my addr 172.21.15.57:0/2333320414 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:33.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.678+0000 7f5fe37fe700 1 -- 172.21.15.57:0/2333320414 >> v1:172.21.15.181:6789/0 conn(0x7f5fe40b0880 legacy=0x7f5fe40aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:33.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.678+0000 7f5fe37fe700 1 -- 172.21.15.57:0/2333320414 >> v1:172.21.15.57:6789/0 conn(0x7f5fe406b7b0 legacy=0x7f5fe40ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.678+0000 7f5fe37fe700 1 -- 172.21.15.57:0/2333320414 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe40b46d0 con 0x7f5fe40ad900 2023-11-28T12:03:33.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.678+0000 7f5fe1ffb700 1 -- 172.21.15.57:0/2333320414 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513766849 0 0) 0x7f5fe40726c0 con 0x7f5fe406b7b0 2023-11-28T12:03:33.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.678+0000 7f5fe37fe700 1 --2- 172.21.15.57:0/2333320414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fe40ad900 0x7f5fe4072180 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009d90 tx=0x7f5fdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319cf80daa4f5297 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:33.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.679+0000 7f5fe1ffb700 1 -- 172.21.15.57:0/2333320414 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00e070 con 0x7f5fe40ad900 2023-11-28T12:03:33.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.679+0000 7f5fe1ffb700 1 -- 172.21.15.57:0/2333320414 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc00bb90 con 0x7f5fe40ad900 2023-11-28T12:03:33.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.679+0000 7f5fe1ffb700 1 -- 172.21.15.57:0/2333320414 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc003a20 con 0x7f5fe40ad900 2023-11-28T12:03:33.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.679+0000 7f5fe95c3700 1 -- 172.21.15.57:0/2333320414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fe40ad900 msgr2=0x7f5fe4072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.679+0000 7f5fe95c3700 1 --2- 172.21.15.57:0/2333320414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fe40ad900 0x7f5fe4072180 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009d90 tx=0x7f5fdc00b330 comp rx=0 tx=0).stop 2023-11-28T12:03:33.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.679+0000 7f5fe95c3700 1 -- 172.21.15.57:0/2333320414 shutdown_connections 2023-11-28T12:03:33.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.679+0000 7f5fe95c3700 1 --2- 172.21.15.57:0/2333320414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fe40ad900 0x7f5fe4072180 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.679+0000 7f5fe95c3700 1 -- 172.21.15.57:0/2333320414 >> 172.21.15.57:0/2333320414 conn(0x7f5fe40b2fc0 msgr2=0x7f5fe4071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:33.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.679+0000 7f5fe95c3700 1 -- 172.21.15.57:0/2333320414 shutdown_connections 2023-11-28T12:03:33.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.679+0000 7f5fe95c3700 1 -- 172.21.15.57:0/2333320414 wait complete. 2023-11-28T12:03:33.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.681+0000 7f5fe95c3700 1 Processor -- start 2023-11-28T12:03:33.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.681+0000 7f5fe95c3700 1 -- start start 2023-11-28T12:03:33.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.681+0000 7f5fe95c3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fe406b7b0 0x7f5fe40cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:33.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.681+0000 7f5fe95c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fe40ad900 0x7f5fe41e6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:33.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.681+0000 7f5fe95c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fe40b0880 0x7f5fe41e6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:33.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.681+0000 7f5fe95c3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe40997d0 con 0x7f5fe406b7b0 2023-11-28T12:03:33.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.681+0000 7f5fe95c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe41f2f90 con 0x7f5fe40ad900 2023-11-28T12:03:33.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.681+0000 7f5fe95c3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe41f2cb0 con 0x7f5fe40b0880 2023-11-28T12:03:33.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fe2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fe406b7b0 0x7f5fe40cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:33.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fe2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fe406b7b0 0x7f5fe40cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35956/0 (socket says 172.21.15.57:35956) 2023-11-28T12:03:33.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fe2ffd700 1 -- 172.21.15.57:0/1085462046 learned_addr learned my addr 172.21.15.57:0/1085462046 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:33.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fe37fe700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fe40b0880 0x7f5fe41e6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:33.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fe27fc700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fe40ad900 0x7f5fe41e6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:33.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fe2ffd700 1 -- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fe40b0880 msgr2=0x7f5fe41e6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fe2ffd700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fe40b0880 0x7f5fe41e6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fe2ffd700 1 -- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fe40ad900 msgr2=0x7f5fe41e6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fe2ffd700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fe40ad900 0x7f5fe41e6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fe2ffd700 1 -- 172.21.15.57:0/1085462046 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc009a70 con 0x7f5fe406b7b0 2023-11-28T12:03:33.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fe2ffd700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fe406b7b0 0x7f5fe40cfaa0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400c4d0 tx=0x7f5fd400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:33.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fcb7fe700 1 -- 172.21.15.57:0/1085462046 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4011790 con 0x7f5fe406b7b0 2023-11-28T12:03:33.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fcb7fe700 1 -- 172.21.15.57:0/1085462046 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd40118f0 con 0x7f5fe406b7b0 2023-11-28T12:03:33.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fe95c3700 1 -- 172.21.15.57:0/1085462046 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe40b4fe0 con 0x7f5fe406b7b0 2023-11-28T12:03:33.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fcb7fe700 1 -- 172.21.15.57:0/1085462046 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4011ca0 con 0x7f5fe406b7b0 2023-11-28T12:03:33.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.682+0000 7f5fe95c3700 1 -- 172.21.15.57:0/1085462046 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe40b4d00 con 0x7f5fe406b7b0 2023-11-28T12:03:33.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.683+0000 7f5fe95c3700 1 -- 172.21.15.57:0/1085462046 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe40b49b0 con 0x7f5fe406b7b0 2023-11-28T12:03:33.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.684+0000 7f5fcb7fe700 1 -- 172.21.15.57:0/1085462046 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5fd4010050 con 0x7f5fe406b7b0 2023-11-28T12:03:33.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.684+0000 7f5fcb7fe700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5fd00682f0 0x7f5fd006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:33.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.684+0000 7f5fcb7fe700 1 -- 172.21.15.57:0/1085462046 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5fd40859a0 con 0x7f5fe406b7b0 2023-11-28T12:03:33.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.684+0000 7f5fe27fc700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5fd00682f0 0x7f5fd006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:33.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.685+0000 7f5fe27fc700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5fd00682f0 0x7f5fd006a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5fe415c8a0 tx=0x7f5fd8007c90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:33.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.689+0000 7f5fcb7fe700 1 -- 172.21.15.57:0/1085462046 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5fd40521e0 con 0x7f5fe406b7b0 2023-11-28T12:03:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.895+0000 7f5fe95c3700 1 -- 172.21.15.57:0/1085462046 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fe40743f0 con 0x7f5fe406b7b0 2023-11-28T12:03:33.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.897+0000 7f5fcb7fe700 1 -- 172.21.15.57:0/1085462046 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17970 (secure 0 0 0) 0x7f5fd4085020 con 0x7f5fe406b7b0 2023-11-28T12:03:33.898 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:33.898 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":37,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:03:33.760012+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:02:51.245666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T12:02:52.363863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:03:33.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.899+0000 7f5fe95c3700 1 -- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5fd00682f0 msgr2=0x7f5fd006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.899+0000 7f5fe95c3700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5fd00682f0 0x7f5fd006a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5fe415c8a0 tx=0x7f5fd8007c90 comp rx=0 tx=0).stop 2023-11-28T12:03:33.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.899+0000 7f5fe95c3700 1 -- 172.21.15.57:0/1085462046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fe406b7b0 msgr2=0x7f5fe40cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:33.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.899+0000 7f5fe95c3700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fe406b7b0 0x7f5fe40cfaa0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400c4d0 tx=0x7f5fd400c890 comp rx=0 tx=0).stop 2023-11-28T12:03:33.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.899+0000 7f5fe95c3700 1 -- 172.21.15.57:0/1085462046 shutdown_connections 2023-11-28T12:03:33.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.899+0000 7f5fe95c3700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5fd00682f0 0x7f5fd006a7a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.899+0000 7f5fe95c3700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fe40b0880 0x7f5fe41e6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.899+0000 7f5fe95c3700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fe406b7b0 0x7f5fe40cfaa0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.899+0000 7f5fe95c3700 1 --2- 172.21.15.57:0/1085462046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fe40ad900 0x7f5fe41e6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:33.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.899+0000 7f5fe95c3700 1 -- 172.21.15.57:0/1085462046 >> 172.21.15.57:0/1085462046 conn(0x7f5fe40b2fc0 msgr2=0x7f5fe40aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:33.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.899+0000 7f5fe95c3700 1 -- 172.21.15.57:0/1085462046 shutdown_connections 2023-11-28T12:03:33.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:33.899+0000 7f5fe95c3700 1 -- 172.21.15.57:0/1085462046 wait complete. 2023-11-28T12:03:33.914 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T12:03:33.914 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:03:34.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.050+0000 7f09e773c700 1 Processor -- start 2023-11-28T12:03:34.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.050+0000 7f09e773c700 1 -- start start 2023-11-28T12:03:34.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e773c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f09e00b0880 0x7f09e006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e773c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f09e0072180 0x7f09e00aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e773c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e00af070 0x7f09e00af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e773c700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e0072620 con 0x7f09e0072180 2023-11-28T12:03:34.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e773c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e00b43f0 con 0x7f09e00b0880 2023-11-28T12:03:34.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e773c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00b46d0 con 0x7f09e00af070 2023-11-28T12:03:34.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e4cd7700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f09e0072180 0x7f09e00aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e4cd7700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f09e0072180 0x7f09e00aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35964/0 (socket says 172.21.15.57:35964) 2023-11-28T12:03:34.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e4cd7700 1 -- 172.21.15.57:0/2393819384 learned_addr learned my addr 172.21.15.57:0/2393819384 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:34.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e5cd9700 1 --2- 172.21.15.57:0/2393819384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e00af070 0x7f09e00af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e54d8700 1 --2- 172.21.15.57:0/2393819384 >> v2:172.21.15.181:3300/0 conn(0x7f09e00b0880 0x7f09e006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e4cd7700 1 -- 172.21.15.57:0/2393819384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e00af070 msgr2=0x7f09e00af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:34.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e4cd7700 1 --2- 172.21.15.57:0/2393819384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e00af070 0x7f09e00af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e4cd7700 1 -- 172.21.15.57:0/2393819384 >> v2:172.21.15.181:3300/0 conn(0x7f09e00b0880 msgr2=0x7f09e006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:34.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e4cd7700 1 --2- 172.21.15.57:0/2393819384 >> v2:172.21.15.181:3300/0 conn(0x7f09e00b0880 0x7f09e006b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.051+0000 7f09e4cd7700 1 -- 172.21.15.57:0/2393819384 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e00b4d00 con 0x7f09e0072180 2023-11-28T12:03:34.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.052+0000 7f09e4cd7700 1 --2- 172.21.15.57:0/2393819384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09e0072180 0x7f09e00aeb30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f09e006efd0 tx=0x7f09d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d6020914e7d8a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:34.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.052+0000 7f09d3fff700 1 -- 172.21.15.57:0/2393819384 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d400b870 con 0x7f09e0072180 2023-11-28T12:03:34.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.052+0000 7f09d3fff700 1 -- 172.21.15.57:0/2393819384 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d400b9d0 con 0x7f09e0072180 2023-11-28T12:03:34.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.052+0000 7f09d3fff700 1 -- 172.21.15.57:0/2393819384 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d400bd60 con 0x7f09e0072180 2023-11-28T12:03:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.052+0000 7f09e773c700 1 -- 172.21.15.57:0/2393819384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09e0072180 msgr2=0x7f09e00aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.052+0000 7f09e773c700 1 --2- 172.21.15.57:0/2393819384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09e0072180 0x7f09e00aeb30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f09e006efd0 tx=0x7f09d4009bd0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.052+0000 7f09e773c700 1 -- 172.21.15.57:0/2393819384 shutdown_connections 2023-11-28T12:03:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.052+0000 7f09e773c700 1 --2- 172.21.15.57:0/2393819384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e00af070 0x7f09e00af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.052+0000 7f09e773c700 1 --2- 172.21.15.57:0/2393819384 >> v2:172.21.15.181:3300/0 conn(0x7f09e00b0880 0x7f09e006b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.052+0000 7f09e773c700 1 --2- 172.21.15.57:0/2393819384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09e0072180 0x7f09e00aeb30 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.052+0000 7f09e773c700 1 -- 172.21.15.57:0/2393819384 >> 172.21.15.57:0/2393819384 conn(0x7f09e00b2fc0 msgr2=0x7f09e0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:34.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.053+0000 7f09e773c700 1 -- 172.21.15.57:0/2393819384 shutdown_connections 2023-11-28T12:03:34.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.053+0000 7f09e773c700 1 -- 172.21.15.57:0/2393819384 wait complete. 2023-11-28T12:03:34.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.054+0000 7f09e773c700 1 Processor -- start 2023-11-28T12:03:34.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.054+0000 7f09e773c700 1 -- start start 2023-11-28T12:03:34.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e773c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e0072180 0x7f09e00ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e773c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09e00af070 0x7f09e01f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e773c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e00b0880 0x7f09e00b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e773c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00cf3e0 con 0x7f09e00af070 2023-11-28T12:03:34.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e773c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00c8a00 con 0x7f09e00b0880 2023-11-28T12:03:34.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e773c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e0099800 con 0x7f09e0072180 2023-11-28T12:03:34.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e4cd7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09e00af070 0x7f09e01f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e4cd7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09e00af070 0x7f09e01f1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35974/0 (socket says 172.21.15.57:35974) 2023-11-28T12:03:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e4cd7700 1 -- 172.21.15.57:0/742087579 learned_addr learned my addr 172.21.15.57:0/742087579 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e54d8700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e0072180 0x7f09e00ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e5cd9700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e00b0880 0x7f09e00b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e4cd7700 1 -- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e0072180 msgr2=0x7f09e00ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e4cd7700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e0072180 0x7f09e00ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e4cd7700 1 -- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e00b0880 msgr2=0x7f09e00b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e4cd7700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e00b0880 0x7f09e00b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e4cd7700 1 -- 172.21.15.57:0/742087579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d4009550 con 0x7f09e00af070 2023-11-28T12:03:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09e4cd7700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09e00af070 0x7f09e01f1e40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f09d4000c00 tx=0x7f09d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.055+0000 7f09d1ffb700 1 -- 172.21.15.57:0/742087579 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d4004520 con 0x7f09e00af070 2023-11-28T12:03:34.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.056+0000 7f09d1ffb700 1 -- 172.21.15.57:0/742087579 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d401e070 con 0x7f09e00af070 2023-11-28T12:03:34.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.056+0000 7f09d1ffb700 1 -- 172.21.15.57:0/742087579 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d40038c0 con 0x7f09e00af070 2023-11-28T12:03:34.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.056+0000 7f09e773c700 1 -- 172.21.15.57:0/742087579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e00b49b0 con 0x7f09e00af070 2023-11-28T12:03:34.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.056+0000 7f09e773c700 1 -- 172.21.15.57:0/742087579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e00b4fe0 con 0x7f09e00af070 2023-11-28T12:03:34.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.057+0000 7f09e773c700 1 -- 172.21.15.57:0/742087579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e00b0c50 con 0x7f09e00af070 2023-11-28T12:03:34.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.057+0000 7f09d1ffb700 1 -- 172.21.15.57:0/742087579 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f09d400f050 con 0x7f09e00af070 2023-11-28T12:03:34.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.058+0000 7f09d1ffb700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f09c40682f0 0x7f09c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.058+0000 7f09d1ffb700 1 -- 172.21.15.57:0/742087579 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f09d4020070 con 0x7f09e00af070 2023-11-28T12:03:34.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.058+0000 7f09e54d8700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f09c40682f0 0x7f09c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.058+0000 7f09e54d8700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f09c40682f0 0x7f09c406a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f09e01e5a60 tx=0x7f09dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:34.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.062+0000 7f09d1ffb700 1 -- 172.21.15.57:0/742087579 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f09d4053a20 con 0x7f09e00af070 2023-11-28T12:03:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.307+0000 7f09e773c700 1 -- 172.21.15.57:0/742087579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09e00743f0 con 0x7f09e00af070 2023-11-28T12:03:34.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.307+0000 7f09d1ffb700 1 -- 172.21.15.57:0/742087579 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f09d4057040 con 0x7f09e00af070 2023-11-28T12:03:34.309 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:34.310 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/340292109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":340292109},{"type":"v1","addr":"172.21.15.181:6836","nonce":340292109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2442321825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2442321825},{"type":"v1","addr":"172.21.15.181:6837","nonce":2442321825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/3452242211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3452242211},{"type":"v1","addr":"172.21.15.57:6833","nonce":3452242211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:02:53.402391+0000","modified":"2023-11-28T12:02:56.540229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.57:6835/968299041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":968299041},{"type":"v1","addr":"172.21.15.57:6835","nonce":968299041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T12:03:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.310+0000 7f09e773c700 1 -- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f09c40682f0 msgr2=0x7f09c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.310+0000 7f09e773c700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f09c40682f0 0x7f09c406a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f09e01e5a60 tx=0x7f09dc008040 comp rx=0 tx=0).stop 2023-11-28T12:03:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.310+0000 7f09e773c700 1 -- 172.21.15.57:0/742087579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09e00af070 msgr2=0x7f09e01f1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.310+0000 7f09e773c700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09e00af070 0x7f09e01f1e40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f09d4000c00 tx=0x7f09d4004750 comp rx=0 tx=0).stop 2023-11-28T12:03:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.310+0000 7f09e773c700 1 -- 172.21.15.57:0/742087579 shutdown_connections 2023-11-28T12:03:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.310+0000 7f09e773c700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f09c40682f0 0x7f09c406a7a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.310+0000 7f09e773c700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e00b0880 0x7f09e00b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.310+0000 7f09e773c700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e0072180 0x7f09e00ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.310+0000 7f09e773c700 1 --2- 172.21.15.57:0/742087579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09e00af070 0x7f09e01f1e40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.310+0000 7f09e773c700 1 -- 172.21.15.57:0/742087579 >> 172.21.15.57:0/742087579 conn(0x7f09e00b2fc0 msgr2=0x7f09e015f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.310+0000 7f09e773c700 1 -- 172.21.15.57:0/742087579 shutdown_connections 2023-11-28T12:03:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.310+0000 7f09e773c700 1 -- 172.21.15.57:0/742087579 wait complete. 2023-11-28T12:03:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 16 2023-11-28T12:03:34.324 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:03:34.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.459+0000 7f68c654c700 1 Processor -- start 2023-11-28T12:03:34.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.459+0000 7f68c654c700 1 -- start start 2023-11-28T12:03:34.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.460+0000 7f68c654c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00cff90 0x7f68c015f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.460+0000 7f68c654c700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c00cfb00 con 0x7f68c0071f70 2023-11-28T12:03:34.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.460+0000 7f68c654c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c015aa20 con 0x7f68c00b4660 2023-11-28T12:03:34.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.460+0000 7f68c654c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c00cf4d0 con 0x7f68c00cff90 2023-11-28T12:03:34.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.460+0000 7f68bf7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f68c0071f70 0x7f68c00b4a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40716/0 (socket says 172.21.15.57:40716) 2023-11-28T12:03:34.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.460+0000 7f68bf7fe700 1 -- 172.21.15.57:0/4119212773 learned_addr learned my addr 172.21.15.57:0/4119212773 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:34.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.460+0000 7f68c4ae9700 1 --2- 172.21.15.57:0/4119212773 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00cff90 0x7f68c015f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.461+0000 7f68beffd700 1 -- 172.21.15.57:0/4119212773 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170987415 0 0) 0x7f68c00cfb00 con 0x7f68c0071f70 2023-11-28T12:03:34.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.461+0000 7f68beffd700 1 -- 172.21.15.57:0/4119212773 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68a80035b0 con 0x7f68c0071f70 2023-11-28T12:03:34.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.461+0000 7f68beffd700 1 -- 172.21.15.57:0/4119212773 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743416589 0 0) 0x7f68c015aa20 con 0x7f68c00b4660 2023-11-28T12:03:34.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.461+0000 7f68beffd700 1 -- 172.21.15.57:0/4119212773 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68c00cfb00 con 0x7f68c00b4660 2023-11-28T12:03:34.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.461+0000 7f68c4ae9700 1 -- 172.21.15.57:0/4119212773 >> v1:172.21.15.181:6789/0 conn(0x7f68c00b4660 legacy=0x7f68c006b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.461+0000 7f68c4ae9700 1 -- 172.21.15.57:0/4119212773 >> v1:172.21.15.57:6789/0 conn(0x7f68c0071f70 legacy=0x7f68c00b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.461+0000 7f68c4ae9700 1 -- 172.21.15.57:0/4119212773 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c0072550 con 0x7f68c00cff90 2023-11-28T12:03:34.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.461+0000 7f68c4ae9700 1 --2- 172.21.15.57:0/4119212773 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00cff90 0x7f68c015f170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009dc0 tx=0x7f68b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c4a89fab2296d5b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:34.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.461+0000 7f68beffd700 1 -- 172.21.15.57:0/4119212773 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b800e070 con 0x7f68c00cff90 2023-11-28T12:03:34.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.461+0000 7f68beffd700 1 -- 172.21.15.57:0/4119212773 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b800bb20 con 0x7f68c00cff90 2023-11-28T12:03:34.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.461+0000 7f68beffd700 1 -- 172.21.15.57:0/4119212773 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8012430 con 0x7f68c00cff90 2023-11-28T12:03:34.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.462+0000 7f68c654c700 1 -- 172.21.15.57:0/4119212773 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00cff90 msgr2=0x7f68c015f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.462+0000 7f68c654c700 1 --2- 172.21.15.57:0/4119212773 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00cff90 0x7f68c015f170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009dc0 tx=0x7f68b800b330 comp rx=0 tx=0).stop 2023-11-28T12:03:34.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.462+0000 7f68c654c700 1 -- 172.21.15.57:0/4119212773 shutdown_connections 2023-11-28T12:03:34.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.462+0000 7f68c654c700 1 --2- 172.21.15.57:0/4119212773 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00cff90 0x7f68c015f170 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.462+0000 7f68c654c700 1 -- 172.21.15.57:0/4119212773 >> 172.21.15.57:0/4119212773 conn(0x7f68c00b2fc0 msgr2=0x7f68c0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:34.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.462+0000 7f68c654c700 1 -- 172.21.15.57:0/4119212773 shutdown_connections 2023-11-28T12:03:34.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.462+0000 7f68c654c700 1 -- 172.21.15.57:0/4119212773 wait complete. 2023-11-28T12:03:34.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.464+0000 7f68c654c700 1 Processor -- start 2023-11-28T12:03:34.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.464+0000 7f68c654c700 1 -- start start 2023-11-28T12:03:34.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68c654c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68c0071f70 0x7f68c00c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68c654c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00b4660 0x7f68c01e5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68c654c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00cff90 0x7f68c00c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68c654c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c00c7c10 con 0x7f68c0071f70 2023-11-28T12:03:34.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68c654c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c00c6d00 con 0x7f68c00cff90 2023-11-28T12:03:34.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68c654c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c00c6a20 con 0x7f68c00b4660 2023-11-28T12:03:34.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68bffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68c0071f70 0x7f68c00c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68bffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68c0071f70 0x7f68c00c5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45880/0 (socket says 172.21.15.57:45880) 2023-11-28T12:03:34.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68bffff700 1 -- 172.21.15.57:0/3243000923 learned_addr learned my addr 172.21.15.57:0/3243000923 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:34.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68bf7fe700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00b4660 0x7f68c01e5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68c4ae9700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00cff90 0x7f68c00c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68bffff700 1 -- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00b4660 msgr2=0x7f68c01e5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68bffff700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00b4660 0x7f68c01e5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68bffff700 1 -- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00cff90 msgr2=0x7f68c00c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68bffff700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00cff90 0x7f68c00c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68bffff700 1 -- 172.21.15.57:0/3243000923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b8009a70 con 0x7f68c0071f70 2023-11-28T12:03:34.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.465+0000 7f68bffff700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68c0071f70 0x7f68c00c5550 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c520 tx=0x7f68b000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:34.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.466+0000 7f68bcff9700 1 -- 172.21.15.57:0/3243000923 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b00052d0 con 0x7f68c0071f70 2023-11-28T12:03:34.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.466+0000 7f68bcff9700 1 -- 172.21.15.57:0/3243000923 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b0005430 con 0x7f68c0071f70 2023-11-28T12:03:34.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.466+0000 7f68c654c700 1 -- 172.21.15.57:0/3243000923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c00b5140 con 0x7f68c0071f70 2023-11-28T12:03:34.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.466+0000 7f68bcff9700 1 -- 172.21.15.57:0/3243000923 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b00038c0 con 0x7f68c0071f70 2023-11-28T12:03:34.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.466+0000 7f68c654c700 1 -- 172.21.15.57:0/3243000923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c00cf790 con 0x7f68c0071f70 2023-11-28T12:03:34.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.467+0000 7f68c654c700 1 -- 172.21.15.57:0/3243000923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c015f6b0 con 0x7f68c0071f70 2023-11-28T12:03:34.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.467+0000 7f68bcff9700 1 -- 172.21.15.57:0/3243000923 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f68b0019030 con 0x7f68c0071f70 2023-11-28T12:03:34.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.468+0000 7f68bcff9700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f68ac0682f0 0x7f68ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.468+0000 7f68bcff9700 1 -- 172.21.15.57:0/3243000923 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f68b0084fd0 con 0x7f68c0071f70 2023-11-28T12:03:34.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.468+0000 7f68bf7fe700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f68ac0682f0 0x7f68ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.468+0000 7f68bf7fe700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f68ac0682f0 0x7f68ac06a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68c00c9640 tx=0x7f68b4007dc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:34.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.472+0000 7f68bcff9700 1 -- 172.21.15.57:0/3243000923 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f68b0051810 con 0x7f68c0071f70 2023-11-28T12:03:34.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.718+0000 7f68c654c700 1 -- 172.21.15.57:0/3243000923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68c00743f0 con 0x7f68c0071f70 2023-11-28T12:03:34.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.719+0000 7f68bcff9700 1 -- 172.21.15.57:0/3243000923 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f68b0054e30 con 0x7f68c0071f70 2023-11-28T12:03:34.720 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:34.720 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/340292109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":340292109},{"type":"v1","addr":"172.21.15.181:6836","nonce":340292109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2442321825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2442321825},{"type":"v1","addr":"172.21.15.181:6837","nonce":2442321825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/3452242211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3452242211},{"type":"v1","addr":"172.21.15.57:6833","nonce":3452242211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:02:53.402391+0000","modified":"2023-11-28T12:02:56.540229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.57:6835/968299041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":968299041},{"type":"v1","addr":"172.21.15.57:6835","nonce":968299041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T12:03:34.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.721+0000 7f68c654c700 1 -- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f68ac0682f0 msgr2=0x7f68ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.721+0000 7f68c654c700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f68ac0682f0 0x7f68ac06a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68c00c9640 tx=0x7f68b4007dc0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.721+0000 7f68c654c700 1 -- 172.21.15.57:0/3243000923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68c0071f70 msgr2=0x7f68c00c5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.721+0000 7f68c654c700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68c0071f70 0x7f68c00c5550 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c520 tx=0x7f68b000c830 comp rx=0 tx=0).stop 2023-11-28T12:03:34.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.721+0000 7f68c654c700 1 -- 172.21.15.57:0/3243000923 shutdown_connections 2023-11-28T12:03:34.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.721+0000 7f68c654c700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f68ac0682f0 0x7f68ac06a7a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.721+0000 7f68c654c700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00cff90 0x7f68c00c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.721+0000 7f68c654c700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68c0071f70 0x7f68c00c5550 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.721+0000 7f68c654c700 1 --2- 172.21.15.57:0/3243000923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68c00b4660 0x7f68c01e5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.721+0000 7f68c654c700 1 -- 172.21.15.57:0/3243000923 >> 172.21.15.57:0/3243000923 conn(0x7f68c00b2fc0 msgr2=0x7f68c0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.721+0000 7f68c654c700 1 -- 172.21.15.57:0/3243000923 shutdown_connections 2023-11-28T12:03:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.721+0000 7f68c654c700 1 -- 172.21.15.57:0/3243000923 wait complete. 2023-11-28T12:03:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 16 2023-11-28T12:03:34.736 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:03:34.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.870+0000 7fd1472f5700 1 Processor -- start 2023-11-28T12:03:34.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.870+0000 7fd1472f5700 1 -- start start 2023-11-28T12:03:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd1472f5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd14006a270 0x7fd140164690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd1472f5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd14006e620 0x7fd1401612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd1472f5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1400b1910 con 0x7fd14006a640 2023-11-28T12:03:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd1472f5700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1400b2590 con 0x7fd14006e620 2023-11-28T12:03:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd1472f5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14006eaa0 con 0x7fd14006a270 2023-11-28T12:03:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd145892700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd14006e620 0x7fd1401612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd145892700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd14006e620 0x7fd1401612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45896/0 (socket says 172.21.15.57:45896) 2023-11-28T12:03:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd145892700 1 -- 172.21.15.57:0/2693486415 learned_addr learned my addr 172.21.15.57:0/2693486415 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd145091700 1 --2- 172.21.15.57:0/2693486415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd14006a270 0x7fd140164690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd145892700 1 -- 172.21.15.57:0/2693486415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd14006a270 msgr2=0x7fd140164690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd145892700 1 --2- 172.21.15.57:0/2693486415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd14006a270 0x7fd140164690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd145892700 1 -- 172.21.15.57:0/2693486415 >> v1:172.21.15.181:6789/0 conn(0x7fd14006a640 legacy=0x7fd14006df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd145892700 1 -- 172.21.15.57:0/2693486415 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1400b1bf0 con 0x7fd14006e620 2023-11-28T12:03:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.871+0000 7fd145892700 1 --2- 172.21.15.57:0/2693486415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd14006e620 0x7fd1401612c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd138009e50 tx=0x7fd13800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd4ff089e13b3db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.872+0000 7fd137fff700 1 -- 172.21.15.57:0/2693486415 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13800e070 con 0x7fd14006e620 2023-11-28T12:03:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.872+0000 7fd137fff700 1 -- 172.21.15.57:0/2693486415 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13800b9c0 con 0x7fd14006e620 2023-11-28T12:03:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.872+0000 7fd137fff700 1 -- 172.21.15.57:0/2693486415 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13800bd70 con 0x7fd14006e620 2023-11-28T12:03:34.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.872+0000 7fd1472f5700 1 -- 172.21.15.57:0/2693486415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd14006e620 msgr2=0x7fd1401612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.872+0000 7fd1472f5700 1 --2- 172.21.15.57:0/2693486415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd14006e620 0x7fd1401612c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd138009e50 tx=0x7fd13800b330 comp rx=0 tx=0).stop 2023-11-28T12:03:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.872+0000 7fd1472f5700 1 -- 172.21.15.57:0/2693486415 shutdown_connections 2023-11-28T12:03:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.872+0000 7fd1472f5700 1 --2- 172.21.15.57:0/2693486415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd14006e620 0x7fd1401612c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.872+0000 7fd1472f5700 1 --2- 172.21.15.57:0/2693486415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd14006a270 0x7fd140164690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.872+0000 7fd1472f5700 1 -- 172.21.15.57:0/2693486415 >> 172.21.15.57:0/2693486415 conn(0x7fd14006c010 msgr2=0x7fd1400cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.872+0000 7fd1472f5700 1 -- 172.21.15.57:0/2693486415 shutdown_connections 2023-11-28T12:03:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.872+0000 7fd1472f5700 1 -- 172.21.15.57:0/2693486415 wait complete. 2023-11-28T12:03:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.874+0000 7fd1472f5700 1 Processor -- start 2023-11-28T12:03:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.874+0000 7fd1472f5700 1 -- start start 2023-11-28T12:03:34.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.875+0000 7fd1472f5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd14006a270 0x7fd1400afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.875+0000 7fd1472f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14006a640 0x7fd140148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.875+0000 7fd1472f5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd14006e620 0x7fd140160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.875+0000 7fd1472f5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14015bb80 con 0x7fd14006a270 2023-11-28T12:03:34.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.875+0000 7fd1472f5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400b0450 con 0x7fd14006a640 2023-11-28T12:03:34.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.875+0000 7fd1472f5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400b0730 con 0x7fd14006e620 2023-11-28T12:03:34.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.875+0000 7fd145091700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd14006a270 0x7fd1400afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.875+0000 7fd145091700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd14006a270 0x7fd1400afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45902/0 (socket says 172.21.15.57:45902) 2023-11-28T12:03:34.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.875+0000 7fd145091700 1 -- 172.21.15.57:0/1837269915 learned_addr learned my addr 172.21.15.57:0/1837269915 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:34.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.875+0000 7fd145892700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd14006e620 0x7fd140160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.875+0000 7fd144890700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14006a640 0x7fd140148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.876+0000 7fd145091700 1 -- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd14006e620 msgr2=0x7fd140160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.876+0000 7fd145091700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd14006e620 0x7fd140160660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.876+0000 7fd145091700 1 -- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14006a640 msgr2=0x7fd140148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:34.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.876+0000 7fd145091700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14006a640 0x7fd140148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:34.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.876+0000 7fd145091700 1 -- 172.21.15.57:0/1837269915 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd138009b30 con 0x7fd14006a270 2023-11-28T12:03:34.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.876+0000 7fd144890700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14006a640 0x7fd140148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:34.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.876+0000 7fd145091700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd14006a270 0x7fd1400afe70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e810 tx=0x7fd13000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:34.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.876+0000 7fd135ffb700 1 -- 172.21.15.57:0/1837269915 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130013610 con 0x7fd14006a270 2023-11-28T12:03:34.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.876+0000 7fd135ffb700 1 -- 172.21.15.57:0/1837269915 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd130013770 con 0x7fd14006a270 2023-11-28T12:03:34.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.876+0000 7fd1472f5700 1 -- 172.21.15.57:0/1837269915 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1400b21b0 con 0x7fd14006a270 2023-11-28T12:03:34.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.876+0000 7fd135ffb700 1 -- 172.21.15.57:0/1837269915 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130013b20 con 0x7fd14006a270 2023-11-28T12:03:34.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.876+0000 7fd1472f5700 1 -- 172.21.15.57:0/1837269915 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1400b1ed0 con 0x7fd14006a270 2023-11-28T12:03:34.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.877+0000 7fd1472f5700 1 -- 172.21.15.57:0/1837269915 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14015e950 con 0x7fd14006a270 2023-11-28T12:03:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.878+0000 7fd135ffb700 1 -- 172.21.15.57:0/1837269915 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd130023070 con 0x7fd14006a270 2023-11-28T12:03:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.878+0000 7fd135ffb700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1240682f0 0x7fd12406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.878+0000 7fd135ffb700 1 -- 172.21.15.57:0/1837269915 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd130085940 con 0x7fd14006a270 2023-11-28T12:03:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.878+0000 7fd144890700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1240682f0 0x7fd12406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.879+0000 7fd144890700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1240682f0 0x7fd12406a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd13c000f60 tx=0x7fd13c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:34.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:34.883+0000 7fd135ffb700 1 -- 172.21.15.57:0/1837269915 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd130052100 con 0x7fd14006a270 2023-11-28T12:03:35.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.085+0000 7fd1472f5700 1 -- 172.21.15.57:0/1837269915 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd14015cc40 con 0x7fd14006a270 2023-11-28T12:03:35.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.086+0000 7fd135ffb700 1 -- 172.21.15.57:0/1837269915 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17970 (secure 0 0 0) 0x7fd130055720 con 0x7fd14006a270 2023-11-28T12:03:35.087 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:35.088 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":37,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:03:33.760012+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:02:51.245666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T12:02:52.363863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:03:35.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.088+0000 7fd1472f5700 1 -- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1240682f0 msgr2=0x7fd12406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:35.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.088+0000 7fd1472f5700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1240682f0 0x7fd12406a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd13c000f60 tx=0x7fd13c009040 comp rx=0 tx=0).stop 2023-11-28T12:03:35.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.088+0000 7fd1472f5700 1 -- 172.21.15.57:0/1837269915 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd14006a270 msgr2=0x7fd1400afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:35.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.088+0000 7fd1472f5700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd14006a270 0x7fd1400afe70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e810 tx=0x7fd13000eb20 comp rx=0 tx=0).stop 2023-11-28T12:03:35.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.088+0000 7fd1472f5700 1 -- 172.21.15.57:0/1837269915 shutdown_connections 2023-11-28T12:03:35.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.088+0000 7fd1472f5700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1240682f0 0x7fd12406a7a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.088+0000 7fd1472f5700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd14006e620 0x7fd140160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.088+0000 7fd1472f5700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd14006a270 0x7fd1400afe70 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.088+0000 7fd1472f5700 1 --2- 172.21.15.57:0/1837269915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14006a640 0x7fd140148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.088+0000 7fd1472f5700 1 -- 172.21.15.57:0/1837269915 >> 172.21.15.57:0/1837269915 conn(0x7fd14006c010 msgr2=0x7fd1400ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:35.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.088+0000 7fd1472f5700 1 -- 172.21.15.57:0/1837269915 shutdown_connections 2023-11-28T12:03:35.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.088+0000 7fd1472f5700 1 -- 172.21.15.57:0/1837269915 wait complete. 2023-11-28T12:03:35.103 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T12:03:35.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.238+0000 7f9fb489a700 1 Processor -- start 2023-11-28T12:03:35.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.238+0000 7f9fb489a700 1 -- start start 2023-11-28T12:03:35.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.239+0000 7f9fb489a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9fb015a590 0x7f9fb006b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:35.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.239+0000 7f9fb489a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb00cd480 con 0x7f9fb00b50d0 2023-11-28T12:03:35.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.239+0000 7f9fb489a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb00cd760 con 0x7f9fb006cb50 2023-11-28T12:03:35.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.239+0000 7f9fb489a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb00cda40 con 0x7f9fb015a590 2023-11-28T12:03:35.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.240+0000 7f9fae59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9fb015a590 0x7f9fb006b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.240+0000 7f9fadd9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f9fb00b50d0 0x7f9fb006c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40724/0 (socket says 172.21.15.57:40724) 2023-11-28T12:03:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.240+0000 7f9fae59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9fb015a590 0x7f9fb006b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45912/0 (socket says 172.21.15.57:45912) 2023-11-28T12:03:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.240+0000 7f9fadd9b700 1 -- 172.21.15.57:0/3016109934 learned_addr learned my addr 172.21.15.57:0/3016109934 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.240+0000 7f9fae59c700 1 -- 172.21.15.57:0/3016109934 >> v1:172.21.15.181:6789/0 conn(0x7f9fb006cb50 legacy=0x7f9fb006a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.240+0000 7f9fae59c700 1 -- 172.21.15.57:0/3016109934 >> v1:172.21.15.57:6789/0 conn(0x7f9fb00b50d0 legacy=0x7f9fb006c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.240+0000 7f9fae59c700 1 -- 172.21.15.57:0/3016109934 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb00cdd20 con 0x7f9fb015a590 2023-11-28T12:03:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.240+0000 7f9fad59a700 1 -- 172.21.15.57:0/3016109934 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975931029 0 0) 0x7f9fb00cd480 con 0x7f9fb00b50d0 2023-11-28T12:03:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.240+0000 7f9fae59c700 1 --2- 172.21.15.57:0/3016109934 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9fb015a590 0x7f9fb006b5b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009840 tx=0x7f9fa0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76974339af21dea8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.240+0000 7f9fad59a700 1 -- 172.21.15.57:0/3016109934 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0004230 con 0x7f9fb015a590 2023-11-28T12:03:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.240+0000 7f9fad59a700 1 -- 172.21.15.57:0/3016109934 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa0004390 con 0x7f9fb015a590 2023-11-28T12:03:35.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.240+0000 7f9fad59a700 1 -- 172.21.15.57:0/3016109934 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa00036a0 con 0x7f9fb015a590 2023-11-28T12:03:35.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.241+0000 7f9fb489a700 1 -- 172.21.15.57:0/3016109934 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9fb015a590 msgr2=0x7f9fb006b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:35.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.241+0000 7f9fb489a700 1 --2- 172.21.15.57:0/3016109934 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9fb015a590 0x7f9fb006b5b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009840 tx=0x7f9fa0009b50 comp rx=0 tx=0).stop 2023-11-28T12:03:35.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.241+0000 7f9fb489a700 1 -- 172.21.15.57:0/3016109934 shutdown_connections 2023-11-28T12:03:35.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.241+0000 7f9fb489a700 1 --2- 172.21.15.57:0/3016109934 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9fb015a590 0x7f9fb006b5b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.241+0000 7f9fb489a700 1 -- 172.21.15.57:0/3016109934 >> 172.21.15.57:0/3016109934 conn(0x7f9fb00cf400 msgr2=0x7f9fb00cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:35.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.241+0000 7f9fb489a700 1 -- 172.21.15.57:0/3016109934 shutdown_connections 2023-11-28T12:03:35.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.241+0000 7f9fb489a700 1 -- 172.21.15.57:0/3016109934 wait complete. 2023-11-28T12:03:35.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.243+0000 7f9fb489a700 1 Processor -- start 2023-11-28T12:03:35.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.243+0000 7f9fb489a700 1 -- start start 2023-11-28T12:03:35.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fb489a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fb006cb50 0x7f9fb01e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:35.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fb489a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9fb00b50d0 0x7f9fb015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:35.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fb489a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fb015a590 0x7f9fb00c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:35.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fb489a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb01e5f00 con 0x7f9fb00b50d0 2023-11-28T12:03:35.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fb489a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb015b1b0 con 0x7f9fb006cb50 2023-11-28T12:03:35.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fb489a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb015b490 con 0x7f9fb015a590 2023-11-28T12:03:35.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fadd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9fb00b50d0 0x7f9fb015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:35.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fadd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9fb00b50d0 0x7f9fb015a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45928/0 (socket says 172.21.15.57:45928) 2023-11-28T12:03:35.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fadd9b700 1 -- 172.21.15.57:0/4127632087 learned_addr learned my addr 172.21.15.57:0/4127632087 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:35.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fae59c700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fb006cb50 0x7f9fb01e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:35.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9faed9d700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fb015a590 0x7f9fb00c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:35.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fadd9b700 1 -- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fb015a590 msgr2=0x7f9fb00c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:35.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fadd9b700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fb015a590 0x7f9fb00c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fadd9b700 1 -- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fb006cb50 msgr2=0x7f9fb01e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:35.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fadd9b700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fb006cb50 0x7f9fb01e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.244+0000 7f9fadd9b700 1 -- 172.21.15.57:0/4127632087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa00094a0 con 0x7f9fb00b50d0 2023-11-28T12:03:35.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.245+0000 7f9fadd9b700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9fb00b50d0 0x7f9fb015a960 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400c520 tx=0x7f9fa400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:35.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.245+0000 7f9f9effd700 1 -- 172.21.15.57:0/4127632087 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4004c80 con 0x7f9fb00b50d0 2023-11-28T12:03:35.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.245+0000 7f9f9effd700 1 -- 172.21.15.57:0/4127632087 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa4004e00 con 0x7f9fb00b50d0 2023-11-28T12:03:35.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.245+0000 7f9fb489a700 1 -- 172.21.15.57:0/4127632087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb006baf0 con 0x7f9fb00b50d0 2023-11-28T12:03:35.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.245+0000 7f9f9effd700 1 -- 172.21.15.57:0/4127632087 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4007310 con 0x7f9fb00b50d0 2023-11-28T12:03:35.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.245+0000 7f9fb489a700 1 -- 172.21.15.57:0/4127632087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb00ce000 con 0x7f9fb00b50d0 2023-11-28T12:03:35.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.246+0000 7f9fb489a700 1 -- 172.21.15.57:0/4127632087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb00724c0 con 0x7f9fb00b50d0 2023-11-28T12:03:35.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.246+0000 7f9f9effd700 1 -- 172.21.15.57:0/4127632087 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9fa401a020 con 0x7f9fb00b50d0 2023-11-28T12:03:35.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.247+0000 7f9f9effd700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9f900682f0 0x7f9f9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:35.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.247+0000 7f9f9effd700 1 -- 172.21.15.57:0/4127632087 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f9fa4085760 con 0x7f9fb00b50d0 2023-11-28T12:03:35.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.247+0000 7f9fae59c700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9f900682f0 0x7f9f9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:35.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.248+0000 7f9fae59c700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9f900682f0 0x7f9f9006a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00090b0 tx=0x7f9fa0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:35.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.251+0000 7f9f9effd700 1 -- 172.21.15.57:0/4127632087 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9fa4051fa0 con 0x7f9fb00b50d0 2023-11-28T12:03:35.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.496+0000 7f9fb489a700 1 -- 172.21.15.57:0/4127632087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9fb01e6040 con 0x7f9fb00b50d0 2023-11-28T12:03:35.772 INFO:tasks.ceph.mon.a.smithi057.stderr:2023-11-28T12:03:35.771+0000 7fb777740700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T12:03:35.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.781+0000 7f9f9effd700 1 -- 172.21.15.57:0/4127632087 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9fb01e6040 con 0x7f9fb00b50d0 2023-11-28T12:03:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.784+0000 7f9fb489a700 1 -- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9f900682f0 msgr2=0x7f9f9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.784+0000 7f9fb489a700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9f900682f0 0x7f9f9006a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00090b0 tx=0x7f9fa0008030 comp rx=0 tx=0).stop 2023-11-28T12:03:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.784+0000 7f9fb489a700 1 -- 172.21.15.57:0/4127632087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9fb00b50d0 msgr2=0x7f9fb015a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.784+0000 7f9fb489a700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9fb00b50d0 0x7f9fb015a960 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400c520 tx=0x7f9fa400c830 comp rx=0 tx=0).stop 2023-11-28T12:03:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.784+0000 7f9fb489a700 1 -- 172.21.15.57:0/4127632087 shutdown_connections 2023-11-28T12:03:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.784+0000 7f9fb489a700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9f900682f0 0x7f9f9006a7a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.784+0000 7f9fb489a700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fb015a590 0x7f9fb00c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.784+0000 7f9fb489a700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fb006cb50 0x7f9fb01e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.784+0000 7f9fb489a700 1 --2- 172.21.15.57:0/4127632087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9fb00b50d0 0x7f9fb015a960 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.784+0000 7f9fb489a700 1 -- 172.21.15.57:0/4127632087 >> 172.21.15.57:0/4127632087 conn(0x7f9fb00cf400 msgr2=0x7f9fb00b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.784+0000 7f9fb489a700 1 -- 172.21.15.57:0/4127632087 shutdown_connections 2023-11-28T12:03:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.784+0000 7f9fb489a700 1 -- 172.21.15.57:0/4127632087 wait complete. 2023-11-28T12:03:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T12:03:35.787 INFO:tasks.ceph.mds.a.smithi057.stderr: -5887> 2023-11-28T12:02:55.914+0000 7f8a54e21700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T12:03:35.801 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T12:03:35.822 INFO:tasks.ceph.mds.a.smithi057.stdout:starting mds.a at 2023-11-28T12:03:35.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.942+0000 7fcdba2c2700 1 Processor -- start 2023-11-28T12:03:35.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.942+0000 7fcdba2c2700 1 -- start start 2023-11-28T12:03:35.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.942+0000 7fcdba2c2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcdb40b2ee0 0x7fcdb40b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:35.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.942+0000 7fcdba2c2700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb4161370 con 0x7fcdb415fe50 2023-11-28T12:03:35.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdba2c2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb40cf700 con 0x7fcdb406d170 2023-11-28T12:03:35.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdba2c2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb406d5c0 con 0x7fcdb40b2ee0 2023-11-28T12:03:35.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb3fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcdb415fe50 0x7fcdb4160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40740/0 (socket says 172.21.15.57:40740) 2023-11-28T12:03:35.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb3fff700 1 -- 172.21.15.57:0/3896935230 learned_addr learned my addr 172.21.15.57:0/3896935230 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:35.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb885f700 1 --2- 172.21.15.57:0/3896935230 >> v2:172.21.15.181:3300/0 conn(0x7fcdb40b2ee0 0x7fcdb40b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:35.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903532006 0 0) 0x7fcdb4161370 con 0x7fcdb415fe50 2023-11-28T12:03:35.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd9c0035b0 con 0x7fcdb415fe50 2023-11-28T12:03:35.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406061580 0 0) 0x7fcdb40cf700 con 0x7fcdb406d170 2023-11-28T12:03:35.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdb4161370 con 0x7fcdb406d170 2023-11-28T12:03:35.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2292619614 0 0) 0x7fcd9c0035b0 con 0x7fcdb415fe50 2023-11-28T12:03:35.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 >> v2:172.21.15.181:3300/0 conn(0x7fcdb40b2ee0 msgr2=0x7fcdb40b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:35.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 --2- 172.21.15.57:0/3896935230 >> v2:172.21.15.181:3300/0 conn(0x7fcdb40b2ee0 0x7fcdb40b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 >> v1:172.21.15.181:6789/0 conn(0x7fcdb406d170 legacy=0x7fcdb40b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:35.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb40cedd0 con 0x7fcdb415fe50 2023-11-28T12:03:35.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fcda40015f0 con 0x7fcdb415fe50 2023-11-28T12:03:35.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 >> v1:172.21.15.57:6789/0 conn(0x7fcdb415fe50 legacy=0x7fcdb4160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:35.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 --2- 172.21.15.57:0/3896935230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd9c003be0 0x7fcd9c003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:35.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 --2- 172.21.15.57:0/3896935230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd9c004850 0x7fcd9c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:35.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 --2- 172.21.15.57:0/3896935230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd9c005200 0x7fcd9c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:35.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c00c0e0 con 0x7fcd9c004850 2023-11-28T12:03:35.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c00c220 con 0x7fcd9c005200 2023-11-28T12:03:35.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.943+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c00c3b0 con 0x7fcd9c003be0 2023-11-28T12:03:35.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb37fe700 1 --2- 172.21.15.57:0/3896935230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd9c004850 0x7fcd9c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:35.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb3fff700 1 --2- 172.21.15.57:0/3896935230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd9c003be0 0x7fcd9c003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:35.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb885f700 1 --2- 172.21.15.57:0/3896935230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd9c005200 0x7fcd9c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:35.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb37fe700 1 -- 172.21.15.57:0/3896935230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd9c003be0 msgr2=0x7fcd9c003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:35.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb37fe700 1 --2- 172.21.15.57:0/3896935230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd9c003be0 0x7fcd9c003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb37fe700 1 -- 172.21.15.57:0/3896935230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd9c005200 msgr2=0x7fcd9c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:35.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb37fe700 1 --2- 172.21.15.57:0/3896935230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd9c005200 0x7fcd9c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb37fe700 1 -- 172.21.15.57:0/3896935230 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c0035b0 con 0x7fcd9c004850 2023-11-28T12:03:35.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb885f700 1 --2- 172.21.15.57:0/3896935230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd9c005200 0x7fcd9c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:35.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb885f700 1 -- 172.21.15.57:0/3896935230 reap_dead start 2023-11-28T12:03:35.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb37fe700 1 --2- 172.21.15.57:0/3896935230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd9c004850 0x7fcd9c004cc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcda8007fd0 tx=0x7fcd9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72554d30b35470b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:35.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8001430 con 0x7fcd9c004850 2023-11-28T12:03:35.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda80052d0 con 0x7fcd9c004850 2023-11-28T12:03:35.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.944+0000 7fcdb2ffd700 1 -- 172.21.15.57:0/3896935230 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8001d40 con 0x7fcd9c004850 2023-11-28T12:03:35.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.945+0000 7fcdba2c2700 1 -- 172.21.15.57:0/3896935230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd9c004850 msgr2=0x7fcd9c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:35.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.945+0000 7fcdba2c2700 1 --2- 172.21.15.57:0/3896935230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd9c004850 0x7fcd9c004cc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcda8007fd0 tx=0x7fcd9c005880 comp rx=0 tx=0).stop 2023-11-28T12:03:35.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.945+0000 7fcdba2c2700 1 -- 172.21.15.57:0/3896935230 shutdown_connections 2023-11-28T12:03:35.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.945+0000 7fcdba2c2700 1 --2- 172.21.15.57:0/3896935230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd9c004850 0x7fcd9c004cc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.945+0000 7fcdba2c2700 1 -- 172.21.15.57:0/3896935230 >> 172.21.15.57:0/3896935230 conn(0x7fcdb406ea80 msgr2=0x7fcdb406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:35.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.945+0000 7fcdba2c2700 1 -- 172.21.15.57:0/3896935230 shutdown_connections 2023-11-28T12:03:35.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.945+0000 7fcdba2c2700 1 -- 172.21.15.57:0/3896935230 wait complete. 2023-11-28T12:03:35.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.946+0000 7fcdba2c2700 1 Processor -- start 2023-11-28T12:03:35.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.946+0000 7fcdba2c2700 1 -- start start 2023-11-28T12:03:35.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.947+0000 7fcdba2c2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd9c004850 0x7fcdb40ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:35.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.947+0000 7fcdba2c2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdb40b25c0 0x7fcdb40aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:35.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.947+0000 7fcdba2c2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdb40c86f0 0x7fcdb414ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:35.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.947+0000 7fcdba2c2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb40ae830 con 0x7fcdb40c86f0 2023-11-28T12:03:35.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.947+0000 7fcdba2c2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb40adc20 con 0x7fcdb40b25c0 2023-11-28T12:03:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.947+0000 7fcdba2c2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb40adf00 con 0x7fcd9c004850 2023-11-28T12:03:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.947+0000 7fcdb885f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdb40c86f0 0x7fcdb414ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.947+0000 7fcdb885f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdb40c86f0 0x7fcdb414ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45954/0 (socket says 172.21.15.57:45954) 2023-11-28T12:03:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.947+0000 7fcdb885f700 1 -- 172.21.15.57:0/2035043044 learned_addr learned my addr 172.21.15.57:0/2035043044 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.947+0000 7fcdb37fe700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdb40b25c0 0x7fcdb40aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.947+0000 7fcdb3fff700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd9c004850 0x7fcdb40ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.948+0000 7fcdb885f700 1 -- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd9c004850 msgr2=0x7fcdb40ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.948+0000 7fcdb885f700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd9c004850 0x7fcdb40ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.948+0000 7fcdb885f700 1 -- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdb40b25c0 msgr2=0x7fcdb40aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.948+0000 7fcdb885f700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdb40b25c0 0x7fcdb40aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:35.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.948+0000 7fcdb885f700 1 -- 172.21.15.57:0/2035043044 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb40cf390 con 0x7fcdb40c86f0 2023-11-28T12:03:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.948+0000 7fcdb885f700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdb40c86f0 0x7fcdb414ccf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4072a20 tx=0x7fcdac0101c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.948+0000 7fcdb0ff9700 1 -- 172.21.15.57:0/2035043044 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac01b070 con 0x7fcdb40c86f0 2023-11-28T12:03:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.948+0000 7fcdb0ff9700 1 -- 172.21.15.57:0/2035043044 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdac00f040 con 0x7fcdb40c86f0 2023-11-28T12:03:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.948+0000 7fcdba2c2700 1 -- 172.21.15.57:0/2035043044 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb40cf0b0 con 0x7fcdb40c86f0 2023-11-28T12:03:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.948+0000 7fcdb0ff9700 1 -- 172.21.15.57:0/2035043044 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00e050 con 0x7fcdb40c86f0 2023-11-28T12:03:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.948+0000 7fcdba2c2700 1 -- 172.21.15.57:0/2035043044 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb40b3b80 con 0x7fcdb40c86f0 2023-11-28T12:03:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.949+0000 7fcdba2c2700 1 -- 172.21.15.57:0/2035043044 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb40b3f20 con 0x7fcdb40c86f0 2023-11-28T12:03:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.950+0000 7fcdb0ff9700 1 -- 172.21.15.57:0/2035043044 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcdac010a60 con 0x7fcdb40c86f0 2023-11-28T12:03:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.950+0000 7fcdb0ff9700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcda0068320 0x7fcda006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.950+0000 7fcdb0ff9700 1 -- 172.21.15.57:0/2035043044 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fcdac085f70 con 0x7fcdb40c86f0 2023-11-28T12:03:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.950+0000 7fcdb3fff700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcda0068320 0x7fcda006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.951+0000 7fcdb3fff700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcda0068320 0x7fcda006a7d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcda4001050 tx=0x7fcda4000f80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:35.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:35.955+0000 7fcdb0ff9700 1 -- 172.21.15.57:0/2035043044 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcdac052750 con 0x7fcdb40c86f0 2023-11-28T12:03:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.196+0000 7fcdba2c2700 1 -- 172.21.15.57:0/2035043044 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcdb40743f0 con 0x7fcdb40c86f0 2023-11-28T12:03:36.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.782+0000 7fcdb0ff9700 1 -- 172.21.15.57:0/2035043044 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcdac055d70 con 0x7fcdb40c86f0 2023-11-28T12:03:36.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.784+0000 7fcdba2c2700 1 -- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcda0068320 msgr2=0x7fcda006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:36.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.784+0000 7fcdba2c2700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcda0068320 0x7fcda006a7d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcda4001050 tx=0x7fcda4000f80 comp rx=0 tx=0).stop 2023-11-28T12:03:36.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.784+0000 7fcdba2c2700 1 -- 172.21.15.57:0/2035043044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdb40c86f0 msgr2=0x7fcdb414ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:36.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.784+0000 7fcdba2c2700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdb40c86f0 0x7fcdb414ccf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4072a20 tx=0x7fcdac0101c0 comp rx=0 tx=0).stop 2023-11-28T12:03:36.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.785+0000 7fcdba2c2700 1 -- 172.21.15.57:0/2035043044 shutdown_connections 2023-11-28T12:03:36.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.785+0000 7fcdba2c2700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcda0068320 0x7fcda006a7d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:36.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.785+0000 7fcdba2c2700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdb40c86f0 0x7fcdb414ccf0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:36.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.785+0000 7fcdba2c2700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd9c004850 0x7fcdb40ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:36.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.785+0000 7fcdba2c2700 1 --2- 172.21.15.57:0/2035043044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdb40b25c0 0x7fcdb40aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:36.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.785+0000 7fcdba2c2700 1 -- 172.21.15.57:0/2035043044 >> 172.21.15.57:0/2035043044 conn(0x7fcdb406ea80 msgr2=0x7fcdb40cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:36.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.785+0000 7fcdba2c2700 1 -- 172.21.15.57:0/2035043044 shutdown_connections 2023-11-28T12:03:36.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.785+0000 7fcdba2c2700 1 -- 172.21.15.57:0/2035043044 wait complete. 2023-11-28T12:03:36.801 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T12:03:36.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.990+0000 7fabe5325700 1 Processor -- start 2023-11-28T12:03:36.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.990+0000 7fabe5325700 1 -- start start 2023-11-28T12:03:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.990+0000 7fabe5325700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fabe015cce0 0x7fabe00b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.990+0000 7fabe5325700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fabe00b22e0 0x7fabe00d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.990+0000 7fabe5325700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe00b32b0 con 0x7fabe00b1c90 2023-11-28T12:03:36.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.991+0000 7fabe5325700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe00b26e0 con 0x7fabe015cce0 2023-11-28T12:03:36.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.991+0000 7fabe5325700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe00b2980 con 0x7fabe00b22e0 2023-11-28T12:03:36.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.991+0000 7fabdeffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fabe015cce0 0x7fabe00b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:36.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.991+0000 7fabdeffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fabe015cce0 0x7fabe00b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45956/0 (socket says 172.21.15.57:45956) 2023-11-28T12:03:36.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.991+0000 7fabdeffd700 1 -- 172.21.15.57:0/4213747204 learned_addr learned my addr 172.21.15.57:0/4213747204 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:36.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.991+0000 7fabde7fc700 1 --2- 172.21.15.57:0/4213747204 >> v2:172.21.15.181:3300/0 conn(0x7fabe00b22e0 0x7fabe00d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:36.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.991+0000 7fabdeffd700 1 -- 172.21.15.57:0/4213747204 >> v2:172.21.15.181:3300/0 conn(0x7fabe00b22e0 msgr2=0x7fabe00d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.991+0000 7fabdeffd700 1 --2- 172.21.15.57:0/4213747204 >> v2:172.21.15.181:3300/0 conn(0x7fabe00b22e0 0x7fabe00d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.991+0000 7fabdeffd700 1 -- 172.21.15.57:0/4213747204 >> v1:172.21.15.181:6789/0 conn(0x7fabe00b1c90 legacy=0x7fabe00afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.991+0000 7fabdeffd700 1 -- 172.21.15.57:0/4213747204 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe00b2c60 con 0x7fabe015cce0 2023-11-28T12:03:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.991+0000 7fabdeffd700 1 --2- 172.21.15.57:0/4213747204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabe015cce0 0x7fabe00b1750 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fabe006d450 tx=0x7fabc8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53582fa037de41fc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.991+0000 7fabddffb700 1 -- 172.21.15.57:0/4213747204 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc800b870 con 0x7fabe015cce0 2023-11-28T12:03:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.991+0000 7fabddffb700 1 -- 172.21.15.57:0/4213747204 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc800b9d0 con 0x7fabe015cce0 2023-11-28T12:03:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.992+0000 7fabddffb700 1 -- 172.21.15.57:0/4213747204 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc800bd60 con 0x7fabe015cce0 2023-11-28T12:03:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.992+0000 7fabe5325700 1 -- 172.21.15.57:0/4213747204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabe015cce0 msgr2=0x7fabe00b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.992+0000 7fabe5325700 1 --2- 172.21.15.57:0/4213747204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabe015cce0 0x7fabe00b1750 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fabe006d450 tx=0x7fabc8009bd0 comp rx=0 tx=0).stop 2023-11-28T12:03:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.992+0000 7fabe5325700 1 -- 172.21.15.57:0/4213747204 shutdown_connections 2023-11-28T12:03:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.992+0000 7fabe5325700 1 --2- 172.21.15.57:0/4213747204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabe015cce0 0x7fabe00b1750 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:36.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.992+0000 7fabe5325700 1 --2- 172.21.15.57:0/4213747204 >> v2:172.21.15.181:3300/0 conn(0x7fabe00b22e0 0x7fabe00d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:36.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.992+0000 7fabe5325700 1 -- 172.21.15.57:0/4213747204 >> 172.21.15.57:0/4213747204 conn(0x7fabe009c4d0 msgr2=0x7fabe00d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:36.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.992+0000 7fabe5325700 1 -- 172.21.15.57:0/4213747204 shutdown_connections 2023-11-28T12:03:36.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.992+0000 7fabe5325700 1 -- 172.21.15.57:0/4213747204 wait complete. 2023-11-28T12:03:36.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.994+0000 7fabe5325700 1 Processor -- start 2023-11-28T12:03:36.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.994+0000 7fabe5325700 1 -- start start 2023-11-28T12:03:36.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.994+0000 7fabe5325700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabe00b1c90 0x7fabe00c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:36.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.994+0000 7fabe5325700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe015cce0 0x7fabe015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:36.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.994+0000 7fabe5325700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe015d5f0 0x7fabe01584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:36.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.994+0000 7fabe5325700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe00b8c70 con 0x7fabe00b1c90 2023-11-28T12:03:36.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.994+0000 7fabe5325700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe015da60 con 0x7fabe015d5f0 2023-11-28T12:03:36.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.994+0000 7fabe5325700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe015dcc0 con 0x7fabe015cce0 2023-11-28T12:03:36.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabdeffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabe00b1c90 0x7fabe00c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:36.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabdeffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabe00b1c90 0x7fabe00c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45972/0 (socket says 172.21.15.57:45972) 2023-11-28T12:03:37.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabdeffd700 1 -- 172.21.15.57:0/2990930565 learned_addr learned my addr 172.21.15.57:0/2990930565 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:37.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabde7fc700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe015cce0 0x7fabe015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:37.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabdeffd700 1 -- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe015cce0 msgr2=0x7fabe015d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:37.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabdeffd700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe015cce0 0x7fabe015d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:37.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabdf7fe700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe015d5f0 0x7fabe01584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:37.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabdeffd700 1 -- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe015d5f0 msgr2=0x7fabe01584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:37.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabdeffd700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe015d5f0 0x7fabe01584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:37.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabdeffd700 1 -- 172.21.15.57:0/2990930565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc8009550 con 0x7fabe00b1c90 2023-11-28T12:03:37.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabdf7fe700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe015d5f0 0x7fabe01584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:37.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabdeffd700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabe00b1c90 0x7fabe00c8c60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fabc8000c00 tx=0x7fabc8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:37.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabd77fe700 1 -- 172.21.15.57:0/2990930565 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8004540 con 0x7fabe00b1c90 2023-11-28T12:03:37.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabd77fe700 1 -- 172.21.15.57:0/2990930565 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc80036a0 con 0x7fabe00b1c90 2023-11-28T12:03:37.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabe5325700 1 -- 172.21.15.57:0/2990930565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe00b4750 con 0x7fabe00b1c90 2023-11-28T12:03:37.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabd77fe700 1 -- 172.21.15.57:0/2990930565 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8003a30 con 0x7fabe00b1c90 2023-11-28T12:03:37.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.995+0000 7fabe5325700 1 -- 172.21.15.57:0/2990930565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe00b2f40 con 0x7fabe00b1c90 2023-11-28T12:03:37.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.996+0000 7fabe5325700 1 -- 172.21.15.57:0/2990930565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe00b48e0 con 0x7fabe00b1c90 2023-11-28T12:03:37.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.997+0000 7fabd77fe700 1 -- 172.21.15.57:0/2990930565 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fabc800f050 con 0x7fabe00b1c90 2023-11-28T12:03:37.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.997+0000 7fabd77fe700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fabc00682f0 0x7fabc006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:37.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.997+0000 7fabd77fe700 1 -- 172.21.15.57:0/2990930565 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fabc8087660 con 0x7fabe00b1c90 2023-11-28T12:03:37.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.998+0000 7fabde7fc700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fabc00682f0 0x7fabc006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:37.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:36.998+0000 7fabde7fc700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fabc00682f0 0x7fabc006a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fabe015afe0 tx=0x7fabd0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:37.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.002+0000 7fabd77fe700 1 -- 172.21.15.57:0/2990930565 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fabc8053e40 con 0x7fabe00b1c90 2023-11-28T12:03:37.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.205+0000 7fabe5325700 1 -- 172.21.15.57:0/2990930565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fabe00743f0 con 0x7fabe00b1c90 2023-11-28T12:03:37.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.787+0000 7fabd77fe700 1 -- 172.21.15.57:0/2990930565 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7fabc8057460 con 0x7fabe00b1c90 2023-11-28T12:03:37.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.791+0000 7fabe5325700 1 -- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fabc00682f0 msgr2=0x7fabc006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:37.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.791+0000 7fabe5325700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fabc00682f0 0x7fabc006a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fabe015afe0 tx=0x7fabd0008040 comp rx=0 tx=0).stop 2023-11-28T12:03:37.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.791+0000 7fabe5325700 1 -- 172.21.15.57:0/2990930565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabe00b1c90 msgr2=0x7fabe00c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:37.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.791+0000 7fabe5325700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabe00b1c90 0x7fabe00c8c60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fabc8000c00 tx=0x7fabc8004750 comp rx=0 tx=0).stop 2023-11-28T12:03:37.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.791+0000 7fabe5325700 1 -- 172.21.15.57:0/2990930565 shutdown_connections 2023-11-28T12:03:37.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.791+0000 7fabe5325700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fabc00682f0 0x7fabc006a7a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:37.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.791+0000 7fabe5325700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe015d5f0 0x7fabe01584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:37.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.791+0000 7fabe5325700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabe00b1c90 0x7fabe00c8c60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:37.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.791+0000 7fabe5325700 1 --2- 172.21.15.57:0/2990930565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe015cce0 0x7fabe015d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:37.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.791+0000 7fabe5325700 1 -- 172.21.15.57:0/2990930565 >> 172.21.15.57:0/2990930565 conn(0x7fabe009c4d0 msgr2=0x7fabe006be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:37.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.791+0000 7fabe5325700 1 -- 172.21.15.57:0/2990930565 shutdown_connections 2023-11-28T12:03:37.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.791+0000 7fabe5325700 1 -- 172.21.15.57:0/2990930565 wait complete. 2023-11-28T12:03:37.793 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' removed 2023-11-28T12:03:37.812 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T12:03:37.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.955+0000 7fd729f4f700 1 Processor -- start 2023-11-28T12:03:37.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.955+0000 7fd729f4f700 1 -- start start 2023-11-28T12:03:37.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd729f4f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd72415cc50 0x7fd7240af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:37.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd729f4f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd724161160 0x7fd724070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:37.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd729f4f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7240cf3e0 con 0x7fd724071150 2023-11-28T12:03:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd729f4f700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7240715d0 con 0x7fd72415cc50 2023-11-28T12:03:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd729f4f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240cd800 con 0x7fd724161160 2023-11-28T12:03:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd7237fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd72415cc50 0x7fd7240af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd7237fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd72415cc50 0x7fd7240af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45976/0 (socket says 172.21.15.57:45976) 2023-11-28T12:03:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd7237fe700 1 -- 172.21.15.57:0/3982200986 learned_addr learned my addr 172.21.15.57:0/3982200986 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd722ffd700 1 --2- 172.21.15.57:0/3982200986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd724161160 0x7fd724070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd7237fe700 1 -- 172.21.15.57:0/3982200986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd724161160 msgr2=0x7fd724070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd7237fe700 1 --2- 172.21.15.57:0/3982200986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd724161160 0x7fd724070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd7237fe700 1 -- 172.21.15.57:0/3982200986 >> v1:172.21.15.181:6789/0 conn(0x7fd724071150 legacy=0x7fd7240c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd7237fe700 1 -- 172.21.15.57:0/3982200986 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7240cdae0 con 0x7fd72415cc50 2023-11-28T12:03:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.956+0000 7fd7237fe700 1 --2- 172.21.15.57:0/3982200986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd72415cc50 0x7fd7240af1f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd7140098f0 tx=0x7fd714009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a29690220bda66 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.957+0000 7fd7227fc700 1 -- 172.21.15.57:0/3982200986 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714011830 con 0x7fd72415cc50 2023-11-28T12:03:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.957+0000 7fd7227fc700 1 -- 172.21.15.57:0/3982200986 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd714011990 con 0x7fd72415cc50 2023-11-28T12:03:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.957+0000 7fd7227fc700 1 -- 172.21.15.57:0/3982200986 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714011d20 con 0x7fd72415cc50 2023-11-28T12:03:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.957+0000 7fd729f4f700 1 -- 172.21.15.57:0/3982200986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd72415cc50 msgr2=0x7fd7240af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.957+0000 7fd729f4f700 1 --2- 172.21.15.57:0/3982200986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd72415cc50 0x7fd7240af1f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd7140098f0 tx=0x7fd714009c00 comp rx=0 tx=0).stop 2023-11-28T12:03:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.957+0000 7fd729f4f700 1 -- 172.21.15.57:0/3982200986 shutdown_connections 2023-11-28T12:03:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.957+0000 7fd729f4f700 1 --2- 172.21.15.57:0/3982200986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd72415cc50 0x7fd7240af1f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.957+0000 7fd729f4f700 1 --2- 172.21.15.57:0/3982200986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd724161160 0x7fd724070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.957+0000 7fd729f4f700 1 -- 172.21.15.57:0/3982200986 >> 172.21.15.57:0/3982200986 conn(0x7fd7240cfc30 msgr2=0x7fd7240d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.957+0000 7fd729f4f700 1 -- 172.21.15.57:0/3982200986 shutdown_connections 2023-11-28T12:03:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.957+0000 7fd729f4f700 1 -- 172.21.15.57:0/3982200986 wait complete. 2023-11-28T12:03:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.958+0000 7fd729f4f700 1 Processor -- start 2023-11-28T12:03:37.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.959+0000 7fd729f4f700 1 -- start start 2023-11-28T12:03:37.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.959+0000 7fd729f4f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd724071150 0x7fd7241f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:37.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.959+0000 7fd729f4f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd72415cc50 0x7fd7240b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:37.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.959+0000 7fd729f4f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd724161160 0x7fd7241574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:37.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.959+0000 7fd729f4f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240b2ba0 con 0x7fd724071150 2023-11-28T12:03:37.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.959+0000 7fd729f4f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240b2df0 con 0x7fd724161160 2023-11-28T12:03:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.959+0000 7fd729f4f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240b30d0 con 0x7fd72415cc50 2023-11-28T12:03:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd7237fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd724071150 0x7fd7241f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd7237fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd724071150 0x7fd7241f5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45988/0 (socket says 172.21.15.57:45988) 2023-11-28T12:03:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd7237fe700 1 -- 172.21.15.57:0/3010483674 learned_addr learned my addr 172.21.15.57:0/3010483674 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd722ffd700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd72415cc50 0x7fd7240b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd723fff700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd724161160 0x7fd7241574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd7237fe700 1 -- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd72415cc50 msgr2=0x7fd7240b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd7237fe700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd72415cc50 0x7fd7240b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd7237fe700 1 -- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd724161160 msgr2=0x7fd7241574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd7237fe700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd724161160 0x7fd7241574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd7237fe700 1 -- 172.21.15.57:0/3010483674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd714009550 con 0x7fd724071150 2023-11-28T12:03:37.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd723fff700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd724161160 0x7fd7241574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:37.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd7237fe700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd724071150 0x7fd7241f5bc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd714000c00 tx=0x7fd714004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:37.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd70bfff700 1 -- 172.21.15.57:0/3010483674 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714004480 con 0x7fd724071150 2023-11-28T12:03:37.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd70bfff700 1 -- 172.21.15.57:0/3010483674 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71401f070 con 0x7fd724071150 2023-11-28T12:03:37.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd70bfff700 1 -- 172.21.15.57:0/3010483674 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7140038c0 con 0x7fd724071150 2023-11-28T12:03:37.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd729f4f700 1 -- 172.21.15.57:0/3010483674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7241616b0 con 0x7fd724071150 2023-11-28T12:03:37.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.960+0000 7fd729f4f700 1 -- 172.21.15.57:0/3010483674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7240cdf70 con 0x7fd724071150 2023-11-28T12:03:37.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.962+0000 7fd729f4f700 1 -- 172.21.15.57:0/3010483674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7240cf610 con 0x7fd724071150 2023-11-28T12:03:37.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.962+0000 7fd70bfff700 1 -- 172.21.15.57:0/3010483674 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd71402a020 con 0x7fd724071150 2023-11-28T12:03:37.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.962+0000 7fd70bfff700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd7100682f0 0x7fd71006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:37.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.963+0000 7fd70bfff700 1 -- 172.21.15.57:0/3010483674 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5739+0+0 (secure 0 0 0) 0x7fd714087180 con 0x7fd724071150 2023-11-28T12:03:37.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.963+0000 7fd722ffd700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd7100682f0 0x7fd71006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:37.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.963+0000 7fd722ffd700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd7100682f0 0x7fd71006a7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd72415bf60 tx=0x7fd718009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:37.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:37.967+0000 7fd70bfff700 1 -- 172.21.15.57:0/3010483674 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd714053aa0 con 0x7fd724071150 2023-11-28T12:03:38.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.166+0000 7fd729f4f700 1 -- 172.21.15.57:0/3010483674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd7240743f0 con 0x7fd724071150 2023-11-28T12:03:38.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.790+0000 7fd70bfff700 1 -- 172.21.15.57:0/3010483674 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd7140570c0 con 0x7fd724071150 2023-11-28T12:03:38.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.796+0000 7fd729f4f700 1 -- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd7100682f0 msgr2=0x7fd71006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:38.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.796+0000 7fd729f4f700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd7100682f0 0x7fd71006a7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd72415bf60 tx=0x7fd718009210 comp rx=0 tx=0).stop 2023-11-28T12:03:38.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.796+0000 7fd729f4f700 1 -- 172.21.15.57:0/3010483674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd724071150 msgr2=0x7fd7241f5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:38.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.796+0000 7fd729f4f700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd724071150 0x7fd7241f5bc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd714000c00 tx=0x7fd714004750 comp rx=0 tx=0).stop 2023-11-28T12:03:38.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.796+0000 7fd729f4f700 1 -- 172.21.15.57:0/3010483674 shutdown_connections 2023-11-28T12:03:38.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.796+0000 7fd729f4f700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd7100682f0 0x7fd71006a7a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:38.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.796+0000 7fd729f4f700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd724161160 0x7fd7241574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:38.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.796+0000 7fd729f4f700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd724071150 0x7fd7241f5bc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:38.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.796+0000 7fd729f4f700 1 --2- 172.21.15.57:0/3010483674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd72415cc50 0x7fd7240b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:38.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.796+0000 7fd729f4f700 1 -- 172.21.15.57:0/3010483674 >> 172.21.15.57:0/3010483674 conn(0x7fd7240cfc30 msgr2=0x7fd7240aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:38.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.796+0000 7fd729f4f700 1 -- 172.21.15.57:0/3010483674 shutdown_connections 2023-11-28T12:03:38.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.796+0000 7fd729f4f700 1 -- 172.21.15.57:0/3010483674 wait complete. 2023-11-28T12:03:38.798 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' removed 2023-11-28T12:03:38.814 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T12:03:38.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.962+0000 7f14de6a7700 1 Processor -- start 2023-11-28T12:03:38.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.962+0000 7f14de6a7700 1 -- start start 2023-11-28T12:03:38.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.962+0000 7f14de6a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14d815a5f0 0x7f14d80d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:38.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.962+0000 7f14de6a7700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d815a9c0 con 0x7f14d80b2ee0 2023-11-28T12:03:38.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.962+0000 7f14de6a7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d80c7b80 con 0x7f14d80cc200 2023-11-28T12:03:38.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.962+0000 7f14de6a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d80cc990 con 0x7f14d815a5f0 2023-11-28T12:03:38.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14dcc44700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f14d80b2ee0 0x7f14d80ced30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40754/0 (socket says 172.21.15.57:40754) 2023-11-28T12:03:38.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14dcc44700 1 -- 172.21.15.57:0/3712865567 learned_addr learned my addr 172.21.15.57:0/3712865567 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:38.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d7fff700 1 --2- 172.21.15.57:0/3712865567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14d815a5f0 0x7f14d80d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:38.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106043748 0 0) 0x7f14d815a9c0 con 0x7f14d80b2ee0 2023-11-28T12:03:38.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14c40035b0 con 0x7f14d80b2ee0 2023-11-28T12:03:38.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1885494349 0 0) 0x7f14c40035b0 con 0x7f14d80b2ee0 2023-11-28T12:03:38.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14d815a5f0 msgr2=0x7f14d80d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:38.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 --2- 172.21.15.57:0/3712865567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14d815a5f0 0x7f14d80d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:38.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 >> v1:172.21.15.181:6789/0 conn(0x7f14d80cc200 legacy=0x7f14d80b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:38.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d80ccc70 con 0x7f14d80b2ee0 2023-11-28T12:03:38.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f14d00019f0 con 0x7f14d80b2ee0 2023-11-28T12:03:38.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 >> v1:172.21.15.57:6789/0 conn(0x7f14d80b2ee0 legacy=0x7f14d80ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:38.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 --2- 172.21.15.57:0/3712865567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14c4003a30 0x7f14c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:38.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 --2- 172.21.15.57:0/3712865567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14d815a5f0 0x7f14c4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:38.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 --2- 172.21.15.57:0/3712865567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c4004d90 0x7f14c4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:38.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d80cc990 con 0x7f14c4004d90 2023-11-28T12:03:38.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c400ba10 con 0x7f14c4003a30 2023-11-28T12:03:38.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.963+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c400bba0 con 0x7f14d815a5f0 2023-11-28T12:03:38.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.964+0000 7f14dcc44700 1 --2- 172.21.15.57:0/3712865567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c4004d90 0x7f14c4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:38.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.964+0000 7f14d7fff700 1 --2- 172.21.15.57:0/3712865567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14c4003a30 0x7f14c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:38.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.964+0000 7f14cffff700 1 --2- 172.21.15.57:0/3712865567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14d815a5f0 0x7f14c4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:38.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.964+0000 7f14dcc44700 1 -- 172.21.15.57:0/3712865567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14d815a5f0 msgr2=0x7f14c4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:38.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.964+0000 7f14dcc44700 1 --2- 172.21.15.57:0/3712865567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14d815a5f0 0x7f14c4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:38.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.964+0000 7f14dcc44700 1 -- 172.21.15.57:0/3712865567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14c4003a30 msgr2=0x7f14c4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:38.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.964+0000 7f14dcc44700 1 --2- 172.21.15.57:0/3712865567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14c4003a30 0x7f14c4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:38.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.964+0000 7f14dcc44700 1 -- 172.21.15.57:0/3712865567 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c40035b0 con 0x7f14c4004d90 2023-11-28T12:03:38.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.964+0000 7f14dcc44700 1 --2- 172.21.15.57:0/3712865567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c4004d90 0x7f14c4005200 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f14d000a7d0 tx=0x7f14c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cb8bbd3d5bdd95 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:38.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.964+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0012600 con 0x7f14c4004d90 2023-11-28T12:03:38.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.964+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d0012760 con 0x7f14c4004d90 2023-11-28T12:03:38.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.964+0000 7f14d77fe700 1 -- 172.21.15.57:0/3712865567 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0012b10 con 0x7f14c4004d90 2023-11-28T12:03:38.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.965+0000 7f14de6a7700 1 -- 172.21.15.57:0/3712865567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c4004d90 msgr2=0x7f14c4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:38.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.965+0000 7f14de6a7700 1 --2- 172.21.15.57:0/3712865567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c4004d90 0x7f14c4005200 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f14d000a7d0 tx=0x7f14c4005880 comp rx=0 tx=0).stop 2023-11-28T12:03:38.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.965+0000 7f14dcc44700 1 -- 172.21.15.57:0/3712865567 reap_dead start 2023-11-28T12:03:38.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.965+0000 7f14de6a7700 1 -- 172.21.15.57:0/3712865567 shutdown_connections 2023-11-28T12:03:38.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.965+0000 7f14de6a7700 1 -- 172.21.15.57:0/3712865567 >> 172.21.15.57:0/3712865567 conn(0x7f14d806ea80 msgr2=0x7f14d806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:38.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.965+0000 7f14de6a7700 1 -- 172.21.15.57:0/3712865567 shutdown_connections 2023-11-28T12:03:38.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.965+0000 7f14de6a7700 1 -- 172.21.15.57:0/3712865567 wait complete. 2023-11-28T12:03:38.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.966+0000 7f14de6a7700 1 Processor -- start 2023-11-28T12:03:38.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.966+0000 7f14de6a7700 1 -- start start 2023-11-28T12:03:38.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14de6a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14c4004d90 0x7f14d80c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:38.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14de6a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14d81ef8f0 0x7f14d81f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:38.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14de6a7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d81f3a10 0x7f14d80b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:38.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14de6a7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d80c84f0 con 0x7f14d81f3a10 2023-11-28T12:03:38.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14de6a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d81efd60 con 0x7f14d81ef8f0 2023-11-28T12:03:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14de6a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d81f4320 con 0x7f14c4004d90 2023-11-28T12:03:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14dcc44700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d81f3a10 0x7f14d80b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14dcc44700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d81f3a10 0x7f14d80b5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46000/0 (socket says 172.21.15.57:46000) 2023-11-28T12:03:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14dcc44700 1 -- 172.21.15.57:0/1520893411 learned_addr learned my addr 172.21.15.57:0/1520893411 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14cffff700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14d81ef8f0 0x7f14d81f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14d7fff700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14c4004d90 0x7f14d80c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14dcc44700 1 -- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14c4004d90 msgr2=0x7f14d80c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14dcc44700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14c4004d90 0x7f14d80c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14dcc44700 1 -- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14d81ef8f0 msgr2=0x7f14d81f34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:38.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14dcc44700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14d81ef8f0 0x7f14d81f34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:38.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.967+0000 7f14dcc44700 1 -- 172.21.15.57:0/1520893411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d80cd230 con 0x7f14d81f3a10 2023-11-28T12:03:38.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.968+0000 7f14dcc44700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d81f3a10 0x7f14d80b5850 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f14d0012000 tx=0x7f14d0001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:38.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.968+0000 7f14d57fa700 1 -- 172.21.15.57:0/1520893411 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d00019b0 con 0x7f14d81f3a10 2023-11-28T12:03:38.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.968+0000 7f14d57fa700 1 -- 172.21.15.57:0/1520893411 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d0027470 con 0x7f14d81f3a10 2023-11-28T12:03:38.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.968+0000 7f14de6a7700 1 -- 172.21.15.57:0/1520893411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d80ccf50 con 0x7f14d81f3a10 2023-11-28T12:03:38.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.968+0000 7f14d57fa700 1 -- 172.21.15.57:0/1520893411 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0027820 con 0x7f14d81f3a10 2023-11-28T12:03:38.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.968+0000 7f14de6a7700 1 -- 172.21.15.57:0/1520893411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d80cd510 con 0x7f14d81f3a10 2023-11-28T12:03:38.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.969+0000 7f14de6a7700 1 -- 172.21.15.57:0/1520893411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d80c8660 con 0x7f14d81f3a10 2023-11-28T12:03:38.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.969+0000 7f14d57fa700 1 -- 172.21.15.57:0/1520893411 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f14d0010070 con 0x7f14d81f3a10 2023-11-28T12:03:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.970+0000 7f14d57fa700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f14b8068320 0x7f14b806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.970+0000 7f14d57fa700 1 -- 172.21.15.57:0/1520893411 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f14d0087870 con 0x7f14d81f3a10 2023-11-28T12:03:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.970+0000 7f14d7fff700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f14b8068320 0x7f14b806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.970+0000 7f14d7fff700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f14b8068320 0x7f14b806a7d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009c60 tx=0x7f14c8009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:38.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:38.974+0000 7f14d57fa700 1 -- 172.21.15.57:0/1520893411 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f14d0054300 con 0x7f14d81f3a10 2023-11-28T12:03:39.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.174+0000 7f14de6a7700 1 -- 172.21.15.57:0/1520893411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f14d80743f0 con 0x7f14d81f3a10 2023-11-28T12:03:39.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.178+0000 7f14d57fa700 1 -- 172.21.15.57:0/1520893411 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f14d0057920 con 0x7f14d81f3a10 2023-11-28T12:03:39.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.180+0000 7f14de6a7700 1 -- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f14b8068320 msgr2=0x7f14b806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:39.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.180+0000 7f14de6a7700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f14b8068320 0x7f14b806a7d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009c60 tx=0x7f14c8009210 comp rx=0 tx=0).stop 2023-11-28T12:03:39.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.180+0000 7f14de6a7700 1 -- 172.21.15.57:0/1520893411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d81f3a10 msgr2=0x7f14d80b5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:39.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.180+0000 7f14de6a7700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d81f3a10 0x7f14d80b5850 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f14d0012000 tx=0x7f14d0001d00 comp rx=0 tx=0).stop 2023-11-28T12:03:39.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.180+0000 7f14de6a7700 1 -- 172.21.15.57:0/1520893411 shutdown_connections 2023-11-28T12:03:39.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.180+0000 7f14de6a7700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f14b8068320 0x7f14b806a7d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.180+0000 7f14de6a7700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d81f3a10 0x7f14d80b5850 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.180+0000 7f14de6a7700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14c4004d90 0x7f14d80c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.180+0000 7f14de6a7700 1 --2- 172.21.15.57:0/1520893411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14d81ef8f0 0x7f14d81f34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.180+0000 7f14de6a7700 1 -- 172.21.15.57:0/1520893411 >> 172.21.15.57:0/1520893411 conn(0x7f14d806ea80 msgr2=0x7f14d8155d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:39.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.180+0000 7f14de6a7700 1 -- 172.21.15.57:0/1520893411 shutdown_connections 2023-11-28T12:03:39.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.180+0000 7f14de6a7700 1 -- 172.21.15.57:0/1520893411 wait complete. 2023-11-28T12:03:39.195 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T12:03:39.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.330+0000 7f4771614700 1 Processor -- start 2023-11-28T12:03:39.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.330+0000 7f4771614700 1 -- start start 2023-11-28T12:03:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.330+0000 7f4771614700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f476c0cd5a0 0x7f476c0cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.330+0000 7f4771614700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f476c0b22a0 0x7f476c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.330+0000 7f4771614700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f476c0d1ee0 con 0x7f476c06ab10 2023-11-28T12:03:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.330+0000 7f4771614700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f476c0cdb10 con 0x7f476c0b22a0 2023-11-28T12:03:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.330+0000 7f4771614700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f476c0d1890 con 0x7f476c0cd5a0 2023-11-28T12:03:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.331+0000 7f476b7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f476c0b22a0 0x7f476c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.331+0000 7f476affd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f476c06ab10 0x7f476c06aee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40766/0 (socket says 172.21.15.57:40766) 2023-11-28T12:03:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.331+0000 7f476affd700 1 -- 172.21.15.57:0/1524091532 learned_addr learned my addr 172.21.15.57:0/1524091532 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.331+0000 7f476a7fc700 1 --2- 172.21.15.57:0/1524091532 >> v2:172.21.15.181:3300/0 conn(0x7f476c0cd5a0 0x7f476c0cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.331+0000 7f476b7fe700 1 -- 172.21.15.57:0/1524091532 >> v2:172.21.15.181:3300/0 conn(0x7f476c0cd5a0 msgr2=0x7f476c0cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.331+0000 7f476b7fe700 1 --2- 172.21.15.57:0/1524091532 >> v2:172.21.15.181:3300/0 conn(0x7f476c0cd5a0 0x7f476c0cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.331+0000 7f476b7fe700 1 -- 172.21.15.57:0/1524091532 >> v1:172.21.15.57:6789/0 conn(0x7f476c06ab10 legacy=0x7f476c06aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.331+0000 7f476b7fe700 1 -- 172.21.15.57:0/1524091532 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0d1b70 con 0x7f476c0b22a0 2023-11-28T12:03:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.331+0000 7f476a7fc700 1 --2- 172.21.15.57:0/1524091532 >> v2:172.21.15.181:3300/0 conn(0x7f476c0cd5a0 0x7f476c0cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.331+0000 7f476b7fe700 1 --2- 172.21.15.57:0/1524091532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f476c0b22a0 0x7f476c0cf980 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f476c0d0a60 tx=0x7f476400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342221f1893d1d15 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.331+0000 7f4769ffb700 1 -- 172.21.15.57:0/1524091532 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476400e070 con 0x7f476c0b22a0 2023-11-28T12:03:39.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.331+0000 7f4769ffb700 1 -- 172.21.15.57:0/1524091532 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476400bbd0 con 0x7f476c0b22a0 2023-11-28T12:03:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.331+0000 7f4769ffb700 1 -- 172.21.15.57:0/1524091532 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764012430 con 0x7f476c0b22a0 2023-11-28T12:03:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.332+0000 7f4771614700 1 -- 172.21.15.57:0/1524091532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f476c0b22a0 msgr2=0x7f476c0cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.332+0000 7f4771614700 1 --2- 172.21.15.57:0/1524091532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f476c0b22a0 0x7f476c0cf980 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f476c0d0a60 tx=0x7f476400b330 comp rx=0 tx=0).stop 2023-11-28T12:03:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.332+0000 7f4771614700 1 -- 172.21.15.57:0/1524091532 shutdown_connections 2023-11-28T12:03:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.332+0000 7f4771614700 1 --2- 172.21.15.57:0/1524091532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f476c0b22a0 0x7f476c0cf980 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.332+0000 7f4771614700 1 --2- 172.21.15.57:0/1524091532 >> v2:172.21.15.181:3300/0 conn(0x7f476c0cd5a0 0x7f476c0cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.332+0000 7f4771614700 1 -- 172.21.15.57:0/1524091532 >> 172.21.15.57:0/1524091532 conn(0x7f476c06c520 msgr2=0x7f476c06c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.332+0000 7f4771614700 1 -- 172.21.15.57:0/1524091532 shutdown_connections 2023-11-28T12:03:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.332+0000 7f4771614700 1 -- 172.21.15.57:0/1524091532 wait complete. 2023-11-28T12:03:39.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.333+0000 7f4771614700 1 Processor -- start 2023-11-28T12:03:39.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.333+0000 7f4771614700 1 -- start start 2023-11-28T12:03:39.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.334+0000 7f4771614700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f476c06ab10 0x7f476c158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.334+0000 7f4771614700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f476c0b22a0 0x7f476c0c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.334+0000 7f4771614700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f476c0cd5a0 0x7f476c0c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.334+0000 7f4771614700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1593f0 con 0x7f476c0cd5a0 2023-11-28T12:03:39.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.334+0000 7f4771614700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1596d0 con 0x7f476c0b22a0 2023-11-28T12:03:39.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.334+0000 7f4771614700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1599b0 con 0x7f476c06ab10 2023-11-28T12:03:39.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.334+0000 7f476b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f476c0cd5a0 0x7f476c0c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.334+0000 7f476b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f476c0cd5a0 0x7f476c0c8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46018/0 (socket says 172.21.15.57:46018) 2023-11-28T12:03:39.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.334+0000 7f476b7fe700 1 -- 172.21.15.57:0/2724539588 learned_addr learned my addr 172.21.15.57:0/2724539588 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:39.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.334+0000 7f476a7fc700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f476c0b22a0 0x7f476c0c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.334+0000 7f476affd700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f476c06ab10 0x7f476c158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.335+0000 7f476b7fe700 1 -- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f476c06ab10 msgr2=0x7f476c158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:39.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.335+0000 7f476b7fe700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f476c06ab10 0x7f476c158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.335+0000 7f476b7fe700 1 -- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f476c0b22a0 msgr2=0x7f476c0c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:39.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.335+0000 7f476b7fe700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f476c0b22a0 0x7f476c0c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.335+0000 7f476b7fe700 1 -- 172.21.15.57:0/2724539588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4764009a70 con 0x7f476c0cd5a0 2023-11-28T12:03:39.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.335+0000 7f476b7fe700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f476c0cd5a0 0x7f476c0c8b20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4764014040 tx=0x7f4764004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:39.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.335+0000 7f47537fe700 1 -- 172.21.15.57:0/2724539588 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764012430 con 0x7f476c0cd5a0 2023-11-28T12:03:39.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.335+0000 7f47537fe700 1 -- 172.21.15.57:0/2724539588 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4764020070 con 0x7f476c0cd5a0 2023-11-28T12:03:39.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.335+0000 7f4771614700 1 -- 172.21.15.57:0/2724539588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476c0b1ec0 con 0x7f476c0cd5a0 2023-11-28T12:03:39.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.335+0000 7f47537fe700 1 -- 172.21.15.57:0/2724539588 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640098f0 con 0x7f476c0cd5a0 2023-11-28T12:03:39.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.335+0000 7f4771614700 1 -- 172.21.15.57:0/2724539588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c15cad0 con 0x7f476c0cd5a0 2023-11-28T12:03:39.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.336+0000 7f4771614700 1 -- 172.21.15.57:0/2724539588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476c0b2050 con 0x7f476c0cd5a0 2023-11-28T12:03:39.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.337+0000 7f47537fe700 1 -- 172.21.15.57:0/2724539588 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f476402e070 con 0x7f476c0cd5a0 2023-11-28T12:03:39.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.337+0000 7f47537fe700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f47580682f0 0x7f475806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.337+0000 7f47537fe700 1 -- 172.21.15.57:0/2724539588 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4764022070 con 0x7f476c0cd5a0 2023-11-28T12:03:39.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.337+0000 7f476affd700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f47580682f0 0x7f475806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.338+0000 7f476affd700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f47580682f0 0x7f475806a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f475c005910 tx=0x7f475c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:39.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.341+0000 7f47537fe700 1 -- 172.21.15.57:0/2724539588 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f47640541e0 con 0x7f476c0cd5a0 2023-11-28T12:03:39.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.628+0000 7f4771614700 1 -- 172.21.15.57:0/2724539588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f476c0743f0 con 0x7f476c0cd5a0 2023-11-28T12:03:39.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.798+0000 7f47537fe700 1 -- 172.21.15.57:0/2724539588 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v118) v1 ==== 142+0+0 (secure 0 0 0) 0x7f4764087070 con 0x7f476c0cd5a0 2023-11-28T12:03:39.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.801+0000 7f4771614700 1 -- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f47580682f0 msgr2=0x7f475806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:39.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.801+0000 7f4771614700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f47580682f0 0x7f475806a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f475c005910 tx=0x7f475c009040 comp rx=0 tx=0).stop 2023-11-28T12:03:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.801+0000 7f4771614700 1 -- 172.21.15.57:0/2724539588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f476c0cd5a0 msgr2=0x7f476c0c8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.801+0000 7f4771614700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f476c0cd5a0 0x7f476c0c8b20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4764014040 tx=0x7f4764004ad0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.801+0000 7f4771614700 1 -- 172.21.15.57:0/2724539588 shutdown_connections 2023-11-28T12:03:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.801+0000 7f4771614700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f47580682f0 0x7f475806a7a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.801+0000 7f4771614700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f476c0cd5a0 0x7f476c0c8b20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.801+0000 7f4771614700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f476c06ab10 0x7f476c158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.801+0000 7f4771614700 1 --2- 172.21.15.57:0/2724539588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f476c0b22a0 0x7f476c0c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.801+0000 7f4771614700 1 -- 172.21.15.57:0/2724539588 >> 172.21.15.57:0/2724539588 conn(0x7f476c06c520 msgr2=0x7f476c0b8570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:39.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.801+0000 7f4771614700 1 -- 172.21.15.57:0/2724539588 shutdown_connections 2023-11-28T12:03:39.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.801+0000 7f4771614700 1 -- 172.21.15.57:0/2724539588 wait complete. 2023-11-28T12:03:39.817 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-28T12:03:39.817 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2023-11-28T12:03:39.817 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T12:03:39.818 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T12:03:39.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.956+0000 7fd91dce3700 1 Processor -- start 2023-11-28T12:03:39.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.956+0000 7fd91dce3700 1 -- start start 2023-11-28T12:03:39.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.956+0000 7fd91dce3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd9180ce040 0x7fd91815d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.956+0000 7fd91dce3700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91813fc90 con 0x7fd9180b1770 2023-11-28T12:03:39.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.956+0000 7fd91dce3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91813f480 con 0x7fd918142db0 2023-11-28T12:03:39.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.956+0000 7fd91dce3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91813f6d0 con 0x7fd9180ce040 2023-11-28T12:03:39.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd917fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd9180b1770 0x7fd9180b1b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40782/0 (socket says 172.21.15.57:40782) 2023-11-28T12:03:39.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd917fff700 1 -- 172.21.15.57:0/4025898479 learned_addr learned my addr 172.21.15.57:0/4025898479 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:39.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9177fe700 1 --2- 172.21.15.57:0/4025898479 >> v2:172.21.15.181:3300/0 conn(0x7fd9180ce040 0x7fd91815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532031882 0 0) 0x7fd91813fc90 con 0x7fd9180b1770 2023-11-28T12:03:39.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9000035b0 con 0x7fd9180b1770 2023-11-28T12:03:39.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1330653846 0 0) 0x7fd9000035b0 con 0x7fd9180b1770 2023-11-28T12:03:39.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 >> v2:172.21.15.181:3300/0 conn(0x7fd9180ce040 msgr2=0x7fd91815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:39.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 --2- 172.21.15.57:0/4025898479 >> v2:172.21.15.181:3300/0 conn(0x7fd9180ce040 0x7fd91815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 >> v1:172.21.15.181:6789/0 conn(0x7fd918142db0 legacy=0x7fd91813ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:39.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91813f9b0 con 0x7fd9180b1770 2023-11-28T12:03:39.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fd9100019f0 con 0x7fd9180b1770 2023-11-28T12:03:39.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 >> v1:172.21.15.57:6789/0 conn(0x7fd9180b1770 legacy=0x7fd9180b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:39.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 --2- 172.21.15.57:0/4025898479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd900003a00 0x7fd900003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 --2- 172.21.15.57:0/4025898479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd900004850 0x7fd900004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 --2- 172.21.15.57:0/4025898479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd900005200 0x7fd90000dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91813f6d0 con 0x7fd900004850 2023-11-28T12:03:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90000e290 con 0x7fd900005200 2023-11-28T12:03:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90000e450 con 0x7fd900003a00 2023-11-28T12:03:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.957+0000 7fd916ffd700 1 --2- 172.21.15.57:0/4025898479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd900004850 0x7fd900004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.958+0000 7fd917fff700 1 --2- 172.21.15.57:0/4025898479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd900005200 0x7fd90000dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.958+0000 7fd9177fe700 1 --2- 172.21.15.57:0/4025898479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd900003a00 0x7fd900003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.958+0000 7fd916ffd700 1 -- 172.21.15.57:0/4025898479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd900003a00 msgr2=0x7fd900003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.958+0000 7fd916ffd700 1 --2- 172.21.15.57:0/4025898479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd900003a00 0x7fd900003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.958+0000 7fd916ffd700 1 -- 172.21.15.57:0/4025898479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd900005200 msgr2=0x7fd90000dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:39.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.958+0000 7fd916ffd700 1 --2- 172.21.15.57:0/4025898479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd900005200 0x7fd90000dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.958+0000 7fd916ffd700 1 -- 172.21.15.57:0/4025898479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9000035b0 con 0x7fd900004850 2023-11-28T12:03:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.958+0000 7fd917fff700 1 -- 172.21.15.57:0/4025898479 reap_dead start 2023-11-28T12:03:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.958+0000 7fd916ffd700 1 --2- 172.21.15.57:0/4025898479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd900004850 0x7fd900004cc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0017c0 tx=0x7fd900005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e99824fc60ff259 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.958+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0052d0 con 0x7fd900004850 2023-11-28T12:03:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.958+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c005450 con 0x7fd900004850 2023-11-28T12:03:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.958+0000 7fd9167fc700 1 -- 172.21.15.57:0/4025898479 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0038c0 con 0x7fd900004850 2023-11-28T12:03:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.959+0000 7fd91dce3700 1 -- 172.21.15.57:0/4025898479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd900004850 msgr2=0x7fd900004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.959+0000 7fd91dce3700 1 --2- 172.21.15.57:0/4025898479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd900004850 0x7fd900004cc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0017c0 tx=0x7fd900005880 comp rx=0 tx=0).stop 2023-11-28T12:03:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.959+0000 7fd91dce3700 1 -- 172.21.15.57:0/4025898479 shutdown_connections 2023-11-28T12:03:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.959+0000 7fd91dce3700 1 --2- 172.21.15.57:0/4025898479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd900004850 0x7fd900004cc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.959+0000 7fd91dce3700 1 -- 172.21.15.57:0/4025898479 >> 172.21.15.57:0/4025898479 conn(0x7fd91806c010 msgr2=0x7fd9180cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.959+0000 7fd91dce3700 1 -- 172.21.15.57:0/4025898479 shutdown_connections 2023-11-28T12:03:39.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.959+0000 7fd91dce3700 1 -- 172.21.15.57:0/4025898479 wait complete. 2023-11-28T12:03:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.960+0000 7fd91dce3700 1 Processor -- start 2023-11-28T12:03:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.960+0000 7fd91dce3700 1 -- start start 2023-11-28T12:03:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.961+0000 7fd91dce3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd900004850 0x7fd91815c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.961+0000 7fd91dce3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9180c6dc0 0x7fd9180c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.961+0000 7fd91dce3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd918141a50 0x7fd918141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.961+0000 7fd91dce3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91806e9d0 con 0x7fd9180c6dc0 2023-11-28T12:03:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.961+0000 7fd91dce3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd918142410 con 0x7fd900004850 2023-11-28T12:03:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.961+0000 7fd91dce3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9181efad0 con 0x7fd918141a50 2023-11-28T12:03:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.961+0000 7fd916ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9180c6dc0 0x7fd9180c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.961+0000 7fd916ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9180c6dc0 0x7fd9180c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46036/0 (socket says 172.21.15.57:46036) 2023-11-28T12:03:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.961+0000 7fd916ffd700 1 -- 172.21.15.57:0/2180145547 learned_addr learned my addr 172.21.15.57:0/2180145547 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.961+0000 7fd9177fe700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd900004850 0x7fd91815c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.961+0000 7fd917fff700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd918141a50 0x7fd918141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.962+0000 7fd916ffd700 1 -- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd918141a50 msgr2=0x7fd918141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.962+0000 7fd916ffd700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd918141a50 0x7fd918141ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.962+0000 7fd916ffd700 1 -- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd900004850 msgr2=0x7fd91815c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.962+0000 7fd916ffd700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd900004850 0x7fd91815c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.962+0000 7fd916ffd700 1 -- 172.21.15.57:0/2180145547 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91815d3e0 con 0x7fd9180c6dc0 2023-11-28T12:03:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.962+0000 7fd916ffd700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9180c6dc0 0x7fd9180c5fd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000c00 tx=0x7fd90c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.962+0000 7fd8fffff700 1 -- 172.21.15.57:0/2180145547 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c001290 con 0x7fd9180c6dc0 2023-11-28T12:03:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.962+0000 7fd8fffff700 1 -- 172.21.15.57:0/2180145547 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c007040 con 0x7fd9180c6dc0 2023-11-28T12:03:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.962+0000 7fd91dce3700 1 -- 172.21.15.57:0/2180145547 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91815d100 con 0x7fd9180c6dc0 2023-11-28T12:03:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.962+0000 7fd8fffff700 1 -- 172.21.15.57:0/2180145547 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0073d0 con 0x7fd9180c6dc0 2023-11-28T12:03:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.962+0000 7fd91dce3700 1 -- 172.21.15.57:0/2180145547 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9180ce410 con 0x7fd9180c6dc0 2023-11-28T12:03:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.963+0000 7fd91dce3700 1 -- 172.21.15.57:0/2180145547 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9181efc80 con 0x7fd9180c6dc0 2023-11-28T12:03:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.963+0000 7fd8fffff700 1 -- 172.21.15.57:0/2180145547 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd90c02d070 con 0x7fd9180c6dc0 2023-11-28T12:03:39.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.964+0000 7fd8fffff700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9040682f0 0x7fd90406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:39.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.964+0000 7fd8fffff700 1 -- 172.21.15.57:0/2180145547 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd90c086040 con 0x7fd9180c6dc0 2023-11-28T12:03:39.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.964+0000 7fd9177fe700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9040682f0 0x7fd90406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:39.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.965+0000 7fd9177fe700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9040682f0 0x7fd90406a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd908009c60 tx=0x7fd908009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:39.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:39.969+0000 7fd8fffff700 1 -- 172.21.15.57:0/2180145547 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd9181efc80 con 0x7fd9180c6dc0 2023-11-28T12:03:40.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.255+0000 7fd91dce3700 1 -- 172.21.15.57:0/2180145547 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fd91815ca50 con 0x7fd9180c6dc0 2023-11-28T12:03:40.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.799+0000 7fd8fffff700 1 -- 172.21.15.57:0/2180145547 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v119) v1 ==== 152+0+0 (secure 0 0 0) 0x7fd90c052ba0 con 0x7fd9180c6dc0 2023-11-28T12:03:40.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.802+0000 7fd91dce3700 1 -- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9040682f0 msgr2=0x7fd90406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:40.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.802+0000 7fd91dce3700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9040682f0 0x7fd90406a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd908009c60 tx=0x7fd908009210 comp rx=0 tx=0).stop 2023-11-28T12:03:40.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.802+0000 7fd91dce3700 1 -- 172.21.15.57:0/2180145547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9180c6dc0 msgr2=0x7fd9180c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:40.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.802+0000 7fd91dce3700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9180c6dc0 0x7fd9180c5fd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000c00 tx=0x7fd90c001120 comp rx=0 tx=0).stop 2023-11-28T12:03:40.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.802+0000 7fd91dce3700 1 -- 172.21.15.57:0/2180145547 shutdown_connections 2023-11-28T12:03:40.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.802+0000 7fd91dce3700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9040682f0 0x7fd90406a7a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:40.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.802+0000 7fd91dce3700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd918141a50 0x7fd918141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:40.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.802+0000 7fd91dce3700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd900004850 0x7fd91815c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:40.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.802+0000 7fd91dce3700 1 --2- 172.21.15.57:0/2180145547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9180c6dc0 0x7fd9180c5fd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:40.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.802+0000 7fd91dce3700 1 -- 172.21.15.57:0/2180145547 >> 172.21.15.57:0/2180145547 conn(0x7fd91806c010 msgr2=0x7fd9180ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:40.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.802+0000 7fd91dce3700 1 -- 172.21.15.57:0/2180145547 shutdown_connections 2023-11-28T12:03:40.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.802+0000 7fd91dce3700 1 -- 172.21.15.57:0/2180145547 wait complete. 2023-11-28T12:03:40.822 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T12:03:40.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.964+0000 7f0616e4d700 1 Processor -- start 2023-11-28T12:03:40.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.964+0000 7f0616e4d700 1 -- start start 2023-11-28T12:03:40.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.964+0000 7f0616e4d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f061015a5f0 0x7f06100d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:40.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.964+0000 7f0616e4d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f06100cc200 0x7f06100b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:40.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.964+0000 7f0616e4d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06100b2d10 0x7f061015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:40.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.964+0000 7f0616e4d700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06100b3190 con 0x7f06100cc200 2023-11-28T12:03:40.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.964+0000 7f0616e4d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f061015a9c0 con 0x7f06100b2d10 2023-11-28T12:03:40.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.964+0000 7f0616e4d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100c7b80 con 0x7f061015a5f0 2023-11-28T12:03:40.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f060ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f06100cc200 0x7f06100b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:40.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f060ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f06100cc200 0x7f06100b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46050/0 (socket says 172.21.15.57:46050) 2023-11-28T12:03:40.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f060ffff700 1 -- 172.21.15.57:0/2977675079 learned_addr learned my addr 172.21.15.57:0/2977675079 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:40.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f0614be9700 1 --2- 172.21.15.57:0/2977675079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f061015a5f0 0x7f06100d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:40.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f06153ea700 1 --2- 172.21.15.57:0/2977675079 >> v2:172.21.15.181:3300/0 conn(0x7f06100b2d10 0x7f061015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:40.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f060ffff700 1 -- 172.21.15.57:0/2977675079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f061015a5f0 msgr2=0x7f06100d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f060ffff700 1 --2- 172.21.15.57:0/2977675079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f061015a5f0 0x7f06100d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f060ffff700 1 -- 172.21.15.57:0/2977675079 >> v2:172.21.15.181:3300/0 conn(0x7f06100b2d10 msgr2=0x7f061015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f060ffff700 1 --2- 172.21.15.57:0/2977675079 >> v2:172.21.15.181:3300/0 conn(0x7f06100b2d10 0x7f061015e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f060ffff700 1 -- 172.21.15.57:0/2977675079 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06100cc990 con 0x7f06100cc200 2023-11-28T12:03:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f060ffff700 1 --2- 172.21.15.57:0/2977675079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06100cc200 0x7f06100b27d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0610073cf0 tx=0x7f05f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124c2e2fc5a83e4c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f060f7fe700 1 -- 172.21.15.57:0/2977675079 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f800b870 con 0x7f06100cc200 2023-11-28T12:03:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f060f7fe700 1 -- 172.21.15.57:0/2977675079 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f800b9d0 con 0x7f06100cc200 2023-11-28T12:03:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.965+0000 7f060f7fe700 1 -- 172.21.15.57:0/2977675079 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f800bd60 con 0x7f06100cc200 2023-11-28T12:03:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.966+0000 7f0616e4d700 1 -- 172.21.15.57:0/2977675079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06100cc200 msgr2=0x7f06100b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.966+0000 7f0616e4d700 1 --2- 172.21.15.57:0/2977675079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06100cc200 0x7f06100b27d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0610073cf0 tx=0x7f05f8009bd0 comp rx=0 tx=0).stop 2023-11-28T12:03:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.966+0000 7f0616e4d700 1 -- 172.21.15.57:0/2977675079 shutdown_connections 2023-11-28T12:03:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.966+0000 7f0616e4d700 1 --2- 172.21.15.57:0/2977675079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f061015a5f0 0x7f06100d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.966+0000 7f0616e4d700 1 --2- 172.21.15.57:0/2977675079 >> v2:172.21.15.181:3300/0 conn(0x7f06100b2d10 0x7f061015e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.966+0000 7f0616e4d700 1 --2- 172.21.15.57:0/2977675079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06100cc200 0x7f06100b27d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.966+0000 7f0616e4d700 1 -- 172.21.15.57:0/2977675079 >> 172.21.15.57:0/2977675079 conn(0x7f061006ea80 msgr2=0x7f061006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.966+0000 7f0616e4d700 1 -- 172.21.15.57:0/2977675079 shutdown_connections 2023-11-28T12:03:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.966+0000 7f0616e4d700 1 -- 172.21.15.57:0/2977675079 wait complete. 2023-11-28T12:03:40.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.967+0000 7f0616e4d700 1 Processor -- start 2023-11-28T12:03:40.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.967+0000 7f0616e4d700 1 -- start start 2023-11-28T12:03:40.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.968+0000 7f0616e4d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06100b2d10 0x7f0610158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.968+0000 7f0616e4d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06100cc200 0x7f06101f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.968+0000 7f0616e4d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f061015a5f0 0x7f06101f0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.968+0000 7f0616e4d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06101576c0 con 0x7f061015a5f0 2023-11-28T12:03:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.968+0000 7f0616e4d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0610157f20 con 0x7f06100cc200 2023-11-28T12:03:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.968+0000 7f0616e4d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0610157960 con 0x7f06100b2d10 2023-11-28T12:03:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.968+0000 7f06153ea700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f061015a5f0 0x7f06101f0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f06153ea700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f061015a5f0 0x7f06101f0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46066/0 (socket says 172.21.15.57:46066) 2023-11-28T12:03:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f06153ea700 1 -- 172.21.15.57:0/414679210 learned_addr learned my addr 172.21.15.57:0/414679210 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f0614be9700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06100b2d10 0x7f0610158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f060ffff700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06100cc200 0x7f06101f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f06153ea700 1 -- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06100b2d10 msgr2=0x7f0610158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:40.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f06153ea700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06100b2d10 0x7f0610158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f06153ea700 1 -- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06100cc200 msgr2=0x7f06101f04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f06153ea700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06100cc200 0x7f06101f04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f06153ea700 1 -- 172.21.15.57:0/414679210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f8009550 con 0x7f061015a5f0 2023-11-28T12:03:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f06153ea700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f061015a5f0 0x7f06101f0a30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0608007e60 tx=0x7f060800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f060d7fa700 1 -- 172.21.15.57:0/414679210 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060800ed30 con 0x7f061015a5f0 2023-11-28T12:03:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f060d7fa700 1 -- 172.21.15.57:0/414679210 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0608014420 con 0x7f061015a5f0 2023-11-28T12:03:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f060d7fa700 1 -- 172.21.15.57:0/414679210 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06080147d0 con 0x7f061015a5f0 2023-11-28T12:03:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f0616e4d700 1 -- 172.21.15.57:0/414679210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06100ccf50 con 0x7f061015a5f0 2023-11-28T12:03:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.969+0000 7f0616e4d700 1 -- 172.21.15.57:0/414679210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06100ccc70 con 0x7f061015a5f0 2023-11-28T12:03:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.970+0000 7f0616e4d700 1 -- 172.21.15.57:0/414679210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06100cd230 con 0x7f061015a5f0 2023-11-28T12:03:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.971+0000 7f060d7fa700 1 -- 172.21.15.57:0/414679210 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0608012050 con 0x7f061015a5f0 2023-11-28T12:03:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.972+0000 7f060d7fa700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f05f00682f0 0x7f05f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.972+0000 7f060d7fa700 1 -- 172.21.15.57:0/414679210 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0608086d80 con 0x7f061015a5f0 2023-11-28T12:03:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.972+0000 7f0614be9700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f05f00682f0 0x7f05f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.973+0000 7f0614be9700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f05f00682f0 0x7f05f006a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f06100b52b0 tx=0x7f0600008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:40.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:40.976+0000 7f060d7fa700 1 -- 172.21.15.57:0/414679210 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0608053890 con 0x7f061015a5f0 2023-11-28T12:03:41.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.178+0000 7f0616e4d700 1 -- 172.21.15.57:0/414679210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f06100743f0 con 0x7f061015a5f0 2023-11-28T12:03:41.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.182+0000 7f060d7fa700 1 -- 172.21.15.57:0/414679210 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0608056eb0 con 0x7f061015a5f0 2023-11-28T12:03:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.184+0000 7f0616e4d700 1 -- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f05f00682f0 msgr2=0x7f05f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.184+0000 7f0616e4d700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f05f00682f0 0x7f05f006a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f06100b52b0 tx=0x7f0600008040 comp rx=0 tx=0).stop 2023-11-28T12:03:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.184+0000 7f0616e4d700 1 -- 172.21.15.57:0/414679210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f061015a5f0 msgr2=0x7f06101f0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.184+0000 7f0616e4d700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f061015a5f0 0x7f06101f0a30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0608007e60 tx=0x7f060800e3c0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.184+0000 7f0616e4d700 1 -- 172.21.15.57:0/414679210 shutdown_connections 2023-11-28T12:03:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.185+0000 7f0616e4d700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f05f00682f0 0x7f05f006a7a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.185+0000 7f0616e4d700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f061015a5f0 0x7f06101f0a30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.185+0000 7f0616e4d700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06100b2d10 0x7f0610158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.185+0000 7f0616e4d700 1 --2- 172.21.15.57:0/414679210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06100cc200 0x7f06101f04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.185+0000 7f0616e4d700 1 -- 172.21.15.57:0/414679210 >> 172.21.15.57:0/414679210 conn(0x7f061006ea80 msgr2=0x7f06100cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.185+0000 7f0616e4d700 1 -- 172.21.15.57:0/414679210 shutdown_connections 2023-11-28T12:03:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.185+0000 7f0616e4d700 1 -- 172.21.15.57:0/414679210 wait complete. 2023-11-28T12:03:41.199 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T12:03:41.199 DEBUG:teuthology.orchestra.run.smithi057:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T12:03:41.239 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T12:03:41.282 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T12:03:41.283 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:03:41.283 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:03:41.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16254224k,nr_inodes=4063556,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15749 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3254388k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T12:03:41.304 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:03:41.305 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:03:41.344 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251064k,nr_inodes=4062766,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14954 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253752k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T12:03:41.344 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:03:41.344 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:03:41.368 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16254224k,nr_inodes=4063556,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15749 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3254388k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T12:03:41.368 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:03:41.368 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:03:41.404 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251064k,nr_inodes=4062766,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14954 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253752k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T12:03:41.404 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T12:03:41.404 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:03:41.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.549+0000 7f2f1845f700 1 Processor -- start 2023-11-28T12:03:41.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.550+0000 7f2f1845f700 1 -- start start 2023-11-28T12:03:41.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.550+0000 7f2f1845f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2f1006ab10 0x7f2f1006aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:41.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f1845f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2f100cd5a0 0x7f2f10160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:41.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f1845f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f100d1ee0 con 0x7f2f100b22a0 2023-11-28T12:03:41.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f1845f700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1006b420 con 0x7f2f100cd5a0 2023-11-28T12:03:41.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f1845f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f100cdb10 con 0x7f2f1006ab10 2023-11-28T12:03:41.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f159fa700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2f100cd5a0 0x7f2f10160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:41.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f169fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2f100b22a0 0x7f2f10160840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40792/0 (socket says 172.21.15.57:40792) 2023-11-28T12:03:41.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f169fc700 1 -- 172.21.15.57:0/2548803344 learned_addr learned my addr 172.21.15.57:0/2548803344 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:41.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f159fa700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2f100cd5a0 0x7f2f10160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46070/0 (socket says 172.21.15.57:46070) 2023-11-28T12:03:41.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f161fb700 1 --2- 172.21.15.57:0/2548803344 >> v2:172.21.15.181:3300/0 conn(0x7f2f1006ab10 0x7f2f1006aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:41.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f159fa700 1 -- 172.21.15.57:0/2548803344 >> v2:172.21.15.181:3300/0 conn(0x7f2f1006ab10 msgr2=0x7f2f1006aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:41.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f159fa700 1 --2- 172.21.15.57:0/2548803344 >> v2:172.21.15.181:3300/0 conn(0x7f2f1006ab10 0x7f2f1006aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f159fa700 1 -- 172.21.15.57:0/2548803344 >> v1:172.21.15.57:6789/0 conn(0x7f2f100b22a0 legacy=0x7f2f10160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:41.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f159fa700 1 -- 172.21.15.57:0/2548803344 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f100d1890 con 0x7f2f100cd5a0 2023-11-28T12:03:41.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f151f9700 1 -- 172.21.15.57:0/2548803344 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577130031 0 0) 0x7f2f100d1ee0 con 0x7f2f100b22a0 2023-11-28T12:03:41.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f159fa700 1 --2- 172.21.15.57:0/2548803344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f100cd5a0 0x7f2f10160300 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2f100d0cb0 tx=0x7f2f00009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d56e356f1a07c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:41.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f151f9700 1 -- 172.21.15.57:0/2548803344 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00011830 con 0x7f2f100cd5a0 2023-11-28T12:03:41.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f151f9700 1 -- 172.21.15.57:0/2548803344 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f00011990 con 0x7f2f100cd5a0 2023-11-28T12:03:41.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.551+0000 7f2f151f9700 1 -- 172.21.15.57:0/2548803344 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00011d20 con 0x7f2f100cd5a0 2023-11-28T12:03:41.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.552+0000 7f2f1845f700 1 -- 172.21.15.57:0/2548803344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f100cd5a0 msgr2=0x7f2f10160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:41.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.552+0000 7f2f1845f700 1 --2- 172.21.15.57:0/2548803344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f100cd5a0 0x7f2f10160300 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2f100d0cb0 tx=0x7f2f00009b20 comp rx=0 tx=0).stop 2023-11-28T12:03:41.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.552+0000 7f2f1845f700 1 -- 172.21.15.57:0/2548803344 shutdown_connections 2023-11-28T12:03:41.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.552+0000 7f2f1845f700 1 --2- 172.21.15.57:0/2548803344 >> v2:172.21.15.181:3300/0 conn(0x7f2f1006ab10 0x7f2f1006aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.552+0000 7f2f1845f700 1 --2- 172.21.15.57:0/2548803344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f100cd5a0 0x7f2f10160300 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.552+0000 7f2f1845f700 1 -- 172.21.15.57:0/2548803344 >> 172.21.15.57:0/2548803344 conn(0x7f2f1006c520 msgr2=0x7f2f1006c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:41.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.552+0000 7f2f1845f700 1 -- 172.21.15.57:0/2548803344 shutdown_connections 2023-11-28T12:03:41.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.552+0000 7f2f1845f700 1 -- 172.21.15.57:0/2548803344 wait complete. 2023-11-28T12:03:41.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.554+0000 7f2f1845f700 1 Processor -- start 2023-11-28T12:03:41.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.554+0000 7f2f1845f700 1 -- start start 2023-11-28T12:03:41.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.554+0000 7f2f1845f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1006ab10 0x7f2f10158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:41.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.554+0000 7f2f1845f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f100b22a0 0x7f2f100cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:41.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.554+0000 7f2f1845f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f100cd5a0 0x7f2f100cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:41.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.554+0000 7f2f1845f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10158ea0 con 0x7f2f100cd5a0 2023-11-28T12:03:41.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.554+0000 7f2f1845f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f101f6530 con 0x7f2f1006ab10 2023-11-28T12:03:41.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.554+0000 7f2f1845f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10158980 con 0x7f2f100b22a0 2023-11-28T12:03:41.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f169fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f100cd5a0 0x7f2f100cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:41.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f169fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f100cd5a0 0x7f2f100cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46080/0 (socket says 172.21.15.57:46080) 2023-11-28T12:03:41.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f169fc700 1 -- 172.21.15.57:0/1784639427 learned_addr learned my addr 172.21.15.57:0/1784639427 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:41.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f161fb700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1006ab10 0x7f2f10158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:41.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f159fa700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f100b22a0 0x7f2f100cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:41.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f169fc700 1 -- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f100b22a0 msgr2=0x7f2f100cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:41.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f169fc700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f100b22a0 0x7f2f100cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f169fc700 1 -- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1006ab10 msgr2=0x7f2f10158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:41.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f169fc700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1006ab10 0x7f2f10158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f169fc700 1 -- 172.21.15.57:0/1784639427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f000094a0 con 0x7f2f100cd5a0 2023-11-28T12:03:41.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f161fb700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1006ab10 0x7f2f10158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:41.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f169fc700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f100cd5a0 0x7f2f100cc570 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2f08002230 tx=0x7f2f0800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:41.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f06ffd700 1 -- 172.21.15.57:0/1784639427 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0800abc0 con 0x7f2f100cd5a0 2023-11-28T12:03:41.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f06ffd700 1 -- 172.21.15.57:0/1784639427 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0800ad40 con 0x7f2f100cd5a0 2023-11-28T12:03:41.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f1845f700 1 -- 172.21.15.57:0/1784639427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1015cad0 con 0x7f2f100cd5a0 2023-11-28T12:03:41.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f06ffd700 1 -- 172.21.15.57:0/1784639427 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f08007310 con 0x7f2f100cd5a0 2023-11-28T12:03:41.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.555+0000 7f2f1845f700 1 -- 172.21.15.57:0/1784639427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f100d1b70 con 0x7f2f100cd5a0 2023-11-28T12:03:41.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.556+0000 7f2f1845f700 1 -- 172.21.15.57:0/1784639427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f100b1ec0 con 0x7f2f100cd5a0 2023-11-28T12:03:41.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.557+0000 7f2f06ffd700 1 -- 172.21.15.57:0/1784639427 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2f0801a030 con 0x7f2f100cd5a0 2023-11-28T12:03:41.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.557+0000 7f2f06ffd700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2ef40682f0 0x7f2ef406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:41.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.557+0000 7f2f06ffd700 1 -- 172.21.15.57:0/1784639427 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2f080856e0 con 0x7f2f100cd5a0 2023-11-28T12:03:41.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.557+0000 7f2f161fb700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2ef40682f0 0x7f2ef406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:41.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.558+0000 7f2f161fb700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2ef40682f0 0x7f2ef406a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c007a50 tx=0x7f2f0c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:41.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.562+0000 7f2f06ffd700 1 -- 172.21.15.57:0/1784639427 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2f08052240 con 0x7f2f100cd5a0 2023-11-28T12:03:41.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.811+0000 7f2f1845f700 1 -- 172.21.15.57:0/1784639427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f100743f0 con 0x7f2f100cd5a0 2023-11-28T12:03:41.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.811+0000 7f2f06ffd700 1 -- 172.21.15.57:0/1784639427 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f2f08018090 con 0x7f2f100cd5a0 2023-11-28T12:03:41.813 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:41.813 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/340292109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":340292109},{"type":"v1","addr":"172.21.15.181:6836","nonce":340292109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2442321825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2442321825},{"type":"v1","addr":"172.21.15.181:6837","nonce":2442321825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/3452242211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3452242211},{"type":"v1","addr":"172.21.15.57:6833","nonce":3452242211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2036956183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2036956183},{"type":"v1","addr":"172.21.15.57:6835","nonce":2036956183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-11-28T12:03:41.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.814+0000 7f2f1845f700 1 -- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2ef40682f0 msgr2=0x7f2ef406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:41.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.814+0000 7f2f1845f700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2ef40682f0 0x7f2ef406a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c007a50 tx=0x7f2f0c008040 comp rx=0 tx=0).stop 2023-11-28T12:03:41.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.814+0000 7f2f1845f700 1 -- 172.21.15.57:0/1784639427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f100cd5a0 msgr2=0x7f2f100cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:41.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.814+0000 7f2f1845f700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f100cd5a0 0x7f2f100cc570 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2f08002230 tx=0x7f2f0800a060 comp rx=0 tx=0).stop 2023-11-28T12:03:41.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.814+0000 7f2f1845f700 1 -- 172.21.15.57:0/1784639427 shutdown_connections 2023-11-28T12:03:41.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.814+0000 7f2f1845f700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2ef40682f0 0x7f2ef406a7a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.814+0000 7f2f1845f700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f100cd5a0 0x7f2f100cc570 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.814+0000 7f2f1845f700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1006ab10 0x7f2f10158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.814+0000 7f2f1845f700 1 --2- 172.21.15.57:0/1784639427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f100b22a0 0x7f2f100cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:41.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.814+0000 7f2f1845f700 1 -- 172.21.15.57:0/1784639427 >> 172.21.15.57:0/1784639427 conn(0x7f2f1006c520 msgr2=0x7f2f100b6730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:41.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.814+0000 7f2f1845f700 1 -- 172.21.15.57:0/1784639427 shutdown_connections 2023-11-28T12:03:41.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:41.814+0000 7f2f1845f700 1 -- 172.21.15.57:0/1784639427 wait complete. 2023-11-28T12:03:41.817 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 18 2023-11-28T12:03:41.829 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T12:03:41.829 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T12:03:41.829 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T12:03:41.829 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:03:41.830 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T12:03:41.830 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T12:03:41.830 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T12:03:41.830 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:03:41.830 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T12:03:41.830 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T12:03:41.830 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T12:03:41.831 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:03:41.831 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T12:03:41.831 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T12:03:41.831 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T12:03:41.831 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:03:41.842 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T12:03:41.842 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T12:03:41.845 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T12:03:41.845 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T12:03:41.847 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T12:03:41.848 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T12:03:41.849 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T12:03:41.849 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T12:03:41.885 INFO:tasks.ceph.mds.a:Started 2023-11-28T12:03:41.885 DEBUG:teuthology.parallel:result is None 2023-11-28T12:03:41.886 INFO:tasks.ceph.mds.c:Started 2023-11-28T12:03:41.886 DEBUG:teuthology.parallel:result is None 2023-11-28T12:03:41.889 INFO:tasks.ceph.mds.b:Started 2023-11-28T12:03:41.889 DEBUG:teuthology.parallel:result is None 2023-11-28T12:03:41.890 INFO:tasks.ceph.mds.d:Started 2023-11-28T12:03:41.890 DEBUG:teuthology.parallel:result is None 2023-11-28T12:03:41.890 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T12:03:41.975 INFO:tasks.ceph.mds.c.smithi057.stdout:starting mds.c at 2023-11-28T12:03:41.975 INFO:tasks.ceph.mds.a.smithi057.stdout:starting mds.a at 2023-11-28T12:03:41.979 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-11-28T12:03:41.980 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-11-28T12:03:42.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.037+0000 7f17aa31a700 1 Processor -- start 2023-11-28T12:03:42.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.038+0000 7f17aa31a700 1 -- start start 2023-11-28T12:03:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.038+0000 7f17aa31a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f17a406b220 0x7f17a40d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.038+0000 7f17aa31a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a40d2d30 con 0x7f17a4160cb0 2023-11-28T12:03:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.038+0000 7f17aa31a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a40d0b40 con 0x7f17a415c8c0 2023-11-28T12:03:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.038+0000 7f17aa31a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a40d2a50 con 0x7f17a406b220 2023-11-28T12:03:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f179bfff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f17a4160cb0 0x7f17a406ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40818/0 (socket says 172.21.15.57:40818) 2023-11-28T12:03:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f179bfff700 1 -- 172.21.15.57:0/3931689908 learned_addr learned my addr 172.21.15.57:0/3931689908 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a88b7700 1 --2- 172.21.15.57:0/3931689908 >> v2:172.21.15.181:3300/0 conn(0x7f17a406b220 0x7f17a40d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 333446001 0 0) 0x7f178c001110 con 0x7f17a4160cb0 2023-11-28T12:03:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17900035b0 con 0x7f17a4160cb0 2023-11-28T12:03:42.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3385167431 0 0) 0x7f17900035b0 con 0x7f17a4160cb0 2023-11-28T12:03:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 >> v2:172.21.15.181:3300/0 conn(0x7f17a406b220 msgr2=0x7f17a40d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 --2- 172.21.15.57:0/3931689908 >> v2:172.21.15.181:3300/0 conn(0x7f17a406b220 0x7f17a40d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 >> v1:172.21.15.181:6789/0 conn(0x7f17a415c8c0 legacy=0x7f17a40b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a40cd760 con 0x7f17a4160cb0 2023-11-28T12:03:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f178c001800 con 0x7f17a4160cb0 2023-11-28T12:03:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 >> v1:172.21.15.57:6789/0 conn(0x7f17a4160cb0 legacy=0x7f17a406ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 --2- 172.21.15.57:0/3931689908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1790003a00 0x7f1790003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 --2- 172.21.15.57:0/3931689908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17900043f0 0x7f17900048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 --2- 172.21.15.57:0/3931689908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17900050b0 0x7f179000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a40d2a50 con 0x7f1790003a00 2023-11-28T12:03:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179000c000 con 0x7f17900050b0 2023-11-28T12:03:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179000c1c0 con 0x7f17900043f0 2023-11-28T12:03:42.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a3fff700 1 --2- 172.21.15.57:0/3931689908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1790003a00 0x7f1790003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f17a88b7700 1 --2- 172.21.15.57:0/3931689908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17900050b0 0x7f179000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.039+0000 7f179bfff700 1 --2- 172.21.15.57:0/3931689908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17900043f0 0x7f17900048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.040+0000 7f17a3fff700 1 -- 172.21.15.57:0/3931689908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17900043f0 msgr2=0x7f17900048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.040+0000 7f17a3fff700 1 --2- 172.21.15.57:0/3931689908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17900043f0 0x7f17900048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.040+0000 7f17a3fff700 1 -- 172.21.15.57:0/3931689908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17900050b0 msgr2=0x7f179000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.040+0000 7f17a3fff700 1 --2- 172.21.15.57:0/3931689908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17900050b0 0x7f179000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.040+0000 7f17a3fff700 1 -- 172.21.15.57:0/3931689908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17900035b0 con 0x7f1790003a00 2023-11-28T12:03:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.040+0000 7f17a88b7700 1 --2- 172.21.15.57:0/3931689908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17900050b0 0x7f179000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:03:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.040+0000 7f17a88b7700 1 -- 172.21.15.57:0/3931689908 reap_dead start 2023-11-28T12:03:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.040+0000 7f17a3fff700 1 --2- 172.21.15.57:0/3931689908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1790003a00 0x7f1790003dd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f17940018d0 tx=0x7f1790005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63390ba5df74bc5c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:42.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.040+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17940052d0 con 0x7f1790003a00 2023-11-28T12:03:42.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.040+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1794005430 con 0x7f1790003a00 2023-11-28T12:03:42.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.040+0000 7f17a37fe700 1 -- 172.21.15.57:0/3931689908 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17940038c0 con 0x7f1790003a00 2023-11-28T12:03:42.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.041+0000 7f17aa31a700 1 -- 172.21.15.57:0/3931689908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1790003a00 msgr2=0x7f1790003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:42.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.041+0000 7f17aa31a700 1 --2- 172.21.15.57:0/3931689908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1790003a00 0x7f1790003dd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f17940018d0 tx=0x7f1790005880 comp rx=0 tx=0).stop 2023-11-28T12:03:42.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.041+0000 7f17aa31a700 1 -- 172.21.15.57:0/3931689908 shutdown_connections 2023-11-28T12:03:42.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.041+0000 7f17aa31a700 1 --2- 172.21.15.57:0/3931689908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1790003a00 0x7f1790003dd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:42.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.041+0000 7f17aa31a700 1 -- 172.21.15.57:0/3931689908 >> 172.21.15.57:0/3931689908 conn(0x7f17a406c520 msgr2=0x7f17a406c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:42.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.041+0000 7f17aa31a700 1 -- 172.21.15.57:0/3931689908 shutdown_connections 2023-11-28T12:03:42.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.041+0000 7f17aa31a700 1 -- 172.21.15.57:0/3931689908 wait complete. 2023-11-28T12:03:42.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.042+0000 7f17aa31a700 1 Processor -- start 2023-11-28T12:03:42.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.043+0000 7f17aa31a700 1 -- start start 2023-11-28T12:03:42.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17aa31a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1790003a00 0x7f17a414d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:42.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17aa31a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a41503d0 0x7f17a415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:42.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17aa31a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17a40b79a0 0x7f17a40b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:42.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17aa31a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a40b8350 con 0x7f17a41503d0 2023-11-28T12:03:42.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17aa31a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a415d4e0 con 0x7f1790003a00 2023-11-28T12:03:42.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17aa31a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a415d7c0 con 0x7f17a40b79a0 2023-11-28T12:03:42.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f179bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a41503d0 0x7f17a415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:42.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f179bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a41503d0 0x7f17a415cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46120/0 (socket says 172.21.15.57:46120) 2023-11-28T12:03:42.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f179bfff700 1 -- 172.21.15.57:0/1466472043 learned_addr learned my addr 172.21.15.57:0/1466472043 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:42.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17a88b7700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17a40b79a0 0x7f17a40b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:42.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17a3fff700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1790003a00 0x7f17a414d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:42.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f179bfff700 1 -- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17a40b79a0 msgr2=0x7f17a40b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:42.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f179bfff700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17a40b79a0 0x7f17a40b7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:42.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f179bfff700 1 -- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1790003a00 msgr2=0x7f17a414d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:42.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f179bfff700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1790003a00 0x7f17a414d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:42.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f179bfff700 1 -- 172.21.15.57:0/1466472043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a4161120 con 0x7f17a41503d0 2023-11-28T12:03:42.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17a88b7700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17a40b79a0 0x7f17a40b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:42.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17a3fff700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1790003a00 0x7f17a414d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:42.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f179bfff700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a41503d0 0x7f17a415cc90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f178c007290 tx=0x7f178c007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:42.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17a17fa700 1 -- 172.21.15.57:0/1466472043 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c00f770 con 0x7f17a41503d0 2023-11-28T12:03:42.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17a17fa700 1 -- 172.21.15.57:0/1466472043 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178c00f8f0 con 0x7f17a41503d0 2023-11-28T12:03:42.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17aa31a700 1 -- 172.21.15.57:0/1466472043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a40cda40 con 0x7f17a41503d0 2023-11-28T12:03:42.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.044+0000 7f17a17fa700 1 -- 172.21.15.57:0/1466472043 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c00fca0 con 0x7f17a41503d0 2023-11-28T12:03:42.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.045+0000 7f17aa31a700 1 -- 172.21.15.57:0/1466472043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a40d2700 con 0x7f17a41503d0 2023-11-28T12:03:42.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.045+0000 7f17aa31a700 1 -- 172.21.15.57:0/1466472043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a4067d70 con 0x7f17a41503d0 2023-11-28T12:03:42.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.046+0000 7f17a17fa700 1 -- 172.21.15.57:0/1466472043 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f178c01d460 con 0x7f17a41503d0 2023-11-28T12:03:42.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.047+0000 7f17a17fa700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f17840682f0 0x7f178406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:42.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.047+0000 7f17a17fa700 1 -- 172.21.15.57:0/1466472043 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f178c0859b0 con 0x7f17a41503d0 2023-11-28T12:03:42.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.047+0000 7f17a3fff700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f17840682f0 0x7f178406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:42.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.047+0000 7f17a3fff700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f17840682f0 0x7f178406a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1794000c00 tx=0x7f1794005000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:42.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.052+0000 7f17a17fa700 1 -- 172.21.15.57:0/1466472043 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f178c0523e0 con 0x7f17a41503d0 2023-11-28T12:03:42.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:42.269+0000 7f17aa31a700 1 -- 172.21.15.57:0/1466472043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f17a40d3500 con 0x7f17a41503d0 2023-11-28T12:03:43.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.190+0000 7f17a17fa700 1 -- 172.21.15.57:0/1466472043 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v45) v1 ==== 97+0+0 (secure 0 0 0) 0x7f17a40d3500 con 0x7f17a41503d0 2023-11-28T12:03:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.193+0000 7f17aa31a700 1 -- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f17840682f0 msgr2=0x7f178406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.193+0000 7f17aa31a700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f17840682f0 0x7f178406a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1794000c00 tx=0x7f1794005000 comp rx=0 tx=0).stop 2023-11-28T12:03:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.193+0000 7f17aa31a700 1 -- 172.21.15.57:0/1466472043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a41503d0 msgr2=0x7f17a415cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.193+0000 7f17aa31a700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a41503d0 0x7f17a415cc90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f178c007290 tx=0x7f178c007650 comp rx=0 tx=0).stop 2023-11-28T12:03:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.194+0000 7f17aa31a700 1 -- 172.21.15.57:0/1466472043 shutdown_connections 2023-11-28T12:03:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.194+0000 7f17aa31a700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f17840682f0 0x7f178406a7a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.194+0000 7f17aa31a700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17a40b79a0 0x7f17a40b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.194+0000 7f17aa31a700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1790003a00 0x7f17a414d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.194+0000 7f17aa31a700 1 --2- 172.21.15.57:0/1466472043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a41503d0 0x7f17a415cc90 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.194+0000 7f17aa31a700 1 -- 172.21.15.57:0/1466472043 >> 172.21.15.57:0/1466472043 conn(0x7f17a406c520 msgr2=0x7f17a4157650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.194+0000 7f17aa31a700 1 -- 172.21.15.57:0/1466472043 shutdown_connections 2023-11-28T12:03:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.194+0000 7f17aa31a700 1 -- 172.21.15.57:0/1466472043 wait complete. 2023-11-28T12:03:43.196 INFO:teuthology.orchestra.run.smithi057.stderr: removed all blocklist entries 2023-11-28T12:03:43.209 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T12:03:43.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.349+0000 7faa30330700 1 Processor -- start 2023-11-28T12:03:43.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.349+0000 7faa30330700 1 -- start start 2023-11-28T12:03:43.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.350+0000 7faa30330700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa2806a270 0x7faa2806a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:43.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.350+0000 7faa30330700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7faa2806e570 0x7faa28070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:43.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.350+0000 7faa30330700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2806ab80 con 0x7faa2815cc80 2023-11-28T12:03:43.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.350+0000 7faa30330700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa280b2490 con 0x7faa2806e570 2023-11-28T12:03:43.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.350+0000 7faa30330700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2806e990 con 0x7faa2806a270 2023-11-28T12:03:43.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.350+0000 7faa2e8cd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7faa2806e570 0x7faa28070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:43.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.350+0000 7faa2d8cb700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7faa2815cc80 0x7faa2806de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40826/0 (socket says 172.21.15.57:40826) 2023-11-28T12:03:43.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.350+0000 7faa2d8cb700 1 -- 172.21.15.57:0/762524999 learned_addr learned my addr 172.21.15.57:0/762524999 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:43.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.350+0000 7faa2e0cc700 1 --2- 172.21.15.57:0/762524999 >> v2:172.21.15.181:3300/0 conn(0x7faa2806a270 0x7faa2806a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.351+0000 7faa2e8cd700 1 -- 172.21.15.57:0/762524999 >> v2:172.21.15.181:3300/0 conn(0x7faa2806a270 msgr2=0x7faa2806a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.351+0000 7faa2e8cd700 1 --2- 172.21.15.57:0/762524999 >> v2:172.21.15.181:3300/0 conn(0x7faa2806a270 0x7faa2806a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.351+0000 7faa2e8cd700 1 -- 172.21.15.57:0/762524999 >> v1:172.21.15.57:6789/0 conn(0x7faa2815cc80 legacy=0x7faa2806de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.351+0000 7faa2d0ca700 1 -- 172.21.15.57:0/762524999 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856155185 0 0) 0x7faa2806ab80 con 0x7faa2815cc80 2023-11-28T12:03:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.351+0000 7faa2e8cd700 1 -- 172.21.15.57:0/762524999 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa280b1810 con 0x7faa2806e570 2023-11-28T12:03:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.351+0000 7faa2e8cd700 1 --2- 172.21.15.57:0/762524999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2806e570 0x7faa28070d20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faa28067380 tx=0x7faa2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380e86d12330e9dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.351+0000 7faa2d0ca700 1 -- 172.21.15.57:0/762524999 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2000e070 con 0x7faa2806e570 2023-11-28T12:03:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.351+0000 7faa2d0ca700 1 -- 172.21.15.57:0/762524999 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2000bae0 con 0x7faa2806e570 2023-11-28T12:03:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.351+0000 7faa2d0ca700 1 -- 172.21.15.57:0/762524999 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2000be90 con 0x7faa2806e570 2023-11-28T12:03:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.352+0000 7faa30330700 1 -- 172.21.15.57:0/762524999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2806e570 msgr2=0x7faa28070d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.352+0000 7faa30330700 1 --2- 172.21.15.57:0/762524999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2806e570 0x7faa28070d20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faa28067380 tx=0x7faa2000b330 comp rx=0 tx=0).stop 2023-11-28T12:03:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.352+0000 7faa30330700 1 -- 172.21.15.57:0/762524999 shutdown_connections 2023-11-28T12:03:43.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.352+0000 7faa30330700 1 --2- 172.21.15.57:0/762524999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2806e570 0x7faa28070d20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.352+0000 7faa30330700 1 --2- 172.21.15.57:0/762524999 >> v2:172.21.15.181:3300/0 conn(0x7faa2806a270 0x7faa2806a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.352+0000 7faa30330700 1 -- 172.21.15.57:0/762524999 >> 172.21.15.57:0/762524999 conn(0x7faa2806c010 msgr2=0x7faa280cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.352+0000 7faa30330700 1 -- 172.21.15.57:0/762524999 shutdown_connections 2023-11-28T12:03:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.352+0000 7faa30330700 1 -- 172.21.15.57:0/762524999 wait complete. 2023-11-28T12:03:43.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.353+0000 7faa30330700 1 Processor -- start 2023-11-28T12:03:43.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.353+0000 7faa30330700 1 -- start start 2023-11-28T12:03:43.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa30330700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2806a270 0x7faa280b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:43.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa30330700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2806e570 0x7faa280b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:43.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa30330700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa2815cc80 0x7faa281f1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:43.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa30330700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa280b3370 con 0x7faa2806e570 2023-11-28T12:03:43.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa30330700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa280b3090 con 0x7faa2806a270 2023-11-28T12:03:43.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa30330700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa28142a70 con 0x7faa2815cc80 2023-11-28T12:03:43.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa2d8cb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2806e570 0x7faa280b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:43.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa2d8cb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2806e570 0x7faa280b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46150/0 (socket says 172.21.15.57:46150) 2023-11-28T12:03:43.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa2d8cb700 1 -- 172.21.15.57:0/2088831327 learned_addr learned my addr 172.21.15.57:0/2088831327 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:43.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa2e0cc700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2806a270 0x7faa280b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:43.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa2e8cd700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa2815cc80 0x7faa281f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:43.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa2d8cb700 1 -- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa2815cc80 msgr2=0x7faa281f1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:43.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa2d8cb700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa2815cc80 0x7faa281f1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa2d8cb700 1 -- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2806a270 msgr2=0x7faa280b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:43.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa2d8cb700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2806a270 0x7faa280b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa2d8cb700 1 -- 172.21.15.57:0/2088831327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa20009a70 con 0x7faa2806e570 2023-11-28T12:03:43.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa2e8cd700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa2815cc80 0x7faa281f1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:43.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.354+0000 7faa2d8cb700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2806e570 0x7faa280b0460 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c550 tx=0x7faa1c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:43.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.355+0000 7faa1affd700 1 -- 172.21.15.57:0/2088831327 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c004c80 con 0x7faa2806e570 2023-11-28T12:03:43.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.355+0000 7faa1affd700 1 -- 172.21.15.57:0/2088831327 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c004e00 con 0x7faa2806e570 2023-11-28T12:03:43.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.355+0000 7faa30330700 1 -- 172.21.15.57:0/2088831327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa280b1dd0 con 0x7faa2806e570 2023-11-28T12:03:43.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.355+0000 7faa1affd700 1 -- 172.21.15.57:0/2088831327 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c007310 con 0x7faa2806e570 2023-11-28T12:03:43.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.355+0000 7faa30330700 1 -- 172.21.15.57:0/2088831327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa280b1af0 con 0x7faa2806e570 2023-11-28T12:03:43.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.356+0000 7faa30330700 1 -- 172.21.15.57:0/2088831327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa280b20b0 con 0x7faa2806e570 2023-11-28T12:03:43.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.356+0000 7faa1affd700 1 -- 172.21.15.57:0/2088831327 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faa1c01a020 con 0x7faa2806e570 2023-11-28T12:03:43.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.357+0000 7faa1affd700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faa0c0682f0 0x7faa0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:43.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.357+0000 7faa1affd700 1 -- 172.21.15.57:0/2088831327 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faa1c085450 con 0x7faa2806e570 2023-11-28T12:03:43.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.357+0000 7faa2e0cc700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faa0c0682f0 0x7faa0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:43.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.357+0000 7faa2e0cc700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faa0c0682f0 0x7faa0c06a7a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faa24009b40 tx=0x7faa24009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:43.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.361+0000 7faa1affd700 1 -- 172.21.15.57:0/2088831327 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faa1c052000 con 0x7faa2806e570 2023-11-28T12:03:43.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.642+0000 7faa30330700 1 -- 172.21.15.57:0/2088831327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7faa281428d0 con 0x7faa2806e570 2023-11-28T12:03:43.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.642+0000 7faa1affd700 1 -- 172.21.15.57:0/2088831327 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5298 (secure 0 0 0) 0x7faa281428d0 con 0x7faa2806e570 2023-11-28T12:03:43.643 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:43.643 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:03:43.643 INFO:teuthology.orchestra.run.smithi057.stdout: "auth_dump": [ 2023-11-28T12:03:43.643 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.643 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.a", 2023-11-28T12:03:43.643 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVlFqRBJBAAxYPlgmjdxnRzi0h9JBB1BQ==", 2023-11-28T12:03:43.643 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.643 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:03:43.643 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:03:43.643 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:03:43.644 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.644 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.644 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.644 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.644 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.b", 2023-11-28T12:03:43.644 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVl0d0ZMhAAykXowuVhIqRSjBr/XMvanw==", 2023-11-28T12:03:43.644 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.644 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:03:43.644 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:03:43.644 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:03:43.644 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.645 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.645 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.645 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.645 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.c", 2023-11-28T12:03:43.645 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVl4Z2qLBAACYGN7Glu6Bv2yLio50flgQ==", 2023-11-28T12:03:43.645 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.645 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:03:43.645 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:03:43.645 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:03:43.645 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.646 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.646 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.646 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.646 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.d", 2023-11-28T12:03:43.646 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVl2P5jORAAR7iZXovJiiCXuxlaaZJ75Q==", 2023-11-28T12:03:43.646 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.646 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:03:43.646 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:03:43.646 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:03:43.647 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.647 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.647 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.647 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.647 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.0", 2023-11-28T12:03:43.647 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQB31mVlUtyoIRAAVGG65BAriZ4e4QHUYCJXdg==", 2023-11-28T12:03:43.648 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.648 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:03:43.648 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:03:43.648 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.648 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.648 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.648 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.648 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.1", 2023-11-28T12:03:43.648 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQB61mVlNA98IRAArcXcx4glfEvQ1gxAXQYxHA==", 2023-11-28T12:03:43.648 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.648 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:03:43.649 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:03:43.649 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.649 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.649 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.649 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.649 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.2", 2023-11-28T12:03:43.649 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQB91mVlzthPIhAAFgsWUTI/2kmVCVMgcZrIdA==", 2023-11-28T12:03:43.649 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.649 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:03:43.649 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:03:43.650 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.650 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.650 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.650 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.650 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.3", 2023-11-28T12:03:43.650 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCA1mVlZPuzJBAAyAAUBUEitzTq7fPdxj5gzQ==", 2023-11-28T12:03:43.650 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.650 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:03:43.650 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:03:43.650 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.650 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.651 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.651 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.651 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.4", 2023-11-28T12:03:43.651 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCG1mVlWibrKhAAzuTgZ4ayPw3imRALW/Y4FQ==", 2023-11-28T12:03:43.651 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.651 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:03:43.651 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:03:43.651 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.651 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.651 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.651 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.652 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.5", 2023-11-28T12:03:43.652 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCJ1mVlnX7bKBAAYYUYhDSDTVKkrzVKw78+Ug==", 2023-11-28T12:03:43.652 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.652 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:03:43.652 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:03:43.652 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.652 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.652 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.652 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.652 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.6", 2023-11-28T12:03:43.652 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCM1mVlE7aUJxAAXM3axU5A9IEXt2/pNcvJ9Q==", 2023-11-28T12:03:43.652 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.653 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:03:43.653 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:03:43.653 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.653 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.653 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.653 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.653 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.7", 2023-11-28T12:03:43.653 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCP1mVlSApqJhAA93hHHLl+o8cKTgdCaG+0Bg==", 2023-11-28T12:03:43.653 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.653 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:03:43.653 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:03:43.654 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.654 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.654 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.654 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.654 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.0", 2023-11-28T12:03:43.654 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBx1mVlmhvFAhAAnKh5ldWFcp+P85m8HR2vug==", 2023-11-28T12:03:43.654 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.654 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:03:43.654 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow r", 2023-11-28T12:03:43.654 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T12:03:43.654 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.655 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.655 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.655 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.1", 2023-11-28T12:03:43.655 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBx1mVlMEgaChAAGCjKh3RcPIVp2UWKGqmqRA==", 2023-11-28T12:03:43.655 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.655 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:03:43.655 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-11-28T12:03:43.655 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-11-28T12:03:43.655 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-11-28T12:03:43.655 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.655 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.655 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.656 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.2", 2023-11-28T12:03:43.656 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBx1mVlqGCwBhAA6ZJBICxFbxIjYlgA80ekuA==", 2023-11-28T12:03:43.656 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.656 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:03:43.656 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-11-28T12:03:43.656 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-11-28T12:03:43.656 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-11-28T12:03:43.656 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.656 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.656 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.656 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.3", 2023-11-28T12:03:43.656 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBx1mVldQXGDxAAkY9/3er6vMwer5oWETJvfg==", 2023-11-28T12:03:43.657 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.657 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:03:43.657 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-11-28T12:03:43.657 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-11-28T12:03:43.657 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-11-28T12:03:43.657 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.657 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.658 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.658 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.admin", 2023-11-28T12:03:43.658 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVlzqiCAxAAIOe5RWQ+gxWKaK64A5jhlA==", 2023-11-28T12:03:43.658 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.658 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-11-28T12:03:43.659 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:03:43.659 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:03:43.659 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.659 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.659 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.660 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.660 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mgr.x", 2023-11-28T12:03:43.660 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVl+yMTIBAAUgQtxbubBwh+1DLDFEaSKQ==", 2023-11-28T12:03:43.660 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.660 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-11-28T12:03:43.660 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile mgr", 2023-11-28T12:03:43.660 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.660 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.660 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:43.660 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:43.660 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mgr.y", 2023-11-28T12:03:43.660 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVlEPXqGxAAxVwmP5CKh4D1Hv+kYm7fxg==", 2023-11-28T12:03:43.661 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:03:43.661 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-11-28T12:03:43.661 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile mgr", 2023-11-28T12:03:43.661 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:03:43.661 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.661 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:43.661 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-11-28T12:03:43.661 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:03:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.644+0000 7faa30330700 1 -- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faa0c0682f0 msgr2=0x7faa0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.644+0000 7faa30330700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faa0c0682f0 0x7faa0c06a7a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faa24009b40 tx=0x7faa24009210 comp rx=0 tx=0).stop 2023-11-28T12:03:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.644+0000 7faa30330700 1 -- 172.21.15.57:0/2088831327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2806e570 msgr2=0x7faa280b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.644+0000 7faa30330700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2806e570 0x7faa280b0460 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c550 tx=0x7faa1c00c860 comp rx=0 tx=0).stop 2023-11-28T12:03:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.644+0000 7faa30330700 1 -- 172.21.15.57:0/2088831327 shutdown_connections 2023-11-28T12:03:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.644+0000 7faa30330700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faa0c0682f0 0x7faa0c06a7a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.644+0000 7faa30330700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa2815cc80 0x7faa281f1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.644+0000 7faa30330700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2806a270 0x7faa280b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.644+0000 7faa30330700 1 --2- 172.21.15.57:0/2088831327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2806e570 0x7faa280b0460 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.644+0000 7faa30330700 1 -- 172.21.15.57:0/2088831327 >> 172.21.15.57:0/2088831327 conn(0x7faa2806c010 msgr2=0x7faa280d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:43.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.644+0000 7faa30330700 1 -- 172.21.15.57:0/2088831327 shutdown_connections 2023-11-28T12:03:43.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.644+0000 7faa30330700 1 -- 172.21.15.57:0/2088831327 wait complete. 2023-11-28T12:03:43.664 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:03:43.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.799+0000 7f9b9c31d700 1 Processor -- start 2023-11-28T12:03:43.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.799+0000 7f9b9c31d700 1 -- start start 2023-11-28T12:03:43.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.800+0000 7f9b9c31d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b9415e010 0x7f9b940ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:43.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.800+0000 7f9b9c31d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9415af30 con 0x7f9b9406e400 2023-11-28T12:03:43.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.800+0000 7f9b9c31d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9415ef40 con 0x7f9b940d15f0 2023-11-28T12:03:43.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.800+0000 7f9b9c31d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9415a9d0 con 0x7f9b9415e010 2023-11-28T12:03:43.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.801+0000 7f9b9a0b9700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f9b9406e400 0x7f9b9406e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40842/0 (socket says 172.21.15.57:40842) 2023-11-28T12:03:43.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.801+0000 7f9b9a0b9700 1 -- 172.21.15.57:0/521808013 learned_addr learned my addr 172.21.15.57:0/521808013 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:43.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.801+0000 7f9b9a8ba700 1 --2- 172.21.15.57:0/521808013 >> v2:172.21.15.181:3300/0 conn(0x7f9b9415e010 0x7f9b940ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:43.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.801+0000 7f9b990b7700 1 -- 172.21.15.57:0/521808013 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446407130 0 0) 0x7f9b9415af30 con 0x7f9b9406e400 2023-11-28T12:03:43.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.801+0000 7f9b990b7700 1 -- 172.21.15.57:0/521808013 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b800035b0 con 0x7f9b9406e400 2023-11-28T12:03:43.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.801+0000 7f9b990b7700 1 -- 172.21.15.57:0/521808013 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648617961 0 0) 0x7f9b9415ef40 con 0x7f9b940d15f0 2023-11-28T12:03:43.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.801+0000 7f9b990b7700 1 -- 172.21.15.57:0/521808013 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b9415af30 con 0x7f9b940d15f0 2023-11-28T12:03:43.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.801+0000 7f9b9a8ba700 1 -- 172.21.15.57:0/521808013 >> v1:172.21.15.181:6789/0 conn(0x7f9b940d15f0 legacy=0x7f9b9415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:43.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.801+0000 7f9b9a8ba700 1 -- 172.21.15.57:0/521808013 >> v1:172.21.15.57:6789/0 conn(0x7f9b9406e400 legacy=0x7f9b9406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:43.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.801+0000 7f9b990b7700 1 -- 172.21.15.57:0/521808013 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1050565531 0 0) 0x7f9b800035b0 con 0x7f9b9406e400 2023-11-28T12:03:43.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.801+0000 7f9b9a8ba700 1 -- 172.21.15.57:0/521808013 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9415e980 con 0x7f9b9415e010 2023-11-28T12:03:43.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.801+0000 7f9b9a8ba700 1 --2- 172.21.15.57:0/521808013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9415e010 0x7f9b940ce970 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009d90 tx=0x7f9b8c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cf36d7def66865d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:43.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.802+0000 7f9b990b7700 1 -- 172.21.15.57:0/521808013 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00e070 con 0x7f9b9415e010 2023-11-28T12:03:43.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.802+0000 7f9b990b7700 1 -- 172.21.15.57:0/521808013 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c003a20 con 0x7f9b9415e010 2023-11-28T12:03:43.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.802+0000 7f9b990b7700 1 -- 172.21.15.57:0/521808013 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90001670 con 0x7f9b9415e010 2023-11-28T12:03:43.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.802+0000 7f9b9c31d700 1 -- 172.21.15.57:0/521808013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9415e010 msgr2=0x7f9b940ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:43.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.802+0000 7f9b9c31d700 1 --2- 172.21.15.57:0/521808013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9415e010 0x7f9b940ce970 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009d90 tx=0x7f9b8c00b330 comp rx=0 tx=0).stop 2023-11-28T12:03:43.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.802+0000 7f9b9c31d700 1 -- 172.21.15.57:0/521808013 shutdown_connections 2023-11-28T12:03:43.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.802+0000 7f9b9c31d700 1 --2- 172.21.15.57:0/521808013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9415e010 0x7f9b940ce970 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.802+0000 7f9b9c31d700 1 -- 172.21.15.57:0/521808013 >> 172.21.15.57:0/521808013 conn(0x7f9b9406a270 msgr2=0x7f9b9406a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:43.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.802+0000 7f9b9c31d700 1 -- 172.21.15.57:0/521808013 shutdown_connections 2023-11-28T12:03:43.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.802+0000 7f9b9c31d700 1 -- 172.21.15.57:0/521808013 wait complete. 2023-11-28T12:03:43.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.804+0000 7f9b9c31d700 1 Processor -- start 2023-11-28T12:03:43.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.804+0000 7f9b9c31d700 1 -- start start 2023-11-28T12:03:43.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.804+0000 7f9b9c31d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b9406e400 0x7f9b94151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:43.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.804+0000 7f9b9c31d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b940d15f0 0x7f9b941524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:43.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.804+0000 7f9b9c31d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9415e010 0x7f9b941e5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:43.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.804+0000 7f9b9c31d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b94152cc0 con 0x7f9b9406e400 2023-11-28T12:03:43.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.804+0000 7f9b9c31d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b941e4790 con 0x7f9b9415e010 2023-11-28T12:03:43.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.804+0000 7f9b9c31d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b941529f0 con 0x7f9b940d15f0 2023-11-28T12:03:43.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b9a0b9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b9406e400 0x7f9b94151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:43.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b9a0b9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b9406e400 0x7f9b94151d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46162/0 (socket says 172.21.15.57:46162) 2023-11-28T12:03:43.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b9a0b9700 1 -- 172.21.15.57:0/67022469 learned_addr learned my addr 172.21.15.57:0/67022469 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:43.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b9a8ba700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9415e010 0x7f9b941e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:43.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b998b8700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b940d15f0 0x7f9b941524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:43.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b9a0b9700 1 -- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b940d15f0 msgr2=0x7f9b941524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:43.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b9a0b9700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b940d15f0 0x7f9b941524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b9a0b9700 1 -- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9415e010 msgr2=0x7f9b941e5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:43.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b9a0b9700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9415e010 0x7f9b941e5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:43.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b9a0b9700 1 -- 172.21.15.57:0/67022469 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c009a70 con 0x7f9b9406e400 2023-11-28T12:03:43.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b9a8ba700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9415e010 0x7f9b941e5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:43.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b9a0b9700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b9406e400 0x7f9b94151d00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000c750 tx=0x7f9b9000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:43.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b8affd700 1 -- 172.21.15.57:0/67022469 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90007560 con 0x7f9b9406e400 2023-11-28T12:03:43.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b8affd700 1 -- 172.21.15.57:0/67022469 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b900076c0 con 0x7f9b9406e400 2023-11-28T12:03:43.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b9c31d700 1 -- 172.21.15.57:0/67022469 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9415abc0 con 0x7f9b9406e400 2023-11-28T12:03:43.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b8affd700 1 -- 172.21.15.57:0/67022469 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90004ab0 con 0x7f9b9406e400 2023-11-28T12:03:43.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.805+0000 7f9b9c31d700 1 -- 172.21.15.57:0/67022469 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9415ec60 con 0x7f9b9406e400 2023-11-28T12:03:43.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.806+0000 7f9b9c31d700 1 -- 172.21.15.57:0/67022469 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9415e490 con 0x7f9b9406e400 2023-11-28T12:03:43.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.807+0000 7f9b8affd700 1 -- 172.21.15.57:0/67022469 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9b90010070 con 0x7f9b9406e400 2023-11-28T12:03:43.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.807+0000 7f9b8affd700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9b780682f0 0x7f9b7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:43.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.807+0000 7f9b8affd700 1 -- 172.21.15.57:0/67022469 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9b90084b20 con 0x7f9b9406e400 2023-11-28T12:03:43.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.808+0000 7f9b998b8700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9b780682f0 0x7f9b7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:43.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.808+0000 7f9b998b8700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9b780682f0 0x7f9b7806a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9b940b3400 tx=0x7f9b84009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:43.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:43.812+0000 7f9b8affd700 1 -- 172.21.15.57:0/67022469 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9b900515f0 con 0x7f9b9406e400 2023-11-28T12:03:44.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.015+0000 7f9b9c31d700 1 -- 172.21.15.57:0/67022469 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b940743f0 con 0x7f9b9406e400 2023-11-28T12:03:44.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.016+0000 7f9b8affd700 1 -- 172.21.15.57:0/67022469 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+14209 (secure 0 0 0) 0x7f9b90018020 con 0x7f9b9406e400 2023-11-28T12:03:44.016 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:44.016 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":45,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:03:43.186713+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:03:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.018+0000 7f9b9c31d700 1 -- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9b780682f0 msgr2=0x7f9b7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.018+0000 7f9b9c31d700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9b780682f0 0x7f9b7806a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9b940b3400 tx=0x7f9b84009040 comp rx=0 tx=0).stop 2023-11-28T12:03:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.018+0000 7f9b9c31d700 1 -- 172.21.15.57:0/67022469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b9406e400 msgr2=0x7f9b94151d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.018+0000 7f9b9c31d700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b9406e400 0x7f9b94151d00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000c750 tx=0x7f9b9000cb10 comp rx=0 tx=0).stop 2023-11-28T12:03:44.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.018+0000 7f9b9c31d700 1 -- 172.21.15.57:0/67022469 shutdown_connections 2023-11-28T12:03:44.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.018+0000 7f9b9c31d700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9b780682f0 0x7f9b7806a7a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.018+0000 7f9b9c31d700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9415e010 0x7f9b941e5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.018+0000 7f9b9c31d700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b9406e400 0x7f9b94151d00 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.018+0000 7f9b9c31d700 1 --2- 172.21.15.57:0/67022469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b940d15f0 0x7f9b941524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.018+0000 7f9b9c31d700 1 -- 172.21.15.57:0/67022469 >> 172.21.15.57:0/67022469 conn(0x7f9b9406a270 msgr2=0x7f9b94152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:44.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.018+0000 7f9b9c31d700 1 -- 172.21.15.57:0/67022469 shutdown_connections 2023-11-28T12:03:44.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.018+0000 7f9b9c31d700 1 -- 172.21.15.57:0/67022469 wait complete. 2023-11-28T12:03:44.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:03:44.032 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:03:44.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.171+0000 7fb2a6294700 1 Processor -- start 2023-11-28T12:03:44.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.171+0000 7fb2a6294700 1 -- start start 2023-11-28T12:03:44.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.172+0000 7fb2a6294700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb2a015a4d0 0x7fb2a006a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.172+0000 7fb2a6294700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2a00cdf20 0x7fb2a00cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.172+0000 7fb2a6294700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a015e0d0 con 0x7fb2a006a7b0 2023-11-28T12:03:44.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.172+0000 7fb2a6294700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a00ce360 con 0x7fb2a015a4d0 2023-11-28T12:03:44.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.172+0000 7fb2a6294700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a006ac30 con 0x7fb2a00cdf20 2023-11-28T12:03:44.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.172+0000 7fb2a4831700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2a00cdf20 0x7fb2a00cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.172+0000 7fb29ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb2a015a4d0 0x7fb2a006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.172+0000 7fb2a4831700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2a00cdf20 0x7fb2a00cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:50410/0 (socket says 172.21.15.57:50410) 2023-11-28T12:03:44.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.172+0000 7fb2a4831700 1 -- 172.21.15.57:0/2172867275 learned_addr learned my addr 172.21.15.57:0/2172867275 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:44.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.173+0000 7fb2a4831700 1 -- 172.21.15.57:0/2172867275 >> v2:172.21.15.181:3300/0 conn(0x7fb2a015a4d0 msgr2=0x7fb2a006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:44.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.173+0000 7fb2a4831700 1 --2- 172.21.15.57:0/2172867275 >> v2:172.21.15.181:3300/0 conn(0x7fb2a015a4d0 0x7fb2a006a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.173+0000 7fb2a4831700 1 -- 172.21.15.57:0/2172867275 >> v1:172.21.15.181:6789/0 conn(0x7fb2a006a7b0 legacy=0x7fb2a00cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.173+0000 7fb2a4831700 1 -- 172.21.15.57:0/2172867275 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00ce560 con 0x7fb2a00cdf20 2023-11-28T12:03:44.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.173+0000 7fb29ffff700 1 --2- 172.21.15.57:0/2172867275 >> v2:172.21.15.181:3300/0 conn(0x7fb2a015a4d0 0x7fb2a006a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:03:44.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.173+0000 7fb29effd700 1 -- 172.21.15.57:0/2172867275 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963597356 0 0) 0x7fb2a015e0d0 con 0x7fb2a006a7b0 2023-11-28T12:03:44.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.173+0000 7fb2a4831700 1 --2- 172.21.15.57:0/2172867275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2a00cdf20 0x7fb2a00cb010 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb298009e50 tx=0x7fb29800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b2f03c02a9e444 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:44.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.173+0000 7fb29effd700 1 -- 172.21.15.57:0/2172867275 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29800e070 con 0x7fb2a00cdf20 2023-11-28T12:03:44.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.173+0000 7fb29effd700 1 -- 172.21.15.57:0/2172867275 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29800b910 con 0x7fb2a00cdf20 2023-11-28T12:03:44.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.173+0000 7fb29effd700 1 -- 172.21.15.57:0/2172867275 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29800bcc0 con 0x7fb2a00cdf20 2023-11-28T12:03:44.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.174+0000 7fb2a6294700 1 -- 172.21.15.57:0/2172867275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2a00cdf20 msgr2=0x7fb2a00cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.174+0000 7fb2a6294700 1 --2- 172.21.15.57:0/2172867275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2a00cdf20 0x7fb2a00cb010 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb298009e50 tx=0x7fb29800b330 comp rx=0 tx=0).stop 2023-11-28T12:03:44.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.174+0000 7fb2a6294700 1 -- 172.21.15.57:0/2172867275 shutdown_connections 2023-11-28T12:03:44.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.174+0000 7fb2a6294700 1 --2- 172.21.15.57:0/2172867275 >> v2:172.21.15.181:3300/0 conn(0x7fb2a015a4d0 0x7fb2a006a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.174+0000 7fb2a6294700 1 --2- 172.21.15.57:0/2172867275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2a00cdf20 0x7fb2a00cb010 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.174+0000 7fb2a6294700 1 -- 172.21.15.57:0/2172867275 >> 172.21.15.57:0/2172867275 conn(0x7fb2a006c010 msgr2=0x7fb2a00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:44.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.174+0000 7fb2a6294700 1 -- 172.21.15.57:0/2172867275 shutdown_connections 2023-11-28T12:03:44.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.174+0000 7fb2a6294700 1 -- 172.21.15.57:0/2172867275 wait complete. 2023-11-28T12:03:44.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.175+0000 7fb2a6294700 1 Processor -- start 2023-11-28T12:03:44.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.175+0000 7fb2a6294700 1 -- start start 2023-11-28T12:03:44.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.176+0000 7fb2a6294700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a006a7b0 0x7fb2a0151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.176+0000 7fb2a6294700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2a00cdf20 0x7fb2a00b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.176+0000 7fb2a6294700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2a015a4d0 0x7fb2a00b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.176+0000 7fb2a6294700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00b0400 con 0x7fb2a006a7b0 2023-11-28T12:03:44.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.176+0000 7fb2a6294700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00b2590 con 0x7fb2a015a4d0 2023-11-28T12:03:44.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.176+0000 7fb2a6294700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00c7810 con 0x7fb2a00cdf20 2023-11-28T12:03:44.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.176+0000 7fb29ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a006a7b0 0x7fb2a0151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.176+0000 7fb29ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a006a7b0 0x7fb2a0151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46176/0 (socket says 172.21.15.57:46176) 2023-11-28T12:03:44.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.176+0000 7fb29ffff700 1 -- 172.21.15.57:0/1937477406 learned_addr learned my addr 172.21.15.57:0/1937477406 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:44.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.176+0000 7fb29f7fe700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2a00cdf20 0x7fb2a00b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.176+0000 7fb2a4831700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2a015a4d0 0x7fb2a00b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.177+0000 7fb29ffff700 1 -- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2a00cdf20 msgr2=0x7fb2a00b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.177+0000 7fb29ffff700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2a00cdf20 0x7fb2a00b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.177+0000 7fb29ffff700 1 -- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2a015a4d0 msgr2=0x7fb2a00b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.177+0000 7fb29ffff700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2a015a4d0 0x7fb2a00b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.177+0000 7fb29ffff700 1 -- 172.21.15.57:0/1937477406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2900094a0 con 0x7fb2a006a7b0 2023-11-28T12:03:44.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.177+0000 7fb2a4831700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2a015a4d0 0x7fb2a00b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:44.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.177+0000 7fb29f7fe700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2a00cdf20 0x7fb2a00b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:44.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.177+0000 7fb29ffff700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a006a7b0 0x7fb2a0151760 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb29000ec10 tx=0x7fb290009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:44.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.177+0000 7fb29cff9700 1 -- 172.21.15.57:0/1937477406 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29000c8e0 con 0x7fb2a006a7b0 2023-11-28T12:03:44.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.177+0000 7fb29cff9700 1 -- 172.21.15.57:0/1937477406 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29000ca40 con 0x7fb2a006a7b0 2023-11-28T12:03:44.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.177+0000 7fb2a6294700 1 -- 172.21.15.57:0/1937477406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb298009b30 con 0x7fb2a006a7b0 2023-11-28T12:03:44.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.177+0000 7fb29cff9700 1 -- 172.21.15.57:0/1937477406 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29000cdd0 con 0x7fb2a006a7b0 2023-11-28T12:03:44.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.177+0000 7fb2a6294700 1 -- 172.21.15.57:0/1937477406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a015e600 con 0x7fb2a006a7b0 2023-11-28T12:03:44.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.178+0000 7fb2a6294700 1 -- 172.21.15.57:0/1937477406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a015e320 con 0x7fb2a006a7b0 2023-11-28T12:03:44.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.178+0000 7fb29cff9700 1 -- 172.21.15.57:0/1937477406 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb290019030 con 0x7fb2a006a7b0 2023-11-28T12:03:44.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.179+0000 7fb29cff9700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb28c0682f0 0x7fb28c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.179+0000 7fb29cff9700 1 -- 172.21.15.57:0/1937477406 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb290085960 con 0x7fb2a006a7b0 2023-11-28T12:03:44.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.179+0000 7fb29f7fe700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb28c0682f0 0x7fb28c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.180+0000 7fb29f7fe700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb28c0682f0 0x7fb28c06a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00c7200 tx=0x7fb294009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:44.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.184+0000 7fb29cff9700 1 -- 172.21.15.57:0/1937477406 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb290052510 con 0x7fb2a006a7b0 2023-11-28T12:03:44.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.385+0000 7fb2a6294700 1 -- 172.21.15.57:0/1937477406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb2a015e8e0 con 0x7fb2a006a7b0 2023-11-28T12:03:44.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.386+0000 7fb29cff9700 1 -- 172.21.15.57:0/1937477406 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb290055b30 con 0x7fb2a006a7b0 2023-11-28T12:03:44.386 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:03:44.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.388+0000 7fb2a6294700 1 -- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb28c0682f0 msgr2=0x7fb28c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.388+0000 7fb2a6294700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb28c0682f0 0x7fb28c06a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00c7200 tx=0x7fb294009040 comp rx=0 tx=0).stop 2023-11-28T12:03:44.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.388+0000 7fb2a6294700 1 -- 172.21.15.57:0/1937477406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a006a7b0 msgr2=0x7fb2a0151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.388+0000 7fb2a6294700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a006a7b0 0x7fb2a0151760 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb29000ec10 tx=0x7fb290009e20 comp rx=0 tx=0).stop 2023-11-28T12:03:44.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.388+0000 7fb2a6294700 1 -- 172.21.15.57:0/1937477406 shutdown_connections 2023-11-28T12:03:44.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.388+0000 7fb2a6294700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb28c0682f0 0x7fb28c06a7a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.388+0000 7fb2a6294700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2a015a4d0 0x7fb2a00b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.388+0000 7fb2a6294700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a006a7b0 0x7fb2a0151760 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.388+0000 7fb2a6294700 1 --2- 172.21.15.57:0/1937477406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2a00cdf20 0x7fb2a00b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.388+0000 7fb2a6294700 1 -- 172.21.15.57:0/1937477406 >> 172.21.15.57:0/1937477406 conn(0x7fb2a006c010 msgr2=0x7fb2a00b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:44.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.388+0000 7fb2a6294700 1 -- 172.21.15.57:0/1937477406 shutdown_connections 2023-11-28T12:03:44.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.388+0000 7fb2a6294700 1 -- 172.21.15.57:0/1937477406 wait complete. 2023-11-28T12:03:44.403 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:03:44.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.542+0000 7f38a1144700 1 Processor -- start 2023-11-28T12:03:44.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.542+0000 7f38a1144700 1 -- start start 2023-11-28T12:03:44.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.543+0000 7f38a1144700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f389c0cec00 0x7f389c065fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.543+0000 7f38a1144700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f389c160300 0x7f389c15d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.543+0000 7f38a1144700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389c15ceb0 con 0x7f389c15d880 2023-11-28T12:03:44.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.543+0000 7f38a1144700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f389c1608c0 con 0x7f389c160300 2023-11-28T12:03:44.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.543+0000 7f38a1144700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c160ba0 con 0x7f389c0cec00 2023-11-28T12:03:44.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.543+0000 7f389a59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f389c160300 0x7f389c15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.543+0000 7f389b59e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f389c15d880 0x7f389c0cac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:33774/0 (socket says 172.21.15.57:33774) 2023-11-28T12:03:44.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.543+0000 7f389b59e700 1 -- 172.21.15.57:0/297547641 learned_addr learned my addr 172.21.15.57:0/297547641 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:44.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.543+0000 7f389ad9d700 1 --2- 172.21.15.57:0/297547641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f389c0cec00 0x7f389c065fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.544+0000 7f389ad9d700 1 -- 172.21.15.57:0/297547641 >> v2:172.21.15.181:3300/0 conn(0x7f389c160300 msgr2=0x7f389c15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:44.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.544+0000 7f389ad9d700 1 --2- 172.21.15.57:0/297547641 >> v2:172.21.15.181:3300/0 conn(0x7f389c160300 0x7f389c15d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.544+0000 7f389ad9d700 1 -- 172.21.15.57:0/297547641 >> v1:172.21.15.181:6789/0 conn(0x7f389c15d880 legacy=0x7f389c0cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.544+0000 7f3899d9b700 1 -- 172.21.15.57:0/297547641 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280555035 0 0) 0x7f389c15ceb0 con 0x7f389c15d880 2023-11-28T12:03:44.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.544+0000 7f389ad9d700 1 -- 172.21.15.57:0/297547641 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c160e80 con 0x7f389c0cec00 2023-11-28T12:03:44.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.544+0000 7f389a59c700 1 --2- 172.21.15.57:0/297547641 >> v2:172.21.15.181:3300/0 conn(0x7f389c160300 0x7f389c15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:03:44.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.544+0000 7f389ad9d700 1 --2- 172.21.15.57:0/297547641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f389c0cec00 0x7f389c065fd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3894009840 tx=0x7f3894009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d420d1539c9da6b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:44.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.544+0000 7f3899d9b700 1 -- 172.21.15.57:0/297547641 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894011830 con 0x7f389c0cec00 2023-11-28T12:03:44.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.544+0000 7f3899d9b700 1 -- 172.21.15.57:0/297547641 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3894011990 con 0x7f389c0cec00 2023-11-28T12:03:44.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.544+0000 7f3899d9b700 1 -- 172.21.15.57:0/297547641 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894011d20 con 0x7f389c0cec00 2023-11-28T12:03:44.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.545+0000 7f38a1144700 1 -- 172.21.15.57:0/297547641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f389c0cec00 msgr2=0x7f389c065fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.545+0000 7f38a1144700 1 --2- 172.21.15.57:0/297547641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f389c0cec00 0x7f389c065fd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3894009840 tx=0x7f3894009b50 comp rx=0 tx=0).stop 2023-11-28T12:03:44.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.545+0000 7f38a1144700 1 -- 172.21.15.57:0/297547641 shutdown_connections 2023-11-28T12:03:44.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.545+0000 7f38a1144700 1 --2- 172.21.15.57:0/297547641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f389c0cec00 0x7f389c065fd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.545+0000 7f38a1144700 1 --2- 172.21.15.57:0/297547641 >> v2:172.21.15.181:3300/0 conn(0x7f389c160300 0x7f389c15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.545+0000 7f38a1144700 1 -- 172.21.15.57:0/297547641 >> 172.21.15.57:0/297547641 conn(0x7f389c0cf400 msgr2=0x7f389c0cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:44.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.545+0000 7f38a1144700 1 -- 172.21.15.57:0/297547641 shutdown_connections 2023-11-28T12:03:44.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.545+0000 7f38a1144700 1 -- 172.21.15.57:0/297547641 wait complete. 2023-11-28T12:03:44.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.547+0000 7f38a1144700 1 Processor -- start 2023-11-28T12:03:44.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.547+0000 7f38a1144700 1 -- start start 2023-11-28T12:03:44.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f38a1144700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f389c0cec00 0x7f389c154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f38a1144700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f389c15d880 0x7f389c0722a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f38a1144700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f389c160300 0x7f389c154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f38a1144700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c154590 con 0x7f389c0cec00 2023-11-28T12:03:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f38a1144700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0b2250 con 0x7f389c160300 2023-11-28T12:03:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f38a1144700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0b24f0 con 0x7f389c15d880 2023-11-28T12:03:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f389ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f389c0cec00 0x7f389c154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f389ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f389c0cec00 0x7f389c154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44260/0 (socket says 172.21.15.57:44260) 2023-11-28T12:03:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f389ad9d700 1 -- 172.21.15.57:0/2071872235 learned_addr learned my addr 172.21.15.57:0/2071872235 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f389a59c700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f389c15d880 0x7f389c0722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f389b59e700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f389c160300 0x7f389c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f389ad9d700 1 -- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f389c15d880 msgr2=0x7f389c0722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f389ad9d700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f389c15d880 0x7f389c0722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f389ad9d700 1 -- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f389c160300 msgr2=0x7f389c154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f389ad9d700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f389c160300 0x7f389c154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.548+0000 7f389ad9d700 1 -- 172.21.15.57:0/2071872235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38840094a0 con 0x7f389c0cec00 2023-11-28T12:03:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.549+0000 7f389ad9d700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f389c0cec00 0x7f389c154750 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3894009b90 tx=0x7f3894004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.549+0000 7f38937fe700 1 -- 172.21.15.57:0/2071872235 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38940038a0 con 0x7f389c0cec00 2023-11-28T12:03:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.549+0000 7f38937fe700 1 -- 172.21.15.57:0/2071872235 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389401e070 con 0x7f389c0cec00 2023-11-28T12:03:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.549+0000 7f38a1144700 1 -- 172.21.15.57:0/2071872235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38940094a0 con 0x7f389c0cec00 2023-11-28T12:03:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.549+0000 7f38937fe700 1 -- 172.21.15.57:0/2071872235 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894003c40 con 0x7f389c0cec00 2023-11-28T12:03:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.549+0000 7f38a1144700 1 -- 172.21.15.57:0/2071872235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389c066510 con 0x7f389c0cec00 2023-11-28T12:03:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.550+0000 7f38a1144700 1 -- 172.21.15.57:0/2071872235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389c161160 con 0x7f389c0cec00 2023-11-28T12:03:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.550+0000 7f38937fe700 1 -- 172.21.15.57:0/2071872235 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f389400f050 con 0x7f389c0cec00 2023-11-28T12:03:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.551+0000 7f38937fe700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f387c0682f0 0x7f387c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.551+0000 7f38937fe700 1 -- 172.21.15.57:0/2071872235 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3894086a00 con 0x7f389c0cec00 2023-11-28T12:03:44.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.551+0000 7f389a59c700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f387c0682f0 0x7f387c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.552+0000 7f389a59c700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f387c0682f0 0x7f387c06a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f389c1f5bf0 tx=0x7f3884008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:44.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.555+0000 7f38937fe700 1 -- 172.21.15.57:0/2071872235 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f38940534c0 con 0x7f389c0cec00 2023-11-28T12:03:44.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.757+0000 7f38a1144700 1 -- 172.21.15.57:0/2071872235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f389c0cefd0 con 0x7f389c0cec00 2023-11-28T12:03:44.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.757+0000 7f38937fe700 1 -- 172.21.15.57:0/2071872235 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38940150a0 con 0x7f389c0cec00 2023-11-28T12:03:44.757 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:03:44.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.759+0000 7f38a1144700 1 -- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f387c0682f0 msgr2=0x7f387c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.759+0000 7f38a1144700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f387c0682f0 0x7f387c06a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f389c1f5bf0 tx=0x7f3884008040 comp rx=0 tx=0).stop 2023-11-28T12:03:44.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.759+0000 7f38a1144700 1 -- 172.21.15.57:0/2071872235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f389c0cec00 msgr2=0x7f389c154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.759+0000 7f38a1144700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f389c0cec00 0x7f389c154750 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3894009b90 tx=0x7f3894004750 comp rx=0 tx=0).stop 2023-11-28T12:03:44.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.760+0000 7f38a1144700 1 -- 172.21.15.57:0/2071872235 shutdown_connections 2023-11-28T12:03:44.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.760+0000 7f38a1144700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f387c0682f0 0x7f387c06a7a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.760+0000 7f38a1144700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f389c160300 0x7f389c154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.760+0000 7f38a1144700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f389c0cec00 0x7f389c154750 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.760+0000 7f38a1144700 1 --2- 172.21.15.57:0/2071872235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f389c15d880 0x7f389c0722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.760+0000 7f38a1144700 1 -- 172.21.15.57:0/2071872235 >> 172.21.15.57:0/2071872235 conn(0x7f389c0cf400 msgr2=0x7f389c0af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:44.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.760+0000 7f38a1144700 1 -- 172.21.15.57:0/2071872235 shutdown_connections 2023-11-28T12:03:44.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.760+0000 7f38a1144700 1 -- 172.21.15.57:0/2071872235 wait complete. 2023-11-28T12:03:44.773 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T12:03:44.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.909+0000 7fcc449bc700 1 Processor -- start 2023-11-28T12:03:44.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.909+0000 7fcc449bc700 1 -- start start 2023-11-28T12:03:44.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.910+0000 7fcc449bc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcc40071f70 0x7fcc400acb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.910+0000 7fcc449bc700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc400af0c0 con 0x7fcc4006a270 2023-11-28T12:03:44.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.910+0000 7fcc449bc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc400b0590 con 0x7fcc400ae520 2023-11-28T12:03:44.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.910+0000 7fcc449bc700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc400723f0 con 0x7fcc40071f70 2023-11-28T12:03:44.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.910+0000 7fcc3e59c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcc4006a270 0x7fcc4006a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54892/0 (socket says 172.21.15.57:54892) 2023-11-28T12:03:44.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.910+0000 7fcc3ed9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcc40071f70 0x7fcc400acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.910+0000 7fcc3e59c700 1 -- 172.21.15.57:0/1518507586 learned_addr learned my addr 172.21.15.57:0/1518507586 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:44.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.911+0000 7fcc3d59a700 1 -- 172.21.15.57:0/1518507586 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997091441 0 0) 0x7fcc400af0c0 con 0x7fcc4006a270 2023-11-28T12:03:44.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.911+0000 7fcc3d59a700 1 -- 172.21.15.57:0/1518507586 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc2c0034d0 con 0x7fcc4006a270 2023-11-28T12:03:44.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.911+0000 7fcc3ed9d700 1 -- 172.21.15.57:0/1518507586 >> v1:172.21.15.181:6789/0 conn(0x7fcc400ae520 legacy=0x7fcc400ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:44.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.911+0000 7fcc3ed9d700 1 -- 172.21.15.57:0/1518507586 >> v1:172.21.15.57:6789/0 conn(0x7fcc4006a270 legacy=0x7fcc4006a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.911+0000 7fcc3ed9d700 1 -- 172.21.15.57:0/1518507586 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc40159280 con 0x7fcc40071f70 2023-11-28T12:03:44.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.911+0000 7fcc3ed9d700 1 --2- 172.21.15.57:0/1518507586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc40071f70 0x7fcc400acb30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009d90 tx=0x7fcc3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de80951d1d7ca7c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:44.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.911+0000 7fcc3d59a700 1 -- 172.21.15.57:0/1518507586 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3800e070 con 0x7fcc40071f70 2023-11-28T12:03:44.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.911+0000 7fcc3d59a700 1 -- 172.21.15.57:0/1518507586 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3800bb20 con 0x7fcc40071f70 2023-11-28T12:03:44.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.911+0000 7fcc3d59a700 1 -- 172.21.15.57:0/1518507586 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38012430 con 0x7fcc40071f70 2023-11-28T12:03:44.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.912+0000 7fcc449bc700 1 -- 172.21.15.57:0/1518507586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc40071f70 msgr2=0x7fcc400acb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.912+0000 7fcc449bc700 1 --2- 172.21.15.57:0/1518507586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc40071f70 0x7fcc400acb30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009d90 tx=0x7fcc3800b330 comp rx=0 tx=0).stop 2023-11-28T12:03:44.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.912+0000 7fcc449bc700 1 -- 172.21.15.57:0/1518507586 shutdown_connections 2023-11-28T12:03:44.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.912+0000 7fcc449bc700 1 --2- 172.21.15.57:0/1518507586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc40071f70 0x7fcc400acb30 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.912+0000 7fcc449bc700 1 -- 172.21.15.57:0/1518507586 >> 172.21.15.57:0/1518507586 conn(0x7fcc400b2fc0 msgr2=0x7fcc40071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:44.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.912+0000 7fcc449bc700 1 -- 172.21.15.57:0/1518507586 shutdown_connections 2023-11-28T12:03:44.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.912+0000 7fcc449bc700 1 -- 172.21.15.57:0/1518507586 wait complete. 2023-11-28T12:03:44.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.913+0000 7fcc449bc700 1 Processor -- start 2023-11-28T12:03:44.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.913+0000 7fcc449bc700 1 -- start start 2023-11-28T12:03:44.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc449bc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc4006a270 0x7fcc401f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc449bc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc40071f70 0x7fcc400c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc449bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc400ae520 0x7fcc401f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc449bc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc400cc7d0 con 0x7fcc400ae520 2023-11-28T12:03:44.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc449bc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc401f5be0 con 0x7fcc40071f70 2023-11-28T12:03:44.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc449bc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc401f5ec0 con 0x7fcc4006a270 2023-11-28T12:03:44.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc3ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc400ae520 0x7fcc401f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc3ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc400ae520 0x7fcc401f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44280/0 (socket says 172.21.15.57:44280) 2023-11-28T12:03:44.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc3ed9d700 1 -- 172.21.15.57:0/100582665 learned_addr learned my addr 172.21.15.57:0/100582665 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:44.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc3e59c700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc4006a270 0x7fcc401f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc3dd9b700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc40071f70 0x7fcc400c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc3ed9d700 1 -- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc4006a270 msgr2=0x7fcc401f66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc3ed9d700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc4006a270 0x7fcc401f66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc3ed9d700 1 -- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc40071f70 msgr2=0x7fcc400c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:44.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc3ed9d700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc40071f70 0x7fcc400c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:44.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc3ed9d700 1 -- 172.21.15.57:0/100582665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc38009a70 con 0x7fcc400ae520 2023-11-28T12:03:44.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.914+0000 7fcc3dd9b700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc40071f70 0x7fcc400c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:44.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.915+0000 7fcc3ed9d700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc400ae520 0x7fcc401f1e00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcc40073cc0 tx=0x7fcc38004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:44.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.915+0000 7fcc36ffd700 1 -- 172.21.15.57:0/100582665 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc380098f0 con 0x7fcc400ae520 2023-11-28T12:03:44.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.915+0000 7fcc36ffd700 1 -- 172.21.15.57:0/100582665 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc38020070 con 0x7fcc400ae520 2023-11-28T12:03:44.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.915+0000 7fcc449bc700 1 -- 172.21.15.57:0/100582665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc40072670 con 0x7fcc400ae520 2023-11-28T12:03:44.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.915+0000 7fcc36ffd700 1 -- 172.21.15.57:0/100582665 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc380276a0 con 0x7fcc400ae520 2023-11-28T12:03:44.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.915+0000 7fcc449bc700 1 -- 172.21.15.57:0/100582665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc400b0d10 con 0x7fcc400ae520 2023-11-28T12:03:44.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.916+0000 7fcc449bc700 1 -- 172.21.15.57:0/100582665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc400afc60 con 0x7fcc400ae520 2023-11-28T12:03:44.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.916+0000 7fcc36ffd700 1 -- 172.21.15.57:0/100582665 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcc38012430 con 0x7fcc400ae520 2023-11-28T12:03:44.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.917+0000 7fcc36ffd700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcc200682f0 0x7fcc2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:44.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.917+0000 7fcc36ffd700 1 -- 172.21.15.57:0/100582665 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcc38022070 con 0x7fcc400ae520 2023-11-28T12:03:44.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.917+0000 7fcc3e59c700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcc200682f0 0x7fcc2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:44.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.917+0000 7fcc3e59c700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcc200682f0 0x7fcc2006a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcc300011e0 tx=0x7fcc30009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:44.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:44.921+0000 7fcc36ffd700 1 -- 172.21.15.57:0/100582665 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcc380545c0 con 0x7fcc400ae520 2023-11-28T12:03:45.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.123+0000 7fcc449bc700 1 -- 172.21.15.57:0/100582665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcc4006b700 con 0x7fcc400ae520 2023-11-28T12:03:45.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.123+0000 7fcc36ffd700 1 -- 172.21.15.57:0/100582665 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v45) v1 ==== 84+0+126 (secure 0 0 0) 0x7fcc38057be0 con 0x7fcc400ae520 2023-11-28T12:03:45.123 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:45.124 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2023-11-28T12:03:45.124 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:45.124 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 1, 2023-11-28T12:03:45.124 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": "rbd" 2023-11-28T12:03:45.124 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:45.124 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:03:45.124 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 2, 2023-11-28T12:03:45.124 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": ".mgr" 2023-11-28T12:03:45.124 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:45.124 INFO:teuthology.orchestra.run.smithi057.stdout:] 2023-11-28T12:03:45.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.125+0000 7fcc449bc700 1 -- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcc200682f0 msgr2=0x7fcc2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:45.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.125+0000 7fcc449bc700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcc200682f0 0x7fcc2006a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcc300011e0 tx=0x7fcc30009040 comp rx=0 tx=0).stop 2023-11-28T12:03:45.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.125+0000 7fcc449bc700 1 -- 172.21.15.57:0/100582665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc400ae520 msgr2=0x7fcc401f1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:45.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.125+0000 7fcc449bc700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc400ae520 0x7fcc401f1e00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcc40073cc0 tx=0x7fcc38004ad0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.126+0000 7fcc449bc700 1 -- 172.21.15.57:0/100582665 shutdown_connections 2023-11-28T12:03:45.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.126+0000 7fcc449bc700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcc200682f0 0x7fcc2006a7a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.126+0000 7fcc449bc700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc400ae520 0x7fcc401f1e00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.126+0000 7fcc449bc700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc4006a270 0x7fcc401f66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.126+0000 7fcc449bc700 1 --2- 172.21.15.57:0/100582665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc40071f70 0x7fcc400c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.126+0000 7fcc449bc700 1 -- 172.21.15.57:0/100582665 >> 172.21.15.57:0/100582665 conn(0x7fcc400b2fc0 msgr2=0x7fcc400c2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:45.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.126+0000 7fcc449bc700 1 -- 172.21.15.57:0/100582665 shutdown_connections 2023-11-28T12:03:45.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.126+0000 7fcc449bc700 1 -- 172.21.15.57:0/100582665 wait complete. 2023-11-28T12:03:45.139 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T12:03:45.140 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T12:03:45.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.279+0000 7f999c449700 1 Processor -- start 2023-11-28T12:03:45.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.279+0000 7f999c449700 1 -- start start 2023-11-28T12:03:45.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.280+0000 7f999c449700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f999415cce0 0x7f999406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:45.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.280+0000 7f999c449700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f999415dc40 0x7f9994070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:45.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.280+0000 7f999c449700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9994071150 0x7f99940c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:45.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.280+0000 7f999c449700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99940715d0 con 0x7f9994071150 2023-11-28T12:03:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.280+0000 7f999c449700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f999415d0b0 con 0x7f999415cce0 2023-11-28T12:03:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.280+0000 7f999c449700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99940cdc10 con 0x7f999415dc40 2023-11-28T12:03:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.280+0000 7f999a9e6700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9994071150 0x7f99940c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.280+0000 7f999a9e6700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9994071150 0x7f99940c4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44292/0 (socket says 172.21.15.57:44292) 2023-11-28T12:03:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.280+0000 7f999a9e6700 1 -- 172.21.15.57:0/1642131658 learned_addr learned my addr 172.21.15.57:0/1642131658 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.280+0000 7f999a1e5700 1 --2- 172.21.15.57:0/1642131658 >> v2:172.21.15.181:3300/0 conn(0x7f999415cce0 0x7f999406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.280+0000 7f99999e4700 1 --2- 172.21.15.57:0/1642131658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f999415dc40 0x7f9994070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.281+0000 7f999a9e6700 1 -- 172.21.15.57:0/1642131658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f999415dc40 msgr2=0x7f9994070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.281+0000 7f999a9e6700 1 --2- 172.21.15.57:0/1642131658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f999415dc40 0x7f9994070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.281+0000 7f999a9e6700 1 -- 172.21.15.57:0/1642131658 >> v2:172.21.15.181:3300/0 conn(0x7f999415cce0 msgr2=0x7f999406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.281+0000 7f999a9e6700 1 --2- 172.21.15.57:0/1642131658 >> v2:172.21.15.181:3300/0 conn(0x7f999415cce0 0x7f999406b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.281+0000 7f999a9e6700 1 -- 172.21.15.57:0/1642131658 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99940cfa10 con 0x7f9994071150 2023-11-28T12:03:45.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.281+0000 7f999a9e6700 1 --2- 172.21.15.57:0/1642131658 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9994071150 0x7f99940c4b10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9994071f20 tx=0x7f998c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddfab03572c02df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:45.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.281+0000 7f99991e3700 1 -- 172.21.15.57:0/1642131658 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c00e070 con 0x7f9994071150 2023-11-28T12:03:45.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.281+0000 7f99991e3700 1 -- 172.21.15.57:0/1642131658 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998c00b8d0 con 0x7f9994071150 2023-11-28T12:03:45.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.281+0000 7f99991e3700 1 -- 172.21.15.57:0/1642131658 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c00bc80 con 0x7f9994071150 2023-11-28T12:03:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.282+0000 7f999c449700 1 -- 172.21.15.57:0/1642131658 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9994071150 msgr2=0x7f99940c4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.282+0000 7f999c449700 1 --2- 172.21.15.57:0/1642131658 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9994071150 0x7f99940c4b10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9994071f20 tx=0x7f998c00b330 comp rx=0 tx=0).stop 2023-11-28T12:03:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.282+0000 7f999c449700 1 -- 172.21.15.57:0/1642131658 shutdown_connections 2023-11-28T12:03:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.282+0000 7f999c449700 1 --2- 172.21.15.57:0/1642131658 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9994071150 0x7f99940c4b10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.282+0000 7f999c449700 1 --2- 172.21.15.57:0/1642131658 >> v2:172.21.15.181:3300/0 conn(0x7f999415cce0 0x7f999406b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.282+0000 7f999c449700 1 --2- 172.21.15.57:0/1642131658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f999415dc40 0x7f9994070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.282+0000 7f999c449700 1 -- 172.21.15.57:0/1642131658 >> 172.21.15.57:0/1642131658 conn(0x7f99940cfc30 msgr2=0x7f99940d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.282+0000 7f999c449700 1 -- 172.21.15.57:0/1642131658 shutdown_connections 2023-11-28T12:03:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.282+0000 7f999c449700 1 -- 172.21.15.57:0/1642131658 wait complete. 2023-11-28T12:03:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.283+0000 7f999c449700 1 Processor -- start 2023-11-28T12:03:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.283+0000 7f999c449700 1 -- start start 2023-11-28T12:03:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999c449700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9994071150 0x7f9994157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999c449700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f999415cce0 0x7f999415f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999c449700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f999415dc40 0x7f999415fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999c449700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99941f66a0 con 0x7f999415dc40 2023-11-28T12:03:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999c449700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99941f5a20 con 0x7f999415cce0 2023-11-28T12:03:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999c449700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99941f5d00 con 0x7f9994071150 2023-11-28T12:03:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999a9e6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f999415dc40 0x7f999415fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999a9e6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f999415dc40 0x7f999415fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44298/0 (socket says 172.21.15.57:44298) 2023-11-28T12:03:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999a9e6700 1 -- 172.21.15.57:0/39821808 learned_addr learned my addr 172.21.15.57:0/39821808 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999a1e5700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9994071150 0x7f9994157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:45.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f99999e4700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f999415cce0 0x7f999415f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:45.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999a9e6700 1 -- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9994071150 msgr2=0x7f9994157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:45.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999a9e6700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9994071150 0x7f9994157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999a9e6700 1 -- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f999415cce0 msgr2=0x7f999415f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:45.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999a9e6700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f999415cce0 0x7f999415f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.284+0000 7f999a9e6700 1 -- 172.21.15.57:0/39821808 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c009b30 con 0x7f999415dc40 2023-11-28T12:03:45.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.285+0000 7f999a9e6700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f999415dc40 0x7f999415fbc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9994071740 tx=0x7f998c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:45.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.285+0000 7f998affd700 1 -- 172.21.15.57:0/39821808 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c003e10 con 0x7f999415dc40 2023-11-28T12:03:45.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.285+0000 7f998affd700 1 -- 172.21.15.57:0/39821808 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998c004250 con 0x7f999415dc40 2023-11-28T12:03:45.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.285+0000 7f999c449700 1 -- 172.21.15.57:0/39821808 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99940cd8a0 con 0x7f999415dc40 2023-11-28T12:03:45.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.285+0000 7f998affd700 1 -- 172.21.15.57:0/39821808 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c004600 con 0x7f999415dc40 2023-11-28T12:03:45.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.285+0000 7f999c449700 1 -- 172.21.15.57:0/39821808 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99940cf730 con 0x7f999415dc40 2023-11-28T12:03:45.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.286+0000 7f999c449700 1 -- 172.21.15.57:0/39821808 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99940cf3e0 con 0x7f999415dc40 2023-11-28T12:03:45.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.286+0000 7f998affd700 1 -- 172.21.15.57:0/39821808 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f998c004760 con 0x7f999415dc40 2023-11-28T12:03:45.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.287+0000 7f998affd700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f99780682f0 0x7f997806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:45.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.287+0000 7f998affd700 1 -- 172.21.15.57:0/39821808 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f998c086b40 con 0x7f999415dc40 2023-11-28T12:03:45.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.287+0000 7f999a1e5700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f99780682f0 0x7f997806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:45.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.287+0000 7f999a1e5700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f99780682f0 0x7f997806a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f99941f58e0 tx=0x7f9990008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:45.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.291+0000 7f998affd700 1 -- 172.21.15.57:0/39821808 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f998c0537a0 con 0x7f999415dc40 2023-11-28T12:03:45.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.488+0000 7f999c449700 1 -- 172.21.15.57:0/39821808 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f99941f67e0 con 0x7f999415dc40 2023-11-28T12:03:45.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.822+0000 7f998affd700 1 -- 172.21.15.57:0/39821808 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v46) v1 ==== 138+0+0 (secure 0 0 0) 0x7f99941f67e0 con 0x7f999415dc40 2023-11-28T12:03:45.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.827+0000 7f999c449700 1 -- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f99780682f0 msgr2=0x7f997806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:45.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.827+0000 7f999c449700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f99780682f0 0x7f997806a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f99941f58e0 tx=0x7f9990008040 comp rx=0 tx=0).stop 2023-11-28T12:03:45.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.827+0000 7f999c449700 1 -- 172.21.15.57:0/39821808 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f999415dc40 msgr2=0x7f999415fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:45.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.827+0000 7f999c449700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f999415dc40 0x7f999415fbc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9994071740 tx=0x7f998c004ad0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.827+0000 7f999c449700 1 -- 172.21.15.57:0/39821808 shutdown_connections 2023-11-28T12:03:45.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.827+0000 7f999c449700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f99780682f0 0x7f997806a7a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.827+0000 7f999c449700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f999415dc40 0x7f999415fbc0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.827+0000 7f999c449700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9994071150 0x7f9994157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.827+0000 7f999c449700 1 --2- 172.21.15.57:0/39821808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f999415cce0 0x7f999415f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.827+0000 7f999c449700 1 -- 172.21.15.57:0/39821808 >> 172.21.15.57:0/39821808 conn(0x7f99940cfc30 msgr2=0x7f99940b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:45.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.828+0000 7f999c449700 1 -- 172.21.15.57:0/39821808 shutdown_connections 2023-11-28T12:03:45.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.828+0000 7f999c449700 1 -- 172.21.15.57:0/39821808 wait complete. 2023-11-28T12:03:45.830 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' created 2023-11-28T12:03:45.845 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T12:03:45.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.991+0000 7fe000ea5700 1 Processor -- start 2023-11-28T12:03:45.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.991+0000 7fe000ea5700 1 -- start start 2023-11-28T12:03:45.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fe000ea5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdffc15a4d0 0x7fdffc06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:45.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fe000ea5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdffc06a7b0 0x7fdffc0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:45.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fe000ea5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdffc0cdd50 0x7fdffc0b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:45.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fe000ea5700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdffc06ac30 con 0x7fdffc06a7b0 2023-11-28T12:03:45.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fe000ea5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdffc15e0d0 con 0x7fdffc15a4d0 2023-11-28T12:03:45.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fe000ea5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc0ce560 con 0x7fdffc0cdd50 2023-11-28T12:03:45.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fdff9d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdffc06a7b0 0x7fdffc0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:45.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fdff9d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdffc06a7b0 0x7fdffc0cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44306/0 (socket says 172.21.15.57:44306) 2023-11-28T12:03:45.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fdff9d9b700 1 -- 172.21.15.57:0/3151395004 learned_addr learned my addr 172.21.15.57:0/3151395004 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:45.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fdffad9d700 1 --2- 172.21.15.57:0/3151395004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdffc0cdd50 0x7fdffc0b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:45.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fdffa59c700 1 --2- 172.21.15.57:0/3151395004 >> v2:172.21.15.181:3300/0 conn(0x7fdffc15a4d0 0x7fdffc06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:45.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fdff9d9b700 1 -- 172.21.15.57:0/3151395004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdffc0cdd50 msgr2=0x7fdffc0b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:45.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fdff9d9b700 1 --2- 172.21.15.57:0/3151395004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdffc0cdd50 0x7fdffc0b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fdff9d9b700 1 -- 172.21.15.57:0/3151395004 >> v2:172.21.15.181:3300/0 conn(0x7fdffc15a4d0 msgr2=0x7fdffc06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:45.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fdff9d9b700 1 --2- 172.21.15.57:0/3151395004 >> v2:172.21.15.181:3300/0 conn(0x7fdffc15a4d0 0x7fdffc06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fdff9d9b700 1 -- 172.21.15.57:0/3151395004 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc15e320 con 0x7fdffc06a7b0 2023-11-28T12:03:45.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.992+0000 7fdff9d9b700 1 --2- 172.21.15.57:0/3151395004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc06a7b0 0x7fdffc0cd810 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdffc06b140 tx=0x7fdff0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a39e337340798b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:45.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.993+0000 7fdff959a700 1 -- 172.21.15.57:0/3151395004 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff000b870 con 0x7fdffc06a7b0 2023-11-28T12:03:45.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.993+0000 7fdff959a700 1 -- 172.21.15.57:0/3151395004 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff000b9d0 con 0x7fdffc06a7b0 2023-11-28T12:03:45.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.993+0000 7fdff959a700 1 -- 172.21.15.57:0/3151395004 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff000bd60 con 0x7fdffc06a7b0 2023-11-28T12:03:45.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.993+0000 7fe000ea5700 1 -- 172.21.15.57:0/3151395004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc06a7b0 msgr2=0x7fdffc0cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:45.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.993+0000 7fe000ea5700 1 --2- 172.21.15.57:0/3151395004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc06a7b0 0x7fdffc0cd810 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdffc06b140 tx=0x7fdff0009bd0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.993+0000 7fe000ea5700 1 -- 172.21.15.57:0/3151395004 shutdown_connections 2023-11-28T12:03:45.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.993+0000 7fe000ea5700 1 --2- 172.21.15.57:0/3151395004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdffc0cdd50 0x7fdffc0b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.993+0000 7fe000ea5700 1 --2- 172.21.15.57:0/3151395004 >> v2:172.21.15.181:3300/0 conn(0x7fdffc15a4d0 0x7fdffc06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.993+0000 7fe000ea5700 1 --2- 172.21.15.57:0/3151395004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc06a7b0 0x7fdffc0cd810 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.993+0000 7fe000ea5700 1 -- 172.21.15.57:0/3151395004 >> 172.21.15.57:0/3151395004 conn(0x7fdffc06c010 msgr2=0x7fdffc0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:45.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.993+0000 7fe000ea5700 1 -- 172.21.15.57:0/3151395004 shutdown_connections 2023-11-28T12:03:45.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.993+0000 7fe000ea5700 1 -- 172.21.15.57:0/3151395004 wait complete. 2023-11-28T12:03:45.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.995+0000 7fe000ea5700 1 Processor -- start 2023-11-28T12:03:45.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.995+0000 7fe000ea5700 1 -- start start 2023-11-28T12:03:45.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fe000ea5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdffc06a7b0 0x7fdffc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:45.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fe000ea5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdffc0cdd50 0x7fdffc147700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:45.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fe000ea5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc15a4d0 0x7fdffc153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:45.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fe000ea5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc15c240 con 0x7fdffc15a4d0 2023-11-28T12:03:45.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fe000ea5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc0c9d10 con 0x7fdffc0cdd50 2023-11-28T12:03:45.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fe000ea5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc15c420 con 0x7fdffc06a7b0 2023-11-28T12:03:45.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdffad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc15a4d0 0x7fdffc153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:45.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdffad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc15a4d0 0x7fdffc153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44318/0 (socket says 172.21.15.57:44318) 2023-11-28T12:03:45.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdffad9d700 1 -- 172.21.15.57:0/3972903559 learned_addr learned my addr 172.21.15.57:0/3972903559 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:45.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdffa59c700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdffc06a7b0 0x7fdffc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:45.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdff9d9b700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdffc0cdd50 0x7fdffc147700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:45.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdffad9d700 1 -- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdffc06a7b0 msgr2=0x7fdffc15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:45.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdffad9d700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdffc06a7b0 0x7fdffc15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdffad9d700 1 -- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdffc0cdd50 msgr2=0x7fdffc147700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:45.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdffad9d700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdffc0cdd50 0x7fdffc147700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:45.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdffad9d700 1 -- 172.21.15.57:0/3972903559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff0009550 con 0x7fdffc15a4d0 2023-11-28T12:03:45.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdffad9d700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc15a4d0 0x7fdffc153cc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdff4007e60 tx=0x7fdff400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:45.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdfeaffd700 1 -- 172.21.15.57:0/3972903559 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff400ed30 con 0x7fdffc15a4d0 2023-11-28T12:03:45.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdfeaffd700 1 -- 172.21.15.57:0/3972903559 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff4014420 con 0x7fdffc15a4d0 2023-11-28T12:03:45.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fe000ea5700 1 -- 172.21.15.57:0/3972903559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdffc15e8e0 con 0x7fdffc15a4d0 2023-11-28T12:03:45.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.996+0000 7fdfeaffd700 1 -- 172.21.15.57:0/3972903559 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff40147d0 con 0x7fdffc15a4d0 2023-11-28T12:03:46.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.997+0000 7fe000ea5700 1 -- 172.21.15.57:0/3972903559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdffc15e600 con 0x7fdffc15a4d0 2023-11-28T12:03:46.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.998+0000 7fe000ea5700 1 -- 172.21.15.57:0/3972903559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdffc15a8a0 con 0x7fdffc15a4d0 2023-11-28T12:03:46.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.998+0000 7fdfeaffd700 1 -- 172.21.15.57:0/3972903559 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdff4012050 con 0x7fdffc15a4d0 2023-11-28T12:03:46.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.998+0000 7fdfeaffd700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdfdc0682f0 0x7fdfdc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:46.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.999+0000 7fdfeaffd700 1 -- 172.21.15.57:0/3972903559 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fdff4085fb0 con 0x7fdffc15a4d0 2023-11-28T12:03:46.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.999+0000 7fdffa59c700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdfdc0682f0 0x7fdfdc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:46.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:45.999+0000 7fdffa59c700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdfdc0682f0 0x7fdfdc06a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdfec006b20 tx=0x7fdfec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:46.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.003+0000 7fdfeaffd700 1 -- 172.21.15.57:0/3972903559 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdff40529f0 con 0x7fdffc15a4d0 2023-11-28T12:03:46.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.201+0000 7fe000ea5700 1 -- 172.21.15.57:0/3972903559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdffc0743f0 con 0x7fdffc15a4d0 2023-11-28T12:03:46.393 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:03:46.396 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:03:46.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.826+0000 7fdfeaffd700 1 -- 172.21.15.57:0/3972903559 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v47) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdff4056010 con 0x7fdffc15a4d0 2023-11-28T12:03:46.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.830+0000 7fe000ea5700 1 -- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdfdc0682f0 msgr2=0x7fdfdc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:46.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.830+0000 7fe000ea5700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdfdc0682f0 0x7fdfdc06a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdfec006b20 tx=0x7fdfec008040 comp rx=0 tx=0).stop 2023-11-28T12:03:46.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.830+0000 7fe000ea5700 1 -- 172.21.15.57:0/3972903559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc15a4d0 msgr2=0x7fdffc153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:46.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.830+0000 7fe000ea5700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc15a4d0 0x7fdffc153cc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdff4007e60 tx=0x7fdff400e3c0 comp rx=0 tx=0).stop 2023-11-28T12:03:46.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.831+0000 7fe000ea5700 1 -- 172.21.15.57:0/3972903559 shutdown_connections 2023-11-28T12:03:46.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.831+0000 7fe000ea5700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdfdc0682f0 0x7fdfdc06a7a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:46.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.831+0000 7fe000ea5700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc15a4d0 0x7fdffc153cc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:46.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.831+0000 7fe000ea5700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdffc06a7b0 0x7fdffc15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:46.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.831+0000 7fe000ea5700 1 --2- 172.21.15.57:0/3972903559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdffc0cdd50 0x7fdffc147700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:46.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.831+0000 7fe000ea5700 1 -- 172.21.15.57:0/3972903559 >> 172.21.15.57:0/3972903559 conn(0x7fdffc06c010 msgr2=0x7fdffc1550c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:46.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.832+0000 7fe000ea5700 1 -- 172.21.15.57:0/3972903559 shutdown_connections 2023-11-28T12:03:46.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:46.833+0000 7fe000ea5700 1 -- 172.21.15.57:0/3972903559 wait complete. 2023-11-28T12:03:46.845 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' created 2023-11-28T12:03:46.861 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T12:03:47.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.003+0000 7f8eb5511700 1 Processor -- start 2023-11-28T12:03:47.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.003+0000 7f8eb5511700 1 -- start start 2023-11-28T12:03:47.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.004+0000 7f8eb5511700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8eb00cdf20 0x7f8eb0165420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.004+0000 7f8eb5511700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb015e0d0 con 0x7f8eb015a4d0 2023-11-28T12:03:47.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.004+0000 7f8eb5511700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb00cc020 con 0x7f8eb006a980 2023-11-28T12:03:47.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.004+0000 7f8eb5511700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00ce380 con 0x7f8eb00cdf20 2023-11-28T12:03:47.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.004+0000 7f8eaeffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8eb015a4d0 0x7f8eb006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54898/0 (socket says 172.21.15.57:54898) 2023-11-28T12:03:47.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.004+0000 7f8eaeffd700 1 -- 172.21.15.57:0/1937847674 learned_addr learned my addr 172.21.15.57:0/1937847674 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:47.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.005+0000 7f8eaf7fe700 1 --2- 172.21.15.57:0/1937847674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8eb00cdf20 0x7f8eb0165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.005+0000 7f8eadffb700 1 -- 172.21.15.57:0/1937847674 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104516995 0 0) 0x7f8eb015e0d0 con 0x7f8eb015a4d0 2023-11-28T12:03:47.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.005+0000 7f8eadffb700 1 -- 172.21.15.57:0/1937847674 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e980035b0 con 0x7f8eb015a4d0 2023-11-28T12:03:47.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.005+0000 7f8eadffb700 1 -- 172.21.15.57:0/1937847674 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591250028 0 0) 0x7f8eb00cc020 con 0x7f8eb006a980 2023-11-28T12:03:47.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.005+0000 7f8eadffb700 1 -- 172.21.15.57:0/1937847674 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8eb015e0d0 con 0x7f8eb006a980 2023-11-28T12:03:47.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.005+0000 7f8eaf7fe700 1 -- 172.21.15.57:0/1937847674 >> v1:172.21.15.181:6789/0 conn(0x7f8eb006a980 legacy=0x7f8eb00cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.005+0000 7f8eaf7fe700 1 -- 172.21.15.57:0/1937847674 >> v1:172.21.15.57:6789/0 conn(0x7f8eb015a4d0 legacy=0x7f8eb006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.005+0000 7f8eaf7fe700 1 -- 172.21.15.57:0/1937847674 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb00ce560 con 0x7f8eb00cdf20 2023-11-28T12:03:47.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.005+0000 7f8eaf7fe700 1 --2- 172.21.15.57:0/1937847674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8eb00cdf20 0x7f8eb0165420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009dc0 tx=0x7f8ea800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd97d6fcc0eb0fec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:47.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.005+0000 7f8eadffb700 1 -- 172.21.15.57:0/1937847674 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea800e070 con 0x7f8eb00cdf20 2023-11-28T12:03:47.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.005+0000 7f8eadffb700 1 -- 172.21.15.57:0/1937847674 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea800bb20 con 0x7f8eb00cdf20 2023-11-28T12:03:47.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.005+0000 7f8eadffb700 1 -- 172.21.15.57:0/1937847674 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8012430 con 0x7f8eb00cdf20 2023-11-28T12:03:47.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.006+0000 7f8eb5511700 1 -- 172.21.15.57:0/1937847674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8eb00cdf20 msgr2=0x7f8eb0165420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.006+0000 7f8eb5511700 1 --2- 172.21.15.57:0/1937847674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8eb00cdf20 0x7f8eb0165420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009dc0 tx=0x7f8ea800b330 comp rx=0 tx=0).stop 2023-11-28T12:03:47.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.006+0000 7f8eb5511700 1 -- 172.21.15.57:0/1937847674 shutdown_connections 2023-11-28T12:03:47.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.006+0000 7f8eb5511700 1 --2- 172.21.15.57:0/1937847674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8eb00cdf20 0x7f8eb0165420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.006+0000 7f8eb5511700 1 -- 172.21.15.57:0/1937847674 >> 172.21.15.57:0/1937847674 conn(0x7f8eb006c010 msgr2=0x7f8eb00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:47.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.006+0000 7f8eb5511700 1 -- 172.21.15.57:0/1937847674 shutdown_connections 2023-11-28T12:03:47.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.006+0000 7f8eb5511700 1 -- 172.21.15.57:0/1937847674 wait complete. 2023-11-28T12:03:47.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.008+0000 7f8eb5511700 1 Processor -- start 2023-11-28T12:03:47.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.008+0000 7f8eb5511700 1 -- start start 2023-11-28T12:03:47.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.008+0000 7f8eb5511700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eb006a980 0x7f8eb0143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.008+0000 7f8eb5511700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8eb00cdf20 0x7f8eb015a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.008+0000 7f8eb5511700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eb015a4d0 0x7f8eb01f0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.008+0000 7f8eb5511700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb015ae70 con 0x7f8eb006a980 2023-11-28T12:03:47.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.008+0000 7f8eb5511700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00af9e0 con 0x7f8eb015a4d0 2023-11-28T12:03:47.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.008+0000 7f8eb5511700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00afcc0 con 0x7f8eb00cdf20 2023-11-28T12:03:47.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.008+0000 7f8eaeffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eb006a980 0x7f8eb0143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8eaeffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eb006a980 0x7f8eb0143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44328/0 (socket says 172.21.15.57:44328) 2023-11-28T12:03:47.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8eaeffd700 1 -- 172.21.15.57:0/1596838532 learned_addr learned my addr 172.21.15.57:0/1596838532 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:47.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8eaf7fe700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eb015a4d0 0x7f8eb01f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8eae7fc700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8eb00cdf20 0x7f8eb015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8eaeffd700 1 -- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8eb00cdf20 msgr2=0x7f8eb015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8eaeffd700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8eb00cdf20 0x7f8eb015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8eaeffd700 1 -- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eb015a4d0 msgr2=0x7f8eb01f0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8eaeffd700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eb015a4d0 0x7f8eb01f0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8eaeffd700 1 -- 172.21.15.57:0/1596838532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea8009a70 con 0x7f8eb006a980 2023-11-28T12:03:47.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8eaf7fe700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eb015a4d0 0x7f8eb01f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:47.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8eaeffd700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eb006a980 0x7f8eb0143510 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0001eb0 tx=0x7f8ea000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:47.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8e977fe700 1 -- 172.21.15.57:0/1596838532 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0005d70 con 0x7f8eb006a980 2023-11-28T12:03:47.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8e977fe700 1 -- 172.21.15.57:0/1596838532 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea00070d0 con 0x7f8eb006a980 2023-11-28T12:03:47.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8eb5511700 1 -- 172.21.15.57:0/1596838532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb015e600 con 0x7f8eb006a980 2023-11-28T12:03:47.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8e977fe700 1 -- 172.21.15.57:0/1596838532 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0007460 con 0x7f8eb006a980 2023-11-28T12:03:47.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.009+0000 7f8eb5511700 1 -- 172.21.15.57:0/1596838532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb015e320 con 0x7f8eb006a980 2023-11-28T12:03:47.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.010+0000 7f8eb5511700 1 -- 172.21.15.57:0/1596838532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb015e8e0 con 0x7f8eb006a980 2023-11-28T12:03:47.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.011+0000 7f8e977fe700 1 -- 172.21.15.57:0/1596838532 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ea0019030 con 0x7f8eb006a980 2023-11-28T12:03:47.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.011+0000 7f8e977fe700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8e9c0682f0 0x7f8e9c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.012+0000 7f8e977fe700 1 -- 172.21.15.57:0/1596838532 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f8ea00850a0 con 0x7f8eb006a980 2023-11-28T12:03:47.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.012+0000 7f8eae7fc700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8e9c0682f0 0x7f8e9c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.012+0000 7f8eae7fc700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8e9c0682f0 0x7f8e9c06a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0143a50 tx=0x7f8ea4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:47.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.016+0000 7f8e977fe700 1 -- 172.21.15.57:0/1596838532 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8ea00518d0 con 0x7f8eb006a980 2023-11-28T12:03:47.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.263+0000 7f8eb5511700 1 -- 172.21.15.57:0/1596838532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8eb00743f0 con 0x7f8eb006a980 2023-11-28T12:03:47.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.268+0000 7f8e977fe700 1 -- 172.21.15.57:0/1596838532 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8ea0017020 con 0x7f8eb006a980 2023-11-28T12:03:47.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.274+0000 7f8eb5511700 1 -- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8e9c0682f0 msgr2=0x7f8e9c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.274+0000 7f8eb5511700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8e9c0682f0 0x7f8e9c06a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0143a50 tx=0x7f8ea4009040 comp rx=0 tx=0).stop 2023-11-28T12:03:47.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.274+0000 7f8eb5511700 1 -- 172.21.15.57:0/1596838532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eb006a980 msgr2=0x7f8eb0143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.274+0000 7f8eb5511700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eb006a980 0x7f8eb0143510 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0001eb0 tx=0x7f8ea000c860 comp rx=0 tx=0).stop 2023-11-28T12:03:47.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.275+0000 7f8eb5511700 1 -- 172.21.15.57:0/1596838532 shutdown_connections 2023-11-28T12:03:47.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.275+0000 7f8eb5511700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8e9c0682f0 0x7f8e9c06a7a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.275+0000 7f8eb5511700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eb015a4d0 0x7f8eb01f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.275+0000 7f8eb5511700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8eb006a980 0x7f8eb0143510 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.275+0000 7f8eb5511700 1 --2- 172.21.15.57:0/1596838532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8eb00cdf20 0x7f8eb015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.275+0000 7f8eb5511700 1 -- 172.21.15.57:0/1596838532 >> 172.21.15.57:0/1596838532 conn(0x7f8eb006c010 msgr2=0x7f8eb00cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:47.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.278+0000 7f8eb5511700 1 -- 172.21.15.57:0/1596838532 shutdown_connections 2023-11-28T12:03:47.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.278+0000 7f8eb5511700 1 -- 172.21.15.57:0/1596838532 wait complete. 2023-11-28T12:03:47.290 INFO:teuthology.orchestra.run.smithi057.stderr:new fs with metadata pool 7 and data pool 8 2023-11-28T12:03:47.306 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:03:47.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.442+0000 7febb3a85700 1 Processor -- start 2023-11-28T12:03:47.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.442+0000 7febb3a85700 1 -- start start 2023-11-28T12:03:47.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb3a85700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7febac0cf0d0 0x7febac070f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb3a85700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7febac15ea70 0x7febac0cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb3a85700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac0cf4a0 con 0x7febac06a270 2023-11-28T12:03:47.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb3a85700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac15a9c0 con 0x7febac15ea70 2023-11-28T12:03:47.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb3a85700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac0caf80 con 0x7febac0cf0d0 2023-11-28T12:03:47.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb1020700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7febac15ea70 0x7febac0cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb1020700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7febac15ea70 0x7febac0cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44334/0 (socket says 172.21.15.57:44334) 2023-11-28T12:03:47.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb1020700 1 -- 172.21.15.57:0/196644890 learned_addr learned my addr 172.21.15.57:0/196644890 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:47.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb1821700 1 --2- 172.21.15.57:0/196644890 >> v2:172.21.15.181:3300/0 conn(0x7febac0cf0d0 0x7febac070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb1020700 1 -- 172.21.15.57:0/196644890 >> v2:172.21.15.181:3300/0 conn(0x7febac0cf0d0 msgr2=0x7febac070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:47.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb1020700 1 --2- 172.21.15.57:0/196644890 >> v2:172.21.15.181:3300/0 conn(0x7febac0cf0d0 0x7febac070f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb1020700 1 -- 172.21.15.57:0/196644890 >> v1:172.21.15.181:6789/0 conn(0x7febac06a270 legacy=0x7febac06a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb1020700 1 -- 172.21.15.57:0/196644890 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac15efe0 con 0x7febac15ea70 2023-11-28T12:03:47.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.443+0000 7febb1020700 1 --2- 172.21.15.57:0/196644890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febac15ea70 0x7febac0cbfe0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7febac06b140 tx=0x7feb9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b3354d68172568 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:47.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.444+0000 7febb081f700 1 -- 172.21.15.57:0/196644890 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c011830 con 0x7febac15ea70 2023-11-28T12:03:47.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.444+0000 7febb081f700 1 -- 172.21.15.57:0/196644890 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c011990 con 0x7febac15ea70 2023-11-28T12:03:47.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.444+0000 7febb081f700 1 -- 172.21.15.57:0/196644890 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c011d20 con 0x7febac15ea70 2023-11-28T12:03:47.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.444+0000 7febb3a85700 1 -- 172.21.15.57:0/196644890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febac15ea70 msgr2=0x7febac0cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.444+0000 7febb3a85700 1 --2- 172.21.15.57:0/196644890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febac15ea70 0x7febac0cbfe0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7febac06b140 tx=0x7feb9c009bd0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.444+0000 7febb3a85700 1 -- 172.21.15.57:0/196644890 shutdown_connections 2023-11-28T12:03:47.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.444+0000 7febb3a85700 1 --2- 172.21.15.57:0/196644890 >> v2:172.21.15.181:3300/0 conn(0x7febac0cf0d0 0x7febac070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.444+0000 7febb3a85700 1 --2- 172.21.15.57:0/196644890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febac15ea70 0x7febac0cbfe0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.444+0000 7febb3a85700 1 -- 172.21.15.57:0/196644890 >> 172.21.15.57:0/196644890 conn(0x7febac06bfd0 msgr2=0x7febac0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:47.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.444+0000 7febb3a85700 1 -- 172.21.15.57:0/196644890 shutdown_connections 2023-11-28T12:03:47.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.444+0000 7febb3a85700 1 -- 172.21.15.57:0/196644890 wait complete. 2023-11-28T12:03:47.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.446+0000 7febb3a85700 1 Processor -- start 2023-11-28T12:03:47.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.446+0000 7febb3a85700 1 -- start start 2023-11-28T12:03:47.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.446+0000 7febb3a85700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febac06a270 0x7febac0c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.446+0000 7febb3a85700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febac0cf0d0 0x7febac0b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.446+0000 7febb3a85700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febac15ea70 0x7febac14bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.446+0000 7febb3a85700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0c3c20 con 0x7febac0cf0d0 2023-11-28T12:03:47.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.446+0000 7febb3a85700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0b5910 con 0x7febac06a270 2023-11-28T12:03:47.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.446+0000 7febb3a85700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac14c440 con 0x7febac15ea70 2023-11-28T12:03:47.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7febb1020700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febac0cf0d0 0x7febac0b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7febb1020700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febac0cf0d0 0x7febac0b53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44340/0 (socket says 172.21.15.57:44340) 2023-11-28T12:03:47.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7febb1020700 1 -- 172.21.15.57:0/2051375646 learned_addr learned my addr 172.21.15.57:0/2051375646 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:47.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7febb1821700 1 --2- 172.21.15.57:0/2051375646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febac06a270 0x7febac0c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7febb1020700 1 -- 172.21.15.57:0/2051375646 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febac15ea70 msgr2=0x7febac14bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7febb1020700 1 --2- 172.21.15.57:0/2051375646 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febac15ea70 0x7febac14bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7febb1020700 1 -- 172.21.15.57:0/2051375646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febac06a270 msgr2=0x7febac0c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7febb1020700 1 --2- 172.21.15.57:0/2051375646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febac06a270 0x7febac0c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7febb1020700 1 -- 172.21.15.57:0/2051375646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c009550 con 0x7febac0cf0d0 2023-11-28T12:03:47.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7febb1020700 1 --2- 172.21.15.57:0/2051375646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febac0cf0d0 0x7febac0b53d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009c10 tx=0x7feb9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:47.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7feba27fc700 1 -- 172.21.15.57:0/2051375646 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0036a0 con 0x7febac0cf0d0 2023-11-28T12:03:47.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7feba27fc700 1 -- 172.21.15.57:0/2051375646 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c01e070 con 0x7febac0cf0d0 2023-11-28T12:03:47.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7febb3a85700 1 -- 172.21.15.57:0/2051375646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac0cb500 con 0x7febac0cf0d0 2023-11-28T12:03:47.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7feba27fc700 1 -- 172.21.15.57:0/2051375646 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c003a40 con 0x7febac0cf0d0 2023-11-28T12:03:47.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.447+0000 7febb3a85700 1 -- 172.21.15.57:0/2051375646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac0cb220 con 0x7febac0cf0d0 2023-11-28T12:03:47.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.448+0000 7febb3a85700 1 -- 172.21.15.57:0/2051375646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac0cb7e0 con 0x7febac0cf0d0 2023-11-28T12:03:47.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.449+0000 7feba27fc700 1 -- 172.21.15.57:0/2051375646 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feb9c00f050 con 0x7febac0cf0d0 2023-11-28T12:03:47.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.449+0000 7feba27fc700 1 --2- 172.21.15.57:0/2051375646 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb900682f0 0x7feb9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.449+0000 7feba27fc700 1 -- 172.21.15.57:0/2051375646 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feb9c086dc0 con 0x7febac0cf0d0 2023-11-28T12:03:47.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.449+0000 7febb1821700 1 --2- 172.21.15.57:0/2051375646 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb900682f0 0x7feb9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.450+0000 7febb1821700 1 --2- 172.21.15.57:0/2051375646 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb900682f0 0x7feb9006a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feba8009ab0 tx=0x7feba8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:47.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.454+0000 7feba27fc700 1 -- 172.21.15.57:0/2051375646 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feb9c053620 con 0x7febac0cf0d0 2023-11-28T12:03:47.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.656+0000 7febb3a85700 1 -- 172.21.15.57:0/2051375646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febac0678c0 con 0x7febac0cf0d0 2023-11-28T12:03:47.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.657+0000 7feba27fc700 1 -- 172.21.15.57:0/2051375646 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+17934 (secure 0 0 0) 0x7feb9c056c40 con 0x7febac0cf0d0 2023-11-28T12:03:47.658 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:47.658 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":48,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:03:47.263461+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:03:45.520415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T12:03:46.232534+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:03:47.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.659+0000 7febb3a85700 1 -- 172.21.15.57:0/2051375646 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb900682f0 msgr2=0x7feb9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.659+0000 7febb3a85700 1 --2- 172.21.15.57:0/2051375646 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb900682f0 0x7feb9006a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feba8009ab0 tx=0x7feba8008040 comp rx=0 tx=0).stop 2023-11-28T12:03:47.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.659+0000 7febb3a85700 1 -- 172.21.15.57:0/2051375646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febac0cf0d0 msgr2=0x7febac0b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.659+0000 7febb3a85700 1 --2- 172.21.15.57:0/2051375646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febac0cf0d0 0x7febac0b53d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009c10 tx=0x7feb9c004750 comp rx=0 tx=0).stop 2023-11-28T12:03:47.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.659+0000 7febb3a85700 1 -- 172.21.15.57:0/2051375646 shutdown_connections 2023-11-28T12:03:47.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.659+0000 7febb3a85700 1 --2- 172.21.15.57:0/2051375646 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb900682f0 0x7feb9006a7a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.659+0000 7febb3a85700 1 --2- 172.21.15.57:0/2051375646 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febac15ea70 0x7febac14bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.659+0000 7febb3a85700 1 --2- 172.21.15.57:0/2051375646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febac06a270 0x7febac0c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.659+0000 7febb3a85700 1 --2- 172.21.15.57:0/2051375646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febac0cf0d0 0x7febac0b53d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.659+0000 7febb3a85700 1 -- 172.21.15.57:0/2051375646 >> 172.21.15.57:0/2051375646 conn(0x7febac06bfd0 msgr2=0x7febac15c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.659+0000 7febb3a85700 1 -- 172.21.15.57:0/2051375646 shutdown_connections 2023-11-28T12:03:47.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.659+0000 7febb3a85700 1 -- 172.21.15.57:0/2051375646 wait complete. 2023-11-28T12:03:47.674 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:03:47.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.810+0000 7feaa597b700 1 Processor -- start 2023-11-28T12:03:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.810+0000 7feaa597b700 1 -- start start 2023-11-28T12:03:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7feaa597b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa00b22a0 0x7feaa00cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7feaa597b700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa00d1ee0 con 0x7feaa00cd5a0 2023-11-28T12:03:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7feaa597b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa00cdb10 con 0x7feaa006ab10 2023-11-28T12:03:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7feaa597b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa00d1890 con 0x7feaa00b22a0 2023-11-28T12:03:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7fea967fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7feaa00cd5a0 0x7feaa00cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54914/0 (socket says 172.21.15.57:54914) 2023-11-28T12:03:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7fea967fc700 1 -- 172.21.15.57:0/2408819441 learned_addr learned my addr 172.21.15.57:0/2408819441 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7fea9f7fe700 1 --2- 172.21.15.57:0/2408819441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa00b22a0 0x7feaa00cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7fea9e7fc700 1 -- 172.21.15.57:0/2408819441 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332404983 0 0) 0x7feaa00d1ee0 con 0x7feaa00cd5a0 2023-11-28T12:03:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7fea9e7fc700 1 -- 172.21.15.57:0/2408819441 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea8c0035b0 con 0x7feaa00cd5a0 2023-11-28T12:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7fea9e7fc700 1 -- 172.21.15.57:0/2408819441 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593914244 0 0) 0x7feaa00cdb10 con 0x7feaa006ab10 2023-11-28T12:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7fea9e7fc700 1 -- 172.21.15.57:0/2408819441 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaa00d1ee0 con 0x7feaa006ab10 2023-11-28T12:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7fea9f7fe700 1 -- 172.21.15.57:0/2408819441 >> v1:172.21.15.181:6789/0 conn(0x7feaa006ab10 legacy=0x7feaa006aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7fea9f7fe700 1 -- 172.21.15.57:0/2408819441 >> v1:172.21.15.57:6789/0 conn(0x7feaa00cd5a0 legacy=0x7feaa00cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.811+0000 7fea9f7fe700 1 -- 172.21.15.57:0/2408819441 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa00d1b70 con 0x7feaa00b22a0 2023-11-28T12:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.812+0000 7fea9f7fe700 1 --2- 172.21.15.57:0/2408819441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa00b22a0 0x7feaa00cfb50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fea98009dc0 tx=0x7fea9800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47e9fb88055a9dad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.812+0000 7fea9e7fc700 1 -- 172.21.15.57:0/2408819441 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9800e070 con 0x7feaa00b22a0 2023-11-28T12:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.812+0000 7fea9e7fc700 1 -- 172.21.15.57:0/2408819441 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9800bb20 con 0x7feaa00b22a0 2023-11-28T12:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.812+0000 7fea9e7fc700 1 -- 172.21.15.57:0/2408819441 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98012430 con 0x7feaa00b22a0 2023-11-28T12:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.812+0000 7feaa597b700 1 -- 172.21.15.57:0/2408819441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa00b22a0 msgr2=0x7feaa00cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.812+0000 7feaa597b700 1 --2- 172.21.15.57:0/2408819441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa00b22a0 0x7feaa00cfb50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fea98009dc0 tx=0x7fea9800b330 comp rx=0 tx=0).stop 2023-11-28T12:03:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.812+0000 7feaa597b700 1 -- 172.21.15.57:0/2408819441 shutdown_connections 2023-11-28T12:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.812+0000 7feaa597b700 1 --2- 172.21.15.57:0/2408819441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa00b22a0 0x7feaa00cfb50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.812+0000 7feaa597b700 1 -- 172.21.15.57:0/2408819441 >> 172.21.15.57:0/2408819441 conn(0x7feaa006c520 msgr2=0x7feaa006c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.812+0000 7feaa597b700 1 -- 172.21.15.57:0/2408819441 shutdown_connections 2023-11-28T12:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.812+0000 7feaa597b700 1 -- 172.21.15.57:0/2408819441 wait complete. 2023-11-28T12:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.814+0000 7feaa597b700 1 Processor -- start 2023-11-28T12:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.814+0000 7feaa597b700 1 -- start start 2023-11-28T12:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7feaa597b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa006ab10 0x7feaa00c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7feaa597b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa00b22a0 0x7feaa01573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7feaa597b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaa00cd5a0 0x7feaa01f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7feaa597b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa00c7c00 con 0x7feaa00cd5a0 2023-11-28T12:03:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7feaa597b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa00c8b90 con 0x7feaa00b22a0 2023-11-28T12:03:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7feaa597b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa01f37d0 con 0x7feaa006ab10 2023-11-28T12:03:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7fea9f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaa00cd5a0 0x7feaa01f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7fea9f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaa00cd5a0 0x7feaa01f2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44356/0 (socket says 172.21.15.57:44356) 2023-11-28T12:03:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7fea9f7fe700 1 -- 172.21.15.57:0/2728188163 learned_addr learned my addr 172.21.15.57:0/2728188163 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7fea9effd700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa006ab10 0x7feaa00c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7fea967fc700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa00b22a0 0x7feaa01573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7fea9f7fe700 1 -- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa006ab10 msgr2=0x7feaa00c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7fea9f7fe700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa006ab10 0x7feaa00c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7fea9f7fe700 1 -- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa00b22a0 msgr2=0x7feaa01573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7fea9f7fe700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa00b22a0 0x7feaa01573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7fea9f7fe700 1 -- 172.21.15.57:0/2728188163 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea98009a70 con 0x7feaa00cd5a0 2023-11-28T12:03:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7fea9effd700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa006ab10 0x7feaa00c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7fea967fc700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa00b22a0 0x7feaa01573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.815+0000 7fea9f7fe700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaa00cd5a0 0x7feaa01f2d20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fea9800b370 tx=0x7fea98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.816+0000 7fea9cff9700 1 -- 172.21.15.57:0/2728188163 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9801b810 con 0x7feaa00cd5a0 2023-11-28T12:03:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.816+0000 7fea9cff9700 1 -- 172.21.15.57:0/2728188163 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea98020070 con 0x7feaa00cd5a0 2023-11-28T12:03:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.816+0000 7feaa597b700 1 -- 172.21.15.57:0/2728188163 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa00b1ec0 con 0x7feaa00cd5a0 2023-11-28T12:03:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.816+0000 7fea9cff9700 1 -- 172.21.15.57:0/2728188163 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9801bbf0 con 0x7feaa00cd5a0 2023-11-28T12:03:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.816+0000 7feaa597b700 1 -- 172.21.15.57:0/2728188163 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa015cad0 con 0x7feaa00cd5a0 2023-11-28T12:03:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.817+0000 7feaa597b700 1 -- 172.21.15.57:0/2728188163 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa00b2050 con 0x7feaa00cd5a0 2023-11-28T12:03:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.817+0000 7fea9cff9700 1 -- 172.21.15.57:0/2728188163 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fea9801bd50 con 0x7feaa00cd5a0 2023-11-28T12:03:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.818+0000 7fea9cff9700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fea800682f0 0x7fea8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.818+0000 7fea9cff9700 1 -- 172.21.15.57:0/2728188163 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea98022070 con 0x7feaa00cd5a0 2023-11-28T12:03:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.818+0000 7fea9effd700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fea800682f0 0x7fea8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.818+0000 7fea9effd700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fea800682f0 0x7fea8006a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fea90007fd0 tx=0x7fea90009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:47.822+0000 7fea9cff9700 1 -- 172.21.15.57:0/2728188163 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fea98054b60 con 0x7feaa00cd5a0 2023-11-28T12:03:48.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.022+0000 7feaa597b700 1 -- 172.21.15.57:0/2728188163 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feaa0068450 con 0x7feaa00cd5a0 2023-11-28T12:03:48.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.023+0000 7fea9cff9700 1 -- 172.21.15.57:0/2728188163 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+17934 (secure 0 0 0) 0x7fea98058180 con 0x7feaa00cd5a0 2023-11-28T12:03:48.024 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:48.024 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":48,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:03:47.263461+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:03:45.520415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T12:03:46.232534+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:03:48.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.026+0000 7feaa597b700 1 -- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fea800682f0 msgr2=0x7fea8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:48.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.026+0000 7feaa597b700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fea800682f0 0x7fea8006a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fea90007fd0 tx=0x7fea90009040 comp rx=0 tx=0).stop 2023-11-28T12:03:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.026+0000 7feaa597b700 1 -- 172.21.15.57:0/2728188163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaa00cd5a0 msgr2=0x7feaa01f2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.026+0000 7feaa597b700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaa00cd5a0 0x7feaa01f2d20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fea9800b370 tx=0x7fea98004ad0 comp rx=0 tx=0).stop 2023-11-28T12:03:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.026+0000 7feaa597b700 1 -- 172.21.15.57:0/2728188163 shutdown_connections 2023-11-28T12:03:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.026+0000 7feaa597b700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fea800682f0 0x7fea8006a7a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.026+0000 7feaa597b700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaa00cd5a0 0x7feaa01f2d20 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.026+0000 7feaa597b700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa006ab10 0x7feaa00c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.026+0000 7feaa597b700 1 --2- 172.21.15.57:0/2728188163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa00b22a0 0x7feaa01573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.026+0000 7feaa597b700 1 -- 172.21.15.57:0/2728188163 >> 172.21.15.57:0/2728188163 conn(0x7feaa006c520 msgr2=0x7feaa00b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:48.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.026+0000 7feaa597b700 1 -- 172.21.15.57:0/2728188163 shutdown_connections 2023-11-28T12:03:48.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.026+0000 7feaa597b700 1 -- 172.21.15.57:0/2728188163 wait complete. 2023-11-28T12:03:48.041 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T12:03:48.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.181+0000 7f3af0f50700 1 Processor -- start 2023-11-28T12:03:48.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.181+0000 7f3af0f50700 1 -- start start 2023-11-28T12:03:48.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.182+0000 7f3af0f50700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3aec15a5f0 0x7f3aec0d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:48.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.182+0000 7f3af0f50700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3aec0cc200 0x7f3aec0b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:48.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.182+0000 7f3af0f50700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aec15a9c0 con 0x7f3aec0b2d10 2023-11-28T12:03:48.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.182+0000 7f3af0f50700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aec0b3190 con 0x7f3aec0cc200 2023-11-28T12:03:48.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.182+0000 7f3af0f50700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aec0c7b80 con 0x7f3aec15a5f0 2023-11-28T12:03:48.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.182+0000 7f3ae9d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3aec0cc200 0x7f3aec0b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:48.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.183+0000 7f3ae9d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3aec0cc200 0x7f3aec0b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44358/0 (socket says 172.21.15.57:44358) 2023-11-28T12:03:48.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.183+0000 7f3ae9d9b700 1 -- 172.21.15.57:0/3440660039 learned_addr learned my addr 172.21.15.57:0/3440660039 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:48.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.183+0000 7f3aea59c700 1 --2- 172.21.15.57:0/3440660039 >> v2:172.21.15.181:3300/0 conn(0x7f3aec15a5f0 0x7f3aec0d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:48.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.183+0000 7f3ae9d9b700 1 -- 172.21.15.57:0/3440660039 >> v2:172.21.15.181:3300/0 conn(0x7f3aec15a5f0 msgr2=0x7f3aec0d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:48.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.183+0000 7f3ae9d9b700 1 --2- 172.21.15.57:0/3440660039 >> v2:172.21.15.181:3300/0 conn(0x7f3aec15a5f0 0x7f3aec0d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:48.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.183+0000 7f3ae9d9b700 1 -- 172.21.15.57:0/3440660039 >> v1:172.21.15.181:6789/0 conn(0x7f3aec0b2d10 legacy=0x7f3aec15e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:48.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.183+0000 7f3ae9d9b700 1 -- 172.21.15.57:0/3440660039 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec0cc990 con 0x7f3aec0cc200 2023-11-28T12:03:48.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.183+0000 7f3ae9d9b700 1 --2- 172.21.15.57:0/3440660039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aec0cc200 0x7f3aec0b27d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0d1d40 tx=0x7f3adc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9c3107303e7bc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:48.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.183+0000 7f3ae959a700 1 -- 172.21.15.57:0/3440660039 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc00b870 con 0x7f3aec0cc200 2023-11-28T12:03:48.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.183+0000 7f3ae959a700 1 -- 172.21.15.57:0/3440660039 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc00b9d0 con 0x7f3aec0cc200 2023-11-28T12:03:48.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.183+0000 7f3ae959a700 1 -- 172.21.15.57:0/3440660039 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc00bd60 con 0x7f3aec0cc200 2023-11-28T12:03:48.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.184+0000 7f3af0f50700 1 -- 172.21.15.57:0/3440660039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aec0cc200 msgr2=0x7f3aec0b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:48.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.184+0000 7f3af0f50700 1 --2- 172.21.15.57:0/3440660039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aec0cc200 0x7f3aec0b27d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0d1d40 tx=0x7f3adc009bd0 comp rx=0 tx=0).stop 2023-11-28T12:03:48.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.184+0000 7f3af0f50700 1 -- 172.21.15.57:0/3440660039 shutdown_connections 2023-11-28T12:03:48.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.184+0000 7f3af0f50700 1 --2- 172.21.15.57:0/3440660039 >> v2:172.21.15.181:3300/0 conn(0x7f3aec15a5f0 0x7f3aec0d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:48.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.184+0000 7f3af0f50700 1 --2- 172.21.15.57:0/3440660039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aec0cc200 0x7f3aec0b27d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:48.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.184+0000 7f3af0f50700 1 -- 172.21.15.57:0/3440660039 >> 172.21.15.57:0/3440660039 conn(0x7f3aec06ea80 msgr2=0x7f3aec06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:48.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.184+0000 7f3af0f50700 1 -- 172.21.15.57:0/3440660039 shutdown_connections 2023-11-28T12:03:48.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.184+0000 7f3af0f50700 1 -- 172.21.15.57:0/3440660039 wait complete. 2023-11-28T12:03:48.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.185+0000 7f3af0f50700 1 Processor -- start 2023-11-28T12:03:48.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.185+0000 7f3af0f50700 1 -- start start 2023-11-28T12:03:48.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.186+0000 7f3af0f50700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec0b2d10 0x7f3aec06bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:48.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.186+0000 7f3af0f50700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3aec0cc200 0x7f3aec0afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:48.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.186+0000 7f3af0f50700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aec15a5f0 0x7f3aec0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:48.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.186+0000 7f3af0f50700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec06cf10 con 0x7f3aec15a5f0 2023-11-28T12:03:48.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.186+0000 7f3af0f50700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec0c4ae0 con 0x7f3aec0b2d10 2023-11-28T12:03:48.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.186+0000 7f3af0f50700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec0c4dc0 con 0x7f3aec0cc200 2023-11-28T12:03:48.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.186+0000 7f3aead9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aec15a5f0 0x7f3aec0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:48.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.186+0000 7f3aead9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aec15a5f0 0x7f3aec0b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44374/0 (socket says 172.21.15.57:44374) 2023-11-28T12:03:48.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.186+0000 7f3aead9d700 1 -- 172.21.15.57:0/1618550018 learned_addr learned my addr 172.21.15.57:0/1618550018 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:48.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.186+0000 7f3aea59c700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec0b2d10 0x7f3aec06bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:48.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.186+0000 7f3ae9d9b700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3aec0cc200 0x7f3aec0afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:48.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.187+0000 7f3aead9d700 1 -- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3aec0cc200 msgr2=0x7f3aec0afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:48.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.187+0000 7f3aead9d700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3aec0cc200 0x7f3aec0afd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:48.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.187+0000 7f3aead9d700 1 -- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec0b2d10 msgr2=0x7f3aec06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:48.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.187+0000 7f3aead9d700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec0b2d10 0x7f3aec06bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:48.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.187+0000 7f3aead9d700 1 -- 172.21.15.57:0/1618550018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3adc009550 con 0x7f3aec15a5f0 2023-11-28T12:03:48.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.187+0000 7f3aead9d700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aec15a5f0 0x7f3aec0b0ea0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400cb50 tx=0x7f3ae400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:48.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.187+0000 7f3ae2ffd700 1 -- 172.21.15.57:0/1618550018 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4003c00 con 0x7f3aec15a5f0 2023-11-28T12:03:48.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.187+0000 7f3ae2ffd700 1 -- 172.21.15.57:0/1618550018 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae4003d60 con 0x7f3aec15a5f0 2023-11-28T12:03:48.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.187+0000 7f3af0f50700 1 -- 172.21.15.57:0/1618550018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aec0ccf50 con 0x7f3aec15a5f0 2023-11-28T12:03:48.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.187+0000 7f3ae2ffd700 1 -- 172.21.15.57:0/1618550018 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae400aa20 con 0x7f3aec15a5f0 2023-11-28T12:03:48.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.187+0000 7f3af0f50700 1 -- 172.21.15.57:0/1618550018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aec0ccc70 con 0x7f3aec15a5f0 2023-11-28T12:03:48.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.188+0000 7f3af0f50700 1 -- 172.21.15.57:0/1618550018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aec0cd230 con 0x7f3aec15a5f0 2023-11-28T12:03:48.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.189+0000 7f3ae2ffd700 1 -- 172.21.15.57:0/1618550018 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3ae4010050 con 0x7f3aec15a5f0 2023-11-28T12:03:48.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.189+0000 7f3ae2ffd700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3acc0682f0 0x7f3acc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:48.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.189+0000 7f3ae2ffd700 1 -- 172.21.15.57:0/1618550018 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3ae4085b20 con 0x7f3aec15a5f0 2023-11-28T12:03:48.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.189+0000 7f3aea59c700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3acc0682f0 0x7f3acc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:48.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.190+0000 7f3aea59c700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3acc0682f0 0x7f3acc06a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40099d0 tx=0x7f3ad4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:48.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.194+0000 7f3ae2ffd700 1 -- 172.21.15.57:0/1618550018 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3ae4052380 con 0x7f3aec15a5f0 2023-11-28T12:03:48.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:48.447+0000 7f3af0f50700 1 -- 172.21.15.57:0/1618550018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3aec0743f0 con 0x7f3aec15a5f0 2023-11-28T12:03:49.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.285+0000 7f3ae2ffd700 1 -- 172.21.15.57:0/1618550018 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3ae40559a0 con 0x7f3aec15a5f0 2023-11-28T12:03:49.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.288+0000 7f3af0f50700 1 -- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3acc0682f0 msgr2=0x7f3acc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:49.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.288+0000 7f3af0f50700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3acc0682f0 0x7f3acc06a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40099d0 tx=0x7f3ad4008040 comp rx=0 tx=0).stop 2023-11-28T12:03:49.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.288+0000 7f3af0f50700 1 -- 172.21.15.57:0/1618550018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aec15a5f0 msgr2=0x7f3aec0b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:49.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.288+0000 7f3af0f50700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aec15a5f0 0x7f3aec0b0ea0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400cb50 tx=0x7f3ae400ce60 comp rx=0 tx=0).stop 2023-11-28T12:03:49.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.288+0000 7f3af0f50700 1 -- 172.21.15.57:0/1618550018 shutdown_connections 2023-11-28T12:03:49.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.288+0000 7f3af0f50700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3acc0682f0 0x7f3acc06a7a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:49.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.288+0000 7f3af0f50700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aec15a5f0 0x7f3aec0b0ea0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:49.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.288+0000 7f3af0f50700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aec0b2d10 0x7f3aec06bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:49.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.288+0000 7f3af0f50700 1 --2- 172.21.15.57:0/1618550018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3aec0cc200 0x7f3aec0afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:49.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.288+0000 7f3af0f50700 1 -- 172.21.15.57:0/1618550018 >> 172.21.15.57:0/1618550018 conn(0x7f3aec06ea80 msgr2=0x7f3aec0b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:49.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.288+0000 7f3af0f50700 1 -- 172.21.15.57:0/1618550018 shutdown_connections 2023-11-28T12:03:49.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.288+0000 7f3af0f50700 1 -- 172.21.15.57:0/1618550018 wait complete. 2023-11-28T12:03:49.305 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T12:03:49.305 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T12:03:49.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.446+0000 7f07a1f3d700 1 Processor -- start 2023-11-28T12:03:49.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.446+0000 7f07a1f3d700 1 -- start start 2023-11-28T12:03:49.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.447+0000 7f07a1f3d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f079c159710 0x7f079c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:49.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.447+0000 7f07a1f3d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c06a310 con 0x7f079c0cee40 2023-11-28T12:03:49.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.447+0000 7f07a1f3d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0ce7e0 con 0x7f079c0c9a50 2023-11-28T12:03:49.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.447+0000 7f07a1f3d700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079c0cf2c0 con 0x7f079c159710 2023-11-28T12:03:49.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.447+0000 7f079b7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f079c159710 0x7f079c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:49.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.447+0000 7f079affd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f079c0cee40 0x7f079c0cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54928/0 (socket says 172.21.15.57:54928) 2023-11-28T12:03:49.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.447+0000 7f079affd700 1 -- 172.21.15.57:0/2681265762 learned_addr learned my addr 172.21.15.57:0/2681265762 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:49.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.447+0000 7f079b7fe700 1 --2- 172.21.15.57:0/2681265762 >> v2:172.21.15.57:3300/0 conn(0x7f079c159710 0x7f079c070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44382/0 (socket says 172.21.15.57:44382) 2023-11-28T12:03:49.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.447+0000 7f079b7fe700 1 -- 172.21.15.57:0/2681265762 >> v1:172.21.15.181:6789/0 conn(0x7f079c0c9a50 legacy=0x7f079c06adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:49.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.447+0000 7f079b7fe700 1 -- 172.21.15.57:0/2681265762 >> v1:172.21.15.57:6789/0 conn(0x7f079c0cee40 legacy=0x7f079c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:49.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.448+0000 7f079b7fe700 1 -- 172.21.15.57:0/2681265762 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c06a5b0 con 0x7f079c159710 2023-11-28T12:03:49.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.447+0000 7f079a7fc700 1 -- 172.21.15.57:0/2681265762 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412860451 0 0) 0x7f079c06a310 con 0x7f079c0cee40 2023-11-28T12:03:49.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.448+0000 7f079b7fe700 1 --2- 172.21.15.57:0/2681265762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f079c159710 0x7f079c070370 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f078400bb10 tx=0x7f078400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17ed7e497280efd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:49.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.448+0000 7f079a7fc700 1 -- 172.21.15.57:0/2681265762 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400dcf0 con 0x7f079c159710 2023-11-28T12:03:49.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.448+0000 7f079a7fc700 1 -- 172.21.15.57:0/2681265762 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078400de50 con 0x7f079c159710 2023-11-28T12:03:49.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.448+0000 7f079a7fc700 1 -- 172.21.15.57:0/2681265762 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784010690 con 0x7f079c159710 2023-11-28T12:03:49.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.448+0000 7f07a1f3d700 1 -- 172.21.15.57:0/2681265762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f079c159710 msgr2=0x7f079c070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:49.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.448+0000 7f07a1f3d700 1 --2- 172.21.15.57:0/2681265762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f079c159710 0x7f079c070370 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f078400bb10 tx=0x7f078400be20 comp rx=0 tx=0).stop 2023-11-28T12:03:49.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.448+0000 7f07a1f3d700 1 -- 172.21.15.57:0/2681265762 shutdown_connections 2023-11-28T12:03:49.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.448+0000 7f07a1f3d700 1 --2- 172.21.15.57:0/2681265762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f079c159710 0x7f079c070370 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:49.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.448+0000 7f07a1f3d700 1 -- 172.21.15.57:0/2681265762 >> 172.21.15.57:0/2681265762 conn(0x7f079c0cf400 msgr2=0x7f079c0cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:49.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.449+0000 7f07a1f3d700 1 -- 172.21.15.57:0/2681265762 shutdown_connections 2023-11-28T12:03:49.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.449+0000 7f07a1f3d700 1 -- 172.21.15.57:0/2681265762 wait complete. 2023-11-28T12:03:49.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.450+0000 7f07a1f3d700 1 Processor -- start 2023-11-28T12:03:49.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.451+0000 7f07a1f3d700 1 -- start start 2023-11-28T12:03:49.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.451+0000 7f07a1f3d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f079c0c9a50 0x7f079c1e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:49.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.451+0000 7f07a1f3d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f079c0cee40 0x7f079c0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:49.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.451+0000 7f07a1f3d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f079c159710 0x7f079c14d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:49.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.451+0000 7f07a1f3d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c1e2450 con 0x7f079c0cee40 2023-11-28T12:03:49.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.451+0000 7f07a1f3d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c156f90 con 0x7f079c159710 2023-11-28T12:03:49.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.451+0000 7f07a1f3d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079c15acd0 con 0x7f079c0c9a50 2023-11-28T12:03:49.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f079affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f079c0cee40 0x7f079c0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:49.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f079affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f079c0cee40 0x7f079c0b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44394/0 (socket says 172.21.15.57:44394) 2023-11-28T12:03:49.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f079affd700 1 -- 172.21.15.57:0/1497406043 learned_addr learned my addr 172.21.15.57:0/1497406043 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:49.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f079bfff700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f079c159710 0x7f079c14d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:49.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f079b7fe700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f079c0c9a50 0x7f079c1e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:49.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f079affd700 1 -- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f079c0c9a50 msgr2=0x7f079c1e1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:49.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f079affd700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f079c0c9a50 0x7f079c1e1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:49.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f079affd700 1 -- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f079c159710 msgr2=0x7f079c14d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:49.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f079affd700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f079c159710 0x7f079c14d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:49.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f079affd700 1 -- 172.21.15.57:0/1497406043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078400b740 con 0x7f079c0cee40 2023-11-28T12:03:49.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f079b7fe700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f079c0c9a50 0x7f079c1e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:49.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f079affd700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f079c0cee40 0x7f079c0b44d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f078c00c520 tx=0x7f078c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:49.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f0793fff700 1 -- 172.21.15.57:0/1497406043 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c004c80 con 0x7f079c0cee40 2023-11-28T12:03:49.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f0793fff700 1 -- 172.21.15.57:0/1497406043 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c004e00 con 0x7f079c0cee40 2023-11-28T12:03:49.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f07a1f3d700 1 -- 172.21.15.57:0/1497406043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079c0ceac0 con 0x7f079c0cee40 2023-11-28T12:03:49.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f0793fff700 1 -- 172.21.15.57:0/1497406043 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c007310 con 0x7f079c0cee40 2023-11-28T12:03:49.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.452+0000 7f07a1f3d700 1 -- 172.21.15.57:0/1497406043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c06a890 con 0x7f079c0cee40 2023-11-28T12:03:49.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.453+0000 7f07a1f3d700 1 -- 172.21.15.57:0/1497406043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079c0c9eb0 con 0x7f079c0cee40 2023-11-28T12:03:49.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.454+0000 7f0793fff700 1 -- 172.21.15.57:0/1497406043 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f078c01a020 con 0x7f079c0cee40 2023-11-28T12:03:49.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.454+0000 7f0793fff700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f077c0682f0 0x7f077c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:49.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.454+0000 7f0793fff700 1 -- 172.21.15.57:0/1497406043 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f078c085810 con 0x7f079c0cee40 2023-11-28T12:03:49.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.454+0000 7f079b7fe700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f077c0682f0 0x7f077c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:49.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.455+0000 7f079b7fe700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f077c0682f0 0x7f077c06a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f078400b2c0 tx=0x7f0784013000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:49.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.459+0000 7f0793fff700 1 -- 172.21.15.57:0/1497406043 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f078c052070 con 0x7f079c0cee40 2023-11-28T12:03:49.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:49.706+0000 7f07a1f3d700 1 -- 172.21.15.57:0/1497406043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f079c0743f0 con 0x7f079c0cee40 2023-11-28T12:03:50.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.302+0000 7f0793fff700 1 -- 172.21.15.57:0/1497406043 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f078c055690 con 0x7f079c0cee40 2023-11-28T12:03:50.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.304+0000 7f07a1f3d700 1 -- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f077c0682f0 msgr2=0x7f077c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:50.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.304+0000 7f07a1f3d700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f077c0682f0 0x7f077c06a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f078400b2c0 tx=0x7f0784013000 comp rx=0 tx=0).stop 2023-11-28T12:03:50.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.304+0000 7f07a1f3d700 1 -- 172.21.15.57:0/1497406043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f079c0cee40 msgr2=0x7f079c0b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:50.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.304+0000 7f07a1f3d700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f079c0cee40 0x7f079c0b44d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f078c00c520 tx=0x7f078c00c8e0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.304+0000 7f07a1f3d700 1 -- 172.21.15.57:0/1497406043 shutdown_connections 2023-11-28T12:03:50.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.304+0000 7f07a1f3d700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f077c0682f0 0x7f077c06a7a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.304+0000 7f07a1f3d700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f079c159710 0x7f079c14d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.304+0000 7f07a1f3d700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f079c0c9a50 0x7f079c1e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.304+0000 7f07a1f3d700 1 --2- 172.21.15.57:0/1497406043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f079c0cee40 0x7f079c0b44d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.304+0000 7f07a1f3d700 1 -- 172.21.15.57:0/1497406043 >> 172.21.15.57:0/1497406043 conn(0x7f079c0cf400 msgr2=0x7f079c1589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:50.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.305+0000 7f07a1f3d700 1 -- 172.21.15.57:0/1497406043 shutdown_connections 2023-11-28T12:03:50.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.305+0000 7f07a1f3d700 1 -- 172.21.15.57:0/1497406043 wait complete. 2023-11-28T12:03:50.321 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T12:03:50.321 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:03:50.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.459+0000 7f73b2c8c700 1 Processor -- start 2023-11-28T12:03:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.459+0000 7f73b2c8c700 1 -- start start 2023-11-28T12:03:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b2c8c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f73ac06e400 0x7f73ac06e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b2c8c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ac15e010 0x7f73ac0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b2c8c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ac15af30 con 0x7f73ac0d15f0 2023-11-28T12:03:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b2c8c700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ac15a9d0 con 0x7f73ac06e400 2023-11-28T12:03:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b2c8c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac1584e0 con 0x7f73ac15e010 2023-11-28T12:03:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b0a28700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f73ac06e400 0x7f73ac06e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b0a28700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f73ac06e400 0x7f73ac06e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44396/0 (socket says 172.21.15.57:44396) 2023-11-28T12:03:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b0a28700 1 -- 172.21.15.57:0/3228965800 learned_addr learned my addr 172.21.15.57:0/3228965800 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:50.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b1229700 1 --2- 172.21.15.57:0/3228965800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ac15e010 0x7f73ac0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b0a28700 1 -- 172.21.15.57:0/3228965800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ac15e010 msgr2=0x7f73ac0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b0a28700 1 --2- 172.21.15.57:0/3228965800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ac15e010 0x7f73ac0ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b0a28700 1 -- 172.21.15.57:0/3228965800 >> v1:172.21.15.181:6789/0 conn(0x7f73ac0d15f0 legacy=0x7f73ac15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b0a28700 1 -- 172.21.15.57:0/3228965800 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac15ef40 con 0x7f73ac06e400 2023-11-28T12:03:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73b0a28700 1 --2- 172.21.15.57:0/3228965800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ac06e400 0x7f73ac06e7d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f73a00098f0 tx=0x7f73a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620bab805023bfd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73ab7fe700 1 -- 172.21.15.57:0/3228965800 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000b8a0 con 0x7f73ac06e400 2023-11-28T12:03:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73ab7fe700 1 -- 172.21.15.57:0/3228965800 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a000ba00 con 0x7f73ac06e400 2023-11-28T12:03:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.460+0000 7f73ab7fe700 1 -- 172.21.15.57:0/3228965800 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000bd90 con 0x7f73ac06e400 2023-11-28T12:03:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.461+0000 7f73b2c8c700 1 -- 172.21.15.57:0/3228965800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ac06e400 msgr2=0x7f73ac06e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.461+0000 7f73b2c8c700 1 --2- 172.21.15.57:0/3228965800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ac06e400 0x7f73ac06e7d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f73a00098f0 tx=0x7f73a0009c00 comp rx=0 tx=0).stop 2023-11-28T12:03:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.461+0000 7f73b2c8c700 1 -- 172.21.15.57:0/3228965800 shutdown_connections 2023-11-28T12:03:50.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.461+0000 7f73b2c8c700 1 --2- 172.21.15.57:0/3228965800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ac06e400 0x7f73ac06e7d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.461+0000 7f73b2c8c700 1 --2- 172.21.15.57:0/3228965800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ac15e010 0x7f73ac0ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.461+0000 7f73b2c8c700 1 -- 172.21.15.57:0/3228965800 >> 172.21.15.57:0/3228965800 conn(0x7f73ac06a270 msgr2=0x7f73ac06a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.461+0000 7f73b2c8c700 1 -- 172.21.15.57:0/3228965800 shutdown_connections 2023-11-28T12:03:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.461+0000 7f73b2c8c700 1 -- 172.21.15.57:0/3228965800 wait complete. 2023-11-28T12:03:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.462+0000 7f73b2c8c700 1 Processor -- start 2023-11-28T12:03:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.462+0000 7f73b2c8c700 1 -- start start 2023-11-28T12:03:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.463+0000 7f73b2c8c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ac06e400 0x7f73ac15bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.463+0000 7f73b2c8c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ac0d15f0 0x7f73ac14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.463+0000 7f73b2c8c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ac15e010 0x7f73ac14a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.463+0000 7f73b2c8c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac0b0760 con 0x7f73ac15e010 2023-11-28T12:03:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.463+0000 7f73b2c8c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac15c140 con 0x7f73ac0d15f0 2023-11-28T12:03:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.463+0000 7f73b2c8c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac15c420 con 0x7f73ac06e400 2023-11-28T12:03:50.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.463+0000 7f73b1229700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ac15e010 0x7f73ac14a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.463+0000 7f73b1229700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ac15e010 0x7f73ac14a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44406/0 (socket says 172.21.15.57:44406) 2023-11-28T12:03:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.463+0000 7f73b1229700 1 -- 172.21.15.57:0/1789510953 learned_addr learned my addr 172.21.15.57:0/1789510953 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.464+0000 7f73b1229700 1 -- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ac06e400 msgr2=0x7f73ac15bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.464+0000 7f73b0a28700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ac06e400 0x7f73ac15bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.464+0000 7f73abfff700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ac0d15f0 0x7f73ac14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.464+0000 7f73b1229700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ac06e400 0x7f73ac15bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.464+0000 7f73b1229700 1 -- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ac0d15f0 msgr2=0x7f73ac14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.464+0000 7f73b1229700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ac0d15f0 0x7f73ac14fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.464+0000 7f73b1229700 1 -- 172.21.15.57:0/1789510953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a0009550 con 0x7f73ac15e010 2023-11-28T12:03:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.464+0000 7f73b1229700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ac15e010 0x7f73ac14a800 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f73a4007e60 tx=0x7f73a400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:50.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.464+0000 7f73a97fa700 1 -- 172.21.15.57:0/1789510953 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a400ede0 con 0x7f73ac15e010 2023-11-28T12:03:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.464+0000 7f73a97fa700 1 -- 172.21.15.57:0/1789510953 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a4014420 con 0x7f73ac15e010 2023-11-28T12:03:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.464+0000 7f73b2c8c700 1 -- 172.21.15.57:0/1789510953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ac15e980 con 0x7f73ac15e010 2023-11-28T12:03:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.464+0000 7f73a97fa700 1 -- 172.21.15.57:0/1789510953 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a40147d0 con 0x7f73ac15e010 2023-11-28T12:03:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.464+0000 7f73b2c8c700 1 -- 172.21.15.57:0/1789510953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ac06ed10 con 0x7f73ac15e010 2023-11-28T12:03:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.465+0000 7f73b2c8c700 1 -- 172.21.15.57:0/1789510953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac15ec60 con 0x7f73ac15e010 2023-11-28T12:03:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.466+0000 7f73a97fa700 1 -- 172.21.15.57:0/1789510953 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f73a4012050 con 0x7f73ac15e010 2023-11-28T12:03:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.466+0000 7f73a97fa700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f73900682f0 0x7f739006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.466+0000 7f73a97fa700 1 -- 172.21.15.57:0/1789510953 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f73a40862b0 con 0x7f73ac15e010 2023-11-28T12:03:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.466+0000 7f73b0a28700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f73900682f0 0x7f739006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:50.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.467+0000 7f73b0a28700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f73900682f0 0x7f739006a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009c40 tx=0x7f73a0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:50.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.470+0000 7f73a97fa700 1 -- 172.21.15.57:0/1789510953 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f73a4052b10 con 0x7f73ac15e010 2023-11-28T12:03:50.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.716+0000 7f73b2c8c700 1 -- 172.21.15.57:0/1789510953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73ac0b0910 con 0x7f73ac15e010 2023-11-28T12:03:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.717+0000 7f73a97fa700 1 -- 172.21.15.57:0/1789510953 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f73ac0b0910 con 0x7f73ac15e010 2023-11-28T12:03:50.718 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:50.718 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1773283092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1773283092},{"type":"v1","addr":"172.21.15.181:6837","nonce":1773283092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/3718600227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3718600227},{"type":"v1","addr":"172.21.15.57:6834","nonce":3718600227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/762261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":762261630},{"type":"v1","addr":"172.21.15.57:6835","nonce":762261630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:03:47.263787+0000","modified":"2023-11-28T12:03:50.285215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3437917192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3437917192},{"type":"v1","addr":"172.21.15.181:6836","nonce":3437917192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T12:03:50.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.719+0000 7f73b2c8c700 1 -- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f73900682f0 msgr2=0x7f739006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:50.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.719+0000 7f73b2c8c700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f73900682f0 0x7f739006a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009c40 tx=0x7f73a0008030 comp rx=0 tx=0).stop 2023-11-28T12:03:50.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.719+0000 7f73b2c8c700 1 -- 172.21.15.57:0/1789510953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ac15e010 msgr2=0x7f73ac14a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:50.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.719+0000 7f73b2c8c700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ac15e010 0x7f73ac14a800 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f73a4007e60 tx=0x7f73a400e3c0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.719+0000 7f73b2c8c700 1 -- 172.21.15.57:0/1789510953 shutdown_connections 2023-11-28T12:03:50.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.719+0000 7f73b2c8c700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f73900682f0 0x7f739006a7a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.719+0000 7f73b2c8c700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ac15e010 0x7f73ac14a800 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.719+0000 7f73b2c8c700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ac06e400 0x7f73ac15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.719+0000 7f73b2c8c700 1 --2- 172.21.15.57:0/1789510953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ac0d15f0 0x7f73ac14fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.719+0000 7f73b2c8c700 1 -- 172.21.15.57:0/1789510953 >> 172.21.15.57:0/1789510953 conn(0x7f73ac06a270 msgr2=0x7f73ac153a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:50.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.719+0000 7f73b2c8c700 1 -- 172.21.15.57:0/1789510953 shutdown_connections 2023-11-28T12:03:50.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.719+0000 7f73b2c8c700 1 -- 172.21.15.57:0/1789510953 wait complete. 2023-11-28T12:03:50.722 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2023-11-28T12:03:50.734 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:03:50.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.868+0000 7f579a46d700 1 Processor -- start 2023-11-28T12:03:50.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.869+0000 7f579a46d700 1 -- start start 2023-11-28T12:03:50.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.869+0000 7f579a46d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f579406a270 0x7f579406a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:50.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.869+0000 7f579a46d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f579406e570 0x7f5794070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:50.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.869+0000 7f579a46d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579406ab80 con 0x7f579415cc80 2023-11-28T12:03:50.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.869+0000 7f579a46d700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57940b2490 con 0x7f579406e570 2023-11-28T12:03:50.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.869+0000 7f579a46d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579406e990 con 0x7f579406a270 2023-11-28T12:03:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.870+0000 7f5798a0a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f579406e570 0x7f5794070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.870+0000 7f5798a0a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f579406e570 0x7f5794070d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44408/0 (socket says 172.21.15.57:44408) 2023-11-28T12:03:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.870+0000 7f5798a0a700 1 -- 172.21.15.57:0/2500132947 learned_addr learned my addr 172.21.15.57:0/2500132947 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.870+0000 7f5793fff700 1 --2- 172.21.15.57:0/2500132947 >> v2:172.21.15.181:3300/0 conn(0x7f579406a270 0x7f579406a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.870+0000 7f5798a0a700 1 -- 172.21.15.57:0/2500132947 >> v2:172.21.15.181:3300/0 conn(0x7f579406a270 msgr2=0x7f579406a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.870+0000 7f5798a0a700 1 --2- 172.21.15.57:0/2500132947 >> v2:172.21.15.181:3300/0 conn(0x7f579406a270 0x7f579406a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.870+0000 7f5798a0a700 1 -- 172.21.15.57:0/2500132947 >> v1:172.21.15.181:6789/0 conn(0x7f579415cc80 legacy=0x7f579406de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.870+0000 7f5798a0a700 1 -- 172.21.15.57:0/2500132947 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57940b1810 con 0x7f579406e570 2023-11-28T12:03:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.870+0000 7f5798a0a700 1 --2- 172.21.15.57:0/2500132947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f579406e570 0x7f5794070d20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5794067380 tx=0x7f578c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c561dcda22b3bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.870+0000 7f5792ffd700 1 -- 172.21.15.57:0/2500132947 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c00e070 con 0x7f579406e570 2023-11-28T12:03:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.870+0000 7f5792ffd700 1 -- 172.21.15.57:0/2500132947 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c00b8d0 con 0x7f579406e570 2023-11-28T12:03:50.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.870+0000 7f5792ffd700 1 -- 172.21.15.57:0/2500132947 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c00bc80 con 0x7f579406e570 2023-11-28T12:03:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.871+0000 7f579a46d700 1 -- 172.21.15.57:0/2500132947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f579406e570 msgr2=0x7f5794070d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.871+0000 7f579a46d700 1 --2- 172.21.15.57:0/2500132947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f579406e570 0x7f5794070d20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5794067380 tx=0x7f578c00b330 comp rx=0 tx=0).stop 2023-11-28T12:03:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.871+0000 7f579a46d700 1 -- 172.21.15.57:0/2500132947 shutdown_connections 2023-11-28T12:03:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.871+0000 7f579a46d700 1 --2- 172.21.15.57:0/2500132947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f579406e570 0x7f5794070d20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.871+0000 7f579a46d700 1 --2- 172.21.15.57:0/2500132947 >> v2:172.21.15.181:3300/0 conn(0x7f579406a270 0x7f579406a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.871+0000 7f579a46d700 1 -- 172.21.15.57:0/2500132947 >> 172.21.15.57:0/2500132947 conn(0x7f579406c010 msgr2=0x7f57940cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.871+0000 7f579a46d700 1 -- 172.21.15.57:0/2500132947 shutdown_connections 2023-11-28T12:03:50.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.871+0000 7f579a46d700 1 -- 172.21.15.57:0/2500132947 wait complete. 2023-11-28T12:03:50.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.873+0000 7f579a46d700 1 Processor -- start 2023-11-28T12:03:50.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.873+0000 7f579a46d700 1 -- start start 2023-11-28T12:03:50.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f579a46d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f579406a270 0x7f5794142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:50.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f579a46d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f579406e570 0x7f579415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:50.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f579a46d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f579415cc80 0x7f57941e4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:50.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f579a46d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57940c4780 con 0x7f579406a270 2023-11-28T12:03:50.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f579a46d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57940c4a60 con 0x7f579415cc80 2023-11-28T12:03:50.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f579a46d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57940c4d40 con 0x7f579406e570 2023-11-28T12:03:50.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f5793fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f579406a270 0x7f5794142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:50.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f5793fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f579406a270 0x7f5794142e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44424/0 (socket says 172.21.15.57:44424) 2023-11-28T12:03:50.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f5793fff700 1 -- 172.21.15.57:0/581540317 learned_addr learned my addr 172.21.15.57:0/581540317 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f5798a0a700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f579415cc80 0x7f57941e4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f57937fe700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f579406e570 0x7f579415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f5793fff700 1 -- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f579406e570 msgr2=0x7f579415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f5793fff700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f579406e570 0x7f579415c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f5793fff700 1 -- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f579415cc80 msgr2=0x7f57941e4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f5793fff700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f579415cc80 0x7f57941e4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f5793fff700 1 -- 172.21.15.57:0/581540317 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c009b30 con 0x7f579406a270 2023-11-28T12:03:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f5798a0a700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f579415cc80 0x7f57941e4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.874+0000 7f5793fff700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f579406a270 0x7f5794142e50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f578400e870 tx=0x7f578400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:50.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.875+0000 7f5790ff9700 1 -- 172.21.15.57:0/581540317 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784009590 con 0x7f579406a270 2023-11-28T12:03:50.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.875+0000 7f5790ff9700 1 -- 172.21.15.57:0/581540317 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578400ee70 con 0x7f579406a270 2023-11-28T12:03:50.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.875+0000 7f579a46d700 1 -- 172.21.15.57:0/581540317 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57940b1dd0 con 0x7f579406a270 2023-11-28T12:03:50.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.875+0000 7f5790ff9700 1 -- 172.21.15.57:0/581540317 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578400ca50 con 0x7f579406a270 2023-11-28T12:03:50.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.875+0000 7f579a46d700 1 -- 172.21.15.57:0/581540317 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57940b1af0 con 0x7f579406a270 2023-11-28T12:03:50.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.876+0000 7f579a46d700 1 -- 172.21.15.57:0/581540317 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57940b20b0 con 0x7f579406a270 2023-11-28T12:03:50.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.876+0000 7f5790ff9700 1 -- 172.21.15.57:0/581540317 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5784019030 con 0x7f579406a270 2023-11-28T12:03:50.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.877+0000 7f5790ff9700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f57800682f0 0x7f578006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:50.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.877+0000 7f5790ff9700 1 -- 172.21.15.57:0/581540317 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5784085b20 con 0x7f579406a270 2023-11-28T12:03:50.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.877+0000 7f57937fe700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f57800682f0 0x7f578006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:50.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.877+0000 7f57937fe700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f57800682f0 0x7f578006a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5788007fd0 tx=0x7f5788009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:50.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:50.881+0000 7f5790ff9700 1 -- 172.21.15.57:0/581540317 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5784052380 con 0x7f579406a270 2023-11-28T12:03:51.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.081+0000 7f579a46d700 1 -- 172.21.15.57:0/581540317 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57940c5670 con 0x7f579406a270 2023-11-28T12:03:51.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.082+0000 7f5790ff9700 1 -- 172.21.15.57:0/581540317 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+17921 (secure 0 0 0) 0x7f57840559a0 con 0x7f579406a270 2023-11-28T12:03:51.083 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:51.083 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":49,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:03:48.265874+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:03:45.520415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T12:03:46.232534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:03:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.084+0000 7f579a46d700 1 -- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f57800682f0 msgr2=0x7f578006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.084+0000 7f579a46d700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f57800682f0 0x7f578006a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5788007fd0 tx=0x7f5788009040 comp rx=0 tx=0).stop 2023-11-28T12:03:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.084+0000 7f579a46d700 1 -- 172.21.15.57:0/581540317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f579406a270 msgr2=0x7f5794142e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.084+0000 7f579a46d700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f579406a270 0x7f5794142e50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f578400e870 tx=0x7f578400ec30 comp rx=0 tx=0).stop 2023-11-28T12:03:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.084+0000 7f579a46d700 1 -- 172.21.15.57:0/581540317 shutdown_connections 2023-11-28T12:03:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.084+0000 7f579a46d700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f57800682f0 0x7f578006a7a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.084+0000 7f579a46d700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f579415cc80 0x7f57941e4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.084+0000 7f579a46d700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f579406a270 0x7f5794142e50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.084+0000 7f579a46d700 1 --2- 172.21.15.57:0/581540317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f579406e570 0x7f579415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.084+0000 7f579a46d700 1 -- 172.21.15.57:0/581540317 >> 172.21.15.57:0/581540317 conn(0x7f579406c010 msgr2=0x7f57940cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.084+0000 7f579a46d700 1 -- 172.21.15.57:0/581540317 shutdown_connections 2023-11-28T12:03:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.084+0000 7f579a46d700 1 -- 172.21.15.57:0/581540317 wait complete. 2023-11-28T12:03:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T12:03:51.099 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:03:51.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.235+0000 7fe19ed7f700 1 Processor -- start 2023-11-28T12:03:51.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.235+0000 7fe19ed7f700 1 -- start start 2023-11-28T12:03:51.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.235+0000 7fe19ed7f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1980cd5a0 0x7fe1980cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:51.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.235+0000 7fe19ed7f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1980d1ee0 con 0x7fe19806ab10 2023-11-28T12:03:51.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.235+0000 7fe19ed7f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1980cdb10 con 0x7fe1980b22a0 2023-11-28T12:03:51.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.235+0000 7fe19ed7f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1980d1890 con 0x7fe1980cd5a0 2023-11-28T12:03:51.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.236+0000 7fe19cb1b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe19806ab10 0x7fe19806aee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54934/0 (socket says 172.21.15.57:54934) 2023-11-28T12:03:51.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.236+0000 7fe19cb1b700 1 -- 172.21.15.57:0/50581782 learned_addr learned my addr 172.21.15.57:0/50581782 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:51.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.236+0000 7fe197fff700 1 --2- 172.21.15.57:0/50581782 >> v2:172.21.15.181:3300/0 conn(0x7fe1980cd5a0 0x7fe1980cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:51.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.236+0000 7fe1977fe700 1 -- 172.21.15.57:0/50581782 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458238330 0 0) 0x7fe1980d1ee0 con 0x7fe19806ab10 2023-11-28T12:03:51.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.236+0000 7fe1977fe700 1 -- 172.21.15.57:0/50581782 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1840035b0 con 0x7fe19806ab10 2023-11-28T12:03:51.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.236+0000 7fe1977fe700 1 -- 172.21.15.57:0/50581782 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3092021777 0 0) 0x7fe1840035b0 con 0x7fe19806ab10 2023-11-28T12:03:51.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.236+0000 7fe197fff700 1 -- 172.21.15.57:0/50581782 >> v1:172.21.15.181:6789/0 conn(0x7fe1980b22a0 legacy=0x7fe1980cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.236+0000 7fe197fff700 1 -- 172.21.15.57:0/50581782 >> v1:172.21.15.57:6789/0 conn(0x7fe19806ab10 legacy=0x7fe19806aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.236+0000 7fe197fff700 1 -- 172.21.15.57:0/50581782 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1980d1b70 con 0x7fe1980cd5a0 2023-11-28T12:03:51.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.236+0000 7fe197fff700 1 --2- 172.21.15.57:0/50581782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1980cd5a0 0x7fe1980cf440 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe180009840 tx=0x7fe180009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af94fd3afea92559 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:51.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.237+0000 7fe1977fe700 1 -- 172.21.15.57:0/50581782 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188001670 con 0x7fe1980cd5a0 2023-11-28T12:03:51.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.237+0000 7fe1977fe700 1 -- 172.21.15.57:0/50581782 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe180004030 con 0x7fe1980cd5a0 2023-11-28T12:03:51.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.237+0000 7fe1977fe700 1 -- 172.21.15.57:0/50581782 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1800043c0 con 0x7fe1980cd5a0 2023-11-28T12:03:51.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.237+0000 7fe19ed7f700 1 -- 172.21.15.57:0/50581782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1980cd5a0 msgr2=0x7fe1980cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.237+0000 7fe19ed7f700 1 --2- 172.21.15.57:0/50581782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1980cd5a0 0x7fe1980cf440 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe180009840 tx=0x7fe180009b50 comp rx=0 tx=0).stop 2023-11-28T12:03:51.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.237+0000 7fe19ed7f700 1 -- 172.21.15.57:0/50581782 shutdown_connections 2023-11-28T12:03:51.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.237+0000 7fe19ed7f700 1 --2- 172.21.15.57:0/50581782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1980cd5a0 0x7fe1980cf440 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.237+0000 7fe19ed7f700 1 -- 172.21.15.57:0/50581782 >> 172.21.15.57:0/50581782 conn(0x7fe19806c520 msgr2=0x7fe19806c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:51.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.237+0000 7fe19ed7f700 1 -- 172.21.15.57:0/50581782 shutdown_connections 2023-11-28T12:03:51.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.237+0000 7fe19ed7f700 1 -- 172.21.15.57:0/50581782 wait complete. 2023-11-28T12:03:51.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.239+0000 7fe19ed7f700 1 Processor -- start 2023-11-28T12:03:51.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.239+0000 7fe19ed7f700 1 -- start start 2023-11-28T12:03:51.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.239+0000 7fe19ed7f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe19806ab10 0x7fe19815bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:51.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.239+0000 7fe19ed7f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1980b22a0 0x7fe1980cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:51.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.239+0000 7fe19ed7f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1980cd5a0 0x7fe198149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:51.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.239+0000 7fe19ed7f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1980ccfb0 con 0x7fe1980cd5a0 2023-11-28T12:03:51.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.239+0000 7fe19ed7f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19815ba40 con 0x7fe19806ab10 2023-11-28T12:03:51.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.239+0000 7fe19ed7f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1981496d0 con 0x7fe1980b22a0 2023-11-28T12:03:51.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe19d31c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1980cd5a0 0x7fe198149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:51.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe19d31c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1980cd5a0 0x7fe198149190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44436/0 (socket says 172.21.15.57:44436) 2023-11-28T12:03:51.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe19d31c700 1 -- 172.21.15.57:0/1852075256 learned_addr learned my addr 172.21.15.57:0/1852075256 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:51.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe19cb1b700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe19806ab10 0x7fe19815bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:51.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe197fff700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1980b22a0 0x7fe1980cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:51.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe19d31c700 1 -- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1980b22a0 msgr2=0x7fe1980cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe19d31c700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1980b22a0 0x7fe1980cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe19d31c700 1 -- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe19806ab10 msgr2=0x7fe19815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe19d31c700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe19806ab10 0x7fe19815bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe19d31c700 1 -- 172.21.15.57:0/1852075256 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1800094a0 con 0x7fe1980cd5a0 2023-11-28T12:03:51.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe19d31c700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1980cd5a0 0x7fe198149190 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe19000c8c0 tx=0x7fe19000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:51.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe1957fa700 1 -- 172.21.15.57:0/1852075256 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190003e30 con 0x7fe1980cd5a0 2023-11-28T12:03:51.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe1957fa700 1 -- 172.21.15.57:0/1852075256 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe190005070 con 0x7fe1980cd5a0 2023-11-28T12:03:51.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe19ed7f700 1 -- 172.21.15.57:0/1852075256 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1980b1ec0 con 0x7fe1980cd5a0 2023-11-28T12:03:51.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe1957fa700 1 -- 172.21.15.57:0/1852075256 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190005420 con 0x7fe1980cd5a0 2023-11-28T12:03:51.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.240+0000 7fe19ed7f700 1 -- 172.21.15.57:0/1852075256 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19815cad0 con 0x7fe1980cd5a0 2023-11-28T12:03:51.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.241+0000 7fe19ed7f700 1 -- 172.21.15.57:0/1852075256 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1980b2050 con 0x7fe1980cd5a0 2023-11-28T12:03:51.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.242+0000 7fe1957fa700 1 -- 172.21.15.57:0/1852075256 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe190010070 con 0x7fe1980cd5a0 2023-11-28T12:03:51.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.242+0000 7fe1957fa700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1780682f0 0x7fe17806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:51.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.242+0000 7fe1957fa700 1 -- 172.21.15.57:0/1852075256 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe1900858e0 con 0x7fe1980cd5a0 2023-11-28T12:03:51.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.243+0000 7fe19cb1b700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1780682f0 0x7fe17806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:51.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.243+0000 7fe19cb1b700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1780682f0 0x7fe17806a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe188000f60 tx=0x7fe188001e40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:51.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.247+0000 7fe1957fa700 1 -- 172.21.15.57:0/1852075256 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe1900520c0 con 0x7fe1980cd5a0 2023-11-28T12:03:51.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.444+0000 7fe19ed7f700 1 -- 172.21.15.57:0/1852075256 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1980743f0 con 0x7fe1780682f0 2023-11-28T12:03:51.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.453+0000 7fe1957fa700 1 -- 172.21.15.57:0/1852075256 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+301454 (secure 0 0 0) 0x7fe1980743f0 con 0x7fe1780682f0 2023-11-28T12:03:51.457 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:51.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.457+0000 7fe19ed7f700 1 -- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1780682f0 msgr2=0x7fe17806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.457+0000 7fe19ed7f700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1780682f0 0x7fe17806a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe188000f60 tx=0x7fe188001e40 comp rx=0 tx=0).stop 2023-11-28T12:03:51.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.457+0000 7fe19ed7f700 1 -- 172.21.15.57:0/1852075256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1980cd5a0 msgr2=0x7fe198149190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.457+0000 7fe19ed7f700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1980cd5a0 0x7fe198149190 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe19000c8c0 tx=0x7fe19000cbd0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.457+0000 7fe19ed7f700 1 -- 172.21.15.57:0/1852075256 shutdown_connections 2023-11-28T12:03:51.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.457+0000 7fe19ed7f700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1780682f0 0x7fe17806a7a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.457+0000 7fe19ed7f700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1980cd5a0 0x7fe198149190 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.457+0000 7fe19ed7f700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe19806ab10 0x7fe19815bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.457+0000 7fe19ed7f700 1 --2- 172.21.15.57:0/1852075256 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1980b22a0 0x7fe1980cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.457+0000 7fe19ed7f700 1 -- 172.21.15.57:0/1852075256 >> 172.21.15.57:0/1852075256 conn(0x7fe19806c520 msgr2=0x7fe1980b6120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:51.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.457+0000 7fe19ed7f700 1 -- 172.21.15.57:0/1852075256 shutdown_connections 2023-11-28T12:03:51.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.457+0000 7fe19ed7f700 1 -- 172.21.15.57:0/1852075256 wait complete. 2023-11-28T12:03:51.461 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:03:51.472 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2023-11-28T12:03:49.690550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":72636,"kb_used_data":5372,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":754902084,"statfs":{"total":773094113280,"available":773019734016,"internally_reserved":0,"allocated":5500928,"data_stored":3017542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000920"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824423+0000","last_change":"2023-11-28T12:03:45.824423+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824423+0000","last_undegraded":"2023-11-28T12:03:45.824423+0000","last_fullsized":"2023-11-28T12:03:45.824423+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:00.642126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827796+0000","last_change":"2023-11-28T12:03:45.827796+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827796+0000","last_undegraded":"2023-11-28T12:03:45.827796+0000","last_fullsized":"2023-11-28T12:03:45.827796+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:39.291687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827234+0000","last_change":"2023-11-28T12:03:45.827234+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827234+0000","last_undegraded":"2023-11-28T12:03:45.827234+0000","last_fullsized":"2023-11-28T12:03:45.827234+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:59.530852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823414+0000","last_change":"2023-11-28T12:03:45.823414+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823414+0000","last_undegraded":"2023-11-28T12:03:45.823414+0000","last_fullsized":"2023-11-28T12:03:45.823414+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:10.717376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.828214+0000","last_change":"2023-11-28T12:03:45.828214+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.828214+0000","last_undegraded":"2023-11-28T12:03:45.828214+0000","last_fullsized":"2023-11-28T12:03:45.828214+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:56.656303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826655+0000","last_change":"2023-11-28T12:03:45.826655+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826655+0000","last_undegraded":"2023-11-28T12:03:45.826655+0000","last_fullsized":"2023-11-28T12:03:45.826655+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:46.170695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823319+0000","last_change":"2023-11-28T12:03:45.823319+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823319+0000","last_undegraded":"2023-11-28T12:03:45.823319+0000","last_fullsized":"2023-11-28T12:03:45.823319+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:13:44.179722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826298+0000","last_change":"2023-11-28T12:03:45.826298+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826298+0000","last_undegraded":"2023-11-28T12:03:45.826298+0000","last_fullsized":"2023-11-28T12:03:45.826298+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:16.229157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825677+0000","last_change":"2023-11-28T12:03:45.825677+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825677+0000","last_undegraded":"2023-11-28T12:03:45.825677+0000","last_fullsized":"2023-11-28T12:03:45.825677+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:57.070237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826611+0000","last_change":"2023-11-28T12:03:45.826611+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826611+0000","last_undegraded":"2023-11-28T12:03:45.826611+0000","last_fullsized":"2023-11-28T12:03:45.826611+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:40.312689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827494+0000","last_change":"2023-11-28T12:03:45.827494+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827494+0000","last_undegraded":"2023-11-28T12:03:45.827494+0000","last_fullsized":"2023-11-28T12:03:45.827494+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:05.212437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824983+0000","last_change":"2023-11-28T12:03:45.824983+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824983+0000","last_undegraded":"2023-11-28T12:03:45.824983+0000","last_fullsized":"2023-11-28T12:03:45.824983+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:25.548629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826279+0000","last_change":"2023-11-28T12:03:45.826279+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826279+0000","last_undegraded":"2023-11-28T12:03:45.826279+0000","last_fullsized":"2023-11-28T12:03:45.826279+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:56:00.591608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823223+0000","last_change":"2023-11-28T12:03:45.823223+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823223+0000","last_undegraded":"2023-11-28T12:03:45.823223+0000","last_fullsized":"2023-11-28T12:03:45.823223+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:18:55.370673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825578+0000","last_change":"2023-11-28T12:03:45.825578+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825578+0000","last_undegraded":"2023-11-28T12:03:45.825578+0000","last_fullsized":"2023-11-28T12:03:45.825578+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:49.199732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824871+0000","last_change":"2023-11-28T12:03:45.824871+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824871+0000","last_undegraded":"2023-11-28T12:03:45.824871+0000","last_fullsized":"2023-11-28T12:03:45.824871+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:16.159397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826448+0000","last_change":"2023-11-28T12:03:45.826448+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826448+0000","last_undegraded":"2023-11-28T12:03:45.826448+0000","last_fullsized":"2023-11-28T12:03:45.826448+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:14.009538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826989+0000","last_change":"2023-11-28T12:03:45.826989+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826989+0000","last_undegraded":"2023-11-28T12:03:45.826989+0000","last_fullsized":"2023-11-28T12:03:45.826989+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:01.829853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823534+0000","last_change":"2023-11-28T12:03:45.823534+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823534+0000","last_undegraded":"2023-11-28T12:03:45.823534+0000","last_fullsized":"2023-11-28T12:03:45.823534+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:09.144457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824252+0000","last_change":"2023-11-28T12:03:45.824252+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824252+0000","last_undegraded":"2023-11-28T12:03:45.824252+0000","last_fullsized":"2023-11-28T12:03:45.824252+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:24:00.980890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827390+0000","last_change":"2023-11-28T12:03:45.827390+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827390+0000","last_undegraded":"2023-11-28T12:03:45.827390+0000","last_fullsized":"2023-11-28T12:03:45.827390+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:35.254897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824829+0000","last_change":"2023-11-28T12:03:45.824829+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824829+0000","last_undegraded":"2023-11-28T12:03:45.824829+0000","last_fullsized":"2023-11-28T12:03:45.824829+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:12.372093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827105+0000","last_change":"2023-11-28T12:03:45.827105+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827105+0000","last_undegraded":"2023-11-28T12:03:45.827105+0000","last_fullsized":"2023-11-28T12:03:45.827105+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:52:57.245222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825904+0000","last_change":"2023-11-28T12:03:45.825904+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825904+0000","last_undegraded":"2023-11-28T12:03:45.825904+0000","last_fullsized":"2023-11-28T12:03:45.825904+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:41.743469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823370+0000","last_change":"2023-11-28T12:03:45.823370+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823370+0000","last_undegraded":"2023-11-28T12:03:45.823370+0000","last_fullsized":"2023-11-28T12:03:45.823370+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:49.866215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":38,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.827185+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:03:45.827185+0000","last_peered":"2023-11-28T12:03:45.827185+0000","last_clean":"2023-11-28T12:03:45.827185+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:03:45.827185+0000","last_undegraded":"2023-11-28T12:03:45.827185+0000","last_fullsized":"2023-11-28T12:03:45.827185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823311+0000","last_change":"2023-11-28T12:03:45.823311+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823311+0000","last_undegraded":"2023-11-28T12:03:45.823311+0000","last_fullsized":"2023-11-28T12:03:45.823311+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:35.699402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.821980+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:03:45.821980+0000","last_peered":"2023-11-28T12:03:45.821980+0000","last_clean":"2023-11-28T12:03:45.821980+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:03:45.821980+0000","last_undegraded":"2023-11-28T12:03:45.821980+0000","last_fullsized":"2023-11-28T12:03:45.821980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825622+0000","last_change":"2023-11-28T12:03:45.825622+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825622+0000","last_undegraded":"2023-11-28T12:03:45.825622+0000","last_fullsized":"2023-11-28T12:03:45.825622+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:10.717376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.828667+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:03:45.828667+0000","last_peered":"2023-11-28T12:03:45.828667+0000","last_clean":"2023-11-28T12:03:45.828667+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:03:45.828667+0000","last_undegraded":"2023-11-28T12:03:45.828667+0000","last_fullsized":"2023-11-28T12:03:45.828667+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":184,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362741+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:03:43.362741+0000","last_peered":"2023-11-28T12:03:43.362741+0000","last_clean":"2023-11-28T12:03:43.362741+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:03:43.362741+0000","last_undegraded":"2023-11-28T12:03:43.362741+0000","last_fullsized":"2023-11-28T12:03:43.362741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.828612+0000","last_change":"2023-11-28T12:03:45.828612+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.828612+0000","last_undegraded":"2023-11-28T12:03:45.828612+0000","last_fullsized":"2023-11-28T12:03:45.828612+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:46.170695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":35,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.827814+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:03:45.827814+0000","last_peered":"2023-11-28T12:03:45.827814+0000","last_clean":"2023-11-28T12:03:45.827814+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:03:45.827814+0000","last_undegraded":"2023-11-28T12:03:45.827814+0000","last_fullsized":"2023-11-28T12:03:45.827814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827650+0000","last_change":"2023-11-28T12:03:45.827650+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827650+0000","last_undegraded":"2023-11-28T12:03:45.827650+0000","last_fullsized":"2023-11-28T12:03:45.827650+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:54.463672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362680+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:03:43.362680+0000","last_peered":"2023-11-28T12:03:43.362680+0000","last_clean":"2023-11-28T12:03:43.362680+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:03:43.362680+0000","last_undegraded":"2023-11-28T12:03:43.362680+0000","last_fullsized":"2023-11-28T12:03:43.362680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823878+0000","last_change":"2023-11-28T12:03:45.823878+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823878+0000","last_undegraded":"2023-11-28T12:03:45.823878+0000","last_fullsized":"2023-11-28T12:03:45.823878+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:21.499561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.822971+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:03:45.822971+0000","last_peered":"2023-11-28T12:03:45.822971+0000","last_clean":"2023-11-28T12:03:45.822971+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:03:45.822971+0000","last_undegraded":"2023-11-28T12:03:45.822971+0000","last_fullsized":"2023-11-28T12:03:45.822971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823246+0000","last_change":"2023-11-28T12:03:45.823246+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823246+0000","last_undegraded":"2023-11-28T12:03:45.823246+0000","last_fullsized":"2023-11-28T12:03:45.823246+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:05.212437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362659+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:03:43.362659+0000","last_peered":"2023-11-28T12:03:43.362659+0000","last_clean":"2023-11-28T12:03:43.362659+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:03:43.362659+0000","last_undegraded":"2023-11-28T12:03:43.362659+0000","last_fullsized":"2023-11-28T12:03:43.362659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826474+0000","last_change":"2023-11-28T12:03:45.826474+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826474+0000","last_undegraded":"2023-11-28T12:03:45.826474+0000","last_fullsized":"2023-11-28T12:03:45.826474+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:14.009538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362731+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:03:43.362731+0000","last_peered":"2023-11-28T12:03:43.362731+0000","last_clean":"2023-11-28T12:03:43.362731+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:03:43.362731+0000","last_undegraded":"2023-11-28T12:03:43.362731+0000","last_fullsized":"2023-11-28T12:03:43.362731+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9936,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94361904,"statfs":{"total":96636764160,"available":96626589696,"internally_reserved":0,"allocated":1122304,"data_stored":814570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.66200000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.46000000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.751,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.751}]},{"osd":1,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.872},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.82399999999999995,"15min":0.86699999999999999},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79100000000000004,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.79100000000000004}]},{"osd":2,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.81000000000000005,"15min":0.85099999999999998},"min":{"1min":0.41399999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73499999999999999,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80400000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84199999999999997,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.84199999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76500000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.61199999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.5}]},{"osd":5,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51700000000000002,"15min":0.52300000000000002},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76500000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72699999999999998,"5min":0.752,"15min":0.752},"last":0.52400000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.46600000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.69099999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8664,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363176,"statfs":{"total":96636764160,"available":96627892224,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.435,"5min":0.307,"15min":0.307},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.437}]},{"osd":1,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.39700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.34000000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.53900000000000003,"15min":0.52400000000000002},"min":{"1min":0.47299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.36599999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56100000000000005,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.48099999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.45000000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.497}]},{"osd":5,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.54600000000000004}]},{"osd":7,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.38100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.40200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.56000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9820,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362020,"statfs":{"total":96636764160,"available":96626708480,"internally_reserved":0,"allocated":1134592,"data_stored":820740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.40200000000000002,"15min":0.40999999999999998},"min":{"1min":0.26900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.55100000000000005,"5min":1.2,"15min":1.2},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40400000000000003,"15min":0.41099999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45900000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.30099999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36699999999999999,"15min":0.36899999999999999},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.45100000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.39100000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.47999999999999998,"15min":0.48799999999999999},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.53600000000000003,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.432},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45100000000000001,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67000000000000004,"5min":1.03,"15min":1.03},"last":0.36399999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63700000000000001,"15min":0.64400000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.70699999999999996,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.63300000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.628,"15min":0.63},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78000000000000003,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.82499999999999996,"5min":0.998,"15min":0.998},"last":0.68899999999999995}]},{"osd":6,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.80400000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.77200000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.65700000000000003}]},{"osd":7,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.81200000000000006,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.55200000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.84999999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8792,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363048,"statfs":{"total":96636764160,"available":96627761152,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.55200000000000005,"15min":0.57999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58599999999999997,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45200000000000001,"15min":0.45800000000000002},"min":{"1min":0.28599999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56399999999999995,"5min":1.667,"15min":1.667},"last":0.443}]},{"osd":2,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.57199999999999995,"15min":0.59599999999999997},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.68000000000000005,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.438,"15min":0.44},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44400000000000001,"15min":0.44800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.628,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.64200000000000002,"15min":0.67000000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76100000000000001,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.48399999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63,"15min":0.628},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92400000000000004,"5min":1.536,"15min":1.536},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.75900000000000001,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98199999999999998,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.626}]},{"osd":5,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.56000000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.61799999999999999},"min":{"1min":0.47399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86599999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.53500000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.032,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.501,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.014,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.59299999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92800000000000005,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.52100000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.997,"5min":1.605,"15min":1.605},"last":0.55800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8664,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363176,"statfs":{"total":96636764160,"available":96627892224,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.375},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60499999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39900000000000002,"15min":0.39700000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58199999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.38800000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40899999999999997},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56699999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.439,"15min":0.435},"min":{"1min":0.33500000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.63300000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.316}]},{"osd":4,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81399999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63,"15min":0.63500000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77600000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.79000000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66100000000000003,"15min":0.66900000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83399999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8984,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362856,"statfs":{"total":96636764160,"available":96627564544,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53100000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.44500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58499999999999996,"5min":1.238,"15min":1.238},"last":0.30599999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58699999999999997,"5min":1.53,"15min":1.53},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.502,"15min":0.51600000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57099999999999995,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.36899999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48799999999999999,"15min":0.496},"min":{"1min":0.36199999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55200000000000005,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.47899999999999998,"15min":0.48599999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60499999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77300000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64600000000000002,"15min":0.65200000000000002},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88500000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83699999999999997,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.80200000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86899999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.48199999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84299999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.51300000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.86799999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.81599999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56200000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8920,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362920,"statfs":{"total":96636764160,"available":96627630080,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.503,"15min":0.495},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69099999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.38400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48199999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65900000000000003,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73099999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.435}]},{"osd":2,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73199999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.39800000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52400000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.433,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.752,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.497},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.50700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.746,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73099999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64400000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.35199999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.49299999999999999,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61199999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71199999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41799999999999998}]},{"osd":7,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70999999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8856,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362984,"statfs":{"total":96636764160,"available":96627695616,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.39000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.65800000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.61599999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67000000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.629},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.88300000000000001,"15min":0.94499999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61499999999999999,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.60499999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71499999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57699999999999996,"15min":0.58099999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69999999999999996,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66300000000000003}]},{"osd":3,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.434,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69699999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.70599999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.69499999999999995}]},{"osd":4,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64500000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.38400000000000001}]},{"osd":6,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.35199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.497},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63900000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42099999999999999}]},{"osd":7,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.375,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66200000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.442},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68799999999999994,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:03:51.477 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:03:51.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.649+0000 7f2520454700 1 Processor -- start 2023-11-28T12:03:51.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.649+0000 7f2520454700 1 -- start start 2023-11-28T12:03:51.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.650+0000 7f2520454700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25180ce040 0x7f251815d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.650+0000 7f2520454700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251813fc90 con 0x7f2518142db0 2023-11-28T12:03:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.650+0000 7f2520454700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251813f480 con 0x7f25180b1770 2023-11-28T12:03:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.650+0000 7f2520454700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251813f6d0 con 0x7f25180ce040 2023-11-28T12:03:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.650+0000 7f251d9ef700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2518142db0 0x7f251813ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54938/0 (socket says 172.21.15.57:54938) 2023-11-28T12:03:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.650+0000 7f251d9ef700 1 -- 172.21.15.57:0/2943841229 learned_addr learned my addr 172.21.15.57:0/2943841229 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251e1f0700 1 --2- 172.21.15.57:0/2943841229 >> v2:172.21.15.181:3300/0 conn(0x7f25180ce040 0x7f251815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147096668 0 0) 0x7f251813fc90 con 0x7f2518142db0 2023-11-28T12:03:51.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25040035b0 con 0x7f2518142db0 2023-11-28T12:03:51.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1576802956 0 0) 0x7f251813f480 con 0x7f25180b1770 2023-11-28T12:03:51.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f251813fc90 con 0x7f25180b1770 2023-11-28T12:03:51.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2092136422 0 0) 0x7f25040035b0 con 0x7f2518142db0 2023-11-28T12:03:51.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 >> v2:172.21.15.181:3300/0 conn(0x7f25180ce040 msgr2=0x7f251815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:51.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 --2- 172.21.15.57:0/2943841229 >> v2:172.21.15.181:3300/0 conn(0x7f25180ce040 0x7f251815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 >> v1:172.21.15.181:6789/0 conn(0x7f25180b1770 legacy=0x7f25180b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251813f9b0 con 0x7f2518142db0 2023-11-28T12:03:51.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f250c001670 con 0x7f2518142db0 2023-11-28T12:03:51.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 >> v1:172.21.15.57:6789/0 conn(0x7f2518142db0 legacy=0x7f251813ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 --2- 172.21.15.57:0/2943841229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2504003be0 0x7f2504003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:51.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 --2- 172.21.15.57:0/2943841229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2504004850 0x7f2504004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:51.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 --2- 172.21.15.57:0/2943841229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2504005200 0x7f250400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:51.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251813f6d0 con 0x7f2504003be0 2023-11-28T12:03:51.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250400c0e0 con 0x7f2504005200 2023-11-28T12:03:51.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250400c270 con 0x7f2504004850 2023-11-28T12:03:51.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251e1f0700 1 --2- 172.21.15.57:0/2943841229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2504003be0 0x7f2504003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:51.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251d9ef700 1 --2- 172.21.15.57:0/2943841229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2504004850 0x7f2504004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:51.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251e9f1700 1 --2- 172.21.15.57:0/2943841229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2504005200 0x7f250400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:51.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251e1f0700 1 -- 172.21.15.57:0/2943841229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2504004850 msgr2=0x7f2504004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251e1f0700 1 --2- 172.21.15.57:0/2943841229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2504004850 0x7f2504004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251e1f0700 1 -- 172.21.15.57:0/2943841229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2504005200 msgr2=0x7f250400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.651+0000 7f251e1f0700 1 --2- 172.21.15.57:0/2943841229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2504005200 0x7f250400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.652+0000 7f251e1f0700 1 -- 172.21.15.57:0/2943841229 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25040035b0 con 0x7f2504003be0 2023-11-28T12:03:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.652+0000 7f251e9f1700 1 -- 172.21.15.57:0/2943841229 reap_dead start 2023-11-28T12:03:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.652+0000 7f251e1f0700 1 --2- 172.21.15.57:0/2943841229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2504003be0 0x7f2504003fd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2514009b30 tx=0x7f2504005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ad72fe05fc95e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.652+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251400e6c0 con 0x7f2504003be0 2023-11-28T12:03:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.652+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251400e820 con 0x7f2504003be0 2023-11-28T12:03:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.652+0000 7f251d1ee700 1 -- 172.21.15.57:0/2943841229 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251400eb90 con 0x7f2504003be0 2023-11-28T12:03:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.652+0000 7f2520454700 1 -- 172.21.15.57:0/2943841229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2504003be0 msgr2=0x7f2504003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.652+0000 7f2520454700 1 --2- 172.21.15.57:0/2943841229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2504003be0 0x7f2504003fd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2514009b30 tx=0x7f2504005880 comp rx=0 tx=0).stop 2023-11-28T12:03:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.653+0000 7f2520454700 1 -- 172.21.15.57:0/2943841229 shutdown_connections 2023-11-28T12:03:51.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.653+0000 7f2520454700 1 --2- 172.21.15.57:0/2943841229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2504003be0 0x7f2504003fd0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.653+0000 7f2520454700 1 -- 172.21.15.57:0/2943841229 >> 172.21.15.57:0/2943841229 conn(0x7f251806c010 msgr2=0x7f25180cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:51.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.653+0000 7f2520454700 1 -- 172.21.15.57:0/2943841229 shutdown_connections 2023-11-28T12:03:51.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.653+0000 7f2520454700 1 -- 172.21.15.57:0/2943841229 wait complete. 2023-11-28T12:03:51.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.654+0000 7f2520454700 1 Processor -- start 2023-11-28T12:03:51.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.654+0000 7f2520454700 1 -- start start 2023-11-28T12:03:51.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f2520454700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2504003be0 0x7f25181f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:51.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f2520454700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25180c1310 0x7f251813cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:51.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f2520454700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f251814fd00 0x7f25180c0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:51.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f2520454700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251813d440 con 0x7f25180c1310 2023-11-28T12:03:51.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f2520454700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251813be60 con 0x7f2504003be0 2023-11-28T12:03:51.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f2520454700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251813c140 con 0x7f251814fd00 2023-11-28T12:03:51.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f251d9ef700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25180c1310 0x7f251813cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:51.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f251d9ef700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25180c1310 0x7f251813cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44454/0 (socket says 172.21.15.57:44454) 2023-11-28T12:03:51.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f251d9ef700 1 -- 172.21.15.57:0/38459395 learned_addr learned my addr 172.21.15.57:0/38459395 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:51.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f251e9f1700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f251814fd00 0x7f25180c0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:51.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f251e1f0700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2504003be0 0x7f25181f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:51.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f251d9ef700 1 -- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f251814fd00 msgr2=0x7f25180c0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f251d9ef700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f251814fd00 0x7f25180c0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f251d9ef700 1 -- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2504003be0 msgr2=0x7f25181f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f251d9ef700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2504003be0 0x7f25181f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f251d9ef700 1 -- 172.21.15.57:0/38459395 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251815d3e0 con 0x7f25180c1310 2023-11-28T12:03:51.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.655+0000 7f251d9ef700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25180c1310 0x7f251813cf00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f250c000fb0 tx=0x7f250c007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:51.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.656+0000 7f250affd700 1 -- 172.21.15.57:0/38459395 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c011580 con 0x7f25180c1310 2023-11-28T12:03:51.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.656+0000 7f250affd700 1 -- 172.21.15.57:0/38459395 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c011700 con 0x7f25180c1310 2023-11-28T12:03:51.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.656+0000 7f2520454700 1 -- 172.21.15.57:0/38459395 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251815d100 con 0x7f25180c1310 2023-11-28T12:03:51.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.656+0000 7f250affd700 1 -- 172.21.15.57:0/38459395 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c011ab0 con 0x7f25180c1310 2023-11-28T12:03:51.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.656+0000 7f2520454700 1 -- 172.21.15.57:0/38459395 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25180ce410 con 0x7f25180c1310 2023-11-28T12:03:51.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.657+0000 7f2520454700 1 -- 172.21.15.57:0/38459395 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25180743f0 con 0x7f25180c1310 2023-11-28T12:03:51.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.657+0000 7f250affd700 1 -- 172.21.15.57:0/38459395 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f250c011c10 con 0x7f25180c1310 2023-11-28T12:03:51.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.658+0000 7f250affd700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f24fc068320 0x7f24fc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:51.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.658+0000 7f250affd700 1 -- 172.21.15.57:0/38459395 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f250c085d80 con 0x7f25180c1310 2023-11-28T12:03:51.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.658+0000 7f251e1f0700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f24fc068320 0x7f24fc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:51.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.658+0000 7f251e1f0700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f24fc068320 0x7f24fc06a7d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f25140090b0 tx=0x7f251401f040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:51.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.662+0000 7f250affd700 1 -- 172.21.15.57:0/38459395 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f250c052560 con 0x7f25180c1310 2023-11-28T12:03:51.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.855+0000 7f2520454700 1 -- 172.21.15.57:0/38459395 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f25180cdae0 con 0x7f24fc068320 2023-11-28T12:03:51.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.864+0000 7f250affd700 1 -- 172.21.15.57:0/38459395 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+301454 (secure 0 0 0) 0x7f25180cdae0 con 0x7f24fc068320 2023-11-28T12:03:51.873 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:51.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.868+0000 7f2520454700 1 -- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f24fc068320 msgr2=0x7f24fc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.868+0000 7f2520454700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f24fc068320 0x7f24fc06a7d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f25140090b0 tx=0x7f251401f040 comp rx=0 tx=0).stop 2023-11-28T12:03:51.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.868+0000 7f2520454700 1 -- 172.21.15.57:0/38459395 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25180c1310 msgr2=0x7f251813cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:51.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.868+0000 7f2520454700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25180c1310 0x7f251813cf00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f250c000fb0 tx=0x7f250c007570 comp rx=0 tx=0).stop 2023-11-28T12:03:51.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.868+0000 7f2520454700 1 -- 172.21.15.57:0/38459395 shutdown_connections 2023-11-28T12:03:51.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.868+0000 7f2520454700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f24fc068320 0x7f24fc06a7d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.868+0000 7f2520454700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f251814fd00 0x7f25180c0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.868+0000 7f2520454700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2504003be0 0x7f25181f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.868+0000 7f2520454700 1 --2- 172.21.15.57:0/38459395 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25180c1310 0x7f251813cf00 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:51.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.868+0000 7f2520454700 1 -- 172.21.15.57:0/38459395 >> 172.21.15.57:0/38459395 conn(0x7f251806c010 msgr2=0x7f25180c2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:51.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.868+0000 7f2520454700 1 -- 172.21.15.57:0/38459395 shutdown_connections 2023-11-28T12:03:51.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:51.868+0000 7f2520454700 1 -- 172.21.15.57:0/38459395 wait complete. 2023-11-28T12:03:51.877 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:03:51.882 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-11-28T12:03:51.690936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":72636,"kb_used_data":5372,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":754902084,"statfs":{"total":773094113280,"available":773019734016,"internally_reserved":0,"allocated":5500928,"data_stored":3017542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.869491"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824423+0000","last_change":"2023-11-28T12:03:45.824423+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824423+0000","last_undegraded":"2023-11-28T12:03:45.824423+0000","last_fullsized":"2023-11-28T12:03:45.824423+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:00.642126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827796+0000","last_change":"2023-11-28T12:03:45.827796+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827796+0000","last_undegraded":"2023-11-28T12:03:45.827796+0000","last_fullsized":"2023-11-28T12:03:45.827796+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:39.291687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827234+0000","last_change":"2023-11-28T12:03:45.827234+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827234+0000","last_undegraded":"2023-11-28T12:03:45.827234+0000","last_fullsized":"2023-11-28T12:03:45.827234+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:59.530852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823414+0000","last_change":"2023-11-28T12:03:45.823414+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823414+0000","last_undegraded":"2023-11-28T12:03:45.823414+0000","last_fullsized":"2023-11-28T12:03:45.823414+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:10.717376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.828214+0000","last_change":"2023-11-28T12:03:45.828214+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.828214+0000","last_undegraded":"2023-11-28T12:03:45.828214+0000","last_fullsized":"2023-11-28T12:03:45.828214+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:56.656303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826655+0000","last_change":"2023-11-28T12:03:45.826655+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826655+0000","last_undegraded":"2023-11-28T12:03:45.826655+0000","last_fullsized":"2023-11-28T12:03:45.826655+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:46.170695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823319+0000","last_change":"2023-11-28T12:03:45.823319+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823319+0000","last_undegraded":"2023-11-28T12:03:45.823319+0000","last_fullsized":"2023-11-28T12:03:45.823319+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:13:44.179722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826298+0000","last_change":"2023-11-28T12:03:45.826298+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826298+0000","last_undegraded":"2023-11-28T12:03:45.826298+0000","last_fullsized":"2023-11-28T12:03:45.826298+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:16.229157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825677+0000","last_change":"2023-11-28T12:03:45.825677+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825677+0000","last_undegraded":"2023-11-28T12:03:45.825677+0000","last_fullsized":"2023-11-28T12:03:45.825677+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:57.070237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826611+0000","last_change":"2023-11-28T12:03:45.826611+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826611+0000","last_undegraded":"2023-11-28T12:03:45.826611+0000","last_fullsized":"2023-11-28T12:03:45.826611+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:40.312689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827494+0000","last_change":"2023-11-28T12:03:45.827494+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827494+0000","last_undegraded":"2023-11-28T12:03:45.827494+0000","last_fullsized":"2023-11-28T12:03:45.827494+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:05.212437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824983+0000","last_change":"2023-11-28T12:03:45.824983+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824983+0000","last_undegraded":"2023-11-28T12:03:45.824983+0000","last_fullsized":"2023-11-28T12:03:45.824983+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:25.548629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826279+0000","last_change":"2023-11-28T12:03:45.826279+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826279+0000","last_undegraded":"2023-11-28T12:03:45.826279+0000","last_fullsized":"2023-11-28T12:03:45.826279+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:56:00.591608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823223+0000","last_change":"2023-11-28T12:03:45.823223+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823223+0000","last_undegraded":"2023-11-28T12:03:45.823223+0000","last_fullsized":"2023-11-28T12:03:45.823223+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:18:55.370673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825578+0000","last_change":"2023-11-28T12:03:45.825578+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825578+0000","last_undegraded":"2023-11-28T12:03:45.825578+0000","last_fullsized":"2023-11-28T12:03:45.825578+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:49.199732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824871+0000","last_change":"2023-11-28T12:03:45.824871+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824871+0000","last_undegraded":"2023-11-28T12:03:45.824871+0000","last_fullsized":"2023-11-28T12:03:45.824871+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:16.159397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826448+0000","last_change":"2023-11-28T12:03:45.826448+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826448+0000","last_undegraded":"2023-11-28T12:03:45.826448+0000","last_fullsized":"2023-11-28T12:03:45.826448+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:14.009538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826989+0000","last_change":"2023-11-28T12:03:45.826989+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826989+0000","last_undegraded":"2023-11-28T12:03:45.826989+0000","last_fullsized":"2023-11-28T12:03:45.826989+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:01.829853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823534+0000","last_change":"2023-11-28T12:03:45.823534+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823534+0000","last_undegraded":"2023-11-28T12:03:45.823534+0000","last_fullsized":"2023-11-28T12:03:45.823534+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:09.144457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824252+0000","last_change":"2023-11-28T12:03:45.824252+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824252+0000","last_undegraded":"2023-11-28T12:03:45.824252+0000","last_fullsized":"2023-11-28T12:03:45.824252+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:24:00.980890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827390+0000","last_change":"2023-11-28T12:03:45.827390+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827390+0000","last_undegraded":"2023-11-28T12:03:45.827390+0000","last_fullsized":"2023-11-28T12:03:45.827390+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:35.254897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824829+0000","last_change":"2023-11-28T12:03:45.824829+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824829+0000","last_undegraded":"2023-11-28T12:03:45.824829+0000","last_fullsized":"2023-11-28T12:03:45.824829+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:12.372093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827105+0000","last_change":"2023-11-28T12:03:45.827105+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827105+0000","last_undegraded":"2023-11-28T12:03:45.827105+0000","last_fullsized":"2023-11-28T12:03:45.827105+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:52:57.245222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825904+0000","last_change":"2023-11-28T12:03:45.825904+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825904+0000","last_undegraded":"2023-11-28T12:03:45.825904+0000","last_fullsized":"2023-11-28T12:03:45.825904+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:41.743469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823370+0000","last_change":"2023-11-28T12:03:45.823370+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823370+0000","last_undegraded":"2023-11-28T12:03:45.823370+0000","last_fullsized":"2023-11-28T12:03:45.823370+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:49.866215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":38,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.827185+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:03:45.827185+0000","last_peered":"2023-11-28T12:03:45.827185+0000","last_clean":"2023-11-28T12:03:45.827185+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:03:45.827185+0000","last_undegraded":"2023-11-28T12:03:45.827185+0000","last_fullsized":"2023-11-28T12:03:45.827185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823311+0000","last_change":"2023-11-28T12:03:45.823311+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823311+0000","last_undegraded":"2023-11-28T12:03:45.823311+0000","last_fullsized":"2023-11-28T12:03:45.823311+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:35.699402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.821980+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:03:45.821980+0000","last_peered":"2023-11-28T12:03:45.821980+0000","last_clean":"2023-11-28T12:03:45.821980+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:03:45.821980+0000","last_undegraded":"2023-11-28T12:03:45.821980+0000","last_fullsized":"2023-11-28T12:03:45.821980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825622+0000","last_change":"2023-11-28T12:03:45.825622+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825622+0000","last_undegraded":"2023-11-28T12:03:45.825622+0000","last_fullsized":"2023-11-28T12:03:45.825622+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:10.717376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.828667+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:03:45.828667+0000","last_peered":"2023-11-28T12:03:45.828667+0000","last_clean":"2023-11-28T12:03:45.828667+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:03:45.828667+0000","last_undegraded":"2023-11-28T12:03:45.828667+0000","last_fullsized":"2023-11-28T12:03:45.828667+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":184,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362741+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:03:43.362741+0000","last_peered":"2023-11-28T12:03:43.362741+0000","last_clean":"2023-11-28T12:03:43.362741+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:03:43.362741+0000","last_undegraded":"2023-11-28T12:03:43.362741+0000","last_fullsized":"2023-11-28T12:03:43.362741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.828612+0000","last_change":"2023-11-28T12:03:45.828612+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.828612+0000","last_undegraded":"2023-11-28T12:03:45.828612+0000","last_fullsized":"2023-11-28T12:03:45.828612+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:46.170695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":35,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.827814+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:03:45.827814+0000","last_peered":"2023-11-28T12:03:45.827814+0000","last_clean":"2023-11-28T12:03:45.827814+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:03:45.827814+0000","last_undegraded":"2023-11-28T12:03:45.827814+0000","last_fullsized":"2023-11-28T12:03:45.827814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827650+0000","last_change":"2023-11-28T12:03:45.827650+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827650+0000","last_undegraded":"2023-11-28T12:03:45.827650+0000","last_fullsized":"2023-11-28T12:03:45.827650+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:54.463672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362680+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:03:43.362680+0000","last_peered":"2023-11-28T12:03:43.362680+0000","last_clean":"2023-11-28T12:03:43.362680+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:03:43.362680+0000","last_undegraded":"2023-11-28T12:03:43.362680+0000","last_fullsized":"2023-11-28T12:03:43.362680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823878+0000","last_change":"2023-11-28T12:03:45.823878+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823878+0000","last_undegraded":"2023-11-28T12:03:45.823878+0000","last_fullsized":"2023-11-28T12:03:45.823878+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:21.499561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.822971+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:03:45.822971+0000","last_peered":"2023-11-28T12:03:45.822971+0000","last_clean":"2023-11-28T12:03:45.822971+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:03:45.822971+0000","last_undegraded":"2023-11-28T12:03:45.822971+0000","last_fullsized":"2023-11-28T12:03:45.822971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823246+0000","last_change":"2023-11-28T12:03:45.823246+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823246+0000","last_undegraded":"2023-11-28T12:03:45.823246+0000","last_fullsized":"2023-11-28T12:03:45.823246+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:05.212437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362659+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:03:43.362659+0000","last_peered":"2023-11-28T12:03:43.362659+0000","last_clean":"2023-11-28T12:03:43.362659+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:03:43.362659+0000","last_undegraded":"2023-11-28T12:03:43.362659+0000","last_fullsized":"2023-11-28T12:03:43.362659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826474+0000","last_change":"2023-11-28T12:03:45.826474+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826474+0000","last_undegraded":"2023-11-28T12:03:45.826474+0000","last_fullsized":"2023-11-28T12:03:45.826474+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:14.009538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362731+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:03:43.362731+0000","last_peered":"2023-11-28T12:03:43.362731+0000","last_clean":"2023-11-28T12:03:43.362731+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:03:43.362731+0000","last_undegraded":"2023-11-28T12:03:43.362731+0000","last_fullsized":"2023-11-28T12:03:43.362731+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9936,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94361904,"statfs":{"total":96636764160,"available":96626589696,"internally_reserved":0,"allocated":1122304,"data_stored":814570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.66200000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.46000000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.751,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.751}]},{"osd":1,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.872},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.82399999999999995,"15min":0.86699999999999999},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79100000000000004,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.79100000000000004}]},{"osd":2,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.81000000000000005,"15min":0.85099999999999998},"min":{"1min":0.41399999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73499999999999999,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80400000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84199999999999997,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.84199999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76500000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.61199999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.5}]},{"osd":5,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51700000000000002,"15min":0.52300000000000002},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76500000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72699999999999998,"5min":0.752,"15min":0.752},"last":0.52400000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.46600000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.69099999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8664,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363176,"statfs":{"total":96636764160,"available":96627892224,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.435,"5min":0.307,"15min":0.307},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.437}]},{"osd":1,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.39700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.34000000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.53900000000000003,"15min":0.52400000000000002},"min":{"1min":0.47299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.36599999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56100000000000005,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.48099999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.45000000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.497}]},{"osd":5,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.54600000000000004}]},{"osd":7,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.38100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.40200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.56000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9820,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362020,"statfs":{"total":96636764160,"available":96626708480,"internally_reserved":0,"allocated":1134592,"data_stored":820740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.40200000000000002,"15min":0.40999999999999998},"min":{"1min":0.26900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.55100000000000005,"5min":1.2,"15min":1.2},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40400000000000003,"15min":0.41099999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45900000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.30099999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36699999999999999,"15min":0.36899999999999999},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.45100000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.39100000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.47999999999999998,"15min":0.48799999999999999},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.53600000000000003,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.432},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45100000000000001,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67000000000000004,"5min":1.03,"15min":1.03},"last":0.36399999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63700000000000001,"15min":0.64400000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.70699999999999996,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.63300000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.628,"15min":0.63},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78000000000000003,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.82499999999999996,"5min":0.998,"15min":0.998},"last":0.68899999999999995}]},{"osd":6,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.80400000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.77200000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.65700000000000003}]},{"osd":7,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.81200000000000006,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.55200000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.84999999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8792,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363048,"statfs":{"total":96636764160,"available":96627761152,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.55200000000000005,"15min":0.57999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58599999999999997,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45200000000000001,"15min":0.45800000000000002},"min":{"1min":0.28599999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56399999999999995,"5min":1.667,"15min":1.667},"last":0.443}]},{"osd":2,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.57199999999999995,"15min":0.59599999999999997},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.68000000000000005,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.438,"15min":0.44},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44400000000000001,"15min":0.44800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.628,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.64200000000000002,"15min":0.67000000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76100000000000001,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.48399999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63,"15min":0.628},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92400000000000004,"5min":1.536,"15min":1.536},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.75900000000000001,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98199999999999998,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.626}]},{"osd":5,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.56000000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.61799999999999999},"min":{"1min":0.47399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86599999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.53500000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.032,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.501,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.014,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.59299999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92800000000000005,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.52100000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.997,"5min":1.605,"15min":1.605},"last":0.55800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8664,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363176,"statfs":{"total":96636764160,"available":96627892224,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.375},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60499999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39900000000000002,"15min":0.39700000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58199999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.38800000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40899999999999997},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56699999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.439,"15min":0.435},"min":{"1min":0.33500000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.63300000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.316}]},{"osd":4,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81399999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63,"15min":0.63500000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77600000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.79000000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66100000000000003,"15min":0.66900000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83399999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8984,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362856,"statfs":{"total":96636764160,"available":96627564544,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53100000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.44500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58499999999999996,"5min":1.238,"15min":1.238},"last":0.30599999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58699999999999997,"5min":1.53,"15min":1.53},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.502,"15min":0.51600000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57099999999999995,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.36899999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48799999999999999,"15min":0.496},"min":{"1min":0.36199999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55200000000000005,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.47899999999999998,"15min":0.48599999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60499999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77300000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64600000000000002,"15min":0.65200000000000002},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88500000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83699999999999997,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.80200000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86899999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.48199999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84299999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.51300000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.86799999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.81599999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56200000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8920,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362920,"statfs":{"total":96636764160,"available":96627630080,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.503,"15min":0.495},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69099999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.38400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48199999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65900000000000003,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73099999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.435}]},{"osd":2,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73199999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.39800000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52400000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.433,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.752,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.497},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.50700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.746,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73099999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64400000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.35199999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.49299999999999999,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61199999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71199999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41799999999999998}]},{"osd":7,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70999999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8856,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362984,"statfs":{"total":96636764160,"available":96627695616,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.39000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.65800000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.61599999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67000000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.629},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.88300000000000001,"15min":0.94499999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61499999999999999,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.60499999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71499999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57699999999999996,"15min":0.58099999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69999999999999996,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66300000000000003}]},{"osd":3,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.434,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69699999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.70599999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.69499999999999995}]},{"osd":4,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64500000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.38400000000000001}]},{"osd":6,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.35199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.497},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63900000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42099999999999999}]},{"osd":7,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.375,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66200000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.442},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68799999999999994,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:03:51.888 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:03:52.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.060+0000 7f63ab2a1700 1 Processor -- start 2023-11-28T12:03:52.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.061+0000 7f63ab2a1700 1 -- start start 2023-11-28T12:03:52.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63ab2a1700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f63a415cc80 0x7f63a406de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63ab2a1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a406e3a0 0x7f63a4070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63ab2a1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a40b1810 con 0x7f63a406a270 2023-11-28T12:03:52.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63ab2a1700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a40b2490 con 0x7f63a415cc80 2023-11-28T12:03:52.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63ab2a1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a40b1af0 con 0x7f63a406e3a0 2023-11-28T12:03:52.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63a883c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f63a415cc80 0x7f63a406de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63a883c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f63a415cc80 0x7f63a406de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44470/0 (socket says 172.21.15.57:44470) 2023-11-28T12:03:52.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63a883c700 1 -- 172.21.15.57:0/4149163408 learned_addr learned my addr 172.21.15.57:0/4149163408 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:52.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63a983e700 1 --2- 172.21.15.57:0/4149163408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a406e3a0 0x7f63a4070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63a883c700 1 -- 172.21.15.57:0/4149163408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a406e3a0 msgr2=0x7f63a4070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:52.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63a883c700 1 --2- 172.21.15.57:0/4149163408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a406e3a0 0x7f63a4070d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63a883c700 1 -- 172.21.15.57:0/4149163408 >> v1:172.21.15.181:6789/0 conn(0x7f63a406a270 legacy=0x7f63a406a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f639bfff700 1 -- 172.21.15.57:0/4149163408 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282487080 0 0) 0x7f63a40b1810 con 0x7f63a406a270 2023-11-28T12:03:52.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63a883c700 1 -- 172.21.15.57:0/4149163408 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a40b1dd0 con 0x7f63a415cc80 2023-11-28T12:03:52.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.062+0000 7f63a883c700 1 --2- 172.21.15.57:0/4149163408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63a415cc80 0x7f63a406de60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f63940098f0 tx=0x7f6394009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6114e1c7bf9405fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:52.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.063+0000 7f639bfff700 1 -- 172.21.15.57:0/4149163408 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394011830 con 0x7f63a415cc80 2023-11-28T12:03:52.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.063+0000 7f639bfff700 1 -- 172.21.15.57:0/4149163408 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6394011990 con 0x7f63a415cc80 2023-11-28T12:03:52.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.063+0000 7f639bfff700 1 -- 172.21.15.57:0/4149163408 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394011d20 con 0x7f63a415cc80 2023-11-28T12:03:52.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.063+0000 7f63ab2a1700 1 -- 172.21.15.57:0/4149163408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63a415cc80 msgr2=0x7f63a406de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.063+0000 7f63ab2a1700 1 --2- 172.21.15.57:0/4149163408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63a415cc80 0x7f63a406de60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f63940098f0 tx=0x7f6394009c00 comp rx=0 tx=0).stop 2023-11-28T12:03:52.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.063+0000 7f63ab2a1700 1 -- 172.21.15.57:0/4149163408 shutdown_connections 2023-11-28T12:03:52.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.063+0000 7f63ab2a1700 1 --2- 172.21.15.57:0/4149163408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a406e3a0 0x7f63a4070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.063+0000 7f63ab2a1700 1 --2- 172.21.15.57:0/4149163408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63a415cc80 0x7f63a406de60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.063+0000 7f63ab2a1700 1 -- 172.21.15.57:0/4149163408 >> 172.21.15.57:0/4149163408 conn(0x7f63a406c010 msgr2=0x7f63a40cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:52.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.063+0000 7f63ab2a1700 1 -- 172.21.15.57:0/4149163408 shutdown_connections 2023-11-28T12:03:52.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.063+0000 7f63ab2a1700 1 -- 172.21.15.57:0/4149163408 wait complete. 2023-11-28T12:03:52.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.065+0000 7f63ab2a1700 1 Processor -- start 2023-11-28T12:03:52.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.065+0000 7f63ab2a1700 1 -- start start 2023-11-28T12:03:52.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.065+0000 7f63ab2a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a406a270 0x7f63a4152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.065+0000 7f63ab2a1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63a406e3a0 0x7f63a40b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.065+0000 7f63ab2a1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a415cc80 0x7f63a415edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.065+0000 7f63ab2a1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a41f2750 con 0x7f63a406e3a0 2023-11-28T12:03:52.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.065+0000 7f63ab2a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a4153550 con 0x7f63a406a270 2023-11-28T12:03:52.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.065+0000 7f63ab2a1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a40b2f70 con 0x7f63a415cc80 2023-11-28T12:03:52.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.065+0000 7f63a883c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63a406e3a0 0x7f63a40b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f63a883c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63a406e3a0 0x7f63a40b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44482/0 (socket says 172.21.15.57:44482) 2023-11-28T12:03:52.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f63a883c700 1 -- 172.21.15.57:0/3119101570 learned_addr learned my addr 172.21.15.57:0/3119101570 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:52.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f63a903d700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a406a270 0x7f63a4152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f63a983e700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a415cc80 0x7f63a415edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f63a883c700 1 -- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a415cc80 msgr2=0x7f63a415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f63a883c700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a415cc80 0x7f63a415edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f63a883c700 1 -- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a406a270 msgr2=0x7f63a4152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f63a883c700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a406a270 0x7f63a4152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f63a883c700 1 -- 172.21.15.57:0/3119101570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394009550 con 0x7f63a406e3a0 2023-11-28T12:03:52.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f63a883c700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63a406e3a0 0x7f63a40b34c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f63a4067450 tx=0x7f6394004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:52.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f6399ffb700 1 -- 172.21.15.57:0/3119101570 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394004510 con 0x7f63a406e3a0 2023-11-28T12:03:52.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f6399ffb700 1 -- 172.21.15.57:0/3119101570 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63940036a0 con 0x7f63a406e3a0 2023-11-28T12:03:52.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f63ab2a1700 1 -- 172.21.15.57:0/3119101570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a406e800 con 0x7f63a406e3a0 2023-11-28T12:03:52.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.066+0000 7f6399ffb700 1 -- 172.21.15.57:0/3119101570 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394003a10 con 0x7f63a406e3a0 2023-11-28T12:03:52.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.067+0000 7f63ab2a1700 1 -- 172.21.15.57:0/3119101570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a40b20b0 con 0x7f63a406e3a0 2023-11-28T12:03:52.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.067+0000 7f63ab2a1700 1 -- 172.21.15.57:0/3119101570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a415daa0 con 0x7f63a406e3a0 2023-11-28T12:03:52.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.068+0000 7f6399ffb700 1 -- 172.21.15.57:0/3119101570 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f639402a020 con 0x7f63a406e3a0 2023-11-28T12:03:52.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.068+0000 7f6399ffb700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63880682f0 0x7f638806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.069+0000 7f6399ffb700 1 -- 172.21.15.57:0/3119101570 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6394087160 con 0x7f63a406e3a0 2023-11-28T12:03:52.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.069+0000 7f63a903d700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63880682f0 0x7f638806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.069+0000 7f63a903d700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63880682f0 0x7f638806a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f63a0000f60 tx=0x7f63a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:52.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.073+0000 7f6399ffb700 1 -- 172.21.15.57:0/3119101570 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6394053940 con 0x7f63a406e3a0 2023-11-28T12:03:52.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.267+0000 7f63ab2a1700 1 -- 172.21.15.57:0/3119101570 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63a40743f0 con 0x7f63880682f0 2023-11-28T12:03:52.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.275+0000 7f6399ffb700 1 -- 172.21.15.57:0/3119101570 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+301454 (secure 0 0 0) 0x7f63a40743f0 con 0x7f63880682f0 2023-11-28T12:03:52.281 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:52.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.279+0000 7f63ab2a1700 1 -- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63880682f0 msgr2=0x7f638806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.279+0000 7f63ab2a1700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63880682f0 0x7f638806a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f63a0000f60 tx=0x7f63a0009040 comp rx=0 tx=0).stop 2023-11-28T12:03:52.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.279+0000 7f63ab2a1700 1 -- 172.21.15.57:0/3119101570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63a406e3a0 msgr2=0x7f63a40b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.279+0000 7f63ab2a1700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63a406e3a0 0x7f63a40b34c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f63a4067450 tx=0x7f6394004750 comp rx=0 tx=0).stop 2023-11-28T12:03:52.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.279+0000 7f63ab2a1700 1 -- 172.21.15.57:0/3119101570 shutdown_connections 2023-11-28T12:03:52.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.279+0000 7f63ab2a1700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63880682f0 0x7f638806a7a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.279+0000 7f63ab2a1700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a415cc80 0x7f63a415edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.279+0000 7f63ab2a1700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a406a270 0x7f63a4152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.279+0000 7f63ab2a1700 1 --2- 172.21.15.57:0/3119101570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63a406e3a0 0x7f63a40b34c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.279+0000 7f63ab2a1700 1 -- 172.21.15.57:0/3119101570 >> 172.21.15.57:0/3119101570 conn(0x7f63a406c010 msgr2=0x7f63a406da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:52.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.279+0000 7f63ab2a1700 1 -- 172.21.15.57:0/3119101570 shutdown_connections 2023-11-28T12:03:52.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.279+0000 7f63ab2a1700 1 -- 172.21.15.57:0/3119101570 wait complete. 2023-11-28T12:03:52.287 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:03:52.293 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-11-28T12:03:51.690936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":72636,"kb_used_data":5372,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":754902084,"statfs":{"total":773094113280,"available":773019734016,"internally_reserved":0,"allocated":5500928,"data_stored":3017542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.869491"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824423+0000","last_change":"2023-11-28T12:03:45.824423+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824423+0000","last_undegraded":"2023-11-28T12:03:45.824423+0000","last_fullsized":"2023-11-28T12:03:45.824423+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:00.642126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827796+0000","last_change":"2023-11-28T12:03:45.827796+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827796+0000","last_undegraded":"2023-11-28T12:03:45.827796+0000","last_fullsized":"2023-11-28T12:03:45.827796+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:39.291687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827234+0000","last_change":"2023-11-28T12:03:45.827234+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827234+0000","last_undegraded":"2023-11-28T12:03:45.827234+0000","last_fullsized":"2023-11-28T12:03:45.827234+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:59.530852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823414+0000","last_change":"2023-11-28T12:03:45.823414+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823414+0000","last_undegraded":"2023-11-28T12:03:45.823414+0000","last_fullsized":"2023-11-28T12:03:45.823414+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:10.717376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.828214+0000","last_change":"2023-11-28T12:03:45.828214+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.828214+0000","last_undegraded":"2023-11-28T12:03:45.828214+0000","last_fullsized":"2023-11-28T12:03:45.828214+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:56.656303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826655+0000","last_change":"2023-11-28T12:03:45.826655+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826655+0000","last_undegraded":"2023-11-28T12:03:45.826655+0000","last_fullsized":"2023-11-28T12:03:45.826655+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:46.170695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823319+0000","last_change":"2023-11-28T12:03:45.823319+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823319+0000","last_undegraded":"2023-11-28T12:03:45.823319+0000","last_fullsized":"2023-11-28T12:03:45.823319+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:13:44.179722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826298+0000","last_change":"2023-11-28T12:03:45.826298+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826298+0000","last_undegraded":"2023-11-28T12:03:45.826298+0000","last_fullsized":"2023-11-28T12:03:45.826298+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:16.229157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825677+0000","last_change":"2023-11-28T12:03:45.825677+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825677+0000","last_undegraded":"2023-11-28T12:03:45.825677+0000","last_fullsized":"2023-11-28T12:03:45.825677+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:57.070237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826611+0000","last_change":"2023-11-28T12:03:45.826611+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826611+0000","last_undegraded":"2023-11-28T12:03:45.826611+0000","last_fullsized":"2023-11-28T12:03:45.826611+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:40.312689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827494+0000","last_change":"2023-11-28T12:03:45.827494+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827494+0000","last_undegraded":"2023-11-28T12:03:45.827494+0000","last_fullsized":"2023-11-28T12:03:45.827494+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:05.212437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824983+0000","last_change":"2023-11-28T12:03:45.824983+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824983+0000","last_undegraded":"2023-11-28T12:03:45.824983+0000","last_fullsized":"2023-11-28T12:03:45.824983+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:25.548629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826279+0000","last_change":"2023-11-28T12:03:45.826279+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826279+0000","last_undegraded":"2023-11-28T12:03:45.826279+0000","last_fullsized":"2023-11-28T12:03:45.826279+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:56:00.591608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823223+0000","last_change":"2023-11-28T12:03:45.823223+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823223+0000","last_undegraded":"2023-11-28T12:03:45.823223+0000","last_fullsized":"2023-11-28T12:03:45.823223+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:18:55.370673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825578+0000","last_change":"2023-11-28T12:03:45.825578+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825578+0000","last_undegraded":"2023-11-28T12:03:45.825578+0000","last_fullsized":"2023-11-28T12:03:45.825578+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:49.199732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824871+0000","last_change":"2023-11-28T12:03:45.824871+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824871+0000","last_undegraded":"2023-11-28T12:03:45.824871+0000","last_fullsized":"2023-11-28T12:03:45.824871+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:16.159397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826448+0000","last_change":"2023-11-28T12:03:45.826448+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826448+0000","last_undegraded":"2023-11-28T12:03:45.826448+0000","last_fullsized":"2023-11-28T12:03:45.826448+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:14.009538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826989+0000","last_change":"2023-11-28T12:03:45.826989+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826989+0000","last_undegraded":"2023-11-28T12:03:45.826989+0000","last_fullsized":"2023-11-28T12:03:45.826989+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:01.829853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823534+0000","last_change":"2023-11-28T12:03:45.823534+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823534+0000","last_undegraded":"2023-11-28T12:03:45.823534+0000","last_fullsized":"2023-11-28T12:03:45.823534+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:09.144457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824252+0000","last_change":"2023-11-28T12:03:45.824252+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824252+0000","last_undegraded":"2023-11-28T12:03:45.824252+0000","last_fullsized":"2023-11-28T12:03:45.824252+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:24:00.980890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827390+0000","last_change":"2023-11-28T12:03:45.827390+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827390+0000","last_undegraded":"2023-11-28T12:03:45.827390+0000","last_fullsized":"2023-11-28T12:03:45.827390+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:35.254897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824829+0000","last_change":"2023-11-28T12:03:45.824829+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824829+0000","last_undegraded":"2023-11-28T12:03:45.824829+0000","last_fullsized":"2023-11-28T12:03:45.824829+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:12.372093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827105+0000","last_change":"2023-11-28T12:03:45.827105+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827105+0000","last_undegraded":"2023-11-28T12:03:45.827105+0000","last_fullsized":"2023-11-28T12:03:45.827105+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:52:57.245222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825904+0000","last_change":"2023-11-28T12:03:45.825904+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825904+0000","last_undegraded":"2023-11-28T12:03:45.825904+0000","last_fullsized":"2023-11-28T12:03:45.825904+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:41.743469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823370+0000","last_change":"2023-11-28T12:03:45.823370+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823370+0000","last_undegraded":"2023-11-28T12:03:45.823370+0000","last_fullsized":"2023-11-28T12:03:45.823370+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:49.866215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":38,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.827185+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:03:45.827185+0000","last_peered":"2023-11-28T12:03:45.827185+0000","last_clean":"2023-11-28T12:03:45.827185+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:03:45.827185+0000","last_undegraded":"2023-11-28T12:03:45.827185+0000","last_fullsized":"2023-11-28T12:03:45.827185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823311+0000","last_change":"2023-11-28T12:03:45.823311+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823311+0000","last_undegraded":"2023-11-28T12:03:45.823311+0000","last_fullsized":"2023-11-28T12:03:45.823311+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:35.699402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.821980+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:03:45.821980+0000","last_peered":"2023-11-28T12:03:45.821980+0000","last_clean":"2023-11-28T12:03:45.821980+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:03:45.821980+0000","last_undegraded":"2023-11-28T12:03:45.821980+0000","last_fullsized":"2023-11-28T12:03:45.821980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825622+0000","last_change":"2023-11-28T12:03:45.825622+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825622+0000","last_undegraded":"2023-11-28T12:03:45.825622+0000","last_fullsized":"2023-11-28T12:03:45.825622+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:10.717376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.828667+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:03:45.828667+0000","last_peered":"2023-11-28T12:03:45.828667+0000","last_clean":"2023-11-28T12:03:45.828667+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:03:45.828667+0000","last_undegraded":"2023-11-28T12:03:45.828667+0000","last_fullsized":"2023-11-28T12:03:45.828667+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":184,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362741+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:03:43.362741+0000","last_peered":"2023-11-28T12:03:43.362741+0000","last_clean":"2023-11-28T12:03:43.362741+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:03:43.362741+0000","last_undegraded":"2023-11-28T12:03:43.362741+0000","last_fullsized":"2023-11-28T12:03:43.362741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.828612+0000","last_change":"2023-11-28T12:03:45.828612+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.828612+0000","last_undegraded":"2023-11-28T12:03:45.828612+0000","last_fullsized":"2023-11-28T12:03:45.828612+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:46.170695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":35,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.827814+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:03:45.827814+0000","last_peered":"2023-11-28T12:03:45.827814+0000","last_clean":"2023-11-28T12:03:45.827814+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:03:45.827814+0000","last_undegraded":"2023-11-28T12:03:45.827814+0000","last_fullsized":"2023-11-28T12:03:45.827814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827650+0000","last_change":"2023-11-28T12:03:45.827650+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827650+0000","last_undegraded":"2023-11-28T12:03:45.827650+0000","last_fullsized":"2023-11-28T12:03:45.827650+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:54.463672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362680+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:03:43.362680+0000","last_peered":"2023-11-28T12:03:43.362680+0000","last_clean":"2023-11-28T12:03:43.362680+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:03:43.362680+0000","last_undegraded":"2023-11-28T12:03:43.362680+0000","last_fullsized":"2023-11-28T12:03:43.362680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823878+0000","last_change":"2023-11-28T12:03:45.823878+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823878+0000","last_undegraded":"2023-11-28T12:03:45.823878+0000","last_fullsized":"2023-11-28T12:03:45.823878+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:21.499561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.822971+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:03:45.822971+0000","last_peered":"2023-11-28T12:03:45.822971+0000","last_clean":"2023-11-28T12:03:45.822971+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:03:45.822971+0000","last_undegraded":"2023-11-28T12:03:45.822971+0000","last_fullsized":"2023-11-28T12:03:45.822971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823246+0000","last_change":"2023-11-28T12:03:45.823246+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823246+0000","last_undegraded":"2023-11-28T12:03:45.823246+0000","last_fullsized":"2023-11-28T12:03:45.823246+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:05.212437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362659+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:03:43.362659+0000","last_peered":"2023-11-28T12:03:43.362659+0000","last_clean":"2023-11-28T12:03:43.362659+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:03:43.362659+0000","last_undegraded":"2023-11-28T12:03:43.362659+0000","last_fullsized":"2023-11-28T12:03:43.362659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826474+0000","last_change":"2023-11-28T12:03:45.826474+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826474+0000","last_undegraded":"2023-11-28T12:03:45.826474+0000","last_fullsized":"2023-11-28T12:03:45.826474+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:14.009538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362731+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:03:43.362731+0000","last_peered":"2023-11-28T12:03:43.362731+0000","last_clean":"2023-11-28T12:03:43.362731+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:03:43.362731+0000","last_undegraded":"2023-11-28T12:03:43.362731+0000","last_fullsized":"2023-11-28T12:03:43.362731+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9936,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94361904,"statfs":{"total":96636764160,"available":96626589696,"internally_reserved":0,"allocated":1122304,"data_stored":814570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.66200000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.46000000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.751,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.751}]},{"osd":1,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.872},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.82399999999999995,"15min":0.86699999999999999},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79100000000000004,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.79100000000000004}]},{"osd":2,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.81000000000000005,"15min":0.85099999999999998},"min":{"1min":0.41399999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73499999999999999,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80400000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84199999999999997,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.84199999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76500000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.61199999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.5}]},{"osd":5,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51700000000000002,"15min":0.52300000000000002},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76500000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72699999999999998,"5min":0.752,"15min":0.752},"last":0.52400000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.46600000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.69099999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8664,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363176,"statfs":{"total":96636764160,"available":96627892224,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.435,"5min":0.307,"15min":0.307},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.437}]},{"osd":1,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.39700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.34000000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.53900000000000003,"15min":0.52400000000000002},"min":{"1min":0.47299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.36599999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56100000000000005,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.48099999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.45000000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.497}]},{"osd":5,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.54600000000000004}]},{"osd":7,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.38100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.40200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.56000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9820,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362020,"statfs":{"total":96636764160,"available":96626708480,"internally_reserved":0,"allocated":1134592,"data_stored":820740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.40200000000000002,"15min":0.40999999999999998},"min":{"1min":0.26900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.55100000000000005,"5min":1.2,"15min":1.2},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40400000000000003,"15min":0.41099999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45900000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.30099999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36699999999999999,"15min":0.36899999999999999},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.45100000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.39100000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.47999999999999998,"15min":0.48799999999999999},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.53600000000000003,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.432},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45100000000000001,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67000000000000004,"5min":1.03,"15min":1.03},"last":0.36399999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63700000000000001,"15min":0.64400000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.70699999999999996,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.63300000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.628,"15min":0.63},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78000000000000003,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.82499999999999996,"5min":0.998,"15min":0.998},"last":0.68899999999999995}]},{"osd":6,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.80400000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.77200000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.65700000000000003}]},{"osd":7,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.81200000000000006,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.55200000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.84999999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8792,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363048,"statfs":{"total":96636764160,"available":96627761152,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.55200000000000005,"15min":0.57999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58599999999999997,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45200000000000001,"15min":0.45800000000000002},"min":{"1min":0.28599999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56399999999999995,"5min":1.667,"15min":1.667},"last":0.443}]},{"osd":2,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.57199999999999995,"15min":0.59599999999999997},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.68000000000000005,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.438,"15min":0.44},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44400000000000001,"15min":0.44800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.628,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.64200000000000002,"15min":0.67000000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76100000000000001,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.48399999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63,"15min":0.628},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92400000000000004,"5min":1.536,"15min":1.536},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.75900000000000001,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98199999999999998,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.626}]},{"osd":5,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.56000000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.61799999999999999},"min":{"1min":0.47399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86599999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.53500000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.032,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.501,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.014,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.59299999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92800000000000005,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.52100000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.997,"5min":1.605,"15min":1.605},"last":0.55800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8664,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363176,"statfs":{"total":96636764160,"available":96627892224,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.375},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60499999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39900000000000002,"15min":0.39700000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58199999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.38800000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40899999999999997},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56699999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.439,"15min":0.435},"min":{"1min":0.33500000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.63300000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.316}]},{"osd":4,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81399999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63,"15min":0.63500000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77600000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.79000000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66100000000000003,"15min":0.66900000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83399999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8984,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362856,"statfs":{"total":96636764160,"available":96627564544,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53100000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.44500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58499999999999996,"5min":1.238,"15min":1.238},"last":0.30599999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58699999999999997,"5min":1.53,"15min":1.53},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.502,"15min":0.51600000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57099999999999995,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.36899999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48799999999999999,"15min":0.496},"min":{"1min":0.36199999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55200000000000005,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.47899999999999998,"15min":0.48599999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60499999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77300000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64600000000000002,"15min":0.65200000000000002},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88500000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83699999999999997,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.80200000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86899999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.48199999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84299999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.51300000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.86799999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.81599999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56200000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8920,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362920,"statfs":{"total":96636764160,"available":96627630080,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.503,"15min":0.495},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69099999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.38400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48199999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65900000000000003,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73099999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.435}]},{"osd":2,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73199999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.39800000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52400000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.433,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.752,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.497},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.50700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.746,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73099999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64400000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.35199999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.49299999999999999,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61199999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71199999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41799999999999998}]},{"osd":7,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70999999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8856,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362984,"statfs":{"total":96636764160,"available":96627695616,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.39000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.65800000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.61599999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67000000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.629},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.88300000000000001,"15min":0.94499999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61499999999999999,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.60499999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71499999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57699999999999996,"15min":0.58099999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69999999999999996,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66300000000000003}]},{"osd":3,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.434,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69699999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.70599999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.69499999999999995}]},{"osd":4,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64500000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.38400000000000001}]},{"osd":6,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.35199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.497},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63900000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42099999999999999}]},{"osd":7,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.375,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66200000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.442},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68799999999999994,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:03:52.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2023-11-28T12:03:52.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2023-11-28T12:03:52.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2023-11-28T12:03:52.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2023-11-28T12:03:52.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2023-11-28T12:03:52.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2023-11-28T12:03:52.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2023-11-28T12:03:52.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2023-11-28T12:03:52.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2023-11-28T12:03:52.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2023-11-28T12:03:52.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2023-11-28T12:03:52.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2023-11-28T12:03:52.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2023-11-28T12:03:52.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2023-11-28T12:03:52.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2023-11-28T12:03:52.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2023-11-28T12:03:52.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2023-11-28T12:03:52.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2023-11-28T12:03:52.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2023-11-28T12:03:52.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2023-11-28T12:03:52.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2023-11-28T12:03:52.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2023-11-28T12:03:52.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2023-11-28T12:03:52.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2023-11-28T12:03:52.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2023-11-28T12:03:52.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2023-11-28T12:03:52.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2023-11-28T12:03:52.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2023-11-28T12:03:52.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2023-11-28T12:03:52.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2023-11-28T12:03:52.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2023-11-28T12:03:52.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2023-11-28T12:03:52.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2023-11-28T12:03:52.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2023-11-28T12:03:52.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.824423+0000', 'last_change': '2023-11-28T12:03:45.824423+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.824423+0000', 'last_undegraded': '2023-11-28T12:03:45.824423+0000', 'last_fullsized': '2023-11-28T12:03:45.824423+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:44:00.642126+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T12:03:52.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2023-11-28T12:03:52.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2023-11-28T12:03:52.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.827796+0000', 'last_change': '2023-11-28T12:03:45.827796+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.827796+0000', 'last_undegraded': '2023-11-28T12:03:45.827796+0000', 'last_fullsized': '2023-11-28T12:03:45.827796+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T12:45:39.291687+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T12:03:52.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2023-11-28T12:03:52.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2023-11-28T12:03:52.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.827234+0000', 'last_change': '2023-11-28T12:03:45.827234+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.827234+0000', 'last_undegraded': '2023-11-28T12:03:45.827234+0000', 'last_fullsized': '2023-11-28T12:03:45.827234+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T12:43:59.530852+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T12:03:52.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2023-11-28T12:03:52.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-11-28T12:03:52.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.823414+0000', 'last_change': '2023-11-28T12:03:45.823414+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.823414+0000', 'last_undegraded': '2023-11-28T12:03:45.823414+0000', 'last_fullsized': '2023-11-28T12:03:45.823414+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T23:24:10.717376+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T12:03:52.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2023-11-28T12:03:52.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2023-11-28T12:03:52.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.828214+0000', 'last_change': '2023-11-28T12:03:45.828214+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.828214+0000', 'last_undegraded': '2023-11-28T12:03:45.828214+0000', 'last_fullsized': '2023-11-28T12:03:45.828214+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T23:10:56.656303+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T12:03:52.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2023-11-28T12:03:52.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2023-11-28T12:03:52.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.826655+0000', 'last_change': '2023-11-28T12:03:45.826655+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.826655+0000', 'last_undegraded': '2023-11-28T12:03:45.826655+0000', 'last_fullsized': '2023-11-28T12:03:45.826655+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:49:46.170695+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T12:03:52.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2023-11-28T12:03:52.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2023-11-28T12:03:52.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.823319+0000', 'last_change': '2023-11-28T12:03:45.823319+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.823319+0000', 'last_undegraded': '2023-11-28T12:03:45.823319+0000', 'last_fullsized': '2023-11-28T12:03:45.823319+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:13:44.179722+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T12:03:52.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2023-11-28T12:03:52.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2023-11-28T12:03:52.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.826298+0000', 'last_change': '2023-11-28T12:03:45.826298+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.826298+0000', 'last_undegraded': '2023-11-28T12:03:45.826298+0000', 'last_fullsized': '2023-11-28T12:03:45.826298+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:19:16.229157+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T12:03:52.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2023-11-28T12:03:52.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2023-11-28T12:03:52.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.825677+0000', 'last_change': '2023-11-28T12:03:45.825677+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.825677+0000', 'last_undegraded': '2023-11-28T12:03:45.825677+0000', 'last_fullsized': '2023-11-28T12:03:45.825677+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:36:57.070237+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T12:03:52.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2023-11-28T12:03:52.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-11-28T12:03:52.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.826611+0000', 'last_change': '2023-11-28T12:03:45.826611+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.826611+0000', 'last_undegraded': '2023-11-28T12:03:45.826611+0000', 'last_fullsized': '2023-11-28T12:03:45.826611+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:14:40.312689+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T12:03:52.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2023-11-28T12:03:52.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2023-11-28T12:03:52.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.827494+0000', 'last_change': '2023-11-28T12:03:45.827494+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.827494+0000', 'last_undegraded': '2023-11-28T12:03:45.827494+0000', 'last_fullsized': '2023-11-28T12:03:45.827494+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:03:05.212437+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T12:03:52.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2023-11-28T12:03:52.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-11-28T12:03:52.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.824983+0000', 'last_change': '2023-11-28T12:03:45.824983+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.824983+0000', 'last_undegraded': '2023-11-28T12:03:45.824983+0000', 'last_fullsized': '2023-11-28T12:03:45.824983+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T12:06:25.548629+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T12:03:52.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2023-11-28T12:03:52.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2023-11-28T12:03:52.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.826279+0000', 'last_change': '2023-11-28T12:03:45.826279+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.826279+0000', 'last_undegraded': '2023-11-28T12:03:45.826279+0000', 'last_fullsized': '2023-11-28T12:03:45.826279+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T23:56:00.591608+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T12:03:52.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2023-11-28T12:03:52.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2023-11-28T12:03:52.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.823223+0000', 'last_change': '2023-11-28T12:03:45.823223+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.823223+0000', 'last_undegraded': '2023-11-28T12:03:45.823223+0000', 'last_fullsized': '2023-11-28T12:03:45.823223+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T12:18:55.370673+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T12:03:52.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2023-11-28T12:03:52.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2023-11-28T12:03:52.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.825578+0000', 'last_change': '2023-11-28T12:03:45.825578+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.825578+0000', 'last_undegraded': '2023-11-28T12:03:45.825578+0000', 'last_fullsized': '2023-11-28T12:03:45.825578+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:43:49.199732+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T12:03:52.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2023-11-28T12:03:52.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2023-11-28T12:03:52.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.824871+0000', 'last_change': '2023-11-28T12:03:45.824871+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.824871+0000', 'last_undegraded': '2023-11-28T12:03:45.824871+0000', 'last_fullsized': '2023-11-28T12:03:45.824871+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T18:50:16.159397+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T12:03:52.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2023-11-28T12:03:52.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2023-11-28T12:03:52.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.826448+0000', 'last_change': '2023-11-28T12:03:45.826448+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.826448+0000', 'last_undegraded': '2023-11-28T12:03:45.826448+0000', 'last_fullsized': '2023-11-28T12:03:45.826448+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:55:14.009538+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T12:03:52.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2023-11-28T12:03:52.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2023-11-28T12:03:52.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.826989+0000', 'last_change': '2023-11-28T12:03:45.826989+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.826989+0000', 'last_undegraded': '2023-11-28T12:03:45.826989+0000', 'last_fullsized': '2023-11-28T12:03:45.826989+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:23:01.829853+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T12:03:52.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2023-11-28T12:03:52.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2023-11-28T12:03:52.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.823534+0000', 'last_change': '2023-11-28T12:03:45.823534+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.823534+0000', 'last_undegraded': '2023-11-28T12:03:45.823534+0000', 'last_fullsized': '2023-11-28T12:03:45.823534+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T18:02:09.144457+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T12:03:52.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2023-11-28T12:03:52.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2023-11-28T12:03:52.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.824252+0000', 'last_change': '2023-11-28T12:03:45.824252+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.824252+0000', 'last_undegraded': '2023-11-28T12:03:45.824252+0000', 'last_fullsized': '2023-11-28T12:03:45.824252+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:24:00.980890+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T12:03:52.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2023-11-28T12:03:52.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2023-11-28T12:03:52.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.827390+0000', 'last_change': '2023-11-28T12:03:45.827390+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.827390+0000', 'last_undegraded': '2023-11-28T12:03:45.827390+0000', 'last_fullsized': '2023-11-28T12:03:45.827390+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:54:35.254897+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T12:03:52.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2023-11-28T12:03:52.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2023-11-28T12:03:52.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.824829+0000', 'last_change': '2023-11-28T12:03:45.824829+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.824829+0000', 'last_undegraded': '2023-11-28T12:03:45.824829+0000', 'last_fullsized': '2023-11-28T12:03:45.824829+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:05:12.372093+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T12:03:52.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2023-11-28T12:03:52.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2023-11-28T12:03:52.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.827105+0000', 'last_change': '2023-11-28T12:03:45.827105+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.827105+0000', 'last_undegraded': '2023-11-28T12:03:45.827105+0000', 'last_fullsized': '2023-11-28T12:03:45.827105+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:52:57.245222+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T12:03:52.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2023-11-28T12:03:52.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2023-11-28T12:03:52.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.825904+0000', 'last_change': '2023-11-28T12:03:45.825904+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.825904+0000', 'last_undegraded': '2023-11-28T12:03:45.825904+0000', 'last_fullsized': '2023-11-28T12:03:45.825904+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T13:26:41.743469+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T12:03:52.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2023-11-28T12:03:52.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-11-28T12:03:52.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.823370+0000', 'last_change': '2023-11-28T12:03:45.823370+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.823370+0000', 'last_undegraded': '2023-11-28T12:03:45.823370+0000', 'last_fullsized': '2023-11-28T12:03:45.823370+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:29:49.866215+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T12:03:52.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2023-11-28T12:03:52.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2023-11-28T12:03:52.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.823311+0000', 'last_change': '2023-11-28T12:03:45.823311+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.823311+0000', 'last_undegraded': '2023-11-28T12:03:45.823311+0000', 'last_fullsized': '2023-11-28T12:03:45.823311+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T22:03:35.699402+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T12:03:52.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2023-11-28T12:03:52.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2023-11-28T12:03:52.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.825622+0000', 'last_change': '2023-11-28T12:03:45.825622+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.825622+0000', 'last_undegraded': '2023-11-28T12:03:45.825622+0000', 'last_fullsized': '2023-11-28T12:03:45.825622+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T23:24:10.717376+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T12:03:52.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2023-11-28T12:03:52.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2023-11-28T12:03:52.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.828612+0000', 'last_change': '2023-11-28T12:03:45.828612+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.828612+0000', 'last_undegraded': '2023-11-28T12:03:45.828612+0000', 'last_fullsized': '2023-11-28T12:03:45.828612+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:49:46.170695+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T12:03:52.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2023-11-28T12:03:52.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2023-11-28T12:03:52.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.827650+0000', 'last_change': '2023-11-28T12:03:45.827650+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.827650+0000', 'last_undegraded': '2023-11-28T12:03:45.827650+0000', 'last_fullsized': '2023-11-28T12:03:45.827650+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:54:54.463672+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T12:03:52.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2023-11-28T12:03:52.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2023-11-28T12:03:52.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.823878+0000', 'last_change': '2023-11-28T12:03:45.823878+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.823878+0000', 'last_undegraded': '2023-11-28T12:03:45.823878+0000', 'last_fullsized': '2023-11-28T12:03:45.823878+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:40:21.499561+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T12:03:52.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2023-11-28T12:03:52.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-11-28T12:03:52.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.823246+0000', 'last_change': '2023-11-28T12:03:45.823246+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.823246+0000', 'last_undegraded': '2023-11-28T12:03:45.823246+0000', 'last_fullsized': '2023-11-28T12:03:45.823246+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:03:05.212437+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T12:03:52.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2023-11-28T12:03:52.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:03:46.820008+0000', 'last_change': '2023-11-28T12:03:46.820008+0000', 'last_active': '2023-11-28T12:03:46.820008+0000', 'last_peered': '2023-11-28T12:03:46.820008+0000', 'last_clean': '2023-11-28T12:03:46.820008+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:46.820008+0000', 'last_undegraded': '2023-11-28T12:03:46.820008+0000', 'last_fullsized': '2023-11-28T12:03:46.820008+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:46.820008+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:03:52.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-11-28T12:03:52.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:03:45.826474+0000', 'last_change': '2023-11-28T12:03:45.826474+0000', 'last_active': '2023-11-28T12:03:45.817147+0000', 'last_peered': '2023-11-28T12:03:45.817147+0000', 'last_clean': '2023-11-28T12:03:45.817147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:03:45.826474+0000', 'last_undegraded': '2023-11-28T12:03:45.826474+0000', 'last_fullsized': '2023-11-28T12:03:45.826474+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'last_clean_scrub_stamp': '2023-11-28T12:03:45.817147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:55:14.009538+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T12:03:52.326 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T12:03:52.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.470+0000 7feeb21f7700 1 Processor -- start 2023-11-28T12:03:52.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.471+0000 7feeb21f7700 1 -- start start 2023-11-28T12:03:52.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.471+0000 7feeb21f7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac0d15f0 0x7feeac15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.471+0000 7feeb21f7700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac15af30 con 0x7feeac06e400 2023-11-28T12:03:52.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.471+0000 7feeb21f7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac15ef40 con 0x7feeac15e010 2023-11-28T12:03:52.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.471+0000 7feeb21f7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac15a9d0 con 0x7feeac0d15f0 2023-11-28T12:03:52.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeab7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7feeac06e400 0x7feeac06e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54950/0 (socket says 172.21.15.57:54950) 2023-11-28T12:03:52.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeab7fe700 1 -- 172.21.15.57:0/2439808428 learned_addr learned my addr 172.21.15.57:0/2439808428 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:52.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaaffd700 1 --2- 172.21.15.57:0/2439808428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac0d15f0 0x7feeac15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524918332 0 0) 0x7feeac15af30 con 0x7feeac06e400 2023-11-28T12:03:52.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee940035b0 con 0x7feeac06e400 2023-11-28T12:03:52.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3296192506 0 0) 0x7fee940035b0 con 0x7feeac06e400 2023-11-28T12:03:52.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac0d15f0 msgr2=0x7feeac15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:52.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 --2- 172.21.15.57:0/2439808428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac0d15f0 0x7feeac15b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 >> v1:172.21.15.181:6789/0 conn(0x7feeac15e010 legacy=0x7feeac0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac15e980 con 0x7feeac06e400 2023-11-28T12:03:52.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fee9c001670 con 0x7feeac06e400 2023-11-28T12:03:52.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaaffd700 1 --2- 172.21.15.57:0/2439808428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac0d15f0 0x7feeac15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:03:52.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 >> v1:172.21.15.57:6789/0 conn(0x7feeac06e400 legacy=0x7feeac06e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 --2- 172.21.15.57:0/2439808428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee94003a30 0x7fee94003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 --2- 172.21.15.57:0/2439808428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac0d15f0 0x7fee94004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 --2- 172.21.15.57:0/2439808428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee94004d90 0x7fee94005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9400ba10 con 0x7fee94004d90 2023-11-28T12:03:52.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9400bb50 con 0x7fee94003a30 2023-11-28T12:03:52.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9400bcc0 con 0x7feeac0d15f0 2023-11-28T12:03:52.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeabfff700 1 --2- 172.21.15.57:0/2439808428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee94004d90 0x7fee94005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.472+0000 7feeab7fe700 1 --2- 172.21.15.57:0/2439808428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee94003a30 0x7fee94003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.473+0000 7feeaaffd700 1 --2- 172.21.15.57:0/2439808428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac0d15f0 0x7fee94004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.473+0000 7feeabfff700 1 -- 172.21.15.57:0/2439808428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac0d15f0 msgr2=0x7fee94004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:52.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.473+0000 7feeabfff700 1 --2- 172.21.15.57:0/2439808428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac0d15f0 0x7fee94004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.473+0000 7feeabfff700 1 -- 172.21.15.57:0/2439808428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee94003a30 msgr2=0x7fee94003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:52.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.473+0000 7feeabfff700 1 --2- 172.21.15.57:0/2439808428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee94003a30 0x7fee94003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.473+0000 7feeabfff700 1 -- 172.21.15.57:0/2439808428 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea00094a0 con 0x7fee94004d90 2023-11-28T12:03:52.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.473+0000 7feeabfff700 1 --2- 172.21.15.57:0/2439808428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee94004d90 0x7fee94005200 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feea400c290 tx=0x7fee94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a587e89e67ed80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:52.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.473+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4001e70 con 0x7fee94004d90 2023-11-28T12:03:52.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.473+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea400cdf0 con 0x7fee94004d90 2023-11-28T12:03:52.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.473+0000 7feeaa7fc700 1 -- 172.21.15.57:0/2439808428 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4012660 con 0x7fee94004d90 2023-11-28T12:03:52.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.474+0000 7feeb21f7700 1 -- 172.21.15.57:0/2439808428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee94004d90 msgr2=0x7fee94005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.474+0000 7feeb21f7700 1 --2- 172.21.15.57:0/2439808428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee94004d90 0x7fee94005200 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feea400c290 tx=0x7fee94005880 comp rx=0 tx=0).stop 2023-11-28T12:03:52.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.474+0000 7feeabfff700 1 -- 172.21.15.57:0/2439808428 reap_dead start 2023-11-28T12:03:52.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.474+0000 7feeb21f7700 1 -- 172.21.15.57:0/2439808428 shutdown_connections 2023-11-28T12:03:52.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.474+0000 7feeb21f7700 1 -- 172.21.15.57:0/2439808428 >> 172.21.15.57:0/2439808428 conn(0x7feeac06a270 msgr2=0x7feeac06a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:52.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.474+0000 7feeb21f7700 1 -- 172.21.15.57:0/2439808428 shutdown_connections 2023-11-28T12:03:52.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.474+0000 7feeb21f7700 1 -- 172.21.15.57:0/2439808428 wait complete. 2023-11-28T12:03:52.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.475+0000 7feeb21f7700 1 Processor -- start 2023-11-28T12:03:52.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.475+0000 7feeb21f7700 1 -- start start 2023-11-28T12:03:52.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeb21f7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac1417e0 0x7feeac0b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeb21f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeac141bb0 0x7feeac0b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeb21f7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac154c50 0x7feeac1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeb21f7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0b3900 con 0x7feeac141bb0 2023-11-28T12:03:52.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeb21f7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac155630 con 0x7feeac1417e0 2023-11-28T12:03:52.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeb21f7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac1e4ab0 con 0x7feeac154c50 2023-11-28T12:03:52.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeaaffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeac141bb0 0x7feeac0b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeaaffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeac141bb0 0x7feeac0b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44508/0 (socket says 172.21.15.57:44508) 2023-11-28T12:03:52.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeaaffd700 1 -- 172.21.15.57:0/2448233942 learned_addr learned my addr 172.21.15.57:0/2448233942 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:52.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeab7fe700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac1417e0 0x7feeac0b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeabfff700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac154c50 0x7feeac1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeaaffd700 1 -- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac154c50 msgr2=0x7feeac1550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeaaffd700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac154c50 0x7feeac1550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeaaffd700 1 -- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac1417e0 msgr2=0x7feeac0b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeaaffd700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac1417e0 0x7feeac0b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeaaffd700 1 -- 172.21.15.57:0/2448233942 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac15abc0 con 0x7feeac141bb0 2023-11-28T12:03:52.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeabfff700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac154c50 0x7feeac1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:52.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.476+0000 7feeab7fe700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac1417e0 0x7feeac0b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:52.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.477+0000 7feeaaffd700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeac141bb0 0x7feeac0b32d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7feea0011d00 tx=0x7feea000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:52.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.477+0000 7fee93fff700 1 -- 172.21.15.57:0/2448233942 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0011ea0 con 0x7feeac141bb0 2023-11-28T12:03:52.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.477+0000 7fee93fff700 1 -- 172.21.15.57:0/2448233942 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea000fd40 con 0x7feeac141bb0 2023-11-28T12:03:52.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.477+0000 7feeb21f7700 1 -- 172.21.15.57:0/2448233942 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeac15ec60 con 0x7feeac141bb0 2023-11-28T12:03:52.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.477+0000 7fee93fff700 1 -- 172.21.15.57:0/2448233942 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0004270 con 0x7feeac141bb0 2023-11-28T12:03:52.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.477+0000 7feeb21f7700 1 -- 172.21.15.57:0/2448233942 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac15e4b0 con 0x7feeac141bb0 2023-11-28T12:03:52.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.478+0000 7feeb21f7700 1 -- 172.21.15.57:0/2448233942 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeac1e4c60 con 0x7feeac141bb0 2023-11-28T12:03:52.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.478+0000 7fee93fff700 1 -- 172.21.15.57:0/2448233942 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feea0024070 con 0x7feeac141bb0 2023-11-28T12:03:52.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.479+0000 7fee93fff700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee980682f0 0x7fee9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.479+0000 7fee93fff700 1 -- 172.21.15.57:0/2448233942 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feea0086140 con 0x7feeac141bb0 2023-11-28T12:03:52.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.479+0000 7feeab7fe700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee980682f0 0x7fee9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.480+0000 7feeab7fe700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee980682f0 0x7fee9806a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0048d0 tx=0x7fee9c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:52.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.483+0000 7fee93fff700 1 -- 172.21.15.57:0/2448233942 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feeac1e4c60 con 0x7feeac141bb0 2023-11-28T12:03:52.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.769+0000 7feeb21f7700 1 -- 172.21.15.57:0/2448233942 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7feeac1e4c20 con 0x7feeac141bb0 2023-11-28T12:03:52.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.769+0000 7fee93fff700 1 -- 172.21.15.57:0/2448233942 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1108 (secure 0 0 0) 0x7feea0022470 con 0x7feeac141bb0 2023-11-28T12:03:52.771 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:52.771 INFO:teuthology.orchestra.run.smithi057.stdout:{"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":147,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":49,"num_osds":8,"num_up_osds":8,"osd_up_since":1701172896,"num_in_osds":8,"osd_in_since":1701172888,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":74379264,"bytes_avail":773019734016,"bytes_total":773094113280,"unknown_pgs_ratio":0.60952383279800415,"inactive_pgs_ratio":0.30476191639900208},"fsmap":{"epoch":24,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"b","status":"up:active","gid":5739}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-11-28T12:02:41.674291+0000","services":{}},"progress_events":{}} 2023-11-28T12:03:52.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.772+0000 7feeb21f7700 1 -- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee980682f0 msgr2=0x7fee9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.772+0000 7feeb21f7700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee980682f0 0x7fee9806a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0048d0 tx=0x7fee9c009040 comp rx=0 tx=0).stop 2023-11-28T12:03:52.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.772+0000 7feeb21f7700 1 -- 172.21.15.57:0/2448233942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeac141bb0 msgr2=0x7feeac0b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.772+0000 7feeb21f7700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeac141bb0 0x7feeac0b32d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7feea0011d00 tx=0x7feea000f520 comp rx=0 tx=0).stop 2023-11-28T12:03:52.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.772+0000 7feeb21f7700 1 -- 172.21.15.57:0/2448233942 shutdown_connections 2023-11-28T12:03:52.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.772+0000 7feeb21f7700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fee980682f0 0x7fee9806a7a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.772+0000 7feeb21f7700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac154c50 0x7feeac1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.772+0000 7feeb21f7700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac1417e0 0x7feeac0b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.772+0000 7feeb21f7700 1 --2- 172.21.15.57:0/2448233942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feeac141bb0 0x7feeac0b32d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.772+0000 7feeb21f7700 1 -- 172.21.15.57:0/2448233942 >> 172.21.15.57:0/2448233942 conn(0x7feeac06a270 msgr2=0x7feeac0b56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:52.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.772+0000 7feeb21f7700 1 -- 172.21.15.57:0/2448233942 shutdown_connections 2023-11-28T12:03:52.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.772+0000 7feeb21f7700 1 -- 172.21.15.57:0/2448233942 wait complete. 2023-11-28T12:03:52.787 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T12:03:52.787 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:03:52.821 INFO:tasks.ceph.mds.b.smithi181.stderr:2023-11-28T12:03:52.820+0000 7f4db2f28700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T12:03:52.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.921+0000 7f9374dc2700 1 Processor -- start 2023-11-28T12:03:52.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.921+0000 7f9374dc2700 1 -- start start 2023-11-28T12:03:52.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f9374dc2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f937006a270 0x7f937006a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f9374dc2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9370070d90 0x7f93700711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f9374dc2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f937006da40 0x7f937006dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f9374dc2700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9370151a50 con 0x7f937006a270 2023-11-28T12:03:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f9374dc2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f937015b440 con 0x7f9370070d90 2023-11-28T12:03:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f9374dc2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937006ab80 con 0x7f937006da40 2023-11-28T12:03:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f936e59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f937006a270 0x7f937006a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f936e59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f937006a270 0x7f937006a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44510/0 (socket says 172.21.15.57:44510) 2023-11-28T12:03:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f936e59c700 1 -- 172.21.15.57:0/1794605547 learned_addr learned my addr 172.21.15.57:0/1794605547 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f936dd9b700 1 --2- 172.21.15.57:0/1794605547 >> v2:172.21.15.181:3300/0 conn(0x7f9370070d90 0x7f93700711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f936ed9d700 1 --2- 172.21.15.57:0/1794605547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f937006da40 0x7f937006dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f936e59c700 1 -- 172.21.15.57:0/1794605547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f937006da40 msgr2=0x7f937006dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:52.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f936e59c700 1 --2- 172.21.15.57:0/1794605547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f937006da40 0x7f937006dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f936e59c700 1 -- 172.21.15.57:0/1794605547 >> v2:172.21.15.181:3300/0 conn(0x7f9370070d90 msgr2=0x7f93700711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f936e59c700 1 --2- 172.21.15.57:0/1794605547 >> v2:172.21.15.181:3300/0 conn(0x7f9370070d90 0x7f93700711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.923+0000 7f936e59c700 1 -- 172.21.15.57:0/1794605547 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370071ac0 con 0x7f937006a270 2023-11-28T12:03:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.924+0000 7f936e59c700 1 --2- 172.21.15.57:0/1794605547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f937006a270 0x7f937006a640 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f93700cd4f0 tx=0x7f9360009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b212e7c1a611c648 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.924+0000 7f936d59a700 1 -- 172.21.15.57:0/1794605547 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936000b870 con 0x7f937006a270 2023-11-28T12:03:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.924+0000 7f936d59a700 1 -- 172.21.15.57:0/1794605547 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936000b9d0 con 0x7f937006a270 2023-11-28T12:03:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.924+0000 7f936d59a700 1 -- 172.21.15.57:0/1794605547 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936000bd60 con 0x7f937006a270 2023-11-28T12:03:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.924+0000 7f9374dc2700 1 -- 172.21.15.57:0/1794605547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f937006a270 msgr2=0x7f937006a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.924+0000 7f9374dc2700 1 --2- 172.21.15.57:0/1794605547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f937006a270 0x7f937006a640 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f93700cd4f0 tx=0x7f9360009bd0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.925+0000 7f9374dc2700 1 -- 172.21.15.57:0/1794605547 shutdown_connections 2023-11-28T12:03:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.925+0000 7f9374dc2700 1 --2- 172.21.15.57:0/1794605547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f937006da40 0x7f937006dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.925+0000 7f9374dc2700 1 --2- 172.21.15.57:0/1794605547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f937006a270 0x7f937006a640 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.925+0000 7f9374dc2700 1 --2- 172.21.15.57:0/1794605547 >> v2:172.21.15.181:3300/0 conn(0x7f9370070d90 0x7f93700711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.925+0000 7f9374dc2700 1 -- 172.21.15.57:0/1794605547 >> 172.21.15.57:0/1794605547 conn(0x7f937006c010 msgr2=0x7f93700cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:52.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.925+0000 7f9374dc2700 1 -- 172.21.15.57:0/1794605547 shutdown_connections 2023-11-28T12:03:52.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.925+0000 7f9374dc2700 1 -- 172.21.15.57:0/1794605547 wait complete. 2023-11-28T12:03:52.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.926+0000 7f9374dc2700 1 Processor -- start 2023-11-28T12:03:52.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.926+0000 7f9374dc2700 1 -- start start 2023-11-28T12:03:52.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f9374dc2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937006a270 0x7f9370140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f9374dc2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f937006da40 0x7f93700c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f9374dc2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9370070d90 0x7f93700c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f9374dc2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93700c5c50 con 0x7f937006da40 2023-11-28T12:03:52.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f9374dc2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93700c5e60 con 0x7f937006a270 2023-11-28T12:03:52.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f9374dc2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93700c6140 con 0x7f9370070d90 2023-11-28T12:03:52.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f936dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f937006da40 0x7f93700c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f936dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f937006da40 0x7f93700c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44518/0 (socket says 172.21.15.57:44518) 2023-11-28T12:03:52.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f936dd9b700 1 -- 172.21.15.57:0/3563330791 learned_addr learned my addr 172.21.15.57:0/3563330791 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:52.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f936ed9d700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9370070d90 0x7f93700c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f936e59c700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937006a270 0x7f9370140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f936dd9b700 1 -- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9370070d90 msgr2=0x7f93700c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f936dd9b700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9370070d90 0x7f93700c2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f936dd9b700 1 -- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937006a270 msgr2=0x7f9370140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:52.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f936dd9b700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937006a270 0x7f9370140650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:52.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.927+0000 7f936dd9b700 1 -- 172.21.15.57:0/3563330791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9360009550 con 0x7f937006da40 2023-11-28T12:03:52.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.928+0000 7f936dd9b700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f937006da40 0x7f93700c5710 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f936400d800 tx=0x7f936400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:52.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.928+0000 7f935effd700 1 -- 172.21.15.57:0/3563330791 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93640125f0 con 0x7f937006da40 2023-11-28T12:03:52.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.928+0000 7f935effd700 1 -- 172.21.15.57:0/3563330791 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9364012750 con 0x7f937006da40 2023-11-28T12:03:52.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.928+0000 7f9374dc2700 1 -- 172.21.15.57:0/3563330791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937015b0d0 con 0x7f937006da40 2023-11-28T12:03:52.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.928+0000 7f935effd700 1 -- 172.21.15.57:0/3563330791 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364012b00 con 0x7f937006da40 2023-11-28T12:03:52.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.928+0000 7f9374dc2700 1 -- 172.21.15.57:0/3563330791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937015adf0 con 0x7f937006da40 2023-11-28T12:03:52.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.929+0000 7f9374dc2700 1 -- 172.21.15.57:0/3563330791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f937015aac0 con 0x7f937006da40 2023-11-28T12:03:52.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.929+0000 7f935effd700 1 -- 172.21.15.57:0/3563330791 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9364012c60 con 0x7f937006da40 2023-11-28T12:03:52.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.930+0000 7f935effd700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f93500682f0 0x7f935006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:52.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.930+0000 7f935effd700 1 -- 172.21.15.57:0/3563330791 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9364086240 con 0x7f937006da40 2023-11-28T12:03:52.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.930+0000 7f936e59c700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f93500682f0 0x7f935006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:52.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.931+0000 7f936e59c700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f93500682f0 0x7f935006a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9360000c00 tx=0x7f9360008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:52.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:52.934+0000 7f935effd700 1 -- 172.21.15.57:0/3563330791 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9364052a10 con 0x7f937006da40 2023-11-28T12:03:53.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.129+0000 7f9374dc2700 1 -- 172.21.15.57:0/3563330791 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93700cadf0 con 0x7f93500682f0 2023-11-28T12:03:53.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.138+0000 7f935effd700 1 -- 172.21.15.57:0/3563330791 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+301454 (secure 0 0 0) 0x7f93700cadf0 con 0x7f93500682f0 2023-11-28T12:03:53.142 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:53.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.141+0000 7f9374dc2700 1 -- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f93500682f0 msgr2=0x7f935006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:53.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.141+0000 7f9374dc2700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f93500682f0 0x7f935006a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9360000c00 tx=0x7f9360008030 comp rx=0 tx=0).stop 2023-11-28T12:03:53.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.141+0000 7f9374dc2700 1 -- 172.21.15.57:0/3563330791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f937006da40 msgr2=0x7f93700c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:53.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.141+0000 7f9374dc2700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f937006da40 0x7f93700c5710 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f936400d800 tx=0x7f936400dbc0 comp rx=0 tx=0).stop 2023-11-28T12:03:53.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.141+0000 7f9374dc2700 1 -- 172.21.15.57:0/3563330791 shutdown_connections 2023-11-28T12:03:53.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.141+0000 7f9374dc2700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f93500682f0 0x7f935006a7a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:53.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.141+0000 7f9374dc2700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9370070d90 0x7f93700c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:53.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.141+0000 7f9374dc2700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937006a270 0x7f9370140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:53.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.141+0000 7f9374dc2700 1 --2- 172.21.15.57:0/3563330791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f937006da40 0x7f93700c5710 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:53.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.141+0000 7f9374dc2700 1 -- 172.21.15.57:0/3563330791 >> 172.21.15.57:0/3563330791 conn(0x7f937006c010 msgr2=0x7f93700b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:53.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.141+0000 7f9374dc2700 1 -- 172.21.15.57:0/3563330791 shutdown_connections 2023-11-28T12:03:53.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:53.141+0000 7f9374dc2700 1 -- 172.21.15.57:0/3563330791 wait complete. 2023-11-28T12:03:53.146 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:03:53.156 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-11-28T12:03:51.690936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":72636,"kb_used_data":5372,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":754902084,"statfs":{"total":773094113280,"available":773019734016,"internally_reserved":0,"allocated":5500928,"data_stored":3017542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.869491"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824423+0000","last_change":"2023-11-28T12:03:45.824423+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824423+0000","last_undegraded":"2023-11-28T12:03:45.824423+0000","last_fullsized":"2023-11-28T12:03:45.824423+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:00.642126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827796+0000","last_change":"2023-11-28T12:03:45.827796+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827796+0000","last_undegraded":"2023-11-28T12:03:45.827796+0000","last_fullsized":"2023-11-28T12:03:45.827796+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:39.291687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827234+0000","last_change":"2023-11-28T12:03:45.827234+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827234+0000","last_undegraded":"2023-11-28T12:03:45.827234+0000","last_fullsized":"2023-11-28T12:03:45.827234+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:59.530852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823414+0000","last_change":"2023-11-28T12:03:45.823414+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823414+0000","last_undegraded":"2023-11-28T12:03:45.823414+0000","last_fullsized":"2023-11-28T12:03:45.823414+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:10.717376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.828214+0000","last_change":"2023-11-28T12:03:45.828214+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.828214+0000","last_undegraded":"2023-11-28T12:03:45.828214+0000","last_fullsized":"2023-11-28T12:03:45.828214+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:56.656303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826655+0000","last_change":"2023-11-28T12:03:45.826655+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826655+0000","last_undegraded":"2023-11-28T12:03:45.826655+0000","last_fullsized":"2023-11-28T12:03:45.826655+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:46.170695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823319+0000","last_change":"2023-11-28T12:03:45.823319+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823319+0000","last_undegraded":"2023-11-28T12:03:45.823319+0000","last_fullsized":"2023-11-28T12:03:45.823319+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:13:44.179722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826298+0000","last_change":"2023-11-28T12:03:45.826298+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826298+0000","last_undegraded":"2023-11-28T12:03:45.826298+0000","last_fullsized":"2023-11-28T12:03:45.826298+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:16.229157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825677+0000","last_change":"2023-11-28T12:03:45.825677+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825677+0000","last_undegraded":"2023-11-28T12:03:45.825677+0000","last_fullsized":"2023-11-28T12:03:45.825677+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:57.070237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826611+0000","last_change":"2023-11-28T12:03:45.826611+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826611+0000","last_undegraded":"2023-11-28T12:03:45.826611+0000","last_fullsized":"2023-11-28T12:03:45.826611+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:40.312689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827494+0000","last_change":"2023-11-28T12:03:45.827494+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827494+0000","last_undegraded":"2023-11-28T12:03:45.827494+0000","last_fullsized":"2023-11-28T12:03:45.827494+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:05.212437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824983+0000","last_change":"2023-11-28T12:03:45.824983+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824983+0000","last_undegraded":"2023-11-28T12:03:45.824983+0000","last_fullsized":"2023-11-28T12:03:45.824983+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:25.548629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826279+0000","last_change":"2023-11-28T12:03:45.826279+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826279+0000","last_undegraded":"2023-11-28T12:03:45.826279+0000","last_fullsized":"2023-11-28T12:03:45.826279+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:56:00.591608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823223+0000","last_change":"2023-11-28T12:03:45.823223+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823223+0000","last_undegraded":"2023-11-28T12:03:45.823223+0000","last_fullsized":"2023-11-28T12:03:45.823223+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:18:55.370673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825578+0000","last_change":"2023-11-28T12:03:45.825578+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825578+0000","last_undegraded":"2023-11-28T12:03:45.825578+0000","last_fullsized":"2023-11-28T12:03:45.825578+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:49.199732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824871+0000","last_change":"2023-11-28T12:03:45.824871+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824871+0000","last_undegraded":"2023-11-28T12:03:45.824871+0000","last_fullsized":"2023-11-28T12:03:45.824871+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:16.159397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826448+0000","last_change":"2023-11-28T12:03:45.826448+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826448+0000","last_undegraded":"2023-11-28T12:03:45.826448+0000","last_fullsized":"2023-11-28T12:03:45.826448+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:14.009538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826989+0000","last_change":"2023-11-28T12:03:45.826989+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826989+0000","last_undegraded":"2023-11-28T12:03:45.826989+0000","last_fullsized":"2023-11-28T12:03:45.826989+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:01.829853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823534+0000","last_change":"2023-11-28T12:03:45.823534+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823534+0000","last_undegraded":"2023-11-28T12:03:45.823534+0000","last_fullsized":"2023-11-28T12:03:45.823534+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:09.144457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824252+0000","last_change":"2023-11-28T12:03:45.824252+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824252+0000","last_undegraded":"2023-11-28T12:03:45.824252+0000","last_fullsized":"2023-11-28T12:03:45.824252+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:24:00.980890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827390+0000","last_change":"2023-11-28T12:03:45.827390+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827390+0000","last_undegraded":"2023-11-28T12:03:45.827390+0000","last_fullsized":"2023-11-28T12:03:45.827390+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:35.254897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.824829+0000","last_change":"2023-11-28T12:03:45.824829+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.824829+0000","last_undegraded":"2023-11-28T12:03:45.824829+0000","last_fullsized":"2023-11-28T12:03:45.824829+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:12.372093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827105+0000","last_change":"2023-11-28T12:03:45.827105+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827105+0000","last_undegraded":"2023-11-28T12:03:45.827105+0000","last_fullsized":"2023-11-28T12:03:45.827105+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:52:57.245222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825904+0000","last_change":"2023-11-28T12:03:45.825904+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825904+0000","last_undegraded":"2023-11-28T12:03:45.825904+0000","last_fullsized":"2023-11-28T12:03:45.825904+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:41.743469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823370+0000","last_change":"2023-11-28T12:03:45.823370+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823370+0000","last_undegraded":"2023-11-28T12:03:45.823370+0000","last_fullsized":"2023-11-28T12:03:45.823370+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:49.866215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":38,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.827185+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:03:45.827185+0000","last_peered":"2023-11-28T12:03:45.827185+0000","last_clean":"2023-11-28T12:03:45.827185+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:03:45.827185+0000","last_undegraded":"2023-11-28T12:03:45.827185+0000","last_fullsized":"2023-11-28T12:03:45.827185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823311+0000","last_change":"2023-11-28T12:03:45.823311+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823311+0000","last_undegraded":"2023-11-28T12:03:45.823311+0000","last_fullsized":"2023-11-28T12:03:45.823311+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:35.699402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.821980+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:03:45.821980+0000","last_peered":"2023-11-28T12:03:45.821980+0000","last_clean":"2023-11-28T12:03:45.821980+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:03:45.821980+0000","last_undegraded":"2023-11-28T12:03:45.821980+0000","last_fullsized":"2023-11-28T12:03:45.821980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.825622+0000","last_change":"2023-11-28T12:03:45.825622+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.825622+0000","last_undegraded":"2023-11-28T12:03:45.825622+0000","last_fullsized":"2023-11-28T12:03:45.825622+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:10.717376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.828667+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:03:45.828667+0000","last_peered":"2023-11-28T12:03:45.828667+0000","last_clean":"2023-11-28T12:03:45.828667+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:03:45.828667+0000","last_undegraded":"2023-11-28T12:03:45.828667+0000","last_fullsized":"2023-11-28T12:03:45.828667+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":184,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362741+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:03:43.362741+0000","last_peered":"2023-11-28T12:03:43.362741+0000","last_clean":"2023-11-28T12:03:43.362741+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:03:43.362741+0000","last_undegraded":"2023-11-28T12:03:43.362741+0000","last_fullsized":"2023-11-28T12:03:43.362741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.828612+0000","last_change":"2023-11-28T12:03:45.828612+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.828612+0000","last_undegraded":"2023-11-28T12:03:45.828612+0000","last_fullsized":"2023-11-28T12:03:45.828612+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:46.170695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":35,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.827814+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:03:45.827814+0000","last_peered":"2023-11-28T12:03:45.827814+0000","last_clean":"2023-11-28T12:03:45.827814+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:03:45.827814+0000","last_undegraded":"2023-11-28T12:03:45.827814+0000","last_fullsized":"2023-11-28T12:03:45.827814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.827650+0000","last_change":"2023-11-28T12:03:45.827650+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.827650+0000","last_undegraded":"2023-11-28T12:03:45.827650+0000","last_fullsized":"2023-11-28T12:03:45.827650+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:54.463672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362680+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:03:43.362680+0000","last_peered":"2023-11-28T12:03:43.362680+0000","last_clean":"2023-11-28T12:03:43.362680+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:03:43.362680+0000","last_undegraded":"2023-11-28T12:03:43.362680+0000","last_fullsized":"2023-11-28T12:03:43.362680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823878+0000","last_change":"2023-11-28T12:03:45.823878+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823878+0000","last_undegraded":"2023-11-28T12:03:45.823878+0000","last_fullsized":"2023-11-28T12:03:45.823878+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:21.499561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T12:03:45.822971+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:03:45.822971+0000","last_peered":"2023-11-28T12:03:45.822971+0000","last_clean":"2023-11-28T12:03:45.822971+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:03:45.822971+0000","last_undegraded":"2023-11-28T12:03:45.822971+0000","last_fullsized":"2023-11-28T12:03:45.822971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.823246+0000","last_change":"2023-11-28T12:03:45.823246+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.823246+0000","last_undegraded":"2023-11-28T12:03:45.823246+0000","last_fullsized":"2023-11-28T12:03:45.823246+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:05.212437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362659+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:03:43.362659+0000","last_peered":"2023-11-28T12:03:43.362659+0000","last_clean":"2023-11-28T12:03:43.362659+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:03:43.362659+0000","last_undegraded":"2023-11-28T12:03:43.362659+0000","last_fullsized":"2023-11-28T12:03:43.362659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:03:46.820008+0000","last_change":"2023-11-28T12:03:46.820008+0000","last_active":"2023-11-28T12:03:46.820008+0000","last_peered":"2023-11-28T12:03:46.820008+0000","last_clean":"2023-11-28T12:03:46.820008+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:46.820008+0000","last_undegraded":"2023-11-28T12:03:46.820008+0000","last_fullsized":"2023-11-28T12:03:46.820008+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-28T12:03:45.826474+0000","last_change":"2023-11-28T12:03:45.826474+0000","last_active":"2023-11-28T12:03:45.817147+0000","last_peered":"2023-11-28T12:03:45.817147+0000","last_clean":"2023-11-28T12:03:45.817147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:03:45.826474+0000","last_undegraded":"2023-11-28T12:03:45.826474+0000","last_fullsized":"2023-11-28T12:03:45.826474+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:14.009538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T12:03:43.362731+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:03:43.362731+0000","last_peered":"2023-11-28T12:03:43.362731+0000","last_clean":"2023-11-28T12:03:43.362731+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:03:43.362731+0000","last_undegraded":"2023-11-28T12:03:43.362731+0000","last_fullsized":"2023-11-28T12:03:43.362731+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9936,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94361904,"statfs":{"total":96636764160,"available":96626589696,"internally_reserved":0,"allocated":1122304,"data_stored":814570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.66200000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.46000000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.751,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.751}]},{"osd":1,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.872},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.82399999999999995,"15min":0.86699999999999999},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79100000000000004,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.79100000000000004}]},{"osd":2,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.81000000000000005,"15min":0.85099999999999998},"min":{"1min":0.41399999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73499999999999999,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80400000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84199999999999997,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.84199999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76500000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.61199999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.5}]},{"osd":5,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51700000000000002,"15min":0.52300000000000002},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76500000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72699999999999998,"5min":0.752,"15min":0.752},"last":0.52400000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.46600000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.69099999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8664,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363176,"statfs":{"total":96636764160,"available":96627892224,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.435,"5min":0.307,"15min":0.307},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.437}]},{"osd":1,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.39700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.34000000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.53900000000000003,"15min":0.52400000000000002},"min":{"1min":0.47299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.36599999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56100000000000005,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.48099999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.45000000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.497}]},{"osd":5,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.54600000000000004}]},{"osd":7,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.38100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.40200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.56000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9820,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362020,"statfs":{"total":96636764160,"available":96626708480,"internally_reserved":0,"allocated":1134592,"data_stored":820740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.40200000000000002,"15min":0.40999999999999998},"min":{"1min":0.26900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.55100000000000005,"5min":1.2,"15min":1.2},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40400000000000003,"15min":0.41099999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45900000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.30099999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36699999999999999,"15min":0.36899999999999999},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.45100000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.39100000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.47999999999999998,"15min":0.48799999999999999},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.53600000000000003,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.432},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45100000000000001,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67000000000000004,"5min":1.03,"15min":1.03},"last":0.36399999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63700000000000001,"15min":0.64400000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.70699999999999996,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.63300000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.628,"15min":0.63},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78000000000000003,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.82499999999999996,"5min":0.998,"15min":0.998},"last":0.68899999999999995}]},{"osd":6,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.80400000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.77200000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.65700000000000003}]},{"osd":7,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.81200000000000006,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.55200000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.84999999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8792,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363048,"statfs":{"total":96636764160,"available":96627761152,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.55200000000000005,"15min":0.57999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58599999999999997,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45200000000000001,"15min":0.45800000000000002},"min":{"1min":0.28599999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56399999999999995,"5min":1.667,"15min":1.667},"last":0.443}]},{"osd":2,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.57199999999999995,"15min":0.59599999999999997},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.68000000000000005,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.438,"15min":0.44},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44400000000000001,"15min":0.44800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.628,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.64200000000000002,"15min":0.67000000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76100000000000001,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.48399999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63,"15min":0.628},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92400000000000004,"5min":1.536,"15min":1.536},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.75900000000000001,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98199999999999998,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.626}]},{"osd":5,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.56000000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.61799999999999999},"min":{"1min":0.47399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86599999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.53500000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.032,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.501,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.014,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.59299999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92800000000000005,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.52100000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.997,"5min":1.605,"15min":1.605},"last":0.55800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8664,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363176,"statfs":{"total":96636764160,"available":96627892224,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.375},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60499999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39900000000000002,"15min":0.39700000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58199999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.38800000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40899999999999997},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56699999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.439,"15min":0.435},"min":{"1min":0.33500000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.63300000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.316}]},{"osd":4,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81399999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63,"15min":0.63500000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77600000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.79000000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66100000000000003,"15min":0.66900000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83399999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8984,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362856,"statfs":{"total":96636764160,"available":96627564544,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53100000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.44500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58499999999999996,"5min":1.238,"15min":1.238},"last":0.30599999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58699999999999997,"5min":1.53,"15min":1.53},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.502,"15min":0.51600000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57099999999999995,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.36899999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48799999999999999,"15min":0.496},"min":{"1min":0.36199999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55200000000000005,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.47899999999999998,"15min":0.48599999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60499999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77300000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64600000000000002,"15min":0.65200000000000002},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88500000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83699999999999997,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.80200000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86899999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.48199999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84299999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.51300000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.86799999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.81599999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56200000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8920,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362920,"statfs":{"total":96636764160,"available":96627630080,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.503,"15min":0.495},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69099999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.38400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48199999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65900000000000003,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73099999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.435}]},{"osd":2,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73199999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.39800000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52400000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.433,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.752,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.497},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.50700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.746,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73099999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64400000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.35199999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.49299999999999999,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61199999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71199999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41799999999999998}]},{"osd":7,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70999999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8856,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362984,"statfs":{"total":96636764160,"available":96627695616,"internally_reserved":0,"allocated":540672,"data_stored":230372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.39000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.65800000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.61599999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67000000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.629},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.88300000000000001,"15min":0.94499999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61499999999999999,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.60499999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71499999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57699999999999996,"15min":0.58099999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69999999999999996,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66300000000000003}]},{"osd":3,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.434,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69699999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.70599999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.69499999999999995}]},{"osd":4,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64500000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.38400000000000001}]},{"osd":6,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.35199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.497},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63900000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42099999999999999}]},{"osd":7,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.375,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66200000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.442},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68799999999999994,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:03:56.162 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:03:56.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.306+0000 7f3ee2f12700 1 Processor -- start 2023-11-28T12:03:56.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.306+0000 7f3ee2f12700 1 -- start start 2023-11-28T12:03:56.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.306+0000 7f3ee2f12700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3edc0d15f0 0x7f3edc15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:56.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.306+0000 7f3ee2f12700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3edc15e010 0x7f3edc0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:56.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.307+0000 7f3ee2f12700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3edc15af30 con 0x7f3edc06e400 2023-11-28T12:03:56.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.307+0000 7f3ee2f12700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3edc15a9d0 con 0x7f3edc15e010 2023-11-28T12:03:56.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.307+0000 7f3ee2f12700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc1584e0 con 0x7f3edc0d15f0 2023-11-28T12:03:56.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.307+0000 7f3edbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3edc0d15f0 0x7f3edc15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:56.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.307+0000 7f3ee0cae700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3edc06e400 0x7f3edc06e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:56792/0 (socket says 172.21.15.57:56792) 2023-11-28T12:03:56.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.307+0000 7f3ee14af700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3edc15e010 0x7f3edc0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:56.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.307+0000 7f3ee0cae700 1 -- 172.21.15.57:0/2309537604 learned_addr learned my addr 172.21.15.57:0/2309537604 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:56.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.307+0000 7f3ee14af700 1 -- 172.21.15.57:0/2309537604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3edc0d15f0 msgr2=0x7f3edc15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:56.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.307+0000 7f3ee14af700 1 --2- 172.21.15.57:0/2309537604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3edc0d15f0 0x7f3edc15b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.307+0000 7f3ee14af700 1 -- 172.21.15.57:0/2309537604 >> v1:172.21.15.181:6789/0 conn(0x7f3edc06e400 legacy=0x7f3edc06e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:56.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.307+0000 7f3ee14af700 1 -- 172.21.15.57:0/2309537604 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc15ef40 con 0x7f3edc15e010 2023-11-28T12:03:56.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.307+0000 7f3edbfff700 1 --2- 172.21.15.57:0/2309537604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3edc0d15f0 0x7f3edc15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:03:56.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.307+0000 7f3ee14af700 1 --2- 172.21.15.57:0/2309537604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc15e010 0x7f3edc0ce970 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009d90 tx=0x7f3ed400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91e4c8d9beebfe36 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:56.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.308+0000 7f3edb7fe700 1 -- 172.21.15.57:0/2309537604 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed400e070 con 0x7f3edc15e010 2023-11-28T12:03:56.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.308+0000 7f3edb7fe700 1 -- 172.21.15.57:0/2309537604 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed400bb20 con 0x7f3edc15e010 2023-11-28T12:03:56.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.308+0000 7f3edb7fe700 1 -- 172.21.15.57:0/2309537604 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4012430 con 0x7f3edc15e010 2023-11-28T12:03:56.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.308+0000 7f3ee2f12700 1 -- 172.21.15.57:0/2309537604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc15e010 msgr2=0x7f3edc0ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:56.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.308+0000 7f3ee2f12700 1 --2- 172.21.15.57:0/2309537604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc15e010 0x7f3edc0ce970 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009d90 tx=0x7f3ed400b330 comp rx=0 tx=0).stop 2023-11-28T12:03:56.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.308+0000 7f3ee2f12700 1 -- 172.21.15.57:0/2309537604 shutdown_connections 2023-11-28T12:03:56.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.308+0000 7f3ee2f12700 1 --2- 172.21.15.57:0/2309537604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc15e010 0x7f3edc0ce970 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.308+0000 7f3ee2f12700 1 --2- 172.21.15.57:0/2309537604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3edc0d15f0 0x7f3edc15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.308+0000 7f3ee2f12700 1 -- 172.21.15.57:0/2309537604 >> 172.21.15.57:0/2309537604 conn(0x7f3edc06a270 msgr2=0x7f3edc06a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:56.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.308+0000 7f3ee2f12700 1 -- 172.21.15.57:0/2309537604 shutdown_connections 2023-11-28T12:03:56.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.308+0000 7f3ee2f12700 1 -- 172.21.15.57:0/2309537604 wait complete. 2023-11-28T12:03:56.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.310+0000 7f3ee2f12700 1 Processor -- start 2023-11-28T12:03:56.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.310+0000 7f3ee2f12700 1 -- start start 2023-11-28T12:03:56.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee2f12700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3edc06e400 0x7f3edc14c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:56.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee2f12700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc0d15f0 0x7f3edc06b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:56.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee2f12700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3edc15e010 0x7f3edc151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:56.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee2f12700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc06ccf0 con 0x7f3edc06e400 2023-11-28T12:03:56.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee2f12700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc14c630 con 0x7f3edc0d15f0 2023-11-28T12:03:56.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee2f12700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc06c0e0 con 0x7f3edc15e010 2023-11-28T12:03:56.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee0cae700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3edc06e400 0x7f3edc14c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:56.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee0cae700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3edc06e400 0x7f3edc14c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46470/0 (socket says 172.21.15.57:46470) 2023-11-28T12:03:56.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee0cae700 1 -- 172.21.15.57:0/1687915791 learned_addr learned my addr 172.21.15.57:0/1687915791 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:56.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3edbfff700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc0d15f0 0x7f3edc06b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:56.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee14af700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3edc15e010 0x7f3edc151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:56.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee0cae700 1 -- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3edc15e010 msgr2=0x7f3edc151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:56.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee0cae700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3edc15e010 0x7f3edc151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee0cae700 1 -- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc0d15f0 msgr2=0x7f3edc06b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:56.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee0cae700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc0d15f0 0x7f3edc06b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee0cae700 1 -- 172.21.15.57:0/1687915791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec80094a0 con 0x7f3edc06e400 2023-11-28T12:03:56.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3edbfff700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc0d15f0 0x7f3edc06b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:03:56.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ee0cae700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3edc06e400 0x7f3edc14c0f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000c520 tx=0x7f3ed000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:56.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.311+0000 7f3ed97fa700 1 -- 172.21.15.57:0/1687915791 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0005d70 con 0x7f3edc06e400 2023-11-28T12:03:56.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.312+0000 7f3ed97fa700 1 -- 172.21.15.57:0/1687915791 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed00070d0 con 0x7f3edc06e400 2023-11-28T12:03:56.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.312+0000 7f3ee2f12700 1 -- 172.21.15.57:0/1687915791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed4009a70 con 0x7f3edc06e400 2023-11-28T12:03:56.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.312+0000 7f3ed97fa700 1 -- 172.21.15.57:0/1687915791 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0007440 con 0x7f3edc06e400 2023-11-28T12:03:56.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.312+0000 7f3ee2f12700 1 -- 172.21.15.57:0/1687915791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3edc15ec60 con 0x7f3edc06e400 2023-11-28T12:03:56.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.313+0000 7f3ee2f12700 1 -- 172.21.15.57:0/1687915791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3edc15e980 con 0x7f3edc06e400 2023-11-28T12:03:56.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.314+0000 7f3ed97fa700 1 -- 172.21.15.57:0/1687915791 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3ed0019030 con 0x7f3edc06e400 2023-11-28T12:03:56.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.314+0000 7f3ed97fa700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3ec00682f0 0x7f3ec006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:56.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.314+0000 7f3ed97fa700 1 -- 172.21.15.57:0/1687915791 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3ed0085270 con 0x7f3edc06e400 2023-11-28T12:03:56.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.314+0000 7f3edbfff700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3ec00682f0 0x7f3ec006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:56.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.315+0000 7f3edbfff700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3ec00682f0 0x7f3ec006a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3edc150e60 tx=0x7f3ec8009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:56.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.318+0000 7f3ed97fa700 1 -- 172.21.15.57:0/1687915791 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3ed0051c20 con 0x7f3edc06e400 2023-11-28T12:03:56.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.516+0000 7f3ee2f12700 1 -- 172.21.15.57:0/1687915791 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3edc06ce30 con 0x7f3ec00682f0 2023-11-28T12:03:56.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.525+0000 7f3ed97fa700 1 -- 172.21.15.57:0/1687915791 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312174 (secure 0 0 0) 0x7f3ec8011e60 con 0x7f3ec00682f0 2023-11-28T12:03:56.530 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:56.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.529+0000 7f3ee2f12700 1 -- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3ec00682f0 msgr2=0x7f3ec006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:56.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.529+0000 7f3ee2f12700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3ec00682f0 0x7f3ec006a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3edc150e60 tx=0x7f3ec8009210 comp rx=0 tx=0).stop 2023-11-28T12:03:56.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.529+0000 7f3ee2f12700 1 -- 172.21.15.57:0/1687915791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3edc06e400 msgr2=0x7f3edc14c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:56.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.529+0000 7f3ee2f12700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3edc06e400 0x7f3edc14c0f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000c520 tx=0x7f3ed000c830 comp rx=0 tx=0).stop 2023-11-28T12:03:56.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.529+0000 7f3ee2f12700 1 -- 172.21.15.57:0/1687915791 shutdown_connections 2023-11-28T12:03:56.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.529+0000 7f3ee2f12700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3ec00682f0 0x7f3ec006a7a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.529+0000 7f3ee2f12700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3edc15e010 0x7f3edc151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.529+0000 7f3ee2f12700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3edc06e400 0x7f3edc14c0f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.529+0000 7f3ee2f12700 1 --2- 172.21.15.57:0/1687915791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc0d15f0 0x7f3edc06b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.529+0000 7f3ee2f12700 1 -- 172.21.15.57:0/1687915791 >> 172.21.15.57:0/1687915791 conn(0x7f3edc06a270 msgr2=0x7f3edc155e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:56.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.529+0000 7f3ee2f12700 1 -- 172.21.15.57:0/1687915791 shutdown_connections 2023-11-28T12:03:56.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.529+0000 7f3ee2f12700 1 -- 172.21.15.57:0/1687915791 wait complete. 2023-11-28T12:03:56.534 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:03:56.543 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2023-11-28T12:03:55.692114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":74216,"kb_used_data":5736,"kb_used_omap":0,"kb_used_meta":68416,"kb_avail":754900504,"statfs":{"total":773094113280,"available":773018116096,"internally_reserved":0,"allocated":5873664,"data_stored":3191644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70057984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.424282"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464436+0000","last_change":"2023-11-28T12:03:47.286593+0000","last_active":"2023-11-28T12:03:48.464436+0000","last_peered":"2023-11-28T12:03:48.464436+0000","last_clean":"2023-11-28T12:03:48.464436+0000","last_became_active":"2023-11-28T12:03:47.286336+0000","last_became_peered":"2023-11-28T12:03:47.286336+0000","last_unstale":"2023-11-28T12:03:48.464436+0000","last_undegraded":"2023-11-28T12:03:48.464436+0000","last_fullsized":"2023-11-28T12:03:48.464436+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:15.012399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.465086+0000","last_change":"2023-11-28T12:03:47.285666+0000","last_active":"2023-11-28T12:03:48.465086+0000","last_peered":"2023-11-28T12:03:48.465086+0000","last_clean":"2023-11-28T12:03:48.465086+0000","last_became_active":"2023-11-28T12:03:47.285262+0000","last_became_peered":"2023-11-28T12:03:47.285262+0000","last_unstale":"2023-11-28T12:03:48.465086+0000","last_undegraded":"2023-11-28T12:03:48.465086+0000","last_fullsized":"2023-11-28T12:03:48.465086+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:47.173556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.278979+0000","last_change":"2023-11-28T12:03:47.294167+0000","last_active":"2023-11-28T12:03:48.278979+0000","last_peered":"2023-11-28T12:03:48.278979+0000","last_clean":"2023-11-28T12:03:48.278979+0000","last_became_active":"2023-11-28T12:03:47.293632+0000","last_became_peered":"2023-11-28T12:03:47.293632+0000","last_unstale":"2023-11-28T12:03:48.278979+0000","last_undegraded":"2023-11-28T12:03:48.278979+0000","last_fullsized":"2023-11-28T12:03:48.278979+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:46.276411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.465561+0000","last_change":"2023-11-28T12:03:47.286306+0000","last_active":"2023-11-28T12:03:48.465561+0000","last_peered":"2023-11-28T12:03:48.465561+0000","last_clean":"2023-11-28T12:03:48.465561+0000","last_became_active":"2023-11-28T12:03:47.286032+0000","last_became_peered":"2023-11-28T12:03:47.286032+0000","last_unstale":"2023-11-28T12:03:48.465561+0000","last_undegraded":"2023-11-28T12:03:48.465561+0000","last_fullsized":"2023-11-28T12:03:48.465561+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:17.232018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.278980+0000","last_change":"2023-11-28T12:03:47.279937+0000","last_active":"2023-11-28T12:03:48.278980+0000","last_peered":"2023-11-28T12:03:48.278980+0000","last_clean":"2023-11-28T12:03:48.278980+0000","last_became_active":"2023-11-28T12:03:47.279504+0000","last_became_peered":"2023-11-28T12:03:47.279504+0000","last_unstale":"2023-11-28T12:03:48.278980+0000","last_undegraded":"2023-11-28T12:03:48.278980+0000","last_fullsized":"2023-11-28T12:03:48.278980+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:09:23.839405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.463927+0000","last_change":"2023-11-28T12:03:47.287000+0000","last_active":"2023-11-28T12:03:48.463927+0000","last_peered":"2023-11-28T12:03:48.463927+0000","last_clean":"2023-11-28T12:03:48.463927+0000","last_became_active":"2023-11-28T12:03:47.281366+0000","last_became_peered":"2023-11-28T12:03:47.281366+0000","last_unstale":"2023-11-28T12:03:48.463927+0000","last_undegraded":"2023-11-28T12:03:48.463927+0000","last_fullsized":"2023-11-28T12:03:48.463927+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:47.173556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.278699+0000","last_change":"2023-11-28T12:03:47.288356+0000","last_active":"2023-11-28T12:03:48.278699+0000","last_peered":"2023-11-28T12:03:48.278699+0000","last_clean":"2023-11-28T12:03:48.278699+0000","last_became_active":"2023-11-28T12:03:47.288064+0000","last_became_peered":"2023-11-28T12:03:47.288064+0000","last_unstale":"2023-11-28T12:03:48.278699+0000","last_undegraded":"2023-11-28T12:03:48.278699+0000","last_fullsized":"2023-11-28T12:03:48.278699+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:29:34.401822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.278718+0000","last_change":"2023-11-28T12:03:47.293556+0000","last_active":"2023-11-28T12:03:48.278718+0000","last_peered":"2023-11-28T12:03:48.278718+0000","last_clean":"2023-11-28T12:03:48.278718+0000","last_became_active":"2023-11-28T12:03:47.293291+0000","last_became_peered":"2023-11-28T12:03:47.293291+0000","last_unstale":"2023-11-28T12:03:48.278718+0000","last_undegraded":"2023-11-28T12:03:48.278718+0000","last_fullsized":"2023-11-28T12:03:48.278718+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:03.716221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.280105+0000","last_change":"2023-11-28T12:03:47.290045+0000","last_active":"2023-11-28T12:03:48.280105+0000","last_peered":"2023-11-28T12:03:48.280105+0000","last_clean":"2023-11-28T12:03:48.280105+0000","last_became_active":"2023-11-28T12:03:47.289787+0000","last_became_peered":"2023-11-28T12:03:47.289787+0000","last_unstale":"2023-11-28T12:03:48.280105+0000","last_undegraded":"2023-11-28T12:03:48.280105+0000","last_fullsized":"2023-11-28T12:03:48.280105+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:12.060313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464388+0000","last_change":"2023-11-28T12:03:47.282312+0000","last_active":"2023-11-28T12:03:48.464388+0000","last_peered":"2023-11-28T12:03:48.464388+0000","last_clean":"2023-11-28T12:03:48.464388+0000","last_became_active":"2023-11-28T12:03:47.282076+0000","last_became_peered":"2023-11-28T12:03:47.282076+0000","last_unstale":"2023-11-28T12:03:48.464388+0000","last_undegraded":"2023-11-28T12:03:48.464388+0000","last_fullsized":"2023-11-28T12:03:48.464388+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:17.232018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464107+0000","last_change":"2023-11-28T12:03:47.289327+0000","last_active":"2023-11-28T12:03:48.464107+0000","last_peered":"2023-11-28T12:03:48.464107+0000","last_clean":"2023-11-28T12:03:48.464107+0000","last_became_active":"2023-11-28T12:03:47.289000+0000","last_became_peered":"2023-11-28T12:03:47.289000+0000","last_unstale":"2023-11-28T12:03:48.464107+0000","last_undegraded":"2023-11-28T12:03:48.464107+0000","last_fullsized":"2023-11-28T12:03:48.464107+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:16:48.056097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279573+0000","last_change":"2023-11-28T12:03:47.289005+0000","last_active":"2023-11-28T12:03:48.279573+0000","last_peered":"2023-11-28T12:03:48.279573+0000","last_clean":"2023-11-28T12:03:48.279573+0000","last_became_active":"2023-11-28T12:03:47.288723+0000","last_became_peered":"2023-11-28T12:03:47.288723+0000","last_unstale":"2023-11-28T12:03:48.279573+0000","last_undegraded":"2023-11-28T12:03:48.279573+0000","last_fullsized":"2023-11-28T12:03:48.279573+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:45.654911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464433+0000","last_change":"2023-11-28T12:03:47.282509+0000","last_active":"2023-11-28T12:03:48.464433+0000","last_peered":"2023-11-28T12:03:48.464433+0000","last_clean":"2023-11-28T12:03:48.464433+0000","last_became_active":"2023-11-28T12:03:47.282235+0000","last_became_peered":"2023-11-28T12:03:47.282235+0000","last_unstale":"2023-11-28T12:03:48.464433+0000","last_undegraded":"2023-11-28T12:03:48.464433+0000","last_fullsized":"2023-11-28T12:03:48.464433+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:57.659164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279044+0000","last_change":"2023-11-28T12:03:47.292024+0000","last_active":"2023-11-28T12:03:48.279044+0000","last_peered":"2023-11-28T12:03:48.279044+0000","last_clean":"2023-11-28T12:03:48.279044+0000","last_became_active":"2023-11-28T12:03:47.291747+0000","last_became_peered":"2023-11-28T12:03:47.291747+0000","last_unstale":"2023-11-28T12:03:48.279044+0000","last_undegraded":"2023-11-28T12:03:48.279044+0000","last_fullsized":"2023-11-28T12:03:48.279044+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:39.105387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464735+0000","last_change":"2023-11-28T12:03:47.291162+0000","last_active":"2023-11-28T12:03:48.464735+0000","last_peered":"2023-11-28T12:03:48.464735+0000","last_clean":"2023-11-28T12:03:48.464735+0000","last_became_active":"2023-11-28T12:03:47.290899+0000","last_became_peered":"2023-11-28T12:03:47.290899+0000","last_unstale":"2023-11-28T12:03:48.464735+0000","last_undegraded":"2023-11-28T12:03:48.464735+0000","last_fullsized":"2023-11-28T12:03:48.464735+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:13.374954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464211+0000","last_change":"2023-11-28T12:03:47.281935+0000","last_active":"2023-11-28T12:03:48.464211+0000","last_peered":"2023-11-28T12:03:48.464211+0000","last_clean":"2023-11-28T12:03:48.464211+0000","last_became_active":"2023-11-28T12:03:47.281636+0000","last_became_peered":"2023-11-28T12:03:47.281636+0000","last_unstale":"2023-11-28T12:03:48.464211+0000","last_undegraded":"2023-11-28T12:03:48.464211+0000","last_fullsized":"2023-11-28T12:03:48.464211+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:55.466533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279654+0000","last_change":"2023-11-28T12:03:47.289848+0000","last_active":"2023-11-28T12:03:48.279654+0000","last_peered":"2023-11-28T12:03:48.279654+0000","last_clean":"2023-11-28T12:03:48.279654+0000","last_became_active":"2023-11-28T12:03:47.287999+0000","last_became_peered":"2023-11-28T12:03:47.287999+0000","last_unstale":"2023-11-28T12:03:48.279654+0000","last_undegraded":"2023-11-28T12:03:48.279654+0000","last_fullsized":"2023-11-28T12:03:48.279654+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:11.857406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.465148+0000","last_change":"2023-11-28T12:03:47.285801+0000","last_active":"2023-11-28T12:03:48.465148+0000","last_peered":"2023-11-28T12:03:48.465148+0000","last_clean":"2023-11-28T12:03:48.465148+0000","last_became_active":"2023-11-28T12:03:47.285496+0000","last_became_peered":"2023-11-28T12:03:47.285496+0000","last_unstale":"2023-11-28T12:03:48.465148+0000","last_undegraded":"2023-11-28T12:03:48.465148+0000","last_fullsized":"2023-11-28T12:03:48.465148+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:15.012399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279091+0000","last_change":"2023-11-28T12:03:47.291693+0000","last_active":"2023-11-28T12:03:48.279091+0000","last_peered":"2023-11-28T12:03:48.279091+0000","last_clean":"2023-11-28T12:03:48.279091+0000","last_became_active":"2023-11-28T12:03:47.291397+0000","last_became_peered":"2023-11-28T12:03:47.291397+0000","last_unstale":"2023-11-28T12:03:48.279091+0000","last_undegraded":"2023-11-28T12:03:48.279091+0000","last_fullsized":"2023-11-28T12:03:48.279091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:11.706765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464670+0000","last_change":"2023-11-28T12:03:47.288175+0000","last_active":"2023-11-28T12:03:48.464670+0000","last_peered":"2023-11-28T12:03:48.464670+0000","last_clean":"2023-11-28T12:03:48.464670+0000","last_became_active":"2023-11-28T12:03:47.287888+0000","last_became_peered":"2023-11-28T12:03:47.287888+0000","last_unstale":"2023-11-28T12:03:48.464670+0000","last_undegraded":"2023-11-28T12:03:48.464670+0000","last_fullsized":"2023-11-28T12:03:48.464670+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:06.215298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464324+0000","last_change":"2023-11-28T12:03:47.282479+0000","last_active":"2023-11-28T12:03:48.464324+0000","last_peered":"2023-11-28T12:03:48.464324+0000","last_clean":"2023-11-28T12:03:48.464324+0000","last_became_active":"2023-11-28T12:03:47.282176+0000","last_became_peered":"2023-11-28T12:03:47.282176+0000","last_unstale":"2023-11-28T12:03:48.464324+0000","last_undegraded":"2023-11-28T12:03:48.464324+0000","last_fullsized":"2023-11-28T12:03:48.464324+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:53:49.423445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.278978+0000","last_change":"2023-11-28T12:03:47.291542+0000","last_active":"2023-11-28T12:03:48.278978+0000","last_peered":"2023-11-28T12:03:48.278978+0000","last_clean":"2023-11-28T12:03:48.278978+0000","last_became_active":"2023-11-28T12:03:47.291287+0000","last_became_peered":"2023-11-28T12:03:47.291287+0000","last_unstale":"2023-11-28T12:03:48.278978+0000","last_undegraded":"2023-11-28T12:03:48.278978+0000","last_fullsized":"2023-11-28T12:03:48.278978+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:59:42.269415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.463914+0000","last_change":"2023-11-28T12:03:47.289419+0000","last_active":"2023-11-28T12:03:48.463914+0000","last_peered":"2023-11-28T12:03:48.463914+0000","last_clean":"2023-11-28T12:03:48.463914+0000","last_became_active":"2023-11-28T12:03:47.289013+0000","last_became_peered":"2023-11-28T12:03:47.289013+0000","last_unstale":"2023-11-28T12:03:48.463914+0000","last_undegraded":"2023-11-28T12:03:48.463914+0000","last_fullsized":"2023-11-28T12:03:48.463914+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:24.478694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.278554+0000","last_change":"2023-11-28T12:03:47.291116+0000","last_active":"2023-11-28T12:03:48.278554+0000","last_peered":"2023-11-28T12:03:48.278554+0000","last_clean":"2023-11-28T12:03:48.278554+0000","last_became_active":"2023-11-28T12:03:47.290805+0000","last_became_peered":"2023-11-28T12:03:47.290805+0000","last_unstale":"2023-11-28T12:03:48.278554+0000","last_undegraded":"2023-11-28T12:03:48.278554+0000","last_fullsized":"2023-11-28T12:03:48.278554+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:45.654911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279552+0000","last_change":"2023-11-28T12:03:47.290826+0000","last_active":"2023-11-28T12:03:48.279552+0000","last_peered":"2023-11-28T12:03:48.279552+0000","last_clean":"2023-11-28T12:03:48.279552+0000","last_became_active":"2023-11-28T12:03:47.287742+0000","last_became_peered":"2023-11-28T12:03:47.287742+0000","last_unstale":"2023-11-28T12:03:48.279552+0000","last_undegraded":"2023-11-28T12:03:48.279552+0000","last_fullsized":"2023-11-28T12:03:48.279552+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:59:42.269415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279759+0000","last_change":"2023-11-28T12:03:47.281748+0000","last_active":"2023-11-28T12:03:48.279759+0000","last_peered":"2023-11-28T12:03:48.279759+0000","last_clean":"2023-11-28T12:03:48.279759+0000","last_became_active":"2023-11-28T12:03:47.281295+0000","last_became_peered":"2023-11-28T12:03:47.281295+0000","last_unstale":"2023-11-28T12:03:48.279759+0000","last_undegraded":"2023-11-28T12:03:48.279759+0000","last_fullsized":"2023-11-28T12:03:48.279759+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:03.060240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464649+0000","last_change":"2023-11-28T12:03:47.296071+0000","last_active":"2023-11-28T12:03:48.464649+0000","last_peered":"2023-11-28T12:03:48.464649+0000","last_clean":"2023-11-28T12:03:48.464649+0000","last_became_active":"2023-11-28T12:03:47.295772+0000","last_became_peered":"2023-11-28T12:03:47.295772+0000","last_unstale":"2023-11-28T12:03:48.464649+0000","last_undegraded":"2023-11-28T12:03:48.464649+0000","last_fullsized":"2023-11-28T12:03:48.464649+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:13.374954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464949+0000","last_change":"2023-11-28T12:03:47.282464+0000","last_active":"2023-11-28T12:03:48.464949+0000","last_peered":"2023-11-28T12:03:48.464949+0000","last_clean":"2023-11-28T12:03:48.464949+0000","last_became_active":"2023-11-28T12:03:47.282161+0000","last_became_peered":"2023-11-28T12:03:47.282161+0000","last_unstale":"2023-11-28T12:03:48.464949+0000","last_undegraded":"2023-11-28T12:03:48.464949+0000","last_fullsized":"2023-11-28T12:03:48.464949+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:58.073098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279722+0000","last_change":"2023-11-28T12:03:47.286657+0000","last_active":"2023-11-28T12:03:48.279722+0000","last_peered":"2023-11-28T12:03:48.279722+0000","last_clean":"2023-11-28T12:03:48.279722+0000","last_became_active":"2023-11-28T12:03:47.283467+0000","last_became_peered":"2023-11-28T12:03:47.283467+0000","last_unstale":"2023-11-28T12:03:48.279722+0000","last_undegraded":"2023-11-28T12:03:48.279722+0000","last_fullsized":"2023-11-28T12:03:48.279722+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:52.877085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464866+0000","last_change":"2023-11-28T12:03:47.289314+0000","last_active":"2023-11-28T12:03:48.464866+0000","last_peered":"2023-11-28T12:03:48.464866+0000","last_clean":"2023-11-28T12:03:48.464866+0000","last_became_active":"2023-11-28T12:03:47.289029+0000","last_became_peered":"2023-11-28T12:03:47.289029+0000","last_unstale":"2023-11-28T12:03:48.464866+0000","last_undegraded":"2023-11-28T12:03:48.464866+0000","last_fullsized":"2023-11-28T12:03:48.464866+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:36.702263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279684+0000","last_change":"2023-11-28T12:03:47.291052+0000","last_active":"2023-11-28T12:03:48.279684+0000","last_peered":"2023-11-28T12:03:48.279684+0000","last_clean":"2023-11-28T12:03:48.279684+0000","last_became_active":"2023-11-28T12:03:47.290742+0000","last_became_peered":"2023-11-28T12:03:47.290742+0000","last_unstale":"2023-11-28T12:03:48.279684+0000","last_undegraded":"2023-11-28T12:03:48.279684+0000","last_fullsized":"2023-11-28T12:03:48.279684+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:18:07.151269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464171+0000","last_change":"2023-11-28T12:03:47.286925+0000","last_active":"2023-11-28T12:03:48.464171+0000","last_peered":"2023-11-28T12:03:48.464171+0000","last_clean":"2023-11-28T12:03:48.464171+0000","last_became_active":"2023-11-28T12:03:47.286246+0000","last_became_peered":"2023-11-28T12:03:47.286246+0000","last_unstale":"2023-11-28T12:03:48.464171+0000","last_undegraded":"2023-11-28T12:03:48.464171+0000","last_fullsized":"2023-11-28T12:03:48.464171+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:50.869076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464826+0000","last_change":"2023-11-28T12:03:47.285035+0000","last_active":"2023-11-28T12:03:48.464826+0000","last_peered":"2023-11-28T12:03:48.464826+0000","last_clean":"2023-11-28T12:03:48.464826+0000","last_became_active":"2023-11-28T12:03:47.284803+0000","last_became_peered":"2023-11-28T12:03:47.284803+0000","last_unstale":"2023-11-28T12:03:48.464826+0000","last_undegraded":"2023-11-28T12:03:48.464826+0000","last_fullsized":"2023-11-28T12:03:48.464826+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.315550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.292366+0000","last_change":"2023-11-28T12:03:46.836982+0000","last_active":"2023-11-28T12:03:47.292366+0000","last_peered":"2023-11-28T12:03:47.292366+0000","last_clean":"2023-11-28T12:03:47.292366+0000","last_became_active":"2023-11-28T12:03:46.836718+0000","last_became_peered":"2023-11-28T12:03:46.836718+0000","last_unstale":"2023-11-28T12:03:47.292366+0000","last_undegraded":"2023-11-28T12:03:47.292366+0000","last_fullsized":"2023-11-28T12:03:47.292366+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:00.642126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464806+0000","last_change":"2023-11-28T12:03:47.284970+0000","last_active":"2023-11-28T12:03:48.464806+0000","last_peered":"2023-11-28T12:03:48.464806+0000","last_clean":"2023-11-28T12:03:48.464806+0000","last_became_active":"2023-11-28T12:03:47.284633+0000","last_became_peered":"2023-11-28T12:03:47.284633+0000","last_unstale":"2023-11-28T12:03:48.464806+0000","last_undegraded":"2023-11-28T12:03:48.464806+0000","last_fullsized":"2023-11-28T12:03:48.464806+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:17.232018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.286089+0000","last_change":"2023-11-28T12:03:46.849808+0000","last_active":"2023-11-28T12:03:47.286089+0000","last_peered":"2023-11-28T12:03:47.286089+0000","last_clean":"2023-11-28T12:03:47.286089+0000","last_became_active":"2023-11-28T12:03:46.849525+0000","last_became_peered":"2023-11-28T12:03:46.849525+0000","last_unstale":"2023-11-28T12:03:47.286089+0000","last_undegraded":"2023-11-28T12:03:47.286089+0000","last_fullsized":"2023-11-28T12:03:47.286089+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:39.291687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464337+0000","last_change":"2023-11-28T12:03:47.281472+0000","last_active":"2023-11-28T12:03:48.464337+0000","last_peered":"2023-11-28T12:03:48.464337+0000","last_clean":"2023-11-28T12:03:48.464337+0000","last_became_active":"2023-11-28T12:03:47.281215+0000","last_became_peered":"2023-11-28T12:03:47.281215+0000","last_unstale":"2023-11-28T12:03:48.464337+0000","last_undegraded":"2023-11-28T12:03:48.464337+0000","last_fullsized":"2023-11-28T12:03:48.464337+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:11.720237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.295489+0000","last_change":"2023-11-28T12:03:46.844148+0000","last_active":"2023-11-28T12:03:47.295489+0000","last_peered":"2023-11-28T12:03:47.295489+0000","last_clean":"2023-11-28T12:03:47.295489+0000","last_became_active":"2023-11-28T12:03:46.842846+0000","last_became_peered":"2023-11-28T12:03:46.842846+0000","last_unstale":"2023-11-28T12:03:47.295489+0000","last_undegraded":"2023-11-28T12:03:47.295489+0000","last_fullsized":"2023-11-28T12:03:47.295489+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:59.530852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279260+0000","last_change":"2023-11-28T12:03:47.279052+0000","last_active":"2023-11-28T12:03:48.279260+0000","last_peered":"2023-11-28T12:03:48.279260+0000","last_clean":"2023-11-28T12:03:48.279260+0000","last_became_active":"2023-11-28T12:03:47.278780+0000","last_became_peered":"2023-11-28T12:03:47.278780+0000","last_unstale":"2023-11-28T12:03:48.279260+0000","last_undegraded":"2023-11-28T12:03:48.279260+0000","last_fullsized":"2023-11-28T12:03:48.279260+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:15.373192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.287760+0000","last_change":"2023-11-28T12:03:46.846224+0000","last_active":"2023-11-28T12:03:47.287760+0000","last_peered":"2023-11-28T12:03:47.287760+0000","last_clean":"2023-11-28T12:03:47.287760+0000","last_became_active":"2023-11-28T12:03:46.845925+0000","last_became_peered":"2023-11-28T12:03:46.845925+0000","last_unstale":"2023-11-28T12:03:47.287760+0000","last_undegraded":"2023-11-28T12:03:47.287760+0000","last_fullsized":"2023-11-28T12:03:47.287760+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:10.717376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.278978+0000","last_change":"2023-11-28T12:03:47.293163+0000","last_active":"2023-11-28T12:03:48.278978+0000","last_peered":"2023-11-28T12:03:48.278978+0000","last_clean":"2023-11-28T12:03:48.278978+0000","last_became_active":"2023-11-28T12:03:47.292850+0000","last_became_peered":"2023-11-28T12:03:47.292850+0000","last_unstale":"2023-11-28T12:03:48.278978+0000","last_undegraded":"2023-11-28T12:03:48.278978+0000","last_fullsized":"2023-11-28T12:03:48.278978+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:16.327533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.293597+0000","last_change":"2023-11-28T12:03:46.844271+0000","last_active":"2023-11-28T12:03:47.293597+0000","last_peered":"2023-11-28T12:03:47.293597+0000","last_clean":"2023-11-28T12:03:47.293597+0000","last_became_active":"2023-11-28T12:03:46.843943+0000","last_became_peered":"2023-11-28T12:03:46.843943+0000","last_unstale":"2023-11-28T12:03:47.293597+0000","last_undegraded":"2023-11-28T12:03:47.293597+0000","last_fullsized":"2023-11-28T12:03:47.293597+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:56.656303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.465283+0000","last_change":"2023-11-28T12:03:47.289052+0000","last_active":"2023-11-28T12:03:48.465283+0000","last_peered":"2023-11-28T12:03:48.465283+0000","last_clean":"2023-11-28T12:03:48.465283+0000","last_became_active":"2023-11-28T12:03:47.288734+0000","last_became_peered":"2023-11-28T12:03:47.288734+0000","last_unstale":"2023-11-28T12:03:48.465283+0000","last_undegraded":"2023-11-28T12:03:48.465283+0000","last_fullsized":"2023-11-28T12:03:48.465283+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:17.162258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.279148+0000","last_change":"2023-11-28T12:03:46.839179+0000","last_active":"2023-11-28T12:03:47.279148+0000","last_peered":"2023-11-28T12:03:47.279148+0000","last_clean":"2023-11-28T12:03:47.279148+0000","last_became_active":"2023-11-28T12:03:46.838883+0000","last_became_peered":"2023-11-28T12:03:46.838883+0000","last_unstale":"2023-11-28T12:03:47.279148+0000","last_undegraded":"2023-11-28T12:03:47.279148+0000","last_fullsized":"2023-11-28T12:03:47.279148+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:46.170695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279908+0000","last_change":"2023-11-28T12:03:47.278670+0000","last_active":"2023-11-28T12:03:48.279908+0000","last_peered":"2023-11-28T12:03:48.279908+0000","last_clean":"2023-11-28T12:03:48.279908+0000","last_became_active":"2023-11-28T12:03:47.278386+0000","last_became_peered":"2023-11-28T12:03:47.278386+0000","last_unstale":"2023-11-28T12:03:48.279908+0000","last_undegraded":"2023-11-28T12:03:48.279908+0000","last_fullsized":"2023-11-28T12:03:48.279908+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:57.082966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.273712+0000","last_change":"2023-11-28T12:03:46.837834+0000","last_active":"2023-11-28T12:03:47.273712+0000","last_peered":"2023-11-28T12:03:47.273712+0000","last_clean":"2023-11-28T12:03:47.273712+0000","last_became_active":"2023-11-28T12:03:46.837600+0000","last_became_peered":"2023-11-28T12:03:46.837600+0000","last_unstale":"2023-11-28T12:03:47.273712+0000","last_undegraded":"2023-11-28T12:03:47.273712+0000","last_fullsized":"2023-11-28T12:03:47.273712+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:13:44.179722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464959+0000","last_change":"2023-11-28T12:03:47.293075+0000","last_active":"2023-11-28T12:03:48.464959+0000","last_peered":"2023-11-28T12:03:48.464959+0000","last_clean":"2023-11-28T12:03:48.464959+0000","last_became_active":"2023-11-28T12:03:47.292735+0000","last_became_peered":"2023-11-28T12:03:47.292735+0000","last_unstale":"2023-11-28T12:03:48.464959+0000","last_undegraded":"2023-11-28T12:03:48.464959+0000","last_fullsized":"2023-11-28T12:03:48.464959+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:50.869076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.279917+0000","last_change":"2023-11-28T12:03:46.846591+0000","last_active":"2023-11-28T12:03:47.279917+0000","last_peered":"2023-11-28T12:03:47.279917+0000","last_clean":"2023-11-28T12:03:47.279917+0000","last_became_active":"2023-11-28T12:03:46.846323+0000","last_became_peered":"2023-11-28T12:03:46.846323+0000","last_unstale":"2023-11-28T12:03:47.279917+0000","last_undegraded":"2023-11-28T12:03:47.279917+0000","last_fullsized":"2023-11-28T12:03:47.279917+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:16.229157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.278708+0000","last_change":"2023-11-28T12:03:47.272611+0000","last_active":"2023-11-28T12:03:48.278708+0000","last_peered":"2023-11-28T12:03:48.278708+0000","last_clean":"2023-11-28T12:03:48.278708+0000","last_became_active":"2023-11-28T12:03:47.272337+0000","last_became_peered":"2023-11-28T12:03:47.272337+0000","last_unstale":"2023-11-28T12:03:48.278708+0000","last_undegraded":"2023-11-28T12:03:48.278708+0000","last_fullsized":"2023-11-28T12:03:48.278708+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:02.613808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.276907+0000","last_change":"2023-11-28T12:03:46.835235+0000","last_active":"2023-11-28T12:03:47.276907+0000","last_peered":"2023-11-28T12:03:47.276907+0000","last_clean":"2023-11-28T12:03:47.276907+0000","last_became_active":"2023-11-28T12:03:46.834844+0000","last_became_peered":"2023-11-28T12:03:46.834844+0000","last_unstale":"2023-11-28T12:03:47.276907+0000","last_undegraded":"2023-11-28T12:03:47.276907+0000","last_fullsized":"2023-11-28T12:03:47.276907+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:57.070237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464702+0000","last_change":"2023-11-28T12:03:47.284974+0000","last_active":"2023-11-28T12:03:48.464702+0000","last_peered":"2023-11-28T12:03:48.464702+0000","last_clean":"2023-11-28T12:03:48.464702+0000","last_became_active":"2023-11-28T12:03:47.284638+0000","last_became_peered":"2023-11-28T12:03:47.284638+0000","last_unstale":"2023-11-28T12:03:48.464702+0000","last_undegraded":"2023-11-28T12:03:48.464702+0000","last_fullsized":"2023-11-28T12:03:48.464702+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:02.337299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.268897+0000","last_change":"2023-11-28T12:03:46.842800+0000","last_active":"2023-11-28T12:03:47.268897+0000","last_peered":"2023-11-28T12:03:47.268897+0000","last_clean":"2023-11-28T12:03:47.268897+0000","last_became_active":"2023-11-28T12:03:46.842524+0000","last_became_peered":"2023-11-28T12:03:46.842524+0000","last_unstale":"2023-11-28T12:03:47.268897+0000","last_undegraded":"2023-11-28T12:03:47.268897+0000","last_fullsized":"2023-11-28T12:03:47.268897+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:40.312689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.280284+0000","last_change":"2023-11-28T12:03:47.293709+0000","last_active":"2023-11-28T12:03:48.280284+0000","last_peered":"2023-11-28T12:03:48.280284+0000","last_clean":"2023-11-28T12:03:48.280284+0000","last_became_active":"2023-11-28T12:03:47.293396+0000","last_became_peered":"2023-11-28T12:03:47.293396+0000","last_unstale":"2023-11-28T12:03:48.280284+0000","last_undegraded":"2023-11-28T12:03:48.280284+0000","last_fullsized":"2023-11-28T12:03:48.280284+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:57.082966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.291500+0000","last_change":"2023-11-28T12:03:46.844883+0000","last_active":"2023-11-28T12:03:47.291500+0000","last_peered":"2023-11-28T12:03:47.291500+0000","last_clean":"2023-11-28T12:03:47.291500+0000","last_became_active":"2023-11-28T12:03:46.844616+0000","last_became_peered":"2023-11-28T12:03:46.844616+0000","last_unstale":"2023-11-28T12:03:47.291500+0000","last_undegraded":"2023-11-28T12:03:47.291500+0000","last_fullsized":"2023-11-28T12:03:47.291500+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:05.212437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279045+0000","last_change":"2023-11-28T12:03:47.279875+0000","last_active":"2023-11-28T12:03:48.279045+0000","last_peered":"2023-11-28T12:03:48.279045+0000","last_clean":"2023-11-28T12:03:48.279045+0000","last_became_active":"2023-11-28T12:03:47.279206+0000","last_became_peered":"2023-11-28T12:03:47.279206+0000","last_unstale":"2023-11-28T12:03:48.279045+0000","last_undegraded":"2023-11-28T12:03:48.279045+0000","last_fullsized":"2023-11-28T12:03:48.279045+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:57:38.323846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.300348+0000","last_change":"2023-11-28T12:03:46.839096+0000","last_active":"2023-11-28T12:03:47.300348+0000","last_peered":"2023-11-28T12:03:47.300348+0000","last_clean":"2023-11-28T12:03:47.300348+0000","last_became_active":"2023-11-28T12:03:46.838826+0000","last_became_peered":"2023-11-28T12:03:46.838826+0000","last_unstale":"2023-11-28T12:03:47.300348+0000","last_undegraded":"2023-11-28T12:03:47.300348+0000","last_fullsized":"2023-11-28T12:03:47.300348+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:25.548629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464563+0000","last_change":"2023-11-28T12:03:47.289594+0000","last_active":"2023-11-28T12:03:48.464563+0000","last_peered":"2023-11-28T12:03:48.464563+0000","last_clean":"2023-11-28T12:03:48.464563+0000","last_became_active":"2023-11-28T12:03:47.289314+0000","last_became_peered":"2023-11-28T12:03:47.289314+0000","last_unstale":"2023-11-28T12:03:48.464563+0000","last_undegraded":"2023-11-28T12:03:48.464563+0000","last_fullsized":"2023-11-28T12:03:48.464563+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.315550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.293135+0000","last_change":"2023-11-28T12:03:46.839001+0000","last_active":"2023-11-28T12:03:47.293135+0000","last_peered":"2023-11-28T12:03:47.293135+0000","last_clean":"2023-11-28T12:03:47.293135+0000","last_became_active":"2023-11-28T12:03:46.838670+0000","last_became_peered":"2023-11-28T12:03:46.838670+0000","last_unstale":"2023-11-28T12:03:47.293135+0000","last_undegraded":"2023-11-28T12:03:47.293135+0000","last_fullsized":"2023-11-28T12:03:47.293135+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:56:00.591608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279198+0000","last_change":"2023-11-28T12:03:47.291787+0000","last_active":"2023-11-28T12:03:48.279198+0000","last_peered":"2023-11-28T12:03:48.279198+0000","last_clean":"2023-11-28T12:03:48.279198+0000","last_became_active":"2023-11-28T12:03:47.291499+0000","last_became_peered":"2023-11-28T12:03:47.291499+0000","last_unstale":"2023-11-28T12:03:48.279198+0000","last_undegraded":"2023-11-28T12:03:48.279198+0000","last_fullsized":"2023-11-28T12:03:48.279198+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:12.060313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.273448+0000","last_change":"2023-11-28T12:03:46.845927+0000","last_active":"2023-11-28T12:03:47.273448+0000","last_peered":"2023-11-28T12:03:47.273448+0000","last_clean":"2023-11-28T12:03:47.273448+0000","last_became_active":"2023-11-28T12:03:46.845703+0000","last_became_peered":"2023-11-28T12:03:46.845703+0000","last_unstale":"2023-11-28T12:03:47.273448+0000","last_undegraded":"2023-11-28T12:03:47.273448+0000","last_fullsized":"2023-11-28T12:03:47.273448+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:18:55.370673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279841+0000","last_change":"2023-11-28T12:03:47.279514+0000","last_active":"2023-11-28T12:03:48.279841+0000","last_peered":"2023-11-28T12:03:48.279841+0000","last_clean":"2023-11-28T12:03:48.279841+0000","last_became_active":"2023-11-28T12:03:47.279243+0000","last_became_peered":"2023-11-28T12:03:47.279243+0000","last_unstale":"2023-11-28T12:03:48.279841+0000","last_undegraded":"2023-11-28T12:03:48.279841+0000","last_fullsized":"2023-11-28T12:03:48.279841+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:11.857406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.275978+0000","last_change":"2023-11-28T12:03:46.836808+0000","last_active":"2023-11-28T12:03:47.275978+0000","last_peered":"2023-11-28T12:03:47.275978+0000","last_clean":"2023-11-28T12:03:47.275978+0000","last_became_active":"2023-11-28T12:03:46.836559+0000","last_became_peered":"2023-11-28T12:03:46.836559+0000","last_unstale":"2023-11-28T12:03:47.275978+0000","last_undegraded":"2023-11-28T12:03:47.275978+0000","last_fullsized":"2023-11-28T12:03:47.275978+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:49.199732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279445+0000","last_change":"2023-11-28T12:03:47.290097+0000","last_active":"2023-11-28T12:03:48.279445+0000","last_peered":"2023-11-28T12:03:48.279445+0000","last_clean":"2023-11-28T12:03:48.279445+0000","last_became_active":"2023-11-28T12:03:47.288482+0000","last_became_peered":"2023-11-28T12:03:47.288482+0000","last_unstale":"2023-11-28T12:03:48.279445+0000","last_undegraded":"2023-11-28T12:03:48.279445+0000","last_fullsized":"2023-11-28T12:03:48.279445+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:03.060240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.293867+0000","last_change":"2023-11-28T12:03:46.838635+0000","last_active":"2023-11-28T12:03:47.293867+0000","last_peered":"2023-11-28T12:03:47.293867+0000","last_clean":"2023-11-28T12:03:47.293867+0000","last_became_active":"2023-11-28T12:03:46.837729+0000","last_became_peered":"2023-11-28T12:03:46.837729+0000","last_unstale":"2023-11-28T12:03:47.293867+0000","last_undegraded":"2023-11-28T12:03:47.293867+0000","last_fullsized":"2023-11-28T12:03:47.293867+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:16.159397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.278827+0000","last_change":"2023-11-28T12:03:47.285257+0000","last_active":"2023-11-28T12:03:48.278827+0000","last_peered":"2023-11-28T12:03:48.278827+0000","last_clean":"2023-11-28T12:03:48.278827+0000","last_became_active":"2023-11-28T12:03:47.284789+0000","last_became_peered":"2023-11-28T12:03:47.284789+0000","last_unstale":"2023-11-28T12:03:48.278827+0000","last_undegraded":"2023-11-28T12:03:48.278827+0000","last_fullsized":"2023-11-28T12:03:48.278827+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:52.877085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.293433+0000","last_change":"2023-11-28T12:03:46.847716+0000","last_active":"2023-11-28T12:03:47.293433+0000","last_peered":"2023-11-28T12:03:47.293433+0000","last_clean":"2023-11-28T12:03:47.293433+0000","last_became_active":"2023-11-28T12:03:46.847466+0000","last_became_peered":"2023-11-28T12:03:46.847466+0000","last_unstale":"2023-11-28T12:03:47.293433+0000","last_undegraded":"2023-11-28T12:03:47.293433+0000","last_fullsized":"2023-11-28T12:03:47.293433+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:14.009538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.463793+0000","last_change":"2023-11-28T12:03:47.296130+0000","last_active":"2023-11-28T12:03:48.463793+0000","last_peered":"2023-11-28T12:03:48.463793+0000","last_clean":"2023-11-28T12:03:48.463793+0000","last_became_active":"2023-11-28T12:03:47.295810+0000","last_became_peered":"2023-11-28T12:03:47.295810+0000","last_unstale":"2023-11-28T12:03:48.463793+0000","last_undegraded":"2023-11-28T12:03:48.463793+0000","last_fullsized":"2023-11-28T12:03:48.463793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:02.337299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.286668+0000","last_change":"2023-11-28T12:03:46.840664+0000","last_active":"2023-11-28T12:03:47.286668+0000","last_peered":"2023-11-28T12:03:47.286668+0000","last_clean":"2023-11-28T12:03:47.286668+0000","last_became_active":"2023-11-28T12:03:46.840399+0000","last_became_peered":"2023-11-28T12:03:46.840399+0000","last_unstale":"2023-11-28T12:03:47.286668+0000","last_undegraded":"2023-11-28T12:03:47.286668+0000","last_fullsized":"2023-11-28T12:03:47.286668+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:01.829853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.278758+0000","last_change":"2023-11-28T12:03:47.291473+0000","last_active":"2023-11-28T12:03:48.278758+0000","last_peered":"2023-11-28T12:03:48.278758+0000","last_clean":"2023-11-28T12:03:48.278758+0000","last_became_active":"2023-11-28T12:03:47.291228+0000","last_became_peered":"2023-11-28T12:03:47.291228+0000","last_unstale":"2023-11-28T12:03:48.278758+0000","last_undegraded":"2023-11-28T12:03:48.278758+0000","last_fullsized":"2023-11-28T12:03:48.278758+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:00:13.277155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.291431+0000","last_change":"2023-11-28T12:03:46.846897+0000","last_active":"2023-11-28T12:03:47.291431+0000","last_peered":"2023-11-28T12:03:47.291431+0000","last_clean":"2023-11-28T12:03:47.291431+0000","last_became_active":"2023-11-28T12:03:46.846619+0000","last_became_peered":"2023-11-28T12:03:46.846619+0000","last_unstale":"2023-11-28T12:03:47.291431+0000","last_undegraded":"2023-11-28T12:03:47.291431+0000","last_fullsized":"2023-11-28T12:03:47.291431+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:09.144457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464543+0000","last_change":"2023-11-28T12:03:47.285878+0000","last_active":"2023-11-28T12:03:48.464543+0000","last_peered":"2023-11-28T12:03:48.464543+0000","last_clean":"2023-11-28T12:03:48.464543+0000","last_became_active":"2023-11-28T12:03:47.285376+0000","last_became_peered":"2023-11-28T12:03:47.285376+0000","last_unstale":"2023-11-28T12:03:48.464543+0000","last_undegraded":"2023-11-28T12:03:48.464543+0000","last_fullsized":"2023-11-28T12:03:48.464543+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:50.869076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.280822+0000","last_change":"2023-11-28T12:03:46.846074+0000","last_active":"2023-11-28T12:03:47.280822+0000","last_peered":"2023-11-28T12:03:47.280822+0000","last_clean":"2023-11-28T12:03:47.280822+0000","last_became_active":"2023-11-28T12:03:46.845820+0000","last_became_peered":"2023-11-28T12:03:46.845820+0000","last_unstale":"2023-11-28T12:03:47.280822+0000","last_undegraded":"2023-11-28T12:03:47.280822+0000","last_fullsized":"2023-11-28T12:03:47.280822+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:24:00.980890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464789+0000","last_change":"2023-11-28T12:03:47.287937+0000","last_active":"2023-11-28T12:03:48.464789+0000","last_peered":"2023-11-28T12:03:48.464789+0000","last_clean":"2023-11-28T12:03:48.464789+0000","last_became_active":"2023-11-28T12:03:47.284009+0000","last_became_peered":"2023-11-28T12:03:47.284009+0000","last_unstale":"2023-11-28T12:03:48.464789+0000","last_undegraded":"2023-11-28T12:03:48.464789+0000","last_fullsized":"2023-11-28T12:03:48.464789+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:15.012399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.285710+0000","last_change":"2023-11-28T12:03:46.844044+0000","last_active":"2023-11-28T12:03:47.285710+0000","last_peered":"2023-11-28T12:03:47.285710+0000","last_clean":"2023-11-28T12:03:47.285710+0000","last_became_active":"2023-11-28T12:03:46.843336+0000","last_became_peered":"2023-11-28T12:03:46.843336+0000","last_unstale":"2023-11-28T12:03:47.285710+0000","last_undegraded":"2023-11-28T12:03:47.285710+0000","last_fullsized":"2023-11-28T12:03:47.285710+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:35.254897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464997+0000","last_change":"2023-11-28T12:03:47.288745+0000","last_active":"2023-11-28T12:03:48.464997+0000","last_peered":"2023-11-28T12:03:48.464997+0000","last_clean":"2023-11-28T12:03:48.464997+0000","last_became_active":"2023-11-28T12:03:47.282828+0000","last_became_peered":"2023-11-28T12:03:47.282828+0000","last_unstale":"2023-11-28T12:03:48.464997+0000","last_undegraded":"2023-11-28T12:03:48.464997+0000","last_fullsized":"2023-11-28T12:03:48.464997+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:40.294548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.267400+0000","last_change":"2023-11-28T12:03:46.837597+0000","last_active":"2023-11-28T12:03:47.267400+0000","last_peered":"2023-11-28T12:03:47.267400+0000","last_clean":"2023-11-28T12:03:47.267400+0000","last_became_active":"2023-11-28T12:03:46.837340+0000","last_became_peered":"2023-11-28T12:03:46.837340+0000","last_unstale":"2023-11-28T12:03:47.267400+0000","last_undegraded":"2023-11-28T12:03:47.267400+0000","last_fullsized":"2023-11-28T12:03:47.267400+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:12.372093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279441+0000","last_change":"2023-11-28T12:03:47.285803+0000","last_active":"2023-11-28T12:03:48.279441+0000","last_peered":"2023-11-28T12:03:48.279441+0000","last_clean":"2023-11-28T12:03:48.279441+0000","last_became_active":"2023-11-28T12:03:47.282970+0000","last_became_peered":"2023-11-28T12:03:47.282970+0000","last_unstale":"2023-11-28T12:03:48.279441+0000","last_undegraded":"2023-11-28T12:03:48.279441+0000","last_fullsized":"2023-11-28T12:03:48.279441+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:03.716221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.277491+0000","last_change":"2023-11-28T12:03:46.844778+0000","last_active":"2023-11-28T12:03:47.277491+0000","last_peered":"2023-11-28T12:03:47.277491+0000","last_clean":"2023-11-28T12:03:47.277491+0000","last_became_active":"2023-11-28T12:03:46.844497+0000","last_became_peered":"2023-11-28T12:03:46.844497+0000","last_unstale":"2023-11-28T12:03:47.277491+0000","last_undegraded":"2023-11-28T12:03:47.277491+0000","last_fullsized":"2023-11-28T12:03:47.277491+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:52:57.245222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464553+0000","last_change":"2023-11-28T12:03:47.275279+0000","last_active":"2023-11-28T12:03:48.464553+0000","last_peered":"2023-11-28T12:03:48.464553+0000","last_clean":"2023-11-28T12:03:48.464553+0000","last_became_active":"2023-11-28T12:03:47.275005+0000","last_became_peered":"2023-11-28T12:03:47.275005+0000","last_unstale":"2023-11-28T12:03:48.464553+0000","last_undegraded":"2023-11-28T12:03:48.464553+0000","last_fullsized":"2023-11-28T12:03:48.464553+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:06:24.323902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.275718+0000","last_change":"2023-11-28T12:03:46.837294+0000","last_active":"2023-11-28T12:03:47.275718+0000","last_peered":"2023-11-28T12:03:47.275718+0000","last_clean":"2023-11-28T12:03:47.275718+0000","last_became_active":"2023-11-28T12:03:46.836958+0000","last_became_peered":"2023-11-28T12:03:46.836958+0000","last_unstale":"2023-11-28T12:03:47.275718+0000","last_undegraded":"2023-11-28T12:03:47.275718+0000","last_fullsized":"2023-11-28T12:03:47.275718+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:41.743469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279678+0000","last_change":"2023-11-28T12:03:47.287890+0000","last_active":"2023-11-28T12:03:48.279678+0000","last_peered":"2023-11-28T12:03:48.279678+0000","last_clean":"2023-11-28T12:03:48.279678+0000","last_became_active":"2023-11-28T12:03:47.287582+0000","last_became_peered":"2023-11-28T12:03:47.287582+0000","last_unstale":"2023-11-28T12:03:48.279678+0000","last_undegraded":"2023-11-28T12:03:48.279678+0000","last_fullsized":"2023-11-28T12:03:48.279678+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:31.908591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.268014+0000","last_change":"2023-11-28T12:03:46.839115+0000","last_active":"2023-11-28T12:03:47.268014+0000","last_peered":"2023-11-28T12:03:47.268014+0000","last_clean":"2023-11-28T12:03:47.268014+0000","last_became_active":"2023-11-28T12:03:46.838749+0000","last_became_peered":"2023-11-28T12:03:46.838749+0000","last_unstale":"2023-11-28T12:03:47.268014+0000","last_undegraded":"2023-11-28T12:03:47.268014+0000","last_fullsized":"2023-11-28T12:03:47.268014+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:49.866215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":41,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.268781+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:03:47.268781+0000","last_peered":"2023-11-28T12:03:47.268781+0000","last_clean":"2023-11-28T12:03:47.268781+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:03:47.268781+0000","last_undegraded":"2023-11-28T12:03:47.268781+0000","last_fullsized":"2023-11-28T12:03:47.268781+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.465532+0000","last_change":"2023-11-28T12:03:47.284437+0000","last_active":"2023-11-28T12:03:48.465532+0000","last_peered":"2023-11-28T12:03:48.465532+0000","last_clean":"2023-11-28T12:03:48.465532+0000","last_became_active":"2023-11-28T12:03:47.284099+0000","last_became_peered":"2023-11-28T12:03:47.284099+0000","last_unstale":"2023-11-28T12:03:48.465532+0000","last_undegraded":"2023-11-28T12:03:48.465532+0000","last_fullsized":"2023-11-28T12:03:48.465532+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:47.173556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.267513+0000","last_change":"2023-11-28T12:03:46.843753+0000","last_active":"2023-11-28T12:03:47.267513+0000","last_peered":"2023-11-28T12:03:47.267513+0000","last_clean":"2023-11-28T12:03:47.267513+0000","last_became_active":"2023-11-28T12:03:46.843480+0000","last_became_peered":"2023-11-28T12:03:46.843480+0000","last_unstale":"2023-11-28T12:03:47.267513+0000","last_undegraded":"2023-11-28T12:03:47.267513+0000","last_fullsized":"2023-11-28T12:03:47.267513+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:35.699402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":44,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.267763+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:03:47.267763+0000","last_peered":"2023-11-28T12:03:47.267763+0000","last_clean":"2023-11-28T12:03:47.267763+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:03:47.267763+0000","last_undegraded":"2023-11-28T12:03:47.267763+0000","last_fullsized":"2023-11-28T12:03:47.267763+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464025+0000","last_change":"2023-11-28T12:03:47.281816+0000","last_active":"2023-11-28T12:03:48.464025+0000","last_peered":"2023-11-28T12:03:48.464025+0000","last_clean":"2023-11-28T12:03:48.464025+0000","last_became_active":"2023-11-28T12:03:47.281580+0000","last_became_peered":"2023-11-28T12:03:47.281580+0000","last_unstale":"2023-11-28T12:03:48.464025+0000","last_undegraded":"2023-11-28T12:03:48.464025+0000","last_fullsized":"2023-11-28T12:03:48.464025+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.315550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.276814+0000","last_change":"2023-11-28T12:03:46.839153+0000","last_active":"2023-11-28T12:03:47.276814+0000","last_peered":"2023-11-28T12:03:47.276814+0000","last_clean":"2023-11-28T12:03:47.276814+0000","last_became_active":"2023-11-28T12:03:46.838887+0000","last_became_peered":"2023-11-28T12:03:46.838887+0000","last_unstale":"2023-11-28T12:03:47.276814+0000","last_undegraded":"2023-11-28T12:03:47.276814+0000","last_fullsized":"2023-11-28T12:03:47.276814+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:10.717376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":39,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.272471+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:03:47.272471+0000","last_peered":"2023-11-28T12:03:47.272471+0000","last_clean":"2023-11-28T12:03:47.272471+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:03:47.272471+0000","last_undegraded":"2023-11-28T12:03:47.272471+0000","last_fullsized":"2023-11-28T12:03:47.272471+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":187,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.275972+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:03:47.275972+0000","last_peered":"2023-11-28T12:03:47.275972+0000","last_clean":"2023-11-28T12:03:47.275972+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:03:47.275972+0000","last_undegraded":"2023-11-28T12:03:47.275972+0000","last_fullsized":"2023-11-28T12:03:47.275972+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279357+0000","last_change":"2023-11-28T12:03:47.280796+0000","last_active":"2023-11-28T12:03:48.279357+0000","last_peered":"2023-11-28T12:03:48.279357+0000","last_clean":"2023-11-28T12:03:48.279357+0000","last_became_active":"2023-11-28T12:03:47.280507+0000","last_became_peered":"2023-11-28T12:03:47.280507+0000","last_unstale":"2023-11-28T12:03:48.279357+0000","last_undegraded":"2023-11-28T12:03:48.279357+0000","last_fullsized":"2023-11-28T12:03:48.279357+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:11.706765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.276368+0000","last_change":"2023-11-28T12:03:46.846020+0000","last_active":"2023-11-28T12:03:47.276368+0000","last_peered":"2023-11-28T12:03:47.276368+0000","last_clean":"2023-11-28T12:03:47.276368+0000","last_became_active":"2023-11-28T12:03:46.845725+0000","last_became_peered":"2023-11-28T12:03:46.845725+0000","last_unstale":"2023-11-28T12:03:47.276368+0000","last_undegraded":"2023-11-28T12:03:47.276368+0000","last_fullsized":"2023-11-28T12:03:47.276368+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:49:46.170695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.276003+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:03:47.276003+0000","last_peered":"2023-11-28T12:03:47.276003+0000","last_clean":"2023-11-28T12:03:47.276003+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:03:47.276003+0000","last_undegraded":"2023-11-28T12:03:47.276003+0000","last_fullsized":"2023-11-28T12:03:47.276003+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.465264+0000","last_change":"2023-11-28T12:03:47.286672+0000","last_active":"2023-11-28T12:03:48.465264+0000","last_peered":"2023-11-28T12:03:48.465264+0000","last_clean":"2023-11-28T12:03:48.465264+0000","last_became_active":"2023-11-28T12:03:47.286298+0000","last_became_peered":"2023-11-28T12:03:47.286298+0000","last_unstale":"2023-11-28T12:03:48.465264+0000","last_undegraded":"2023-11-28T12:03:48.465264+0000","last_fullsized":"2023-11-28T12:03:48.465264+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:43:50.202593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.273788+0000","last_change":"2023-11-28T12:03:46.844411+0000","last_active":"2023-11-28T12:03:47.273788+0000","last_peered":"2023-11-28T12:03:47.273788+0000","last_clean":"2023-11-28T12:03:47.273788+0000","last_became_active":"2023-11-28T12:03:46.844078+0000","last_became_peered":"2023-11-28T12:03:46.844078+0000","last_unstale":"2023-11-28T12:03:47.273788+0000","last_undegraded":"2023-11-28T12:03:47.273788+0000","last_fullsized":"2023-11-28T12:03:47.273788+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:54.463672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.275856+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:03:47.275856+0000","last_peered":"2023-11-28T12:03:47.275856+0000","last_clean":"2023-11-28T12:03:47.275856+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:03:47.275856+0000","last_undegraded":"2023-11-28T12:03:47.275856+0000","last_fullsized":"2023-11-28T12:03:47.275856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464281+0000","last_change":"2023-11-28T12:03:47.287163+0000","last_active":"2023-11-28T12:03:48.464281+0000","last_peered":"2023-11-28T12:03:48.464281+0000","last_clean":"2023-11-28T12:03:48.464281+0000","last_became_active":"2023-11-28T12:03:47.286585+0000","last_became_peered":"2023-11-28T12:03:47.286585+0000","last_unstale":"2023-11-28T12:03:48.464281+0000","last_undegraded":"2023-11-28T12:03:48.464281+0000","last_fullsized":"2023-11-28T12:03:48.464281+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:13.374954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.294104+0000","last_change":"2023-11-28T12:03:46.846055+0000","last_active":"2023-11-28T12:03:47.294104+0000","last_peered":"2023-11-28T12:03:47.294104+0000","last_clean":"2023-11-28T12:03:47.294104+0000","last_became_active":"2023-11-28T12:03:46.845760+0000","last_became_peered":"2023-11-28T12:03:46.845760+0000","last_unstale":"2023-11-28T12:03:47.294104+0000","last_undegraded":"2023-11-28T12:03:47.294104+0000","last_fullsized":"2023-11-28T12:03:47.294104+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:21.499561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.277525+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:03:47.277525+0000","last_peered":"2023-11-28T12:03:47.277525+0000","last_clean":"2023-11-28T12:03:47.277525+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:03:47.277525+0000","last_undegraded":"2023-11-28T12:03:47.277525+0000","last_fullsized":"2023-11-28T12:03:47.277525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.280169+0000","last_change":"2023-11-28T12:03:47.280419+0000","last_active":"2023-11-28T12:03:48.280169+0000","last_peered":"2023-11-28T12:03:48.280169+0000","last_clean":"2023-11-28T12:03:48.280169+0000","last_became_active":"2023-11-28T12:03:47.280084+0000","last_became_peered":"2023-11-28T12:03:47.280084+0000","last_unstale":"2023-11-28T12:03:48.280169+0000","last_undegraded":"2023-11-28T12:03:48.280169+0000","last_fullsized":"2023-11-28T12:03:48.280169+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:46.276411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.277666+0000","last_change":"2023-11-28T12:03:46.843100+0000","last_active":"2023-11-28T12:03:47.277666+0000","last_peered":"2023-11-28T12:03:47.277666+0000","last_clean":"2023-11-28T12:03:47.277666+0000","last_became_active":"2023-11-28T12:03:46.842872+0000","last_became_peered":"2023-11-28T12:03:46.842872+0000","last_unstale":"2023-11-28T12:03:47.277666+0000","last_undegraded":"2023-11-28T12:03:47.277666+0000","last_fullsized":"2023-11-28T12:03:47.277666+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:05.212437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.464552+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:03:48.464552+0000","last_peered":"2023-11-28T12:03:48.464552+0000","last_clean":"2023-11-28T12:03:48.464552+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:03:48.464552+0000","last_undegraded":"2023-11-28T12:03:48.464552+0000","last_fullsized":"2023-11-28T12:03:48.464552+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.279308+0000","last_change":"2023-11-28T12:03:47.294454+0000","last_active":"2023-11-28T12:03:48.279308+0000","last_peered":"2023-11-28T12:03:48.279308+0000","last_clean":"2023-11-28T12:03:48.279308+0000","last_became_active":"2023-11-28T12:03:47.293893+0000","last_became_peered":"2023-11-28T12:03:47.293893+0000","last_unstale":"2023-11-28T12:03:48.279308+0000","last_undegraded":"2023-11-28T12:03:48.279308+0000","last_fullsized":"2023-11-28T12:03:48.279308+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:46.820008+0000","last_clean_scrub_stamp":"2023-11-28T12:03:46.820008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:15.373192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T12:03:47.268964+0000","last_change":"2023-11-28T12:03:46.829713+0000","last_active":"2023-11-28T12:03:47.268964+0000","last_peered":"2023-11-28T12:03:47.268964+0000","last_clean":"2023-11-28T12:03:47.268964+0000","last_became_active":"2023-11-28T12:03:46.829405+0000","last_became_peered":"2023-11-28T12:03:46.829405+0000","last_unstale":"2023-11-28T12:03:47.268964+0000","last_undegraded":"2023-11-28T12:03:47.268964+0000","last_fullsized":"2023-11-28T12:03:47.268964+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:03:45.817147+0000","last_clean_scrub_stamp":"2023-11-28T12:03:45.817147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:14.009538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T12:03:48.465304+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:03:48.465304+0000","last_peered":"2023-11-28T12:03:48.465304+0000","last_clean":"2023-11-28T12:03:48.465304+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:03:48.465304+0000","last_undegraded":"2023-11-28T12:03:48.465304+0000","last_fullsized":"2023-11-28T12:03:48.465304+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10184,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361656,"statfs":{"total":96636764160,"available":96626335744,"internally_reserved":0,"allocated":1179648,"data_stored":841273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.66200000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.46000000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.751,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.82399999999999995,"15min":0.86699999999999999},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79100000000000004,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.38800000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.81000000000000005,"15min":0.85099999999999998},"min":{"1min":0.41399999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73499999999999999,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80400000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.41399999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84199999999999997,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.61699999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76500000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.61199999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.59099999999999997}]},{"osd":5,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51700000000000002,"15min":0.52300000000000002},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76500000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72699999999999998,"5min":0.752,"15min":0.752},"last":0.60499999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.46600000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.69099999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8836,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363004,"statfs":{"total":96636764160,"available":96627716096,"internally_reserved":0,"allocated":585728,"data_stored":251417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.435,"5min":0.307,"15min":0.307},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.39700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.58699999999999997}]},{"osd":2,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.53900000000000003,"15min":0.52400000000000002},"min":{"1min":0.47299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.61799999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56100000000000005,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.54600000000000004}]},{"osd":4,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.45000000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.69299999999999995}]},{"osd":7,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.38100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.40200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.64800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9992,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94361848,"statfs":{"total":96636764160,"available":96626532352,"internally_reserved":0,"allocated":1179648,"data_stored":841309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.40200000000000002,"15min":0.40999999999999998},"min":{"1min":0.26900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.55100000000000005,"5min":1.2,"15min":1.2},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40400000000000003,"15min":0.41099999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45900000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.60799999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36699999999999999,"15min":0.36899999999999999},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.45100000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.57499999999999996}]},{"osd":3,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.47999999999999998,"15min":0.48799999999999999},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.53600000000000003,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.505},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45100000000000001,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67000000000000004,"5min":1.03,"15min":1.03},"last":0.54800000000000004}]},{"osd":4,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63700000000000001,"15min":0.64400000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.70699999999999996,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.68400000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.628,"15min":0.63},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78000000000000003,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.82499999999999996,"5min":0.998,"15min":0.998},"last":0.79900000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.80400000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.77200000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.871}]},{"osd":7,"last update":"Tue Nov 28 12:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.81200000000000006,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.55200000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.84999999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.91800000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8964,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362876,"statfs":{"total":96636764160,"available":96627585024,"internally_reserved":0,"allocated":585728,"data_stored":250837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.55200000000000005,"15min":0.57999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58599999999999997,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45200000000000001,"15min":0.45800000000000002},"min":{"1min":0.28599999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56399999999999995,"5min":1.667,"15min":1.667},"last":0.34399999999999997}]},{"osd":2,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.57199999999999995,"15min":0.59599999999999997},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.68000000000000005,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.438,"15min":0.44},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.41099999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44400000000000001,"15min":0.44800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.628,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.64200000000000002,"15min":0.67000000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76100000000000001,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.443}]},{"osd":4,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63,"15min":0.628},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92400000000000004,"5min":1.536,"15min":1.536},"last":0.5},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.75900000000000001,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98199999999999998,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.48699999999999999}]},{"osd":5,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.56000000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.61799999999999999},"min":{"1min":0.47399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86599999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.432}]},{"osd":6,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.53500000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.032,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.501,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.014,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Tue Nov 28 12:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92800000000000005,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.52100000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.997,"5min":1.605,"15min":1.605},"last":0.53400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8904,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362936,"statfs":{"total":96636764160,"available":96627646464,"internally_reserved":0,"allocated":589824,"data_stored":252329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.376,"15min":0.375},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60499999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39900000000000002,"15min":0.39700000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58199999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.38800000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40899999999999997},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56699999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.439,"15min":0.435},"min":{"1min":0.33500000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.63300000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.316}]},{"osd":4,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81399999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63,"15min":0.63500000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77600000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.79000000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66100000000000003,"15min":0.66900000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83399999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9228,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362612,"statfs":{"total":96636764160,"available":96627314688,"internally_reserved":0,"allocated":593920,"data_stored":252351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53100000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.44500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58499999999999996,"5min":1.238,"15min":1.238},"last":0.318}]},{"osd":1,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58699999999999997,"5min":1.53,"15min":1.53},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.502,"15min":0.51600000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57099999999999995,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48799999999999999,"15min":0.496},"min":{"1min":0.36199999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55200000000000005,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.443},{"interface":"front","average":{"1min":0.439,"5min":0.47899999999999998,"15min":0.48599999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60499999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.45300000000000001}]},{"osd":4,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77300000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.624},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64600000000000002,"15min":0.65200000000000002},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88500000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83699999999999997,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.80200000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73599999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86899999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.48199999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84299999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Tue Nov 28 12:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.86799999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.81599999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.76100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9088,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362752,"statfs":{"total":96636764160,"available":96627458048,"internally_reserved":0,"allocated":581632,"data_stored":251347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.503,"15min":0.495},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69099999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.44},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.38400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60199999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65900000000000003,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73099999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.46300000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73199999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.39800000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.433,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.752,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.50700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.746,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58699999999999997}]},{"osd":5,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73099999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64400000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.49299999999999999,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61199999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71199999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57199999999999995}]},{"osd":7,"last update":"Tue Nov 28 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70999999999999996,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.621}]}]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9020,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362820,"statfs":{"total":96636764160,"available":96627527680,"internally_reserved":0,"allocated":577536,"data_stored":250781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.39000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.65800000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67000000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.88300000000000001,"15min":0.94499999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61499999999999999,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.48899999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71499999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57699999999999996,"15min":0.58099999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69999999999999996,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.432}]},{"osd":3,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.434,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69699999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.70599999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52300000000000002}]},{"osd":4,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64500000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.59999999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.35199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63900000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.58799999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.375,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66200000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68799999999999994,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:03:56.549 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T12:03:56.549 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T12:03:56.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.728+0000 7f85e8f7a700 1 Processor -- start 2023-11-28T12:03:56.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.728+0000 7f85e8f7a700 1 -- start start 2023-11-28T12:03:56.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e8f7a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f85e40b1650 0x7f85e40b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:56.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e8f7a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f85e40af0a0 0x7f85e415a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:56.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e8f7a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e4159ec0 con 0x7f85e41599b0 2023-11-28T12:03:56.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e8f7a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e40b1f60 con 0x7f85e40af0a0 2023-11-28T12:03:56.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e8f7a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e415d8a0 con 0x7f85e40b1650 2023-11-28T12:03:56.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e2d9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f85e40af0a0 0x7f85e415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:56.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e2d9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f85e40af0a0 0x7f85e415a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46478/0 (socket says 172.21.15.57:46478) 2023-11-28T12:03:56.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e2d9d700 1 -- 172.21.15.57:0/351586239 learned_addr learned my addr 172.21.15.57:0/351586239 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:56.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e259c700 1 --2- 172.21.15.57:0/351586239 >> v2:172.21.15.181:3300/0 conn(0x7f85e40b1650 0x7f85e40b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:56.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e2d9d700 1 -- 172.21.15.57:0/351586239 >> v2:172.21.15.181:3300/0 conn(0x7f85e40b1650 msgr2=0x7f85e40b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:56.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e2d9d700 1 --2- 172.21.15.57:0/351586239 >> v2:172.21.15.181:3300/0 conn(0x7f85e40b1650 0x7f85e40b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e2d9d700 1 -- 172.21.15.57:0/351586239 >> v1:172.21.15.181:6789/0 conn(0x7f85e41599b0 legacy=0x7f85e415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:56.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e2d9d700 1 -- 172.21.15.57:0/351586239 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e415db80 con 0x7f85e40af0a0 2023-11-28T12:03:56.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.729+0000 7f85e259c700 1 --2- 172.21.15.57:0/351586239 >> v2:172.21.15.181:3300/0 conn(0x7f85e40b1650 0x7f85e40b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:56.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.730+0000 7f85e2d9d700 1 --2- 172.21.15.57:0/351586239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85e40af0a0 0x7f85e415a350 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f85e4067310 tx=0x7f85dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9397088073049994 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:56.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.730+0000 7f85e159a700 1 -- 172.21.15.57:0/351586239 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc00e070 con 0x7f85e40af0a0 2023-11-28T12:03:56.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.730+0000 7f85e159a700 1 -- 172.21.15.57:0/351586239 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc00b9c0 con 0x7f85e40af0a0 2023-11-28T12:03:56.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.730+0000 7f85e159a700 1 -- 172.21.15.57:0/351586239 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc00bd70 con 0x7f85e40af0a0 2023-11-28T12:03:56.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.730+0000 7f85e8f7a700 1 -- 172.21.15.57:0/351586239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85e40af0a0 msgr2=0x7f85e415a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:56.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.730+0000 7f85e8f7a700 1 --2- 172.21.15.57:0/351586239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85e40af0a0 0x7f85e415a350 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f85e4067310 tx=0x7f85dc00b330 comp rx=0 tx=0).stop 2023-11-28T12:03:56.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.730+0000 7f85e8f7a700 1 -- 172.21.15.57:0/351586239 shutdown_connections 2023-11-28T12:03:56.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.730+0000 7f85e8f7a700 1 --2- 172.21.15.57:0/351586239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85e40af0a0 0x7f85e415a350 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.730+0000 7f85e8f7a700 1 --2- 172.21.15.57:0/351586239 >> v2:172.21.15.181:3300/0 conn(0x7f85e40b1650 0x7f85e40b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.730+0000 7f85e8f7a700 1 -- 172.21.15.57:0/351586239 >> 172.21.15.57:0/351586239 conn(0x7f85e406c010 msgr2=0x7f85e40cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:56.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.730+0000 7f85e8f7a700 1 -- 172.21.15.57:0/351586239 shutdown_connections 2023-11-28T12:03:56.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.730+0000 7f85e8f7a700 1 -- 172.21.15.57:0/351586239 wait complete. 2023-11-28T12:03:56.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.732+0000 7f85e8f7a700 1 Processor -- start 2023-11-28T12:03:56.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.732+0000 7f85e8f7a700 1 -- start start 2023-11-28T12:03:56.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e8f7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e40af0a0 0x7f85e41f0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:56.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e8f7a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85e40b1650 0x7f85e40b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:56.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e8f7a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e41f0690 0x7f85e40b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:56.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e8f7a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e41543a0 con 0x7f85e40b1650 2023-11-28T12:03:56.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e8f7a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e4151500 con 0x7f85e40af0a0 2023-11-28T12:03:56.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e8f7a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e4153790 con 0x7f85e41f0690 2023-11-28T12:03:56.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85e40b1650 0x7f85e40b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:56.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85e40b1650 0x7f85e40b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46494/0 (socket says 172.21.15.57:46494) 2023-11-28T12:03:56.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e1d9b700 1 -- 172.21.15.57:0/1237091819 learned_addr learned my addr 172.21.15.57:0/1237091819 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e2d9d700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e41f0690 0x7f85e40b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e259c700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e40af0a0 0x7f85e41f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e1d9b700 1 -- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e41f0690 msgr2=0x7f85e40b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e1d9b700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e41f0690 0x7f85e40b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e1d9b700 1 -- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e40af0a0 msgr2=0x7f85e41f0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e1d9b700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e40af0a0 0x7f85e41f0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e1d9b700 1 -- 172.21.15.57:0/1237091819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc009b30 con 0x7f85e40b1650 2023-11-28T12:03:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85e1d9b700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85e40b1650 0x7f85e40b4a10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f85d800c430 tx=0x7f85d800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.733+0000 7f85d2ffd700 1 -- 172.21.15.57:0/1237091819 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d80052d0 con 0x7f85e40b1650 2023-11-28T12:03:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.734+0000 7f85d2ffd700 1 -- 172.21.15.57:0/1237091819 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d8005430 con 0x7f85e40b1650 2023-11-28T12:03:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.734+0000 7f85e8f7a700 1 -- 172.21.15.57:0/1237091819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e415a890 con 0x7f85e40b1650 2023-11-28T12:03:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.734+0000 7f85d2ffd700 1 -- 172.21.15.57:0/1237091819 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d80038c0 con 0x7f85e40b1650 2023-11-28T12:03:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.734+0000 7f85e8f7a700 1 -- 172.21.15.57:0/1237091819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e415de60 con 0x7f85e40b1650 2023-11-28T12:03:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.734+0000 7f85e8f7a700 1 -- 172.21.15.57:0/1237091819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e415aa20 con 0x7f85e40b1650 2023-11-28T12:03:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.735+0000 7f85d2ffd700 1 -- 172.21.15.57:0/1237091819 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f85d8019030 con 0x7f85e40b1650 2023-11-28T12:03:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.736+0000 7f85d2ffd700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f85c40682f0 0x7f85c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.736+0000 7f85d2ffd700 1 -- 172.21.15.57:0/1237091819 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f85d8084db0 con 0x7f85e40b1650 2023-11-28T12:03:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.736+0000 7f85e259c700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f85c40682f0 0x7f85c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.736+0000 7f85e259c700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f85c40682f0 0x7f85c406a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f85d40099d0 tx=0x7f85d4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:56.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:56.741+0000 7f85d2ffd700 1 -- 172.21.15.57:0/1237091819 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f85d8051610 con 0x7f85e40b1650 2023-11-28T12:03:57.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.022+0000 7f85e8f7a700 1 -- 172.21.15.57:0/1237091819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f85e40743f0 con 0x7f85e40b1650 2023-11-28T12:03:57.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.026+0000 7f85d2ffd700 1 -- 172.21.15.57:0/1237091819 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 184+0+0 (secure 0 0 0) 0x7f85d8054c30 con 0x7f85e40b1650 2023-11-28T12:03:57.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.028+0000 7f85e8f7a700 1 -- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f85c40682f0 msgr2=0x7f85c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:57.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.028+0000 7f85e8f7a700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f85c40682f0 0x7f85c406a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f85d40099d0 tx=0x7f85d4008040 comp rx=0 tx=0).stop 2023-11-28T12:03:57.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.028+0000 7f85e8f7a700 1 -- 172.21.15.57:0/1237091819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85e40b1650 msgr2=0x7f85e40b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:57.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.028+0000 7f85e8f7a700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85e40b1650 0x7f85e40b4a10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f85d800c430 tx=0x7f85d800c740 comp rx=0 tx=0).stop 2023-11-28T12:03:57.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.029+0000 7f85e8f7a700 1 -- 172.21.15.57:0/1237091819 shutdown_connections 2023-11-28T12:03:57.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.029+0000 7f85e8f7a700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f85c40682f0 0x7f85c406a7a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.029+0000 7f85e8f7a700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e41f0690 0x7f85e40b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.029+0000 7f85e8f7a700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e40af0a0 0x7f85e41f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.029+0000 7f85e8f7a700 1 --2- 172.21.15.57:0/1237091819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85e40b1650 0x7f85e40b4a10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.029+0000 7f85e8f7a700 1 -- 172.21.15.57:0/1237091819 >> 172.21.15.57:0/1237091819 conn(0x7f85e406c010 msgr2=0x7f85e40af630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:57.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.029+0000 7f85e8f7a700 1 -- 172.21.15.57:0/1237091819 shutdown_connections 2023-11-28T12:03:57.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.029+0000 7f85e8f7a700 1 -- 172.21.15.57:0/1237091819 wait complete. 2023-11-28T12:03:57.031 INFO:teuthology.orchestra.run.smithi057.stderr:updated caps for client.0 2023-11-28T12:03:57.043 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:03:57.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.178+0000 7f77e1f6b700 1 Processor -- start 2023-11-28T12:03:57.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.178+0000 7f77e1f6b700 1 -- start start 2023-11-28T12:03:57.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.178+0000 7f77e1f6b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f77dc0ce040 0x7f77dc15d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:57.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.178+0000 7f77e1f6b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77dc142db0 0x7f77dc13ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:57.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.178+0000 7f77e1f6b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f77dc0b1770 0x7f77dc0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:57.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.178+0000 7f77e1f6b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77dc0b2130 con 0x7f77dc0ce040 2023-11-28T12:03:57.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.178+0000 7f77e1f6b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77dc13fc90 con 0x7f77dc0b1770 2023-11-28T12:03:57.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77e1f6b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc13f6d0 con 0x7f77dc142db0 2023-11-28T12:03:57.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77db7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f77dc0ce040 0x7f77dc15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:57.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77db7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f77dc0ce040 0x7f77dc15d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46504/0 (socket says 172.21.15.57:46504) 2023-11-28T12:03:57.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77db7fe700 1 -- 172.21.15.57:0/2938368111 learned_addr learned my addr 172.21.15.57:0/2938368111 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:57.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77dbfff700 1 --2- 172.21.15.57:0/2938368111 >> v2:172.21.15.181:3300/0 conn(0x7f77dc0b1770 0x7f77dc0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:57.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77daffd700 1 --2- 172.21.15.57:0/2938368111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77dc142db0 0x7f77dc13ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:57.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77db7fe700 1 -- 172.21.15.57:0/2938368111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77dc142db0 msgr2=0x7f77dc13ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:57.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77db7fe700 1 --2- 172.21.15.57:0/2938368111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77dc142db0 0x7f77dc13ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77db7fe700 1 -- 172.21.15.57:0/2938368111 >> v2:172.21.15.181:3300/0 conn(0x7f77dc0b1770 msgr2=0x7f77dc0b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:57.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77db7fe700 1 --2- 172.21.15.57:0/2938368111 >> v2:172.21.15.181:3300/0 conn(0x7f77dc0b1770 0x7f77dc0b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77db7fe700 1 -- 172.21.15.57:0/2938368111 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc13f9b0 con 0x7f77dc0ce040 2023-11-28T12:03:57.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77db7fe700 1 --2- 172.21.15.57:0/2938368111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77dc0ce040 0x7f77dc15d900 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0c37c0 tx=0x7f77cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210fdf72adeed3cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:57.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77da7fc700 1 -- 172.21.15.57:0/2938368111 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc011830 con 0x7f77dc0ce040 2023-11-28T12:03:57.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77da7fc700 1 -- 172.21.15.57:0/2938368111 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc011990 con 0x7f77dc0ce040 2023-11-28T12:03:57.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.179+0000 7f77da7fc700 1 -- 172.21.15.57:0/2938368111 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc011d20 con 0x7f77dc0ce040 2023-11-28T12:03:57.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.180+0000 7f77e1f6b700 1 -- 172.21.15.57:0/2938368111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77dc0ce040 msgr2=0x7f77dc15d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:57.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.180+0000 7f77e1f6b700 1 --2- 172.21.15.57:0/2938368111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77dc0ce040 0x7f77dc15d900 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0c37c0 tx=0x7f77cc009bd0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.180+0000 7f77e1f6b700 1 -- 172.21.15.57:0/2938368111 shutdown_connections 2023-11-28T12:03:57.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.180+0000 7f77e1f6b700 1 --2- 172.21.15.57:0/2938368111 >> v2:172.21.15.181:3300/0 conn(0x7f77dc0b1770 0x7f77dc0b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.180+0000 7f77e1f6b700 1 --2- 172.21.15.57:0/2938368111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77dc0ce040 0x7f77dc15d900 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.180+0000 7f77e1f6b700 1 --2- 172.21.15.57:0/2938368111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77dc142db0 0x7f77dc13ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.180+0000 7f77e1f6b700 1 -- 172.21.15.57:0/2938368111 >> 172.21.15.57:0/2938368111 conn(0x7f77dc06c010 msgr2=0x7f77dc0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:57.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.180+0000 7f77e1f6b700 1 -- 172.21.15.57:0/2938368111 shutdown_connections 2023-11-28T12:03:57.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.180+0000 7f77e1f6b700 1 -- 172.21.15.57:0/2938368111 wait complete. 2023-11-28T12:03:57.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.181+0000 7f77e1f6b700 1 Processor -- start 2023-11-28T12:03:57.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.182+0000 7f77e1f6b700 1 -- start start 2023-11-28T12:03:57.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.182+0000 7f77e1f6b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77dc0b1770 0x7f77dc15c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:57.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.182+0000 7f77e1f6b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77dc142db0 0x7f77dc0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:57.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.182+0000 7f77e1f6b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77dc150f10 0x7f77dc0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:57.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.182+0000 7f77e1f6b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc15ca40 con 0x7f77dc150f10 2023-11-28T12:03:57.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.182+0000 7f77e1f6b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc159e90 con 0x7f77dc0b1770 2023-11-28T12:03:57.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.182+0000 7f77e1f6b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc15a170 con 0x7f77dc142db0 2023-11-28T12:03:57.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.182+0000 7f77dbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77dc150f10 0x7f77dc0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:57.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77dbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77dc150f10 0x7f77dc0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46512/0 (socket says 172.21.15.57:46512) 2023-11-28T12:03:57.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77dbfff700 1 -- 172.21.15.57:0/1516411701 learned_addr learned my addr 172.21.15.57:0/1516411701 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:57.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77daffd700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77dc142db0 0x7f77dc0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:57.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77db7fe700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77dc0b1770 0x7f77dc15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:57.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77dbfff700 1 -- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77dc142db0 msgr2=0x7f77dc0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:57.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77dbfff700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77dc142db0 0x7f77dc0cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77dbfff700 1 -- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77dc0b1770 msgr2=0x7f77dc15c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:57.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77dbfff700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77dc0b1770 0x7f77dc15c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77dbfff700 1 -- 172.21.15.57:0/1516411701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc009550 con 0x7f77dc150f10 2023-11-28T12:03:57.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77db7fe700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77dc0b1770 0x7f77dc15c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:57.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77daffd700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77dc142db0 0x7f77dc0cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:57.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77dbfff700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77dc150f10 0x7f77dc0cccb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f77d4010e80 tx=0x7f77d400d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:57.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77c3fff700 1 -- 172.21.15.57:0/1516411701 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4012040 con 0x7f77dc150f10 2023-11-28T12:03:57.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77c3fff700 1 -- 172.21.15.57:0/1516411701 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d400d9e0 con 0x7f77dc150f10 2023-11-28T12:03:57.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77e1f6b700 1 -- 172.21.15.57:0/1516411701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77dc15d3e0 con 0x7f77dc150f10 2023-11-28T12:03:57.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77c3fff700 1 -- 172.21.15.57:0/1516411701 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d400dd90 con 0x7f77dc150f10 2023-11-28T12:03:57.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.183+0000 7f77e1f6b700 1 -- 172.21.15.57:0/1516411701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77dc15d100 con 0x7f77dc150f10 2023-11-28T12:03:57.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.184+0000 7f77e1f6b700 1 -- 172.21.15.57:0/1516411701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77dc0ce410 con 0x7f77dc150f10 2023-11-28T12:03:57.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.185+0000 7f77c3fff700 1 -- 172.21.15.57:0/1516411701 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f77d4009540 con 0x7f77dc150f10 2023-11-28T12:03:57.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.185+0000 7f77c3fff700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f77c80682f0 0x7f77c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:57.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.186+0000 7f77c3fff700 1 -- 172.21.15.57:0/1516411701 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f77d40881f0 con 0x7f77dc150f10 2023-11-28T12:03:57.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.186+0000 7f77db7fe700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f77c80682f0 0x7f77c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:57.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.186+0000 7f77db7fe700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f77c80682f0 0x7f77c806a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f77dc066e20 tx=0x7f77cc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:57.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.190+0000 7f77c3fff700 1 -- 172.21.15.57:0/1516411701 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f77d4054960 con 0x7f77dc150f10 2023-11-28T12:03:57.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.436+0000 7f77e1f6b700 1 -- 172.21.15.57:0/1516411701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77dc0743f0 con 0x7f77dc150f10 2023-11-28T12:03:57.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.436+0000 7f77c3fff700 1 -- 172.21.15.57:0/1516411701 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f77d4057f80 con 0x7f77dc150f10 2023-11-28T12:03:57.438 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:03:57.438 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1773283092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1773283092},{"type":"v1","addr":"172.21.15.181:6837","nonce":1773283092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/3718600227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3718600227},{"type":"v1","addr":"172.21.15.57:6834","nonce":3718600227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/762261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":762261630},{"type":"v1","addr":"172.21.15.57:6835","nonce":762261630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:03:47.263787+0000","modified":"2023-11-28T12:03:50.285215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3437917192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3437917192},{"type":"v1","addr":"172.21.15.181:6836","nonce":3437917192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T12:03:57.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.438+0000 7f77e1f6b700 1 -- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f77c80682f0 msgr2=0x7f77c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:57.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.438+0000 7f77e1f6b700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f77c80682f0 0x7f77c806a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f77dc066e20 tx=0x7f77cc008030 comp rx=0 tx=0).stop 2023-11-28T12:03:57.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.439+0000 7f77e1f6b700 1 -- 172.21.15.57:0/1516411701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77dc150f10 msgr2=0x7f77dc0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:57.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.439+0000 7f77e1f6b700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77dc150f10 0x7f77dc0cccb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f77d4010e80 tx=0x7f77d400d390 comp rx=0 tx=0).stop 2023-11-28T12:03:57.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.439+0000 7f77e1f6b700 1 -- 172.21.15.57:0/1516411701 shutdown_connections 2023-11-28T12:03:57.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.439+0000 7f77e1f6b700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f77c80682f0 0x7f77c806a7a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.439+0000 7f77e1f6b700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77dc150f10 0x7f77dc0cccb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.439+0000 7f77e1f6b700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77dc0b1770 0x7f77dc15c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.439+0000 7f77e1f6b700 1 --2- 172.21.15.57:0/1516411701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77dc142db0 0x7f77dc0cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:57.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.439+0000 7f77e1f6b700 1 -- 172.21.15.57:0/1516411701 >> 172.21.15.57:0/1516411701 conn(0x7f77dc06c010 msgr2=0x7f77dc06dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:57.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.439+0000 7f77e1f6b700 1 -- 172.21.15.57:0/1516411701 shutdown_connections 2023-11-28T12:03:57.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:57.439+0000 7f77e1f6b700 1 -- 172.21.15.57:0/1516411701 wait complete. 2023-11-28T12:03:57.441 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2023-11-28T12:03:57.856 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T12:03:47.263787+0000', 'modified': '2023-11-28T12:03:50.285215+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5739}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5739': {'gid': 5739, 'name': 'b', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6836/3437917192', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 3437917192}, {'type': 'v1', 'addr': '172.21.15.181:6836', 'nonce': 3437917192}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T12:03:57.856 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T12:03:57.856 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T12:03:58.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.000+0000 7fa921dfa700 1 Processor -- start 2023-11-28T12:03:58.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.000+0000 7fa921dfa700 1 -- start start 2023-11-28T12:03:58.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.001+0000 7fa921dfa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa91c1599b0 0x7fa91c15d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.001+0000 7fa921dfa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91c0af0a0 0x7fa91c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.001+0000 7fa921dfa700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91c159ec0 con 0x7fa91c0b1650 2023-11-28T12:03:58.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.001+0000 7fa921dfa700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91c15d8a0 con 0x7fa91c1599b0 2023-11-28T12:03:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.001+0000 7fa921dfa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c15db80 con 0x7fa91c0af0a0 2023-11-28T12:03:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.001+0000 7fa91b7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa91c0b1650 0x7fa91c0b1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:51770/0 (socket says 172.21.15.57:51770) 2023-11-28T12:03:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.001+0000 7fa91b7fe700 1 -- 172.21.15.57:0/186897794 learned_addr learned my addr 172.21.15.57:0/186897794 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:03:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.001+0000 7fa91affd700 1 --2- 172.21.15.57:0/186897794 >> v2:172.21.15.181:3300/0 conn(0x7fa91c1599b0 0x7fa91c15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.001+0000 7fa91bfff700 1 --2- 172.21.15.57:0/186897794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91c0af0a0 0x7fa91c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.001+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2397694241 0 0) 0x7fa91c159ec0 con 0x7fa91c0b1650 2023-11-28T12:03:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.001+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9080035b0 con 0x7fa91c0b1650 2023-11-28T12:03:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3850397317 0 0) 0x7fa9080035b0 con 0x7fa91c0b1650 2023-11-28T12:03:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91c0af0a0 msgr2=0x7fa91c15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 --2- 172.21.15.57:0/186897794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91c0af0a0 0x7fa91c15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 >> v2:172.21.15.181:3300/0 conn(0x7fa91c1599b0 msgr2=0x7fa91c15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 --2- 172.21.15.57:0/186897794 >> v2:172.21.15.181:3300/0 conn(0x7fa91c1599b0 0x7fa91c15d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c15de60 con 0x7fa91c0b1650 2023-11-28T12:03:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fa90c001670 con 0x7fa91c0b1650 2023-11-28T12:03:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 >> v1:172.21.15.57:6789/0 conn(0x7fa91c0b1650 legacy=0x7fa91c0b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 --2- 172.21.15.57:0/186897794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa908003a00 0x7fa908003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 --2- 172.21.15.57:0/186897794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91c0af0a0 0x7fa908004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 --2- 172.21.15.57:0/186897794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa908004d90 0x7fa908005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c15d8a0 con 0x7fa908004d90 2023-11-28T12:03:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c15db80 con 0x7fa908003a00 2023-11-28T12:03:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90800ba10 con 0x7fa91c0af0a0 2023-11-28T12:03:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91affd700 1 --2- 172.21.15.57:0/186897794 >> v2:172.21.15.181:3300/0 conn(0x7fa91c1599b0 0x7fa91c15d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:03:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91bfff700 1 --2- 172.21.15.57:0/186897794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa908004d90 0x7fa908005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/186897794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa908003a00 0x7fa908003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91affd700 1 --2- 172.21.15.57:0/186897794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91c0af0a0 0x7fa908004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91bfff700 1 -- 172.21.15.57:0/186897794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91c0af0a0 msgr2=0x7fa908004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91bfff700 1 --2- 172.21.15.57:0/186897794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91c0af0a0 0x7fa908004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91bfff700 1 -- 172.21.15.57:0/186897794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa908003a00 msgr2=0x7fa908003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91bfff700 1 --2- 172.21.15.57:0/186897794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa908003a00 0x7fa908003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91bfff700 1 -- 172.21.15.57:0/186897794 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9080035b0 con 0x7fa908004d90 2023-11-28T12:03:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/186897794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa908003a00 0x7fa908003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:03:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.002+0000 7fa91bfff700 1 --2- 172.21.15.57:0/186897794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa908004d90 0x7fa908005200 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa914009fc0 tx=0x7fa908005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958fda9bd875abc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.003+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91400ebd0 con 0x7fa908004d90 2023-11-28T12:03:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.003+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91400f040 con 0x7fa908004d90 2023-11-28T12:03:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.003+0000 7fa91a7fc700 1 -- 172.21.15.57:0/186897794 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914014480 con 0x7fa908004d90 2023-11-28T12:03:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.003+0000 7fa921dfa700 1 -- 172.21.15.57:0/186897794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa908004d90 msgr2=0x7fa908005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.003+0000 7fa921dfa700 1 --2- 172.21.15.57:0/186897794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa908004d90 0x7fa908005200 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa914009fc0 tx=0x7fa908005880 comp rx=0 tx=0).stop 2023-11-28T12:03:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.003+0000 7fa91bfff700 1 -- 172.21.15.57:0/186897794 reap_dead start 2023-11-28T12:03:58.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.003+0000 7fa921dfa700 1 -- 172.21.15.57:0/186897794 shutdown_connections 2023-11-28T12:03:58.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.003+0000 7fa921dfa700 1 -- 172.21.15.57:0/186897794 >> 172.21.15.57:0/186897794 conn(0x7fa91c06c010 msgr2=0x7fa91c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:58.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.003+0000 7fa921dfa700 1 -- 172.21.15.57:0/186897794 shutdown_connections 2023-11-28T12:03:58.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.003+0000 7fa921dfa700 1 -- 172.21.15.57:0/186897794 wait complete. 2023-11-28T12:03:58.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.005+0000 7fa921dfa700 1 Processor -- start 2023-11-28T12:03:58.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.005+0000 7fa921dfa700 1 -- start start 2023-11-28T12:03:58.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.005+0000 7fa921dfa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91c1534c0 0x7fa91c0c3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa921dfa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c153b50 0x7fa91c0b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa921dfa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa91c0b5750 0x7fa91c070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa921dfa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c15b660 con 0x7fa91c0b5750 2023-11-28T12:03:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa921dfa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c15a9e0 con 0x7fa91c153b50 2023-11-28T12:03:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa921dfa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c15acc0 con 0x7fa91c1534c0 2023-11-28T12:03:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa91c0b5750 0x7fa91c070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa91c0b5750 0x7fa91c070bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46532/0 (socket says 172.21.15.57:46532) 2023-11-28T12:03:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa91bfff700 1 -- 172.21.15.57:0/2839562281 learned_addr learned my addr 172.21.15.57:0/2839562281 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91c1534c0 0x7fa91c0c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa91affd700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c153b50 0x7fa91c0b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa91bfff700 1 -- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91c1534c0 msgr2=0x7fa91c0c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa91bfff700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91c1534c0 0x7fa91c0c3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa91bfff700 1 -- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c153b50 msgr2=0x7fa91c0b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa91bfff700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c153b50 0x7fa91c0b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa91bfff700 1 -- 172.21.15.57:0/2839562281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c15a350 con 0x7fa91c0b5750 2023-11-28T12:03:58.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa91bfff700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa91c0b5750 0x7fa91c070bd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa914016000 tx=0x7fa914009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:58.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa913fff700 1 -- 172.21.15.57:0/2839562281 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914007000 con 0x7fa91c0b5750 2023-11-28T12:03:58.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa913fff700 1 -- 172.21.15.57:0/2839562281 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa914022070 con 0x7fa91c0b5750 2023-11-28T12:03:58.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.006+0000 7fa921dfa700 1 -- 172.21.15.57:0/2839562281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c15d590 con 0x7fa91c0b5750 2023-11-28T12:03:58.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.007+0000 7fa913fff700 1 -- 172.21.15.57:0/2839562281 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914029480 con 0x7fa91c0b5750 2023-11-28T12:03:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.007+0000 7fa921dfa700 1 -- 172.21.15.57:0/2839562281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c15a4e0 con 0x7fa91c0b5750 2023-11-28T12:03:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.008+0000 7fa913fff700 1 -- 172.21.15.57:0/2839562281 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa9140295e0 con 0x7fa91c0b5750 2023-11-28T12:03:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.009+0000 7fa913fff700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa8fc0682f0 0x7fa8fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.009+0000 7fa913fff700 1 -- 172.21.15.57:0/2839562281 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa914024070 con 0x7fa91c0b5750 2023-11-28T12:03:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.009+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa8fc0682f0 0x7fa8fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.010+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa8fc0682f0 0x7fa8fc06a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa90c001030 tx=0x7fa90c007980 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:58.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.011+0000 7fa92df99b80 1 Processor -- start 2023-11-28T12:03:58.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.011+0000 7fa92df99b80 1 -- start start 2023-11-28T12:03:58.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa92df99b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280af7f50 0x560280ab19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa92df99b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280aeeac0 0x560280af8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa92df99b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280ab2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa92df99b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560280a2fde0 con 0x560280ada660 2023-11-28T12:03:58.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa92df99b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028099c140 con 0x560280af7f50 2023-11-28T12:03:58.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa92df99b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x560280a0adb0 con 0x560280aeeac0 2023-11-28T12:03:58.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280ab2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280ab2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46540/0 (socket says 172.21.15.57:46540) 2023-11-28T12:03:58.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa91bfff700 1 -- 172.21.15.57:0/4174177000 learned_addr learned my addr 172.21.15.57:0/4174177000 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:58.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/4174177000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280af7f50 0x560280ab19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa91affd700 1 --2- 172.21.15.57:0/4174177000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280aeeac0 0x560280af8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa91bfff700 1 -- 172.21.15.57:0/4174177000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280aeeac0 msgr2=0x560280af8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa91bfff700 1 --2- 172.21.15.57:0/4174177000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280aeeac0 0x560280af8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa91bfff700 1 -- 172.21.15.57:0/4174177000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280af7f50 msgr2=0x560280ab19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa91bfff700 1 --2- 172.21.15.57:0/4174177000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280af7f50 0x560280ab19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa91bfff700 1 -- 172.21.15.57:0/4174177000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602807f8790 con 0x560280ada660 2023-11-28T12:03:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.012+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/4174177000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280af7f50 0x560280ab19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:58.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.013+0000 7fa91bfff700 1 --2- 172.21.15.57:0/4174177000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280ab2b60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa91405f8f0 tx=0x7fa914057950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3942649b86ebb61 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.013+0000 7fa9117fa700 1 -- 172.21.15.57:0/4174177000 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91405fe20 con 0x560280ada660 2023-11-28T12:03:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.013+0000 7fa9117fa700 1 -- 172.21.15.57:0/4174177000 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa914056130 con 0x560280ada660 2023-11-28T12:03:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.013+0000 7fa9117fa700 1 -- 172.21.15.57:0/4174177000 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91405f990 con 0x560280ada660 2023-11-28T12:03:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.013+0000 7fa92df99b80 1 -- 172.21.15.57:0/4174177000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 msgr2=0x560280ab2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.013+0000 7fa92df99b80 1 --2- 172.21.15.57:0/4174177000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280ab2b60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa91405f8f0 tx=0x7fa914057950 comp rx=0 tx=0).stop 2023-11-28T12:03:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.014+0000 7fa92df99b80 1 -- 172.21.15.57:0/4174177000 shutdown_connections 2023-11-28T12:03:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.014+0000 7fa92df99b80 1 --2- 172.21.15.57:0/4174177000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280ab2b60 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.014+0000 7fa92df99b80 1 --2- 172.21.15.57:0/4174177000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280af7f50 0x560280ab19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.014+0000 7fa92df99b80 1 --2- 172.21.15.57:0/4174177000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280aeeac0 0x560280af8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.014+0000 7fa92df99b80 1 -- 172.21.15.57:0/4174177000 >> 172.21.15.57:0/4174177000 conn(0x560280aa7d20 msgr2=0x560280ab0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.014+0000 7fa92df99b80 1 -- 172.21.15.57:0/4174177000 shutdown_connections 2023-11-28T12:03:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.014+0000 7fa92df99b80 1 -- 172.21.15.57:0/4174177000 wait complete. 2023-11-28T12:03:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.014+0000 7fa92df99b80 1 -- start start 2023-11-28T12:03:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.014+0000 7fa92df99b80 1 Processor -- start 2023-11-28T12:03:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.015+0000 7fa92df99b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280af4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.015+0000 7fa92df99b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280aeeac0 0x560280ad7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.015+0000 7fa92df99b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280af7f50 0x560280af6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.015+0000 7fa92df99b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560280a04120 con 0x560280ada660 2023-11-28T12:03:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.015+0000 7fa92df99b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602807f9390 con 0x560280af7f50 2023-11-28T12:03:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.015+0000 7fa92df99b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56028099c340 con 0x560280aeeac0 2023-11-28T12:03:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.015+0000 7fa92df99b80 10 client.0 _ll_register_callbacks cb 0x5602809a2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:03:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.015+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280af4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.015+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280af4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46550/0 (socket says 172.21.15.57:46550) 2023-11-28T12:03:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.015+0000 7fa91b7fe700 1 -- 172.21.15.57:0/1395901701 learned_addr learned my addr 172.21.15.57:0/1395901701 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa91affd700 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280aeeac0 0x560280ad7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa91bfff700 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280af7f50 0x560280af6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa91b7fe700 1 -- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280aeeac0 msgr2=0x560280ad7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280aeeac0 0x560280ad7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa91b7fe700 1 -- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280af7f50 msgr2=0x560280af6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280af7f50 0x560280af6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa91b7fe700 1 -- 172.21.15.57:0/1395901701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c15a4e0 con 0x560280ada660 2023-11-28T12:03:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa91affd700 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280aeeac0 0x560280ad7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa91bfff700 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280af7f50 0x560280af6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:03:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280af4d90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa90c015790 tx=0x7fa90c001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c2307cdef879b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa8fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:03:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa8fa7fc700 1 -- 172.21.15.57:0/1395901701 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c01a040 con 0x560280ada660 2023-11-28T12:03:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa8fa7fc700 1 -- 172.21.15.57:0/1395901701 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa90c024c50 con 0x560280ada660 2023-11-28T12:03:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa92df99b80 1 -- 172.21.15.57:0/1395901701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560280a08030 con 0x560280ada660 2023-11-28T12:03:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa8fa7fc700 1 -- 172.21.15.57:0/1395901701 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa90c024e10 con 0x560280ada660 2023-11-28T12:03:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa92df99b80 10 client.5892 fetch_fsmap learned FSMap version 24 2023-11-28T12:03:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa92df99b80 1 -- 172.21.15.57:0/1395901701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5602808e50a0 con 0x560280ada660 2023-11-28T12:03:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa8fa7fc700 1 -- 172.21.15.57:0/1395901701 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560280a08030 con 0x560280ada660 2023-11-28T12:03:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa8fa7fc700 1 -- 172.21.15.57:0/1395901701 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa90c020050 con 0x560280ada660 2023-11-28T12:03:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa92df99b80 10 client.5892 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T12:03:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa92df99b80 10 client.5892 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T12:03:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.016+0000 7fa92df99b80 20 client.5892 populate_metadata read hostname 'smithi057' 2023-11-28T12:03:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.017+0000 7fa92df99b80 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x560280adab60 0x560280ad2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.017+0000 7fa92df99b80 4 client.5892 mds_command: new command op to 5739 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T12:03:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.017+0000 7fa92df99b80 1 -- 172.21.15.57:0/1395901701 --> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560280853ef0 con 0x560280adab60 2023-11-28T12:03:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.017+0000 7fa91affd700 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x560280adab60 0x560280ad2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.017+0000 7fa8fa7fc700 10 client.5892 ms_handle_connect on v2:172.21.15.181:6834/3437917192 2023-11-28T12:03:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.017+0000 7fa91affd700 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x560280adab60 0x560280ad2950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17bbd8e4fc4a3b31 server_cookie=a2df7e56f2437cc2 in_seq=0 out_seq=0 2023-11-28T12:03:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa8fa7fc700 1 -- 172.21.15.57:0/1395901701 <== mds.0 v2:172.21.15.181:6834/3437917192 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x560280853ef0 con 0x560280adab60 2023-11-28T12:03:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa8fa7fc700 10 client.5892 handle_command_reply: tid=0 2023-11-28T12:03:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa92df99b80 1 client.5892 shutdown 2023-11-28T12:03:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa92df99b80 1 -- 172.21.15.57:0/1395901701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 msgr2=0x560280af4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa92df99b80 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280af4d90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa90c015790 tx=0x7fa90c001a70 comp rx=0 tx=0).stop 2023-11-28T12:03:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa92df99b80 1 -- 172.21.15.57:0/1395901701 shutdown_connections 2023-11-28T12:03:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa92df99b80 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x560280adab60 0x560280ad2950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa92df99b80 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280af7f50 0x560280af6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa8fa7fc700 0 client.5892 ms_handle_reset on v2:172.21.15.181:6834/3437917192 2023-11-28T12:03:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa92df99b80 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280af4d90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa92df99b80 1 --2- 172.21.15.57:0/1395901701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280aeeac0 0x560280ad7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa92df99b80 1 -- 172.21.15.57:0/1395901701 >> 172.21.15.57:0/1395901701 conn(0x560280aa7d20 msgr2=0x560280ab0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa92df99b80 1 -- 172.21.15.57:0/1395901701 shutdown_connections 2023-11-28T12:03:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa92df99b80 1 -- 172.21.15.57:0/1395901701 wait complete. 2023-11-28T12:03:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.018+0000 7fa92df99b80 20 client.5892 trim_cache size 0 max 16384 2023-11-28T12:03:58.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.036+0000 7fa92df99b80 1 Processor -- start 2023-11-28T12:03:58.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.036+0000 7fa92df99b80 1 -- start start 2023-11-28T12:03:58.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.037+0000 7fa92df99b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280ada660 0x560280ae0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.037+0000 7fa92df99b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280adab60 0x560280aedf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.037+0000 7fa92df99b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280aeeac0 0x560280aee480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.037+0000 7fa92df99b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560280af5a60 con 0x560280aeeac0 2023-11-28T12:03:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.037+0000 7fa92df99b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560280aa93c0 con 0x560280ada660 2023-11-28T12:03:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.037+0000 7fa92df99b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x560280adb100 con 0x560280adab60 2023-11-28T12:03:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.037+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280aeeac0 0x560280aee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280aeeac0 0x560280aee480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46560/0 (socket says 172.21.15.57:46560) 2023-11-28T12:03:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa91bfff700 1 -- 172.21.15.57:0/483272710 learned_addr learned my addr 172.21.15.57:0/483272710 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/483272710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280ada660 0x560280ae0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa91affd700 1 --2- 172.21.15.57:0/483272710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280adab60 0x560280aedf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa91bfff700 1 -- 172.21.15.57:0/483272710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280adab60 msgr2=0x560280aedf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa91bfff700 1 --2- 172.21.15.57:0/483272710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280adab60 0x560280aedf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa91bfff700 1 -- 172.21.15.57:0/483272710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280ada660 msgr2=0x560280ae0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa91bfff700 1 --2- 172.21.15.57:0/483272710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280ada660 0x560280ae0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa91bfff700 1 -- 172.21.15.57:0/483272710 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602809cb2e0 con 0x560280aeeac0 2023-11-28T12:03:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa91affd700 1 --2- 172.21.15.57:0/483272710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280adab60 0x560280aedf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:03:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa91bfff700 1 --2- 172.21.15.57:0/483272710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280aeeac0 0x560280aee480 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa914054290 tx=0x7fa914018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cfb22bce3813c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa911ffb700 1 -- 172.21.15.57:0/483272710 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91405fe20 con 0x560280aeeac0 2023-11-28T12:03:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa911ffb700 1 -- 172.21.15.57:0/483272710 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa914066070 con 0x560280aeeac0 2023-11-28T12:03:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.038+0000 7fa911ffb700 1 -- 172.21.15.57:0/483272710 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91405f990 con 0x560280aeeac0 2023-11-28T12:03:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.039+0000 7fa92df99b80 1 -- 172.21.15.57:0/483272710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280aeeac0 msgr2=0x560280aee480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.039+0000 7fa92df99b80 1 --2- 172.21.15.57:0/483272710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280aeeac0 0x560280aee480 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa914054290 tx=0x7fa914018000 comp rx=0 tx=0).stop 2023-11-28T12:03:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.039+0000 7fa92df99b80 1 -- 172.21.15.57:0/483272710 shutdown_connections 2023-11-28T12:03:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.039+0000 7fa92df99b80 1 --2- 172.21.15.57:0/483272710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280aeeac0 0x560280aee480 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.039+0000 7fa92df99b80 1 --2- 172.21.15.57:0/483272710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280ada660 0x560280ae0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.039+0000 7fa92df99b80 1 --2- 172.21.15.57:0/483272710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280adab60 0x560280aedf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.039+0000 7fa92df99b80 1 -- 172.21.15.57:0/483272710 >> 172.21.15.57:0/483272710 conn(0x560280aa7d20 msgr2=0x560280ab0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.039+0000 7fa92df99b80 1 -- 172.21.15.57:0/483272710 shutdown_connections 2023-11-28T12:03:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.039+0000 7fa92df99b80 1 -- 172.21.15.57:0/483272710 wait complete. 2023-11-28T12:03:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.039+0000 7fa92df99b80 1 -- start start 2023-11-28T12:03:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.040+0000 7fa92df99b80 1 Processor -- start 2023-11-28T12:03:58.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.040+0000 7fa92df99b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280b13790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.040+0000 7fa92df99b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280adab60 0x560280ab0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.040+0000 7fa92df99b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280aeeac0 0x560280ab0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.040+0000 7fa92df99b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560280b14070 con 0x560280ada660 2023-11-28T12:03:58.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.040+0000 7fa92df99b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560280b47d50 con 0x560280aeeac0 2023-11-28T12:03:58.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.040+0000 7fa92df99b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x560280b13e20 con 0x560280adab60 2023-11-28T12:03:58.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.040+0000 7fa92df99b80 10 client.0 _ll_register_callbacks cb 0x5602809a2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:03:58.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280b13790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280b13790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46566/0 (socket says 172.21.15.57:46566) 2023-11-28T12:03:58.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa91b7fe700 1 -- 172.21.15.57:0/3345474435 learned_addr learned my addr 172.21.15.57:0/3345474435 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:58.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa91bfff700 1 --2- 172.21.15.57:0/3345474435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280aeeac0 0x560280ab0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa91affd700 1 --2- 172.21.15.57:0/3345474435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280adab60 0x560280ab0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa91b7fe700 1 -- 172.21.15.57:0/3345474435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280adab60 msgr2=0x560280ab0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:58.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/3345474435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280adab60 0x560280ab0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa91b7fe700 1 -- 172.21.15.57:0/3345474435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280aeeac0 msgr2=0x560280ab0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:58.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/3345474435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280aeeac0 0x560280ab0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa91b7fe700 1 -- 172.21.15.57:0/3345474435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602807f8790 con 0x560280ada660 2023-11-28T12:03:58.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa91b7fe700 1 --2- 172.21.15.57:0/3345474435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280b13790 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa90c000c00 tx=0x7fa90c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a75a5b3d9ce656b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:58.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa8fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:03:58.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa8fa7fc700 1 -- 172.21.15.57:0/3345474435 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0147d0 con 0x560280ada660 2023-11-28T12:03:58.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa8fa7fc700 1 -- 172.21.15.57:0/3345474435 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa90c039c20 con 0x560280ada660 2023-11-28T12:03:58.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa92df99b80 1 -- 172.21.15.57:0/3345474435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa914056130 con 0x560280ada660 2023-11-28T12:03:58.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa8fa7fc700 1 -- 172.21.15.57:0/3345474435 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa90c039de0 con 0x560280ada660 2023-11-28T12:03:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa92df99b80 10 client.5898 fetch_fsmap learned FSMap version 24 2023-11-28T12:03:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.041+0000 7fa92df99b80 1 -- 172.21.15.57:0/3345474435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5602808591d0 con 0x560280ada660 2023-11-28T12:03:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.042+0000 7fa8fa7fc700 1 -- 172.21.15.57:0/3345474435 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa914056130 con 0x560280ada660 2023-11-28T12:03:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.042+0000 7fa8fa7fc700 1 -- 172.21.15.57:0/3345474435 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa90c036040 con 0x560280ada660 2023-11-28T12:03:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.042+0000 7fa92df99b80 10 client.5898 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T12:03:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.042+0000 7fa92df99b80 10 client.5898 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T12:03:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.042+0000 7fa92df99b80 20 client.5898 populate_metadata read hostname 'smithi057' 2023-11-28T12:03:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.042+0000 7fa92df99b80 1 --2- 172.21.15.57:0/3345474435 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x560280af7f50 0x560280b46e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.042+0000 7fa92df99b80 4 client.5898 mds_command: new command op to 5739 tid=0[{"prefix": "status"}] 2023-11-28T12:03:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.042+0000 7fa92df99b80 1 -- 172.21.15.57:0/3345474435 --> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560280889040 con 0x560280af7f50 2023-11-28T12:03:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.042+0000 7fa91affd700 1 --2- 172.21.15.57:0/3345474435 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x560280af7f50 0x560280b46e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.042+0000 7fa8fa7fc700 10 client.5898 ms_handle_connect on v2:172.21.15.181:6834/3437917192 2023-11-28T12:03:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.042+0000 7fa91affd700 1 --2- 172.21.15.57:0/3345474435 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x560280af7f50 0x560280b46e40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6083f17d6ae4cd9 server_cookie=7619beeed0273f79 in_seq=0 out_seq=0 2023-11-28T12:03:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa8fa7fc700 1 -- 172.21.15.57:0/3345474435 <== mds.0 v2:172.21.15.181:6834/3437917192 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x560280889040 con 0x560280af7f50 2023-11-28T12:03:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa8fa7fc700 10 client.5898 handle_command_reply: tid=0 2023-11-28T12:03:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa92df99b80 1 client.5898 shutdown 2023-11-28T12:03:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa92df99b80 1 -- 172.21.15.57:0/3345474435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 msgr2=0x560280b13790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa92df99b80 1 --2- 172.21.15.57:0/3345474435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280b13790 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa90c000c00 tx=0x7fa90c016000 comp rx=0 tx=0).stop 2023-11-28T12:03:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa92df99b80 1 -- 172.21.15.57:0/3345474435 shutdown_connections 2023-11-28T12:03:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa92df99b80 1 --2- 172.21.15.57:0/3345474435 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x560280af7f50 0x560280b46e40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa92df99b80 1 --2- 172.21.15.57:0/3345474435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560280aeeac0 0x560280ab0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa92df99b80 1 --2- 172.21.15.57:0/3345474435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560280ada660 0x560280b13790 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa8fa7fc700 0 client.5898 ms_handle_reset on v2:172.21.15.181:6834/3437917192 2023-11-28T12:03:58.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa92df99b80 1 --2- 172.21.15.57:0/3345474435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560280adab60 0x560280ab0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa92df99b80 1 -- 172.21.15.57:0/3345474435 >> 172.21.15.57:0/3345474435 conn(0x560280aa7d20 msgr2=0x560280b10a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:58.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa92df99b80 1 -- 172.21.15.57:0/3345474435 shutdown_connections 2023-11-28T12:03:58.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa92df99b80 1 -- 172.21.15.57:0/3345474435 wait complete. 2023-11-28T12:03:58.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa92df99b80 20 client.5898 trim_cache size 0 max 16384 2023-11-28T12:03:58.052 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:03:58.052 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "0309b83f-6570-4eee-96fa-279822cd7766", 2023-11-28T12:03:58.053 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2023-11-28T12:03:58.053 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 5739, 2023-11-28T12:03:58.053 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-11-28T12:03:58.053 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-11-28T12:03:58.053 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-11-28T12:03:58.053 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 10.778132033, 2023-11-28T12:03:58.053 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 24, 2023-11-28T12:03:58.053 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 49, 2023-11-28T12:03:58.053 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 49, 2023-11-28T12:03:58.054 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 16.062566246999999 2023-11-28T12:03:58.054 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:03:58.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa921dfa700 1 -- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa8fc0682f0 msgr2=0x7fa8fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa921dfa700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa8fc0682f0 0x7fa8fc06a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa90c001030 tx=0x7fa90c007980 comp rx=0 tx=0).stop 2023-11-28T12:03:58.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa921dfa700 1 -- 172.21.15.57:0/2839562281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa91c0b5750 msgr2=0x7fa91c070bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa921dfa700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa91c0b5750 0x7fa91c070bd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa914016000 tx=0x7fa914009750 comp rx=0 tx=0).stop 2023-11-28T12:03:58.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa921dfa700 1 -- 172.21.15.57:0/2839562281 shutdown_connections 2023-11-28T12:03:58.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa921dfa700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa8fc0682f0 0x7fa8fc06a7a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa921dfa700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa91c0b5750 0x7fa91c070bd0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa921dfa700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91c1534c0 0x7fa91c0c3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa921dfa700 1 --2- 172.21.15.57:0/2839562281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c153b50 0x7fa91c0b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.043+0000 7fa921dfa700 1 -- 172.21.15.57:0/2839562281 >> 172.21.15.57:0/2839562281 conn(0x7fa91c06c010 msgr2=0x7fa91c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:58.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.044+0000 7fa921dfa700 1 -- 172.21.15.57:0/2839562281 shutdown_connections 2023-11-28T12:03:58.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.044+0000 7fa921dfa700 1 -- 172.21.15.57:0/2839562281 wait complete. 2023-11-28T12:03:58.058 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T12:03:58.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.191+0000 7f7d5c4e4700 1 Processor -- start 2023-11-28T12:03:58.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.191+0000 7f7d5c4e4700 1 -- start start 2023-11-28T12:03:58.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.192+0000 7f7d5c4e4700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7d5415cc80 0x7f7d5406de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.192+0000 7f7d5c4e4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7d5406e3a0 0x7f7d54070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.192+0000 7f7d5c4e4700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d540b1810 con 0x7f7d5406a270 2023-11-28T12:03:58.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.192+0000 7f7d5c4e4700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d540b2490 con 0x7f7d5415cc80 2023-11-28T12:03:58.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d5c4e4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d540b1af0 con 0x7f7d5406e3a0 2023-11-28T12:03:58.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d59a7f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7d5415cc80 0x7f7d5406de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d59a7f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7d5415cc80 0x7f7d5406de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46574/0 (socket says 172.21.15.57:46574) 2023-11-28T12:03:58.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d5a280700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f7d5406a270 0x7f7d5406a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:51774/0 (socket says 172.21.15.57:51774) 2023-11-28T12:03:58.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d59a7f700 1 -- 172.21.15.57:0/2066465745 learned_addr learned my addr 172.21.15.57:0/2066465745 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:58.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d5aa81700 1 --2- 172.21.15.57:0/2066465745 >> v2:172.21.15.181:3300/0 conn(0x7f7d5406e3a0 0x7f7d54070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d59a7f700 1 -- 172.21.15.57:0/2066465745 >> v2:172.21.15.181:3300/0 conn(0x7f7d5406e3a0 msgr2=0x7f7d54070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:03:58.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d59a7f700 1 --2- 172.21.15.57:0/2066465745 >> v2:172.21.15.181:3300/0 conn(0x7f7d5406e3a0 0x7f7d54070d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d59a7f700 1 -- 172.21.15.57:0/2066465745 >> v1:172.21.15.57:6789/0 conn(0x7f7d5406a270 legacy=0x7f7d5406a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d59a7f700 1 -- 172.21.15.57:0/2066465745 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d540b1dd0 con 0x7f7d5415cc80 2023-11-28T12:03:58.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d5927e700 1 -- 172.21.15.57:0/2066465745 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798622504 0 0) 0x7f7d540b1810 con 0x7f7d5406a270 2023-11-28T12:03:58.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d59a7f700 1 --2- 172.21.15.57:0/2066465745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d5415cc80 0x7f7d5406de60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7d5406ee30 tx=0x7f7d44009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9e192122933c6b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:58.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d5927e700 1 -- 172.21.15.57:0/2066465745 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4400b870 con 0x7f7d5415cc80 2023-11-28T12:03:58.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d5927e700 1 -- 172.21.15.57:0/2066465745 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4400b9d0 con 0x7f7d5415cc80 2023-11-28T12:03:58.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.193+0000 7f7d5927e700 1 -- 172.21.15.57:0/2066465745 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4400bd60 con 0x7f7d5415cc80 2023-11-28T12:03:58.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.194+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/2066465745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d5415cc80 msgr2=0x7f7d5406de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.194+0000 7f7d5c4e4700 1 --2- 172.21.15.57:0/2066465745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d5415cc80 0x7f7d5406de60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7d5406ee30 tx=0x7f7d44009bd0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.194+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/2066465745 shutdown_connections 2023-11-28T12:03:58.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.194+0000 7f7d5c4e4700 1 --2- 172.21.15.57:0/2066465745 >> v2:172.21.15.181:3300/0 conn(0x7f7d5406e3a0 0x7f7d54070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.194+0000 7f7d5c4e4700 1 --2- 172.21.15.57:0/2066465745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d5415cc80 0x7f7d5406de60 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.194+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/2066465745 >> 172.21.15.57:0/2066465745 conn(0x7f7d5406c010 msgr2=0x7f7d540cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:58.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.194+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/2066465745 shutdown_connections 2023-11-28T12:03:58.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.194+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/2066465745 wait complete. 2023-11-28T12:03:58.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.196+0000 7f7d5c4e4700 1 Processor -- start 2023-11-28T12:03:58.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.196+0000 7f7d5c4e4700 1 -- start start 2023-11-28T12:03:58.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d5c4e4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d5406a270 0x7f7d540b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d5c4e4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d5406e3a0 0x7f7d540cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d5c4e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d5415cc80 0x7f7d540cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d5c4e4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540b2b20 con 0x7f7d5406e3a0 2023-11-28T12:03:58.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d5c4e4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540b33c0 con 0x7f7d5415cc80 2023-11-28T12:03:58.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d5c4e4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5415f430 con 0x7f7d5406a270 2023-11-28T12:03:58.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d59a7f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d5406e3a0 0x7f7d540cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d59a7f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d5406e3a0 0x7f7d540cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46588/0 (socket says 172.21.15.57:46588) 2023-11-28T12:03:58.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d59a7f700 1 -- 172.21.15.57:0/3745352849 learned_addr learned my addr 172.21.15.57:0/3745352849 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:03:58.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d5a280700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d5406a270 0x7f7d540b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d5aa81700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d5415cc80 0x7f7d540cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d59a7f700 1 -- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d5406a270 msgr2=0x7f7d540b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d59a7f700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d5406a270 0x7f7d540b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d59a7f700 1 -- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d5415cc80 msgr2=0x7f7d540cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d59a7f700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d5415cc80 0x7f7d540cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.197+0000 7f7d59a7f700 1 -- 172.21.15.57:0/3745352849 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d44009550 con 0x7f7d5406e3a0 2023-11-28T12:03:58.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.198+0000 7f7d59a7f700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d5406e3a0 0x7f7d540cb370 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009c10 tx=0x7f7d44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:58.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.198+0000 7f7d4affd700 1 -- 172.21.15.57:0/3745352849 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44004250 con 0x7f7d5406e3a0 2023-11-28T12:03:58.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.198+0000 7f7d4affd700 1 -- 172.21.15.57:0/3745352849 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4401f070 con 0x7f7d5406e3a0 2023-11-28T12:03:58.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.198+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/3745352849 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5406e7c0 con 0x7f7d5406e3a0 2023-11-28T12:03:58.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.198+0000 7f7d4affd700 1 -- 172.21.15.57:0/3745352849 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d440045b0 con 0x7f7d5406e3a0 2023-11-28T12:03:58.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.198+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/3745352849 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d540b20b0 con 0x7f7d5406e3a0 2023-11-28T12:03:58.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.199+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/3745352849 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5406e950 con 0x7f7d5406e3a0 2023-11-28T12:03:58.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.200+0000 7f7d4affd700 1 -- 172.21.15.57:0/3745352849 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d4400f050 con 0x7f7d5406e3a0 2023-11-28T12:03:58.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.200+0000 7f7d4affd700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7d380682f0 0x7f7d3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:03:58.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.200+0000 7f7d4affd700 1 -- 172.21.15.57:0/3745352849 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d44087020 con 0x7f7d5406e3a0 2023-11-28T12:03:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.200+0000 7f7d5a280700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7d380682f0 0x7f7d3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:03:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.201+0000 7f7d5a280700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7d380682f0 0x7f7d3806a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7d50000f60 tx=0x7f7d50007e70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:03:58.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.204+0000 7f7d4affd700 1 -- 172.21.15.57:0/3745352849 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7d5406e950 con 0x7f7d5406e3a0 2023-11-28T12:03:58.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.452+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/3745352849 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7d5406e950 con 0x7f7d5406e3a0 2023-11-28T12:03:58.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.452+0000 7f7d4affd700 1 -- 172.21.15.57:0/3745352849 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7d5406e950 con 0x7f7d5406e3a0 2023-11-28T12:03:58.453 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T12:03:58.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.455+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7d380682f0 msgr2=0x7f7d3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.455+0000 7f7d5c4e4700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7d380682f0 0x7f7d3806a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7d50000f60 tx=0x7f7d50007e70 comp rx=0 tx=0).stop 2023-11-28T12:03:58.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.455+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/3745352849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d5406e3a0 msgr2=0x7f7d540cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:03:58.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.455+0000 7f7d5c4e4700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d5406e3a0 0x7f7d540cb370 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009c10 tx=0x7f7d44004750 comp rx=0 tx=0).stop 2023-11-28T12:03:58.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.455+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/3745352849 shutdown_connections 2023-11-28T12:03:58.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.455+0000 7f7d5c4e4700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7d380682f0 0x7f7d3806a7a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.455+0000 7f7d5c4e4700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d5415cc80 0x7f7d540cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.455+0000 7f7d5c4e4700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d5406a270 0x7f7d540b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.455+0000 7f7d5c4e4700 1 --2- 172.21.15.57:0/3745352849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d5406e3a0 0x7f7d540cb370 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:03:58.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.455+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/3745352849 >> 172.21.15.57:0/3745352849 conn(0x7f7d5406c010 msgr2=0x7f7d540afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:03:58.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.455+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/3745352849 shutdown_connections 2023-11-28T12:03:58.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:03:58.455+0000 7f7d5c4e4700 1 -- 172.21.15.57:0/3745352849 wait complete. 2023-11-28T12:03:58.469 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T12:03:58.469 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T12:03:58.470 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2023-11-28T12:03:58.470 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T12:03:58.470 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T12:03:58.470 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T12:03:58.470 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T12:03:58.470 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T12:03:58.470 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T12:03:58.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:58.470 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2023-11-28T12:03:58.492 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T12:03:58.492 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T12:03:58.492 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T12:03:58.492 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:58.493 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-11-28T12:03:58.493 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:58.493 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:03:58.493 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:03:58.493 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f0 2023-11-28T12:03:58.493 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:03:58.493 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:03:58.494 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f1 2023-11-28T12:03:58.494 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T12:03:58.494 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:03:58.494 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f0 2023-11-28T12:03:58.494 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T12:03:58.495 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 41821sec preferred_lft 41821sec 2023-11-28T12:03:58.495 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-11-28T12:03:58.495 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:58.495 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:03:58.495 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:03:58.495 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f1 2023-11-28T12:03:58.496 INFO:teuthology.orchestra.run.smithi057.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T12:03:58.496 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 8e:b7:77:39:69:ff brd ff:ff:ff:ff:ff:ff 2023-11-28T12:03:58.496 INFO:teuthology.orchestra.run.smithi057.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T12:03:58.496 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:58.497 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::3f:caff:fe00:648e/64 scope link 2023-11-28T12:03:58.497 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:58.497 INFO:teuthology.orchestra.run.smithi057.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T12:03:58.497 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether f2:ef:c1:11:c5:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T12:03:58.497 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::f0ef:c1ff:fe11:c5fb/64 scope link 2023-11-28T12:03:58.497 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:58.498 INFO:teuthology.orchestra.run.smithi057.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T12:03:58.498 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 8e:b7:77:39:69:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T12:03:58.498 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::8cb7:77ff:fe39:69ff/64 scope link 2023-11-28T12:03:58.498 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:03:58.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:58.498 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T12:03:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-11-28T12:03:58.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:58.565 DEBUG:teuthology.orchestra.run.smithi057:> route 2023-11-28T12:03:58.630 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2023-11-28T12:03:58.631 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T12:03:58.631 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T12:03:58.631 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T12:03:58.631 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T12:03:58.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:58.634 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-11-28T12:03:58.634 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-11-28T12:03:58.634 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T12:03:58.634 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T12:03:58.634 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T12:03:58.634 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-11-28T12:03:58.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:58.754 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-11-28T12:03:58.777 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T12:03:58.777 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T12:03:58.777 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T12:03:58.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:58.778 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2023-11-28T12:03:58.854 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T12:03:58.854 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:03:58.855 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T12:03:58.918 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T12:03:58.919 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:03:58.919 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T12:03:58.984 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-11-28T12:03:59.059 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:03:59.060 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:03:59.134 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:03:59.135 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:03:59.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:59.136 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-11-28T12:03:59.199 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T12:03:59.199 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/) 2023-11-28T12:03:59.243 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-11-28T12:03:59.279 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:03:59.279 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:03:59.346 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:2023-11-28T12:03:59.345+0000 7f3cc8fd85c0 -1 init, newargv = 0x56505b0c86f0 newargc=15 2023-11-28T12:03:59.346 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[141720]: starting ceph client 2023-11-28T12:03:59.350 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:03:59.352 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:03:59.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:59.352 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-11-28T12:03:59.353 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[141720]: starting fuse 2023-11-28T12:03:59.414 INFO:teuthology.orchestra.run.smithi057.stdout:47 2023-11-28T12:03:59.414 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T12:03:59.415 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:03:59.415 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2023-11-28T12:03:59.415 DEBUG:teuthology.orchestra.run.smithi057:> import re 2023-11-28T12:03:59.415 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-11-28T12:03:59.415 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2023-11-28T12:03:59.415 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:59.415 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2023-11-28T12:03:59.415 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T12:03:59.415 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2023-11-28T12:03:59.416 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T12:03:59.416 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:59.416 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2023-11-28T12:03:59.416 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2023-11-28T12:03:59.416 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2023-11-28T12:03:59.416 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2023-11-28T12:03:59.416 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:59.416 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2023-11-28T12:03:59.416 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T12:03:59.416 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T12:03:59.416 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T12:03:59.416 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2023-11-28T12:03:59.417 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2023-11-28T12:03:59.417 DEBUG:teuthology.orchestra.run.smithi057:> return f 2023-11-28T12:03:59.417 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T12:03:59.417 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:03:59.417 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.0")) 2023-11-28T12:03:59.417 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-11-28T12:03:59.556 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.0.141720.asok 2023-11-28T12:03:59.738 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.141720.asok 2023-11-28T12:03:59.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:59.738 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.141720.asok status 2023-11-28T12:03:59.916 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:03:59.916 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2023-11-28T12:03:59.916 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T12:03:59.916 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T12:03:59.916 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "0", 2023-11-28T12:03:59.916 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2023-11-28T12:03:59.916 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T12:03:59.916 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "141720", 2023-11-28T12:03:59.917 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2023-11-28T12:03:59.917 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:59.917 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2023-11-28T12:03:59.917 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2023-11-28T12:03:59.917 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 5913, 2023-11-28T12:03:59.917 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2023-11-28T12:03:59.917 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2023-11-28T12:03:59.917 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2023-11-28T12:03:59.917 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 5913 2023-11-28T12:03:59.917 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:59.917 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-11-28T12:03:59.918 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-11-28T12:03:59.918 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-11-28T12:03:59.918 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2908934526 2023-11-28T12:03:59.918 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:03:59.918 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:59.918 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-11-28T12:03:59.918 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-11-28T12:03:59.918 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-11-28T12:03:59.918 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2908934526 2023-11-28T12:03:59.918 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:03:59.919 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.5913 192.168.0.1:0/2908934526", 2023-11-28T12:03:59.919 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.1:0/2908934526", 2023-11-28T12:03:59.919 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2023-11-28T12:03:59.919 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 24, 2023-11-28T12:03:59.919 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 49, 2023-11-28T12:03:59.919 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2023-11-28T12:03:59.919 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2023-11-28T12:03:59.919 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2023-11-28T12:03:59.919 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:03:59.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:59.927 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2023-11-28T12:03:59.927 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T12:03:59.947 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2023-11-28T12:03:59.947 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T12:03:59.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:03:59.948 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T12:04:00.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:04:00.022 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2023-11-28T12:04:00.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:04:00.094 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2023-11-28T12:04:00.161 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:04:00.162 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-11-28T12:04:00.162 DEBUG:teuthology.orchestra.run.smithi057:> import stat 2023-11-28T12:04:00.162 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:04:00.162 DEBUG:teuthology.orchestra.run.smithi057:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2023-11-28T12:04:00.162 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-11-28T12:04:00.285 INFO:teuthology.orchestra.run.smithi057.stdout:1099511627777 2023-11-28T12:04:00.473 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:04:00.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.655+0000 7f9e62c66700 1 Processor -- start 2023-11-28T12:04:00.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.656+0000 7f9e62c66700 1 -- start start 2023-11-28T12:04:00.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.657+0000 7f9e62c66700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e5c163430 0x7f9e5c165810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:00.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.657+0000 7f9e62c66700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5c151a50 con 0x7f9e5c06a270 2023-11-28T12:04:00.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.657+0000 7f9e62c66700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5c15b440 con 0x7f9e5c070d90 2023-11-28T12:04:00.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.657+0000 7f9e62c66700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c15f0c0 con 0x7f9e5c163430 2023-11-28T12:04:00.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.657+0000 7f9e60a02700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f9e5c06a270 0x7f9e5c06a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:51786/0 (socket says 172.21.15.57:51786) 2023-11-28T12:04:00.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.657+0000 7f9e60a02700 1 -- 172.21.15.57:0/1686995978 learned_addr learned my addr 172.21.15.57:0/1686995978 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:00.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.657+0000 7f9e61203700 1 --2- 172.21.15.57:0/1686995978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e5c163430 0x7f9e5c165810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:00.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.657+0000 7f9e5b7fe700 1 -- 172.21.15.57:0/1686995978 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546167205 0 0) 0x7f9e5c151a50 con 0x7f9e5c06a270 2023-11-28T12:04:00.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.657+0000 7f9e5b7fe700 1 -- 172.21.15.57:0/1686995978 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e480035b0 con 0x7f9e5c06a270 2023-11-28T12:04:00.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.657+0000 7f9e5b7fe700 1 -- 172.21.15.57:0/1686995978 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731055513 0 0) 0x7f9e5c15b440 con 0x7f9e5c070d90 2023-11-28T12:04:00.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.657+0000 7f9e5b7fe700 1 -- 172.21.15.57:0/1686995978 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e5c151a50 con 0x7f9e5c070d90 2023-11-28T12:04:00.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.658+0000 7f9e5b7fe700 1 -- 172.21.15.57:0/1686995978 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 352421610 0 0) 0x7f9e480035b0 con 0x7f9e5c06a270 2023-11-28T12:04:00.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.658+0000 7f9e61203700 1 -- 172.21.15.57:0/1686995978 >> v1:172.21.15.181:6789/0 conn(0x7f9e5c070d90 legacy=0x7f9e5c0711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:00.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.658+0000 7f9e61203700 1 -- 172.21.15.57:0/1686995978 >> v1:172.21.15.57:6789/0 conn(0x7f9e5c06a270 legacy=0x7f9e5c06a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:00.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.658+0000 7f9e61203700 1 -- 172.21.15.57:0/1686995978 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c071ac0 con 0x7f9e5c163430 2023-11-28T12:04:00.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.658+0000 7f9e61203700 1 --2- 172.21.15.57:0/1686995978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e5c163430 0x7f9e5c165810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009dc0 tx=0x7f9e5400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e940b5a389c0faf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:00.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.658+0000 7f9e5b7fe700 1 -- 172.21.15.57:0/1686995978 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5400e070 con 0x7f9e5c163430 2023-11-28T12:04:00.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.658+0000 7f9e5b7fe700 1 -- 172.21.15.57:0/1686995978 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e54003a20 con 0x7f9e5c163430 2023-11-28T12:04:00.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.658+0000 7f9e5b7fe700 1 -- 172.21.15.57:0/1686995978 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c001670 con 0x7f9e5c163430 2023-11-28T12:04:00.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.659+0000 7f9e62c66700 1 -- 172.21.15.57:0/1686995978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e5c163430 msgr2=0x7f9e5c165810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:00.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.659+0000 7f9e62c66700 1 --2- 172.21.15.57:0/1686995978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e5c163430 0x7f9e5c165810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009dc0 tx=0x7f9e5400b330 comp rx=0 tx=0).stop 2023-11-28T12:04:00.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.659+0000 7f9e62c66700 1 -- 172.21.15.57:0/1686995978 shutdown_connections 2023-11-28T12:04:00.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.659+0000 7f9e62c66700 1 --2- 172.21.15.57:0/1686995978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e5c163430 0x7f9e5c165810 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:00.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.659+0000 7f9e62c66700 1 -- 172.21.15.57:0/1686995978 >> 172.21.15.57:0/1686995978 conn(0x7f9e5c06c010 msgr2=0x7f9e5c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:00.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.659+0000 7f9e62c66700 1 -- 172.21.15.57:0/1686995978 shutdown_connections 2023-11-28T12:04:00.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.659+0000 7f9e62c66700 1 -- 172.21.15.57:0/1686995978 wait complete. 2023-11-28T12:04:00.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.661+0000 7f9e62c66700 1 Processor -- start 2023-11-28T12:04:00.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.661+0000 7f9e62c66700 1 -- start start 2023-11-28T12:04:00.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e62c66700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e5c06a270 0x7f9e5c153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:00.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e62c66700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e5c070d90 0x7f9e5c150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:00.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e62c66700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e5c163430 0x7f9e5c0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:00.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e62c66700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c151030 con 0x7f9e5c163430 2023-11-28T12:04:00.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e62c66700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c0b4b40 con 0x7f9e5c070d90 2023-11-28T12:04:00.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e62c66700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c153ad0 con 0x7f9e5c06a270 2023-11-28T12:04:00.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e61203700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e5c163430 0x7f9e5c0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:00.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e61203700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e5c163430 0x7f9e5c0b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46594/0 (socket says 172.21.15.57:46594) 2023-11-28T12:04:00.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e61203700 1 -- 172.21.15.57:0/4219339912 learned_addr learned my addr 172.21.15.57:0/4219339912 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:00.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e60a02700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e5c06a270 0x7f9e5c153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:00.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e5bfff700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e5c070d90 0x7f9e5c150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:00.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e61203700 1 -- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e5c06a270 msgr2=0x7f9e5c153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:00.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e61203700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e5c06a270 0x7f9e5c153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:00.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e61203700 1 -- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e5c070d90 msgr2=0x7f9e5c150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:00.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e61203700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e5c070d90 0x7f9e5c150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:00.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.662+0000 7f9e61203700 1 -- 172.21.15.57:0/4219339912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e54009a70 con 0x7f9e5c163430 2023-11-28T12:04:00.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.663+0000 7f9e61203700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e5c163430 0x7f9e5c0b3ca0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9e54015040 tx=0x7f9e54004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:00.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.663+0000 7f9e597fa700 1 -- 172.21.15.57:0/4219339912 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54004240 con 0x7f9e5c163430 2023-11-28T12:04:00.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.663+0000 7f9e597fa700 1 -- 172.21.15.57:0/4219339912 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e540043a0 con 0x7f9e5c163430 2023-11-28T12:04:00.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.663+0000 7f9e62c66700 1 -- 172.21.15.57:0/4219339912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5c15b0d0 con 0x7f9e5c163430 2023-11-28T12:04:00.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.663+0000 7f9e597fa700 1 -- 172.21.15.57:0/4219339912 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54004730 con 0x7f9e5c163430 2023-11-28T12:04:00.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.663+0000 7f9e62c66700 1 -- 172.21.15.57:0/4219339912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5c15adf0 con 0x7f9e5c163430 2023-11-28T12:04:00.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.664+0000 7f9e62c66700 1 -- 172.21.15.57:0/4219339912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5c15aac0 con 0x7f9e5c163430 2023-11-28T12:04:00.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.664+0000 7f9e597fa700 1 -- 172.21.15.57:0/4219339912 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9e54018040 con 0x7f9e5c163430 2023-11-28T12:04:00.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.665+0000 7f9e597fa700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9e3c0682f0 0x7f9e3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:00.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.665+0000 7f9e597fa700 1 -- 172.21.15.57:0/4219339912 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e54087100 con 0x7f9e5c163430 2023-11-28T12:04:00.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.665+0000 7f9e60a02700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9e3c0682f0 0x7f9e3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:00.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.666+0000 7f9e60a02700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9e3c0682f0 0x7f9e3c06a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c000f60 tx=0x7f9e4c000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:00.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.670+0000 7f9e597fa700 1 -- 172.21.15.57:0/4219339912 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9e54053960 con 0x7f9e5c163430 2023-11-28T12:04:00.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.917+0000 7f9e62c66700 1 -- 172.21.15.57:0/4219339912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e5c153c40 con 0x7f9e5c163430 2023-11-28T12:04:00.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.917+0000 7f9e597fa700 1 -- 172.21.15.57:0/4219339912 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f9e54056f80 con 0x7f9e5c163430 2023-11-28T12:04:00.919 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:00.919 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1773283092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1773283092},{"type":"v1","addr":"172.21.15.181:6837","nonce":1773283092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/3718600227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3718600227},{"type":"v1","addr":"172.21.15.57:6834","nonce":3718600227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/762261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":762261630},{"type":"v1","addr":"172.21.15.57:6835","nonce":762261630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:03:47.263787+0000","modified":"2023-11-28T12:03:50.285215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3437917192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3437917192},{"type":"v1","addr":"172.21.15.181:6836","nonce":3437917192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T12:04:00.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.920+0000 7f9e62c66700 1 -- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9e3c0682f0 msgr2=0x7f9e3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:00.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.920+0000 7f9e62c66700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9e3c0682f0 0x7f9e3c06a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c000f60 tx=0x7f9e4c000bc0 comp rx=0 tx=0).stop 2023-11-28T12:04:00.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.920+0000 7f9e62c66700 1 -- 172.21.15.57:0/4219339912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e5c163430 msgr2=0x7f9e5c0b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:00.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.920+0000 7f9e62c66700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e5c163430 0x7f9e5c0b3ca0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9e54015040 tx=0x7f9e54004ad0 comp rx=0 tx=0).stop 2023-11-28T12:04:00.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.920+0000 7f9e62c66700 1 -- 172.21.15.57:0/4219339912 shutdown_connections 2023-11-28T12:04:00.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.920+0000 7f9e62c66700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9e3c0682f0 0x7f9e3c06a7a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:00.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.920+0000 7f9e62c66700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e5c163430 0x7f9e5c0b3ca0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:00.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.920+0000 7f9e62c66700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e5c06a270 0x7f9e5c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:00.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.920+0000 7f9e62c66700 1 --2- 172.21.15.57:0/4219339912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e5c070d90 0x7f9e5c150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:00.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.920+0000 7f9e62c66700 1 -- 172.21.15.57:0/4219339912 >> 172.21.15.57:0/4219339912 conn(0x7f9e5c06c010 msgr2=0x7f9e5c06dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:00.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.920+0000 7f9e62c66700 1 -- 172.21.15.57:0/4219339912 shutdown_connections 2023-11-28T12:04:00.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:00.920+0000 7f9e62c66700 1 -- 172.21.15.57:0/4219339912 wait complete. 2023-11-28T12:04:00.922 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2023-11-28T12:04:00.934 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2023-11-28T12:04:01.094 INFO:teuthology.orchestra.run.smithi181.stdout:{"message":"","return_code":0} 2023-11-28T12:04:01.095 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T12:04:01.095 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T12:04:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.149+0000 7f6d422190c0 1 Processor -- start 2023-11-28T12:04:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.149+0000 7f6d422190c0 1 -- start start 2023-11-28T12:04:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.150+0000 7f6d422190c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830639aa10 0x5583064717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.150+0000 7f6d422190c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55830639dc80 0x5583063c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.150+0000 7f6d422190c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583064725a0 0x5583063c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.150+0000 7f6d422190c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583062d6fe0 con 0x55830639dc80 2023-11-28T12:04:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.150+0000 7f6d422190c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558306470eb0 con 0x5583064725a0 2023-11-28T12:04:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.150+0000 7f6d422190c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583063c7330 con 0x55830639aa10 2023-11-28T12:04:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.150+0000 7f6d37b04700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55830639dc80 0x5583063c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.150+0000 7f6d37b04700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55830639dc80 0x5583063c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46604/0 (socket says 172.21.15.57:46604) 2023-11-28T12:04:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.150+0000 7f6d37b04700 1 -- 172.21.15.57:0/3423457429 learned_addr learned my addr 172.21.15.57:0/3423457429 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.150+0000 7f6d38305700 1 --2- 172.21.15.57:0/3423457429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830639aa10 0x5583064717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.150+0000 7f6d38b06700 1 --2- 172.21.15.57:0/3423457429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583064725a0 0x5583063c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.150+0000 7f6d37b04700 1 -- 172.21.15.57:0/3423457429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830639aa10 msgr2=0x5583064717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.150+0000 7f6d37b04700 1 --2- 172.21.15.57:0/3423457429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830639aa10 0x5583064717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d37b04700 1 -- 172.21.15.57:0/3423457429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583064725a0 msgr2=0x5583063c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d37b04700 1 --2- 172.21.15.57:0/3423457429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583064725a0 0x5583063c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d37b04700 1 -- 172.21.15.57:0/3423457429 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c005de0 con 0x55830639dc80 2023-11-28T12:04:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d37b04700 1 --2- 172.21.15.57:0/3423457429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55830639dc80 0x5583063c77c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c015040 tx=0x7f6d2c00d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3e9be51fe1ea9a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d36b02700 1 -- 172.21.15.57:0/3423457429 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c008690 con 0x55830639dc80 2023-11-28T12:04:01.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d36b02700 1 -- 172.21.15.57:0/3423457429 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2c008b60 con 0x55830639dc80 2023-11-28T12:04:01.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d36b02700 1 -- 172.21.15.57:0/3423457429 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c01e450 con 0x55830639dc80 2023-11-28T12:04:01.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d422190c0 1 -- 172.21.15.57:0/3423457429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55830639dc80 msgr2=0x5583063c77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d422190c0 1 --2- 172.21.15.57:0/3423457429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55830639dc80 0x5583063c77c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c015040 tx=0x7f6d2c00d000 comp rx=0 tx=0).stop 2023-11-28T12:04:01.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d422190c0 1 -- 172.21.15.57:0/3423457429 shutdown_connections 2023-11-28T12:04:01.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d422190c0 1 --2- 172.21.15.57:0/3423457429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583064725a0 0x5583063c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d422190c0 1 --2- 172.21.15.57:0/3423457429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830639aa10 0x5583064717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d422190c0 1 --2- 172.21.15.57:0/3423457429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55830639dc80 0x5583063c77c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d422190c0 1 -- 172.21.15.57:0/3423457429 >> 172.21.15.57:0/3423457429 conn(0x5583063e6da0 msgr2=0x558306472970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.151+0000 7f6d422190c0 1 -- 172.21.15.57:0/3423457429 shutdown_connections 2023-11-28T12:04:01.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.152+0000 7f6d422190c0 1 -- 172.21.15.57:0/3423457429 wait complete. 2023-11-28T12:04:01.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.152+0000 7f6d422190c0 1 Processor -- start 2023-11-28T12:04:01.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.153+0000 7f6d422190c0 1 -- start start 2023-11-28T12:04:01.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.153+0000 7f6d422190c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830639aa10 0x558306461d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.153+0000 7f6d422190c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830639dc80 0x558306507bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d422190c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583064725a0 0x558306462430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d422190c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583064fc280 con 0x5583064725a0 2023-11-28T12:04:01.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d422190c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558306462270 con 0x55830639aa10 2023-11-28T12:04:01.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d422190c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x558306474df0 con 0x55830639dc80 2023-11-28T12:04:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d38b06700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583064725a0 0x558306462430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d37b04700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830639dc80 0x558306507bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d38305700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830639aa10 0x558306461d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d38b06700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583064725a0 0x558306462430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46608/0 (socket says 172.21.15.57:46608) 2023-11-28T12:04:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d37b04700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830639dc80 0x558306507bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:58948/0 (socket says 172.21.15.57:58948) 2023-11-28T12:04:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d38b06700 1 -- 172.21.15.57:0/1536371022 learned_addr learned my addr 172.21.15.57:0/1536371022 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d38305700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830639aa10 0x558306461d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:51022/0 (socket says 172.21.15.57:51022) 2023-11-28T12:04:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d38305700 1 -- 172.21.15.57:0/1536371022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830639dc80 msgr2=0x558306507bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d38305700 1 --2- 172.21.15.57:0/1536371022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55830639dc80 0x558306507bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d38305700 1 -- 172.21.15.57:0/1536371022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583064725a0 msgr2=0x558306462430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d38305700 1 --2- 172.21.15.57:0/1536371022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583064725a0 0x558306462430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d38305700 1 -- 172.21.15.57:0/1536371022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583063094f0 con 0x55830639aa10 2023-11-28T12:04:01.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d38b06700 1 --2- 172.21.15.57:0/1536371022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583064725a0 0x558306462430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:01.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d38305700 1 --2- 172.21.15.57:0/1536371022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830639aa10 0x558306461d30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800edb0 tx=0x7f6d2800ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.154+0000 7f6d35b00700 1 -- 172.21.15.57:0/1536371022 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d280029d0 con 0x55830639aa10 2023-11-28T12:04:01.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.155+0000 7f6d35b00700 1 -- 172.21.15.57:0/1536371022 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2800a040 con 0x55830639aa10 2023-11-28T12:04:01.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.155+0000 7f6d422190c0 1 -- 172.21.15.57:0/1536371022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5583062ef120 con 0x55830639aa10 2023-11-28T12:04:01.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.155+0000 7f6d35b00700 1 -- 172.21.15.57:0/1536371022 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800cd70 con 0x55830639aa10 2023-11-28T12:04:01.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.155+0000 7f6d422190c0 1 -- 172.21.15.57:0/1536371022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583062e2830 con 0x55830639aa10 2023-11-28T12:04:01.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.156+0000 7f6d35b00700 1 -- 172.21.15.57:0/1536371022 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6d28011430 con 0x55830639aa10 2023-11-28T12:04:01.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.156+0000 7f6d35b00700 1 --2- 172.21.15.57:0/1536371022 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6d24066c10 0x7f6d240690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.157+0000 7f6d35b00700 1 -- 172.21.15.57:0/1536371022 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6d28084650 con 0x55830639aa10 2023-11-28T12:04:01.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.157+0000 7f6d422190c0 1 --2- 172.21.15.57:0/1536371022 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x558306508110 0x558306472e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.157+0000 7f6d422190c0 1 -- 172.21.15.57:0/1536371022 --> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49) v8 -- 0x5583064fc6a0 con 0x558306508110 2023-11-28T12:04:01.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.157+0000 7f6d37b04700 1 --2- 172.21.15.57:0/1536371022 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6d24066c10 0x7f6d240690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.157+0000 7f6d38b06700 1 --2- 172.21.15.57:0/1536371022 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x558306508110 0x558306472e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.157+0000 7f6d37b04700 1 --2- 172.21.15.57:0/1536371022 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6d24066c10 0x7f6d240690c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c01d000 tx=0x7f6d2c008000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.157+0000 7f6d38b06700 1 --2- 172.21.15.57:0/1536371022 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x558306508110 0x558306472e80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.158+0000 7f6d38b06700 1 -- 172.21.15.57:0/1536371022 <== osd.1 v2:172.21.15.57:6808/120755 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f6d30014420 con 0x558306508110 2023-11-28T12:04:01.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.158+0000 7f6d422190c0 1 -- 172.21.15.57:0/1536371022 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x558306508110 msgr2=0x558306472e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.158+0000 7f6d422190c0 1 --2- 172.21.15.57:0/1536371022 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x558306508110 0x558306472e80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.158+0000 7f6d422190c0 1 -- 172.21.15.57:0/1536371022 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6d24066c10 msgr2=0x7f6d240690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.158+0000 7f6d422190c0 1 --2- 172.21.15.57:0/1536371022 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6d24066c10 0x7f6d240690c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c01d000 tx=0x7f6d2c008000 comp rx=0 tx=0).stop 2023-11-28T12:04:01.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.158+0000 7f6d422190c0 1 -- 172.21.15.57:0/1536371022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830639aa10 msgr2=0x558306461d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.158+0000 7f6d422190c0 1 --2- 172.21.15.57:0/1536371022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55830639aa10 0x558306461d30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800edb0 tx=0x7f6d2800ede0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.158+0000 7f6d38b06700 1 -- 172.21.15.57:0/1536371022 reap_dead start 2023-11-28T12:04:01.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.158+0000 7f6d422190c0 1 -- 172.21.15.57:0/1536371022 shutdown_connections 2023-11-28T12:04:01.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.158+0000 7f6d422190c0 1 -- 172.21.15.57:0/1536371022 >> 172.21.15.57:0/1536371022 conn(0x5583063e6da0 msgr2=0x5583063c6410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.158+0000 7f6d422190c0 1 -- 172.21.15.57:0/1536371022 shutdown_connections 2023-11-28T12:04:01.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.158+0000 7f6d422190c0 1 -- 172.21.15.57:0/1536371022 wait complete. 2023-11-28T12:04:01.172 INFO:teuthology.orchestra.run.smithi057.stdout:Yalphaparent_a 2023-11-28T12:04:01.172 DEBUG:teuthology.orchestra.run.smithi057:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T12:04:01.309 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:04:01.309 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511627777, 2023-11-28T12:04:01.310 INFO:teuthology.orchestra.run.smithi057.stdout: "ancestors": [ 2023-11-28T12:04:01.310 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:01.310 INFO:teuthology.orchestra.run.smithi057.stdout: "dirino": 1099511627776, 2023-11-28T12:04:01.310 INFO:teuthology.orchestra.run.smithi057.stdout: "dname": "alpha", 2023-11-28T12:04:01.310 INFO:teuthology.orchestra.run.smithi057.stdout: "version": 2 2023-11-28T12:04:01.310 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:01.310 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:01.311 INFO:teuthology.orchestra.run.smithi057.stdout: "dirino": 1, 2023-11-28T12:04:01.311 INFO:teuthology.orchestra.run.smithi057.stdout: "dname": "parent_a", 2023-11-28T12:04:01.311 INFO:teuthology.orchestra.run.smithi057.stdout: "version": 4 2023-11-28T12:04:01.311 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:01.311 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-11-28T12:04:01.311 INFO:teuthology.orchestra.run.smithi057.stdout: "pool": 8, 2023-11-28T12:04:01.311 INFO:teuthology.orchestra.run.smithi057.stdout: "old_pools": [] 2023-11-28T12:04:01.311 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:04:01.312 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:01.313 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2023-11-28T12:04:01.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.367+0000 7f129d5a00c0 1 Processor -- start 2023-11-28T12:04:01.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.367+0000 7f129d5a00c0 1 -- start start 2023-11-28T12:04:01.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.367+0000 7f129d5a00c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556db46e7a20 0x556db477ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.367+0000 7f129d5a00c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556db46e7e70 0x556db480cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.367+0000 7f129d5a00c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556db475e210 0x556db480d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.367+0000 7f129d5a00c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db464b000 con 0x556db475e210 2023-11-28T12:04:01.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.367+0000 7f129d5a00c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db45ddfe0 con 0x556db46e7e70 2023-11-28T12:04:01.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.367+0000 7f129d5a00c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556db46ecb80 con 0x556db46e7a20 2023-11-28T12:04:01.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f1293e8d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556db475e210 0x556db480d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f1293e8d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556db475e210 0x556db480d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46614/0 (socket says 172.21.15.57:46614) 2023-11-28T12:04:01.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f1293e8d700 1 -- 172.21.15.57:0/1030355226 learned_addr learned my addr 172.21.15.57:0/1030355226 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f129368c700 1 --2- 172.21.15.57:0/1030355226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556db46e7a20 0x556db477ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f1292e8b700 1 --2- 172.21.15.57:0/1030355226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556db46e7e70 0x556db480cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f1293e8d700 1 -- 172.21.15.57:0/1030355226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556db46e7a20 msgr2=0x556db477ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f1293e8d700 1 --2- 172.21.15.57:0/1030355226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556db46e7a20 0x556db477ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f1293e8d700 1 -- 172.21.15.57:0/1030355226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556db46e7e70 msgr2=0x556db480cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f1293e8d700 1 --2- 172.21.15.57:0/1030355226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556db46e7e70 0x556db480cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f1293e8d700 1 -- 172.21.15.57:0/1030355226 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c00a100 con 0x556db475e210 2023-11-28T12:04:01.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f1293e8d700 1 --2- 172.21.15.57:0/1030355226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556db475e210 0x556db480d190 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f128c013040 tx=0x7f128c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2045d69711364e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f1291e89700 1 -- 172.21.15.57:0/1030355226 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c00f650 con 0x556db475e210 2023-11-28T12:04:01.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f1291e89700 1 -- 172.21.15.57:0/1030355226 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c00fa60 con 0x556db475e210 2023-11-28T12:04:01.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.368+0000 7f1291e89700 1 -- 172.21.15.57:0/1030355226 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c00fe10 con 0x556db475e210 2023-11-28T12:04:01.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.369+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1030355226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556db475e210 msgr2=0x556db480d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.369+0000 7f129d5a00c0 1 --2- 172.21.15.57:0/1030355226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556db475e210 0x556db480d190 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f128c013040 tx=0x7f128c00f000 comp rx=0 tx=0).stop 2023-11-28T12:04:01.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.369+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1030355226 shutdown_connections 2023-11-28T12:04:01.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.369+0000 7f129d5a00c0 1 --2- 172.21.15.57:0/1030355226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556db475e210 0x556db480d190 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.369+0000 7f129d5a00c0 1 --2- 172.21.15.57:0/1030355226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556db46e7a20 0x556db477ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.369+0000 7f129d5a00c0 1 --2- 172.21.15.57:0/1030355226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556db46e7e70 0x556db480cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.369+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1030355226 >> 172.21.15.57:0/1030355226 conn(0x556db46eb150 msgr2=0x556db477deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.369+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1030355226 shutdown_connections 2023-11-28T12:04:01.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.369+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1030355226 wait complete. 2023-11-28T12:04:01.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.370+0000 7f129d5a00c0 1 Processor -- start 2023-11-28T12:04:01.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.370+0000 7f129d5a00c0 1 -- start start 2023-11-28T12:04:01.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.370+0000 7f129d5a00c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556db46e7a20 0x556db46e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.370+0000 7f129d5a00c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556db46e7e70 0x556db47622e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.370+0000 7f129d5a00c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556db475e210 0x556db475cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.370+0000 7f129d5a00c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db46e97e0 con 0x556db475e210 2023-11-28T12:04:01.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.370+0000 7f129d5a00c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556db4762fc0 con 0x556db46e7a20 2023-11-28T12:04:01.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.370+0000 7f129d5a00c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556db46d17d0 con 0x556db46e7e70 2023-11-28T12:04:01.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f1293e8d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556db475e210 0x556db475cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f1293e8d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556db475e210 0x556db475cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46626/0 (socket says 172.21.15.57:46626) 2023-11-28T12:04:01.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f1293e8d700 1 -- 172.21.15.57:0/1635224963 learned_addr learned my addr 172.21.15.57:0/1635224963 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f1292e8b700 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556db46e7e70 0x556db47622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f129368c700 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556db46e7a20 0x556db46e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f1293e8d700 1 -- 172.21.15.57:0/1635224963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556db46e7e70 msgr2=0x556db47622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f1293e8d700 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556db46e7e70 0x556db47622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f1293e8d700 1 -- 172.21.15.57:0/1635224963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556db46e7a20 msgr2=0x556db46e92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f1293e8d700 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556db46e7a20 0x556db46e92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f1293e8d700 1 -- 172.21.15.57:0/1635224963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556db45f6120 con 0x556db475e210 2023-11-28T12:04:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f1293e8d700 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556db475e210 0x556db475cd40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f128c00a4a0 tx=0x7f128c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f1290e87700 1 -- 172.21.15.57:0/1635224963 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c005690 con 0x556db475e210 2023-11-28T12:04:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f1290e87700 1 -- 172.21.15.57:0/1635224963 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c009030 con 0x556db475e210 2023-11-28T12:04:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1635224963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556db45e9830 con 0x556db475e210 2023-11-28T12:04:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f1290e87700 1 -- 172.21.15.57:0/1635224963 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c010870 con 0x556db475e210 2023-11-28T12:04:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.371+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1635224963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556db45f6360 con 0x556db475e210 2023-11-28T12:04:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.373+0000 7f1290e87700 1 -- 172.21.15.57:0/1635224963 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f128c00e050 con 0x556db475e210 2023-11-28T12:04:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.373+0000 7f1290e87700 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1280066c10 0x7f12800690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.373+0000 7f1290e87700 1 -- 172.21.15.57:0/1635224963 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f128c085e50 con 0x556db475e210 2023-11-28T12:04:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.373+0000 7f129368c700 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1280066c10 0x7f12800690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.374+0000 7f129d5a00c0 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x556db4762820 0x556db481d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.374+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1635224963 --> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49) v8 -- 0x556db46d3880 con 0x556db4762820 2023-11-28T12:04:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.374+0000 7f1292e8b700 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x556db4762820 0x556db481d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.374+0000 7f129368c700 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1280066c10 0x7f12800690c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1288000c00 tx=0x7f12880016b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.374+0000 7f1292e8b700 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x556db4762820 0x556db481d2e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.375+0000 7f1292e8b700 1 -- 172.21.15.57:0/1635224963 <== osd.1 v2:172.21.15.57:6808/120755 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v49'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1284011790 con 0x556db4762820 2023-11-28T12:04:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.376+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1635224963 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x556db4762820 msgr2=0x556db481d2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.376+0000 7f129d5a00c0 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x556db4762820 0x556db481d2e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.376+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1635224963 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1280066c10 msgr2=0x7f12800690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.376+0000 7f129d5a00c0 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f1280066c10 0x7f12800690c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1288000c00 tx=0x7f12880016b0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.376+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1635224963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556db475e210 msgr2=0x556db475cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.376+0000 7f129d5a00c0 1 --2- 172.21.15.57:0/1635224963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556db475e210 0x556db475cd40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f128c00a4a0 tx=0x7f128c003330 comp rx=0 tx=0).stop 2023-11-28T12:04:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.376+0000 7f1293e8d700 1 -- 172.21.15.57:0/1635224963 reap_dead start 2023-11-28T12:04:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.376+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1635224963 shutdown_connections 2023-11-28T12:04:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.376+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1635224963 >> 172.21.15.57:0/1635224963 conn(0x556db46eb150 msgr2=0x556db46ed860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.376+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1635224963 shutdown_connections 2023-11-28T12:04:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.376+0000 7f129d5a00c0 1 -- 172.21.15.57:0/1635224963 wait complete. 2023-11-28T12:04:01.385 DEBUG:teuthology.orchestra.run.smithi057:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-28T12:04:01.388 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-11-28T12:04:01.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.528+0000 7f4613a7a700 1 Processor -- start 2023-11-28T12:04:01.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.528+0000 7f4613a7a700 1 -- start start 2023-11-28T12:04:01.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.529+0000 7f4613a7a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f460c15cce0 0x7f460c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.529+0000 7f4613a7a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f460c0b1c90 0x7f460c0b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.529+0000 7f4613a7a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460c0b32b0 con 0x7f460c0b22e0 2023-11-28T12:04:01.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.529+0000 7f4613a7a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460c0b26e0 con 0x7f460c0b1c90 2023-11-28T12:04:01.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.529+0000 7f4613a7a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460c0b2980 con 0x7f460c15cce0 2023-11-28T12:04:01.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.529+0000 7f4612017700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f460c0b1c90 0x7f460c0b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.529+0000 7f4611015700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f460c0b22e0 0x7f460c0d0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:51808/0 (socket says 172.21.15.57:51808) 2023-11-28T12:04:01.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.529+0000 7f4611015700 1 -- 172.21.15.57:0/1619522129 learned_addr learned my addr 172.21.15.57:0/1619522129 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:01.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.530+0000 7f4611816700 1 --2- 172.21.15.57:0/1619522129 >> v2:172.21.15.181:3300/0 conn(0x7f460c15cce0 0x7f460c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.530+0000 7f4610814700 1 -- 172.21.15.57:0/1619522129 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509416367 0 0) 0x7f460c0b32b0 con 0x7f460c0b22e0 2023-11-28T12:04:01.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.530+0000 7f4610814700 1 -- 172.21.15.57:0/1619522129 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f80035b0 con 0x7f460c0b22e0 2023-11-28T12:04:01.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.530+0000 7f4612017700 1 -- 172.21.15.57:0/1619522129 >> v2:172.21.15.181:3300/0 conn(0x7f460c15cce0 msgr2=0x7f460c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.530+0000 7f4612017700 1 --2- 172.21.15.57:0/1619522129 >> v2:172.21.15.181:3300/0 conn(0x7f460c15cce0 0x7f460c0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.530+0000 7f4612017700 1 -- 172.21.15.57:0/1619522129 >> v1:172.21.15.57:6789/0 conn(0x7f460c0b22e0 legacy=0x7f460c0d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.530+0000 7f4612017700 1 -- 172.21.15.57:0/1619522129 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c0b2c60 con 0x7f460c0b1c90 2023-11-28T12:04:01.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.530+0000 7f4611816700 1 --2- 172.21.15.57:0/1619522129 >> v2:172.21.15.181:3300/0 conn(0x7f460c15cce0 0x7f460c0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:01.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.530+0000 7f4612017700 1 --2- 172.21.15.57:0/1619522129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f460c0b1c90 0x7f460c0b3740 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f460c08c360 tx=0x7f460400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b690a83ccbeaa117 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.530+0000 7f4610814700 1 -- 172.21.15.57:0/1619522129 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460400e070 con 0x7f460c0b1c90 2023-11-28T12:04:01.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.530+0000 7f4610814700 1 -- 172.21.15.57:0/1619522129 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460400bb20 con 0x7f460c0b1c90 2023-11-28T12:04:01.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.530+0000 7f4610814700 1 -- 172.21.15.57:0/1619522129 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604012430 con 0x7f460c0b1c90 2023-11-28T12:04:01.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.531+0000 7f4613a7a700 1 -- 172.21.15.57:0/1619522129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f460c0b1c90 msgr2=0x7f460c0b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.531+0000 7f4613a7a700 1 --2- 172.21.15.57:0/1619522129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f460c0b1c90 0x7f460c0b3740 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f460c08c360 tx=0x7f460400b330 comp rx=0 tx=0).stop 2023-11-28T12:04:01.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.531+0000 7f4613a7a700 1 -- 172.21.15.57:0/1619522129 shutdown_connections 2023-11-28T12:04:01.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.531+0000 7f4613a7a700 1 --2- 172.21.15.57:0/1619522129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f460c0b1c90 0x7f460c0b3740 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.531+0000 7f4613a7a700 1 --2- 172.21.15.57:0/1619522129 >> v2:172.21.15.181:3300/0 conn(0x7f460c15cce0 0x7f460c0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.531+0000 7f4613a7a700 1 -- 172.21.15.57:0/1619522129 >> 172.21.15.57:0/1619522129 conn(0x7f460c09c4d0 msgr2=0x7f460c0d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.531+0000 7f4613a7a700 1 -- 172.21.15.57:0/1619522129 shutdown_connections 2023-11-28T12:04:01.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.531+0000 7f4613a7a700 1 -- 172.21.15.57:0/1619522129 wait complete. 2023-11-28T12:04:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.533+0000 7f4613a7a700 1 Processor -- start 2023-11-28T12:04:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.533+0000 7f4613a7a700 1 -- start start 2023-11-28T12:04:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4613a7a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f460c0b1c90 0x7f460c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4613a7a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f460c15cce0 0x7f460c0ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4613a7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f460c0b4720 0x7f460c15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4613a7a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0cb790 con 0x7f460c15cce0 2023-11-28T12:04:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4613a7a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0cab80 con 0x7f460c0b4720 2023-11-28T12:04:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4613a7a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0cae60 con 0x7f460c0b1c90 2023-11-28T12:04:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4611015700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f460c15cce0 0x7f460c0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4611015700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f460c15cce0 0x7f460c0ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46634/0 (socket says 172.21.15.57:46634) 2023-11-28T12:04:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4611015700 1 -- 172.21.15.57:0/1708954414 learned_addr learned my addr 172.21.15.57:0/1708954414 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4611816700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f460c0b1c90 0x7f460c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4612017700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f460c0b4720 0x7f460c15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4611015700 1 -- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f460c0b1c90 msgr2=0x7f460c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4611015700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f460c0b1c90 0x7f460c0cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4611015700 1 -- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f460c0b4720 msgr2=0x7f460c15b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4611015700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f460c0b4720 0x7f460c15b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4611015700 1 -- 172.21.15.57:0/1708954414 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46080094a0 con 0x7f460c15cce0 2023-11-28T12:04:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.534+0000 7f4612017700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f460c0b4720 0x7f460c15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.535+0000 7f4611015700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f460c15cce0 0x7f460c0ca230 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00c700 tx=0x7f45fc00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.535+0000 7f46027fc700 1 -- 172.21.15.57:0/1708954414 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc005d70 con 0x7f460c15cce0 2023-11-28T12:04:01.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.535+0000 7f46027fc700 1 -- 172.21.15.57:0/1708954414 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc0070d0 con 0x7f460c15cce0 2023-11-28T12:04:01.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.535+0000 7f46027fc700 1 -- 172.21.15.57:0/1708954414 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc007460 con 0x7f460c15cce0 2023-11-28T12:04:01.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.535+0000 7f4613a7a700 1 -- 172.21.15.57:0/1708954414 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4604009a70 con 0x7f460c15cce0 2023-11-28T12:04:01.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.535+0000 7f4613a7a700 1 -- 172.21.15.57:0/1708954414 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c0d1440 con 0x7f460c15cce0 2023-11-28T12:04:01.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.537+0000 7f46027fc700 1 -- 172.21.15.57:0/1708954414 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f45fc019030 con 0x7f460c15cce0 2023-11-28T12:04:01.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.537+0000 7f46027fc700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f45f00682c0 0x7f45f006a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.537+0000 7f46027fc700 1 -- 172.21.15.57:0/1708954414 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f45fc0851d0 con 0x7f460c15cce0 2023-11-28T12:04:01.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.537+0000 7f4611816700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f45f00682c0 0x7f45f006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.538+0000 7f4611816700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f45f00682c0 0x7f45f006a770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4608009d00 tx=0x7f4608008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.539+0000 7f461fc19b80 1 Processor -- start 2023-11-28T12:04:01.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.539+0000 7f461fc19b80 1 -- start start 2023-11-28T12:04:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f461fc19b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 0x56117f24dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f461fc19b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f2943f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f461fc19b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 0x56117f24ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f461fc19b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117f1cbeb0 con 0x56117f28ab90 2023-11-28T12:04:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f461fc19b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117f138210 con 0x56117f294020 2023-11-28T12:04:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f461fc19b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56117f1a6e80 con 0x56117f276730 2023-11-28T12:04:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f4611015700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f2943f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f4611015700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f2943f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46640/0 (socket says 172.21.15.57:46640) 2023-11-28T12:04:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f4611015700 1 -- 172.21.15.57:0/1247894396 learned_addr learned my addr 172.21.15.57:0/1247894396 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f4612017700 1 --2- 172.21.15.57:0/1247894396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 0x56117f24ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f4611816700 1 --2- 172.21.15.57:0/1247894396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 0x56117f24dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f4611015700 1 -- 172.21.15.57:0/1247894396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 msgr2=0x56117f24ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f4611015700 1 --2- 172.21.15.57:0/1247894396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 0x56117f24ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f4611015700 1 -- 172.21.15.57:0/1247894396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 msgr2=0x56117f24dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f4611015700 1 --2- 172.21.15.57:0/1247894396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 0x56117f24dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f4611015700 1 -- 172.21.15.57:0/1247894396 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56117ef94860 con 0x56117f28ab90 2023-11-28T12:04:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.540+0000 7f4611015700 1 --2- 172.21.15.57:0/1247894396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f2943f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f45fc055370 tx=0x7f45fc054fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee90c5a70b3b945 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.541+0000 7f45ef7fe700 1 -- 172.21.15.57:0/1247894396 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc05b110 con 0x56117f28ab90 2023-11-28T12:04:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.541+0000 7f45ef7fe700 1 -- 172.21.15.57:0/1247894396 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc054b60 con 0x56117f28ab90 2023-11-28T12:04:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.541+0000 7f45ef7fe700 1 -- 172.21.15.57:0/1247894396 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc05d270 con 0x56117f28ab90 2023-11-28T12:04:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.541+0000 7f461fc19b80 1 -- 172.21.15.57:0/1247894396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 msgr2=0x56117f2943f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.541+0000 7f461fc19b80 1 --2- 172.21.15.57:0/1247894396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f2943f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f45fc055370 tx=0x7f45fc054fd0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.541+0000 7f461fc19b80 1 -- 172.21.15.57:0/1247894396 shutdown_connections 2023-11-28T12:04:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.541+0000 7f461fc19b80 1 --2- 172.21.15.57:0/1247894396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 0x56117f24ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.541+0000 7f461fc19b80 1 --2- 172.21.15.57:0/1247894396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 0x56117f24dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.541+0000 7f461fc19b80 1 --2- 172.21.15.57:0/1247894396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f2943f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.541+0000 7f461fc19b80 1 -- 172.21.15.57:0/1247894396 >> 172.21.15.57:0/1247894396 conn(0x56117f243df0 msgr2=0x56117f24cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.541+0000 7f461fc19b80 1 -- 172.21.15.57:0/1247894396 shutdown_connections 2023-11-28T12:04:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.542+0000 7f461fc19b80 1 -- 172.21.15.57:0/1247894396 wait complete. 2023-11-28T12:04:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.542+0000 7f461fc19b80 1 -- start start 2023-11-28T12:04:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.542+0000 7f461fc19b80 1 Processor -- start 2023-11-28T12:04:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f461fc19b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 0x56117f27ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f461fc19b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f28faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f461fc19b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 0x56117f28ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f461fc19b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117f1a01f0 con 0x56117f28ab90 2023-11-28T12:04:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f461fc19b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117ef95460 con 0x56117f294020 2023-11-28T12:04:01.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f461fc19b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56117f138410 con 0x56117f276730 2023-11-28T12:04:01.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f461fc19b80 10 client.0 _ll_register_callbacks cb 0x56117f13e4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:04:01.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f4611015700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f28faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f4611015700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f28faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46644/0 (socket says 172.21.15.57:46644) 2023-11-28T12:04:01.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f4611015700 1 -- 172.21.15.57:0/713151826 learned_addr learned my addr 172.21.15.57:0/713151826 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f4611816700 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 0x56117f27ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f4612017700 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 0x56117f28ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f4611015700 1 -- 172.21.15.57:0/713151826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 msgr2=0x56117f27ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f4611015700 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 0x56117f27ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f4611015700 1 -- 172.21.15.57:0/713151826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 msgr2=0x56117f28ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f4611015700 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 0x56117f28ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f4611015700 1 -- 172.21.15.57:0/713151826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c0d1440 con 0x56117f28ab90 2023-11-28T12:04:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.544+0000 7f4612017700 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 0x56117f28ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f4611015700 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f28faa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00cec0 tx=0x7f45fc059040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0d8584e73f093e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f45ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:04:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f45ecff9700 1 -- 172.21.15.57:0/713151826 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc06cd80 con 0x56117f28ab90 2023-11-28T12:04:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f45ecff9700 1 -- 172.21.15.57:0/713151826 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f45fc05b1d0 con 0x56117f28ab90 2023-11-28T12:04:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f461fc19b80 1 -- 172.21.15.57:0/713151826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56117f1a4100 con 0x56117f28ab90 2023-11-28T12:04:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f45ecff9700 1 -- 172.21.15.57:0/713151826 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45fc0590b0 con 0x56117f28ab90 2023-11-28T12:04:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f461fc19b80 10 client.5952 fetch_fsmap learned FSMap version 24 2023-11-28T12:04:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f461fc19b80 1 -- 172.21.15.57:0/713151826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56117f081170 con 0x56117f28ab90 2023-11-28T12:04:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f45ecff9700 1 -- 172.21.15.57:0/713151826 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56117f1a4100 con 0x56117f28ab90 2023-11-28T12:04:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f45ecff9700 1 -- 172.21.15.57:0/713151826 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f45fc059210 con 0x56117f28ab90 2023-11-28T12:04:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f461fc19b80 10 client.5952 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T12:04:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f461fc19b80 10 client.5952 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T12:04:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f461fc19b80 20 client.5952 populate_metadata read hostname 'smithi057' 2023-11-28T12:04:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f461fc19b80 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x56117f27b3b0 0x56117f28af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f461fc19b80 4 client.5952 mds_command: new command op to 5739 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T12:04:01.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.545+0000 7f461fc19b80 1 -- 172.21.15.57:0/713151826 --> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56117efeffc0 con 0x56117f27b3b0 2023-11-28T12:04:01.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.546+0000 7f4611816700 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x56117f27b3b0 0x56117f28af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.546+0000 7f45ecff9700 10 client.5952 ms_handle_connect on v2:172.21.15.181:6834/3437917192 2023-11-28T12:04:01.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.546+0000 7f4611816700 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x56117f27b3b0 0x56117f28af80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7522863a86ae06c server_cookie=1d685f4de8f0d65d in_seq=0 out_seq=0 2023-11-28T12:04:01.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.546+0000 7f45ecff9700 1 -- 172.21.15.57:0/713151826 <== mds.0 v2:172.21.15.181:6834/3437917192 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x56117efeffc0 con 0x56117f27b3b0 2023-11-28T12:04:01.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.546+0000 7f45ecff9700 10 client.5952 handle_command_reply: tid=0 2023-11-28T12:04:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.546+0000 7f461fc19b80 1 client.5952 shutdown 2023-11-28T12:04:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.547+0000 7f461fc19b80 1 -- 172.21.15.57:0/713151826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 msgr2=0x56117f28faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.547+0000 7f461fc19b80 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f28faa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00cec0 tx=0x7f45fc059040 comp rx=0 tx=0).stop 2023-11-28T12:04:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.547+0000 7f461fc19b80 1 -- 172.21.15.57:0/713151826 shutdown_connections 2023-11-28T12:04:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.547+0000 7f461fc19b80 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x56117f27b3b0 0x56117f28af80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.547+0000 7f461fc19b80 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 0x56117f28ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.547+0000 7f461fc19b80 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 0x56117f27ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.547+0000 7f45ecff9700 0 client.5952 ms_handle_reset on v2:172.21.15.181:6834/3437917192 2023-11-28T12:04:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.547+0000 7f461fc19b80 1 --2- 172.21.15.57:0/713151826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f28faa0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.547+0000 7f461fc19b80 1 -- 172.21.15.57:0/713151826 >> 172.21.15.57:0/713151826 conn(0x56117f243df0 msgr2=0x56117f275b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.547+0000 7f461fc19b80 1 -- 172.21.15.57:0/713151826 shutdown_connections 2023-11-28T12:04:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.547+0000 7f461fc19b80 1 -- 172.21.15.57:0/713151826 wait complete. 2023-11-28T12:04:01.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.547+0000 7f461fc19b80 20 client.5952 trim_cache size 0 max 16384 2023-11-28T12:04:01.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.555+0000 7fbb58f36700 1 Processor -- start 2023-11-28T12:04:01.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.555+0000 7fbb58f36700 1 -- start start 2023-11-28T12:04:01.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.556+0000 7fbb58f36700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb540b21e0 0x7fbb540b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.556+0000 7fbb58f36700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbb540cf1c0 0x7fbb540d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.556+0000 7fbb58f36700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb540d2b20 con 0x7fbb540cf890 2023-11-28T12:04:01.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.556+0000 7fbb58f36700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb540b25b0 con 0x7fbb540cf1c0 2023-11-28T12:04:01.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.556+0000 7fbb58f36700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540cd600 con 0x7fbb540b21e0 2023-11-28T12:04:01.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.556+0000 7fbb51d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbb540cf1c0 0x7fbb540d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.556+0000 7fbb51d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbb540cf1c0 0x7fbb540d1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46650/0 (socket says 172.21.15.57:46650) 2023-11-28T12:04:01.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.556+0000 7fbb51d9b700 1 -- 172.21.15.57:0/1404788200 learned_addr learned my addr 172.21.15.57:0/1404788200 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.556+0000 7fbb5259c700 1 --2- 172.21.15.57:0/1404788200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb540b21e0 0x7fbb540b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.556+0000 7fbb51d9b700 1 -- 172.21.15.57:0/1404788200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb540b21e0 msgr2=0x7fbb540b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.556+0000 7fbb51d9b700 1 --2- 172.21.15.57:0/1404788200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb540b21e0 0x7fbb540b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.556+0000 7fbb51d9b700 1 -- 172.21.15.57:0/1404788200 >> v1:172.21.15.181:6789/0 conn(0x7fbb540cf890 legacy=0x7fbb540d2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.556+0000 7fbb51d9b700 1 -- 172.21.15.57:0/1404788200 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540cd8e0 con 0x7fbb540cf1c0 2023-11-28T12:04:01.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.557+0000 7fbb51d9b700 1 --2- 172.21.15.57:0/1404788200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb540cf1c0 0x7fbb540d1460 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbb480098f0 tx=0x7fbb48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2073a07d8c8d415d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.557+0000 7fbb5159a700 1 -- 172.21.15.57:0/1404788200 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48011830 con 0x7fbb540cf1c0 2023-11-28T12:04:01.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.557+0000 7fbb5159a700 1 -- 172.21.15.57:0/1404788200 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb48011990 con 0x7fbb540cf1c0 2023-11-28T12:04:01.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.557+0000 7fbb5159a700 1 -- 172.21.15.57:0/1404788200 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48011d20 con 0x7fbb540cf1c0 2023-11-28T12:04:01.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.557+0000 7fbb58f36700 1 -- 172.21.15.57:0/1404788200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb540cf1c0 msgr2=0x7fbb540d1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.557+0000 7fbb58f36700 1 --2- 172.21.15.57:0/1404788200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb540cf1c0 0x7fbb540d1460 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbb480098f0 tx=0x7fbb48009c00 comp rx=0 tx=0).stop 2023-11-28T12:04:01.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.557+0000 7fbb58f36700 1 -- 172.21.15.57:0/1404788200 shutdown_connections 2023-11-28T12:04:01.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.557+0000 7fbb58f36700 1 --2- 172.21.15.57:0/1404788200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb540b21e0 0x7fbb540b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.557+0000 7fbb58f36700 1 --2- 172.21.15.57:0/1404788200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb540cf1c0 0x7fbb540d1460 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.557+0000 7fbb58f36700 1 -- 172.21.15.57:0/1404788200 >> 172.21.15.57:0/1404788200 conn(0x7fbb5406aa80 msgr2=0x7fbb5406ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.557+0000 7fbb58f36700 1 -- 172.21.15.57:0/1404788200 shutdown_connections 2023-11-28T12:04:01.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.558+0000 7fbb58f36700 1 -- 172.21.15.57:0/1404788200 wait complete. 2023-11-28T12:04:01.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.559+0000 7fbb58f36700 1 Processor -- start 2023-11-28T12:04:01.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.559+0000 7fbb58f36700 1 -- start start 2023-11-28T12:04:01.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb58f36700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb540b21e0 0x7fbb5415df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb58f36700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb540cf1c0 0x7fbb540c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb58f36700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb540cf890 0x7fbb5414fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb58f36700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540cbe70 con 0x7fbb540cf1c0 2023-11-28T12:04:01.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb58f36700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5415e810 con 0x7fbb540cf890 2023-11-28T12:04:01.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb58f36700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540cb820 con 0x7fbb540b21e0 2023-11-28T12:04:01.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb51d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb540cf1c0 0x7fbb540c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb5259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb540b21e0 0x7fbb5415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb51d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb540cf1c0 0x7fbb540c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46654/0 (socket says 172.21.15.57:46654) 2023-11-28T12:04:01.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb51d9b700 1 -- 172.21.15.57:0/2453931605 learned_addr learned my addr 172.21.15.57:0/2453931605 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb52d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb540cf890 0x7fbb5414fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb51d9b700 1 -- 172.21.15.57:0/2453931605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb540b21e0 msgr2=0x7fbb5415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb51d9b700 1 --2- 172.21.15.57:0/2453931605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb540b21e0 0x7fbb5415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb51d9b700 1 -- 172.21.15.57:0/2453931605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb540cf890 msgr2=0x7fbb5414fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb51d9b700 1 --2- 172.21.15.57:0/2453931605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb540cf890 0x7fbb5414fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb51d9b700 1 -- 172.21.15.57:0/2453931605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb48009550 con 0x7fbb540cf1c0 2023-11-28T12:04:01.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb5259c700 1 --2- 172.21.15.57:0/2453931605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb540b21e0 0x7fbb5415df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:01.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.560+0000 7fbb51d9b700 1 --2- 172.21.15.57:0/2453931605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb540cf1c0 0x7fbb540c8dd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbb48009c40 tx=0x7fbb48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.561+0000 7fbb42ffd700 1 -- 172.21.15.57:0/2453931605 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48004400 con 0x7fbb540cf1c0 2023-11-28T12:04:01.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.561+0000 7fbb42ffd700 1 -- 172.21.15.57:0/2453931605 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4801f070 con 0x7fbb540cf1c0 2023-11-28T12:04:01.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.561+0000 7fbb58f36700 1 -- 172.21.15.57:0/2453931605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb540d19a0 con 0x7fbb540cf1c0 2023-11-28T12:04:01.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.561+0000 7fbb42ffd700 1 -- 172.21.15.57:0/2453931605 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb480038e0 con 0x7fbb540cf1c0 2023-11-28T12:04:01.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.561+0000 7fbb58f36700 1 -- 172.21.15.57:0/2453931605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb540d27b0 con 0x7fbb540cf1c0 2023-11-28T12:04:01.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.562+0000 7fbb58f36700 1 -- 172.21.15.57:0/2453931605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fbb540cfd10 con 0x7fbb540cf1c0 2023-11-28T12:04:01.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.562+0000 7fbb42ffd700 1 -- 172.21.15.57:0/2453931605 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbb4802a020 con 0x7fbb540cf1c0 2023-11-28T12:04:01.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.563+0000 7fbb42ffd700 1 --2- 172.21.15.57:0/2453931605 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb340682f0 0x7fbb3406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.563+0000 7fbb42ffd700 1 -- 172.21.15.57:0/2453931605 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbb48087740 con 0x7fbb540cf1c0 2023-11-28T12:04:01.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.563+0000 7fbb5259c700 1 --2- 172.21.15.57:0/2453931605 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb340682f0 0x7fbb3406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.563+0000 7fbb42ffd700 1 -- 172.21.15.57:0/2453931605 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+502 (secure 0 0 0) 0x7fbb48087bc0 con 0x7fbb540cf1c0 2023-11-28T12:04:01.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.563+0000 7fbb58f36700 1 -- 172.21.15.57:0/2453931605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fbb540678c0 con 0x7fbb540cf1c0 2023-11-28T12:04:01.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.563+0000 7fbb5259c700 1 --2- 172.21.15.57:0/2453931605 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbb340682f0 0x7fbb3406a7a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009fd0 tx=0x7fbb44009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.566+0000 7f461fc19b80 1 Processor -- start 2023-11-28T12:04:01.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.566+0000 7f461fc19b80 1 -- start start 2023-11-28T12:04:01.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f461fc19b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 0x56117f28f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f461fc19b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f28ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f461fc19b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 0x56117f2744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f461fc19b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117f2438b0 con 0x56117f28ab90 2023-11-28T12:04:01.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f461fc19b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117f297cf0 con 0x56117f294020 2023-11-28T12:04:01.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f461fc19b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56117f214310 con 0x56117f276730 2023-11-28T12:04:01.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f4611015700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f28ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f4611015700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f28ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46656/0 (socket says 172.21.15.57:46656) 2023-11-28T12:04:01.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f4611015700 1 -- 172.21.15.57:0/4288225427 learned_addr learned my addr 172.21.15.57:0/4288225427 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f4611816700 1 --2- 172.21.15.57:0/4288225427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 0x56117f28f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f4612017700 1 --2- 172.21.15.57:0/4288225427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 0x56117f2744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f4611015700 1 -- 172.21.15.57:0/4288225427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 msgr2=0x56117f28f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f4611015700 1 --2- 172.21.15.57:0/4288225427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 0x56117f28f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f4611015700 1 -- 172.21.15.57:0/4288225427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 msgr2=0x56117f2744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f4611015700 1 --2- 172.21.15.57:0/4288225427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 0x56117f2744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f4611015700 1 -- 172.21.15.57:0/4288225427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56117f1673b0 con 0x56117f28ab90 2023-11-28T12:04:01.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.567+0000 7f4611015700 1 --2- 172.21.15.57:0/4288225427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f28ff20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f45fc055370 tx=0x7f45fc07e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d431ba67c8c68436 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.568+0000 7f45effff700 1 -- 172.21.15.57:0/4288225427 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc00e040 con 0x56117f28ab90 2023-11-28T12:04:01.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.568+0000 7f45effff700 1 -- 172.21.15.57:0/4288225427 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc061070 con 0x56117f28ab90 2023-11-28T12:04:01.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.568+0000 7f45effff700 1 -- 172.21.15.57:0/4288225427 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc06cd80 con 0x56117f28ab90 2023-11-28T12:04:01.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.568+0000 7f461fc19b80 1 -- 172.21.15.57:0/4288225427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 msgr2=0x56117f28ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.568+0000 7f461fc19b80 1 --2- 172.21.15.57:0/4288225427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f28ff20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f45fc055370 tx=0x7f45fc07e000 comp rx=0 tx=0).stop 2023-11-28T12:04:01.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.568+0000 7f461fc19b80 1 -- 172.21.15.57:0/4288225427 shutdown_connections 2023-11-28T12:04:01.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.568+0000 7f461fc19b80 1 --2- 172.21.15.57:0/4288225427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f294020 0x56117f2744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.568+0000 7f461fc19b80 1 --2- 172.21.15.57:0/4288225427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f276730 0x56117f28f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.568+0000 7f461fc19b80 1 --2- 172.21.15.57:0/4288225427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f28ab90 0x56117f28ff20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.568+0000 7f461fc19b80 1 -- 172.21.15.57:0/4288225427 >> 172.21.15.57:0/4288225427 conn(0x56117f243df0 msgr2=0x56117f250320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.569+0000 7f461fc19b80 1 -- 172.21.15.57:0/4288225427 shutdown_connections 2023-11-28T12:04:01.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.569+0000 7f461fc19b80 1 -- 172.21.15.57:0/4288225427 wait complete. 2023-11-28T12:04:01.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.569+0000 7f461fc19b80 1 -- start start 2023-11-28T12:04:01.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.569+0000 7f461fc19b80 1 Processor -- start 2023-11-28T12:04:01.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.570+0000 7f461fc19b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f276730 0x56117f2e2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.570+0000 7f461fc19b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f28ab90 0x56117f27a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.570+0000 7f461fc19b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f294020 0x56117f27a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.570+0000 7f461fc19b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117f248ed0 con 0x56117f276730 2023-11-28T12:04:01.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.570+0000 7f461fc19b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117f290480 con 0x56117f28ab90 2023-11-28T12:04:01.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.570+0000 7f461fc19b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56117f2efa30 con 0x56117f294020 2023-11-28T12:04:01.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.570+0000 7f461fc19b80 10 client.0 _ll_register_callbacks cb 0x7f45fc0551b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:04:01.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.570+0000 7f4611816700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f276730 0x56117f2e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.570+0000 7f4611816700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f276730 0x56117f2e2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46658/0 (socket says 172.21.15.57:46658) 2023-11-28T12:04:01.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.570+0000 7f4611816700 1 -- 172.21.15.57:0/2938673243 learned_addr learned my addr 172.21.15.57:0/2938673243 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f4612017700 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f294020 0x56117f27a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f4611015700 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f28ab90 0x56117f27a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f4611816700 1 -- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f294020 msgr2=0x56117f27a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f4611816700 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f294020 0x56117f27a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f4611816700 1 -- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f28ab90 msgr2=0x56117f27a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f4611816700 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f28ab90 0x56117f27a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f4611816700 1 -- 172.21.15.57:0/2938673243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56117f081170 con 0x56117f276730 2023-11-28T12:04:01.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f4612017700 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f294020 0x56117f27a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:01.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f4611015700 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f28ab90 0x56117f27a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:01.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f4611816700 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f276730 0x56117f2e2340 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x56117f28dd10 tx=0x7f4608004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1287135279ef2201 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f45ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:04:01.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f45ecff9700 1 -- 172.21.15.57:0/2938673243 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608028040 con 0x56117f276730 2023-11-28T12:04:01.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f45ecff9700 1 -- 172.21.15.57:0/2938673243 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f460802ae40 con 0x56117f276730 2023-11-28T12:04:01.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f461fc19b80 1 -- 172.21.15.57:0/2938673243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56117f250150 con 0x56117f276730 2023-11-28T12:04:01.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f45ecff9700 1 -- 172.21.15.57:0/2938673243 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4608015440 con 0x56117f276730 2023-11-28T12:04:01.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f461fc19b80 10 client.5964 fetch_fsmap learned FSMap version 24 2023-11-28T12:04:01.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f461fc19b80 1 -- 172.21.15.57:0/2938673243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56117eff52a0 con 0x56117f276730 2023-11-28T12:04:01.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f45ecff9700 1 -- 172.21.15.57:0/2938673243 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56117f250150 con 0x56117f276730 2023-11-28T12:04:01.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f45ecff9700 1 -- 172.21.15.57:0/2938673243 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f46080155a0 con 0x56117f276730 2023-11-28T12:04:01.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f461fc19b80 10 client.5964 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T12:04:01.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f461fc19b80 10 client.5964 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T12:04:01.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f461fc19b80 20 client.5964 populate_metadata read hostname 'smithi057' 2023-11-28T12:04:01.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f461fc19b80 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x56117f28ffd0 0x56117f247dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f461fc19b80 4 client.5964 mds_command: new command op to 5739 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-28T12:04:01.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.571+0000 7f461fc19b80 1 -- 172.21.15.57:0/2938673243 --> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x7f45fc054b60 con 0x56117f28ffd0 2023-11-28T12:04:01.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.572+0000 7f4611015700 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x56117f28ffd0 0x56117f247dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.572+0000 7f45ecff9700 10 client.5964 ms_handle_connect on v2:172.21.15.181:6834/3437917192 2023-11-28T12:04:01.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.572+0000 7f4611015700 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x56117f28ffd0 0x56117f247dc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba90bda5fee8a46 server_cookie=f4d781ec8dcb6125 in_seq=0 out_seq=0 2023-11-28T12:04:01.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f45ecff9700 1 -- 172.21.15.57:0/2938673243 <== mds.0 v2:172.21.15.181:6834/3437917192 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f45fc054b60 con 0x56117f28ffd0 2023-11-28T12:04:01.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f45ecff9700 10 client.5964 handle_command_reply: tid=0 2023-11-28T12:04:01.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f461fc19b80 1 client.5964 shutdown 2023-11-28T12:04:01.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f461fc19b80 1 -- 172.21.15.57:0/2938673243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f276730 msgr2=0x56117f2e2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f461fc19b80 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f276730 0x56117f2e2340 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x56117f28dd10 tx=0x7f4608004750 comp rx=0 tx=0).stop 2023-11-28T12:04:01.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f461fc19b80 1 -- 172.21.15.57:0/2938673243 shutdown_connections 2023-11-28T12:04:01.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f461fc19b80 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x56117f28ffd0 0x56117f247dc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f461fc19b80 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56117f294020 0x56117f27a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f461fc19b80 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56117f276730 0x56117f2e2340 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f45ecff9700 0 client.5964 ms_handle_reset on v2:172.21.15.181:6834/3437917192 2023-11-28T12:04:01.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f461fc19b80 1 --2- 172.21.15.57:0/2938673243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56117f28ab90 0x56117f27a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f461fc19b80 1 -- 172.21.15.57:0/2938673243 >> 172.21.15.57:0/2938673243 conn(0x56117f243df0 msgr2=0x56117f2a5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f461fc19b80 1 -- 172.21.15.57:0/2938673243 shutdown_connections 2023-11-28T12:04:01.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f461fc19b80 1 -- 172.21.15.57:0/2938673243 wait complete. 2023-11-28T12:04:01.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.573+0000 7f461fc19b80 20 client.5964 trim_cache size 0 max 16384 2023-11-28T12:04:01.587 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:04:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2023-11-28T12:04:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "a699a63e-8a31-4605-984d-41dc222169f9", 2023-11-28T12:04:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2023-11-28T12:04:01.588 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:04:01.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.574+0000 7f4613a7a700 1 -- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f45f00682c0 msgr2=0x7f45f006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.574+0000 7f4613a7a700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f45f00682c0 0x7f45f006a770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4608009d00 tx=0x7f4608008040 comp rx=0 tx=0).stop 2023-11-28T12:04:01.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.574+0000 7f4613a7a700 1 -- 172.21.15.57:0/1708954414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f460c15cce0 msgr2=0x7f460c0ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.574+0000 7f4613a7a700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f460c15cce0 0x7f460c0ca230 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00c700 tx=0x7f45fc00cac0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.574+0000 7f4613a7a700 1 -- 172.21.15.57:0/1708954414 shutdown_connections 2023-11-28T12:04:01.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.574+0000 7f4613a7a700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f45f00682c0 0x7f45f006a770 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.574+0000 7f4613a7a700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f460c0b4720 0x7f460c15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.574+0000 7f4613a7a700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f460c0b1c90 0x7f460c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.574+0000 7f4613a7a700 1 --2- 172.21.15.57:0/1708954414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f460c15cce0 0x7f460c0ca230 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.574+0000 7f4613a7a700 1 -- 172.21.15.57:0/1708954414 >> 172.21.15.57:0/1708954414 conn(0x7f460c09c4d0 msgr2=0x7f460c158920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.574+0000 7f4613a7a700 1 -- 172.21.15.57:0/1708954414 shutdown_connections 2023-11-28T12:04:01.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.574+0000 7f4613a7a700 1 -- 172.21.15.57:0/1708954414 wait complete. 2023-11-28T12:04:01.591 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T12:04:01.592 INFO:tasks.ceph.mds.b.smithi181.stderr:2023-11-28T12:04:01.573+0000 7f4daf721700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5913=0-4194304@1} caps={5913=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5913 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x56429dce3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T12:04:01.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.728+0000 7fbcf0aca700 1 Processor -- start 2023-11-28T12:04:01.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.728+0000 7fbcf0aca700 1 -- start start 2023-11-28T12:04:01.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.729+0000 7fbcf0aca700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbcec15a4d0 0x7fbcec06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.729+0000 7fbcf0aca700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbcec06a7b0 0x7fbcec0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.729+0000 7fbcf0aca700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec15e0d0 con 0x7fbcec0cdd50 2023-11-28T12:04:01.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.729+0000 7fbcf0aca700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcec06ac30 con 0x7fbcec06a7b0 2023-11-28T12:04:01.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.729+0000 7fbcf0aca700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcec0cc020 con 0x7fbcec15a4d0 2023-11-28T12:04:01.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.729+0000 7fbce9d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbcec06a7b0 0x7fbcec0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.729+0000 7fbce9d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbcec06a7b0 0x7fbcec0cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46664/0 (socket says 172.21.15.57:46664) 2023-11-28T12:04:01.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.729+0000 7fbce9d9b700 1 -- 172.21.15.57:0/1534732083 learned_addr learned my addr 172.21.15.57:0/1534732083 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.730+0000 7fbcea59c700 1 --2- 172.21.15.57:0/1534732083 >> v2:172.21.15.181:3300/0 conn(0x7fbcec15a4d0 0x7fbcec06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.730+0000 7fbce9d9b700 1 -- 172.21.15.57:0/1534732083 >> v2:172.21.15.181:3300/0 conn(0x7fbcec15a4d0 msgr2=0x7fbcec06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.730+0000 7fbce9d9b700 1 --2- 172.21.15.57:0/1534732083 >> v2:172.21.15.181:3300/0 conn(0x7fbcec15a4d0 0x7fbcec06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.730+0000 7fbce9d9b700 1 -- 172.21.15.57:0/1534732083 >> v1:172.21.15.181:6789/0 conn(0x7fbcec0cdd50 legacy=0x7fbcec0b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.730+0000 7fbce9d9b700 1 -- 172.21.15.57:0/1534732083 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0ce560 con 0x7fbcec06a7b0 2023-11-28T12:04:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.730+0000 7fbcea59c700 1 --2- 172.21.15.57:0/1534732083 >> v2:172.21.15.181:3300/0 conn(0x7fbcec15a4d0 0x7fbcec06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.730+0000 7fbce9d9b700 1 --2- 172.21.15.57:0/1534732083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcec06a7b0 0x7fbcec0cd810 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbcec06b140 tx=0x7fbcd4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66537e0c4ddb672 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.730+0000 7fbce959a700 1 -- 172.21.15.57:0/1534732083 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400b870 con 0x7fbcec06a7b0 2023-11-28T12:04:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.730+0000 7fbce959a700 1 -- 172.21.15.57:0/1534732083 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd400b9d0 con 0x7fbcec06a7b0 2023-11-28T12:04:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.730+0000 7fbce959a700 1 -- 172.21.15.57:0/1534732083 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400bd60 con 0x7fbcec06a7b0 2023-11-28T12:04:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.731+0000 7fbcf0aca700 1 -- 172.21.15.57:0/1534732083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcec06a7b0 msgr2=0x7fbcec0cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.731+0000 7fbcf0aca700 1 --2- 172.21.15.57:0/1534732083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcec06a7b0 0x7fbcec0cd810 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbcec06b140 tx=0x7fbcd4009bd0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.731+0000 7fbcf0aca700 1 -- 172.21.15.57:0/1534732083 shutdown_connections 2023-11-28T12:04:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.731+0000 7fbcf0aca700 1 --2- 172.21.15.57:0/1534732083 >> v2:172.21.15.181:3300/0 conn(0x7fbcec15a4d0 0x7fbcec06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.731+0000 7fbcf0aca700 1 --2- 172.21.15.57:0/1534732083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcec06a7b0 0x7fbcec0cd810 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.731+0000 7fbcf0aca700 1 -- 172.21.15.57:0/1534732083 >> 172.21.15.57:0/1534732083 conn(0x7fbcec06c010 msgr2=0x7fbcec0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.731+0000 7fbcf0aca700 1 -- 172.21.15.57:0/1534732083 shutdown_connections 2023-11-28T12:04:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.731+0000 7fbcf0aca700 1 -- 172.21.15.57:0/1534732083 wait complete. 2023-11-28T12:04:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.732+0000 7fbcf0aca700 1 Processor -- start 2023-11-28T12:04:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.732+0000 7fbcf0aca700 1 -- start start 2023-11-28T12:04:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.733+0000 7fbcf0aca700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcec06a7b0 0x7fbcec0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.733+0000 7fbcf0aca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec0cdd50 0x7fbcec0c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.733+0000 7fbcf0aca700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcec15a4d0 0x7fbcec0c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.733+0000 7fbcf0aca700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec15d7a0 con 0x7fbcec06a7b0 2023-11-28T12:04:01.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.733+0000 7fbcf0aca700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0c73a0 con 0x7fbcec0cdd50 2023-11-28T12:04:01.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.733+0000 7fbcf0aca700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0c7680 con 0x7fbcec15a4d0 2023-11-28T12:04:01.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.733+0000 7fbcea59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcec06a7b0 0x7fbcec0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.733+0000 7fbcea59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcec06a7b0 0x7fbcec0c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46678/0 (socket says 172.21.15.57:46678) 2023-11-28T12:04:01.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.733+0000 7fbcea59c700 1 -- 172.21.15.57:0/3702669081 learned_addr learned my addr 172.21.15.57:0/3702669081 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.733+0000 7fbce9d9b700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec0cdd50 0x7fbcec0c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.733+0000 7fbcead9d700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcec15a4d0 0x7fbcec0c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.734+0000 7fbcea59c700 1 -- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcec15a4d0 msgr2=0x7fbcec0c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.734+0000 7fbcea59c700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcec15a4d0 0x7fbcec0c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.734+0000 7fbcea59c700 1 -- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec0cdd50 msgr2=0x7fbcec0c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.734+0000 7fbcea59c700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec0cdd50 0x7fbcec0c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.734+0000 7fbcea59c700 1 -- 172.21.15.57:0/3702669081 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd4009550 con 0x7fbcec06a7b0 2023-11-28T12:04:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.734+0000 7fbcea59c700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcec06a7b0 0x7fbcec0c6ca0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00e810 tx=0x7fbcdc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.734+0000 7fbce2ffd700 1 -- 172.21.15.57:0/3702669081 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc00c020 con 0x7fbcec06a7b0 2023-11-28T12:04:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.734+0000 7fbce2ffd700 1 -- 172.21.15.57:0/3702669081 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc004250 con 0x7fbcec06a7b0 2023-11-28T12:04:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.734+0000 7fbcf0aca700 1 -- 172.21.15.57:0/3702669081 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec15e600 con 0x7fbcec06a7b0 2023-11-28T12:04:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.734+0000 7fbce2ffd700 1 -- 172.21.15.57:0/3702669081 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0036a0 con 0x7fbcec06a7b0 2023-11-28T12:04:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.734+0000 7fbcf0aca700 1 -- 172.21.15.57:0/3702669081 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec15e320 con 0x7fbcec06a7b0 2023-11-28T12:04:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.736+0000 7fbce2ffd700 1 -- 172.21.15.57:0/3702669081 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbcdc003800 con 0x7fbcec06a7b0 2023-11-28T12:04:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.736+0000 7fbce2ffd700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbccc0682f0 0x7fbccc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.736+0000 7fbce2ffd700 1 -- 172.21.15.57:0/3702669081 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbcdc085820 con 0x7fbcec06a7b0 2023-11-28T12:04:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.736+0000 7fbce9d9b700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbccc0682f0 0x7fbccc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.737+0000 7fbce9d9b700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbccc0682f0 0x7fbccc06a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4000c00 tx=0x7fbcd4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.738+0000 7fbcfcc69b80 1 Processor -- start 2023-11-28T12:04:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.739+0000 7fbcfcc69b80 1 -- start start 2023-11-28T12:04:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.739+0000 7fbcfcc69b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 0x55e7587ca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.739+0000 7fbcfcc69b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587d06e0 0x55e7587f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.739+0000 7fbcfcc69b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587cf630 0x55e758815a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.739+0000 7fbcfcc69b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75874be30 con 0x55e7587cf630 2023-11-28T12:04:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.739+0000 7fbcfcc69b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7586b8190 con 0x55e7587f90f0 2023-11-28T12:04:01.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.739+0000 7fbcfcc69b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e758726e00 con 0x55e7587d06e0 2023-11-28T12:04:01.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbcead9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587cf630 0x55e758815a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbcead9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587cf630 0x55e758815a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46694/0 (socket says 172.21.15.57:46694) 2023-11-28T12:04:01.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbcead9d700 1 -- 172.21.15.57:0/4154018098 learned_addr learned my addr 172.21.15.57:0/4154018098 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbcea59c700 1 --2- 172.21.15.57:0/4154018098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 0x55e7587ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbce9d9b700 1 --2- 172.21.15.57:0/4154018098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587d06e0 0x55e7587f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbcea59c700 1 -- 172.21.15.57:0/4154018098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587d06e0 msgr2=0x55e7587f94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbcea59c700 1 --2- 172.21.15.57:0/4154018098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587d06e0 0x55e7587f94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbcea59c700 1 -- 172.21.15.57:0/4154018098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587cf630 msgr2=0x55e758815a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbcea59c700 1 --2- 172.21.15.57:0/4154018098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587cf630 0x55e758815a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbcea59c700 1 -- 172.21.15.57:0/4154018098 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7585147e0 con 0x55e7587f90f0 2023-11-28T12:04:01.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbcead9d700 1 --2- 172.21.15.57:0/4154018098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587cf630 0x55e758815a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:01.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbcea59c700 1 --2- 172.21.15.57:0/4154018098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 0x55e7587ca780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc05d8a0 tx=0x7fbcdc05ddd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc0e2da75d795b21 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbccbfff700 1 -- 172.21.15.57:0/4154018098 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc055230 con 0x55e7587f90f0 2023-11-28T12:04:01.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbccbfff700 1 -- 172.21.15.57:0/4154018098 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc059040 con 0x55e7587f90f0 2023-11-28T12:04:01.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.740+0000 7fbccbfff700 1 -- 172.21.15.57:0/4154018098 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc05d940 con 0x55e7587f90f0 2023-11-28T12:04:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.741+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/4154018098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 msgr2=0x55e7587ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.741+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/4154018098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 0x55e7587ca780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc05d8a0 tx=0x7fbcdc05ddd0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.741+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/4154018098 shutdown_connections 2023-11-28T12:04:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.741+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/4154018098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587cf630 0x55e758815a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.741+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/4154018098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 0x55e7587ca780 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.741+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/4154018098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587d06e0 0x55e7587f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.741+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/4154018098 >> 172.21.15.57:0/4154018098 conn(0x55e7587c3d70 msgr2=0x55e7587ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.741+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/4154018098 shutdown_connections 2023-11-28T12:04:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.741+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/4154018098 wait complete. 2023-11-28T12:04:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.742+0000 7fbcfcc69b80 1 -- start start 2023-11-28T12:04:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.742+0000 7fbcfcc69b80 1 Processor -- start 2023-11-28T12:04:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.743+0000 7fbcfcc69b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587cf630 0x55e7587ef110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.743+0000 7fbcfcc69b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587d06e0 0x55e7587fa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.743+0000 7fbcfcc69b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 0x55e7587fa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.743+0000 7fbcfcc69b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e758720170 con 0x55e7587d06e0 2023-11-28T12:04:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.743+0000 7fbcfcc69b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7585153e0 con 0x55e7587f90f0 2023-11-28T12:04:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.743+0000 7fbcfcc69b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7586b8390 con 0x55e7587cf630 2023-11-28T12:04:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.743+0000 7fbcfcc69b80 10 client.0 _ll_register_callbacks cb 0x55e7586be450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:04:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcea59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587cf630 0x55e7587ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbce9d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587d06e0 0x55e7587fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcea59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587cf630 0x55e7587ef110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:59060/0 (socket says 172.21.15.57:59060) 2023-11-28T12:04:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcea59c700 1 -- 172.21.15.57:0/2263559135 learned_addr learned my addr 172.21.15.57:0/2263559135 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcead9d700 1 --2- 172.21.15.57:0/2263559135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 0x55e7587fa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcea59c700 1 -- 172.21.15.57:0/2263559135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 msgr2=0x55e7587fa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcea59c700 1 --2- 172.21.15.57:0/2263559135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 0x55e7587fa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcea59c700 1 -- 172.21.15.57:0/2263559135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587d06e0 msgr2=0x55e7587fa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcea59c700 1 --2- 172.21.15.57:0/2263559135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587d06e0 0x55e7587fa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcea59c700 1 -- 172.21.15.57:0/2263559135 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec15e320 con 0x55e7587cf630 2023-11-28T12:04:01.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcea59c700 1 --2- 172.21.15.57:0/2263559135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587cf630 0x55e7587ef110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00ee70 tx=0x7fbcdc01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de3276850d400780 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcc97fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3301/0 2023-11-28T12:04:01.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcc97fa700 1 -- 172.21.15.57:0/2263559135 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc061070 con 0x55e7587cf630 2023-11-28T12:04:01.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcc97fa700 1 -- 172.21.15.57:0/2263559135 <== mon.2 v2:172.21.15.181:3301/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbcdc05db50 con 0x55e7587cf630 2023-11-28T12:04:01.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2263559135 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e758724080 con 0x55e7587cf630 2023-11-28T12:04:01.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcc97fa700 1 -- 172.21.15.57:0/2263559135 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcdc0690b0 con 0x55e7587cf630 2023-11-28T12:04:01.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcfcc69b80 10 client.5399 fetch_fsmap learned FSMap version 24 2023-11-28T12:04:01.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.744+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2263559135 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e7586010f0 con 0x55e7587cf630 2023-11-28T12:04:01.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.745+0000 7fbcc97fa700 1 -- 172.21.15.57:0/2263559135 <== mon.2 v2:172.21.15.181:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e758724080 con 0x55e7587cf630 2023-11-28T12:04:01.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.745+0000 7fbcc97fa700 1 -- 172.21.15.57:0/2263559135 <== mon.2 v2:172.21.15.181:3301/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fbcdc051b30 con 0x55e7587cf630 2023-11-28T12:04:01.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.745+0000 7fbcfcc69b80 10 client.5399 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T12:04:01.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.745+0000 7fbcfcc69b80 10 client.5399 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T12:04:01.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.745+0000 7fbcfcc69b80 20 client.5399 populate_metadata read hostname 'smithi057' 2023-11-28T12:04:01.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.745+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2263559135 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x55e7587ec6a0 0x55e7587eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.745+0000 7fbcfcc69b80 4 client.5399 mds_command: new command op to 5739 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T12:04:01.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.745+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2263559135 --> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e75856ff40 con 0x55e7587ec6a0 2023-11-28T12:04:01.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.745+0000 7fbce9d9b700 1 --2- 172.21.15.57:0/2263559135 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x55e7587ec6a0 0x55e7587eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.745+0000 7fbcc97fa700 10 client.5399 ms_handle_connect on v2:172.21.15.181:6834/3437917192 2023-11-28T12:04:01.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.745+0000 7fbce9d9b700 1 --2- 172.21.15.57:0/2263559135 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x55e7587ec6a0 0x55e7587eb0f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=909e9a7976e73382 server_cookie=f2fab64e07c667eb in_seq=0 out_seq=0 2023-11-28T12:04:01.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcc97fa700 1 -- 172.21.15.57:0/2263559135 <== mds.0 v2:172.21.15.181:6834/3437917192 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55e75856ff40 con 0x55e7587ec6a0 2023-11-28T12:04:01.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcc97fa700 10 client.5399 handle_command_reply: tid=0 2023-11-28T12:04:01.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcfcc69b80 1 client.5399 shutdown 2023-11-28T12:04:01.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2263559135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587cf630 msgr2=0x55e7587ef110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2263559135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587cf630 0x55e7587ef110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00ee70 tx=0x7fbcdc01c000 comp rx=0 tx=0).stop 2023-11-28T12:04:01.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2263559135 shutdown_connections 2023-11-28T12:04:01.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2263559135 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x55e7587ec6a0 0x55e7587eb0f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2263559135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 0x55e7587fa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2263559135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587cf630 0x55e7587ef110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcc97fa700 0 client.5399 ms_handle_reset on v2:172.21.15.181:6834/3437917192 2023-11-28T12:04:01.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2263559135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587d06e0 0x55e7587fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2263559135 >> 172.21.15.57:0/2263559135 conn(0x55e7587c3d70 msgr2=0x55e758814d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2263559135 shutdown_connections 2023-11-28T12:04:01.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2263559135 wait complete. 2023-11-28T12:04:01.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.746+0000 7fbcfcc69b80 20 client.5399 trim_cache size 0 max 16384 2023-11-28T12:04:01.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.764+0000 7fbcfcc69b80 1 Processor -- start 2023-11-28T12:04:01.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.764+0000 7fbcfcc69b80 1 -- start start 2023-11-28T12:04:01.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.765+0000 7fbcfcc69b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587cf630 0x55e7587ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.765+0000 7fbcfcc69b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587d06e0 0x55e7587cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.765+0000 7fbcfcc69b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 0x55e7587cb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.765+0000 7fbcfcc69b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7587cc960 con 0x55e7587d06e0 2023-11-28T12:04:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.765+0000 7fbcfcc69b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7587c3b10 con 0x55e7587f90f0 2023-11-28T12:04:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.765+0000 7fbcfcc69b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7587d01c0 con 0x55e7587cf630 2023-11-28T12:04:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.765+0000 7fbce9d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587d06e0 0x55e7587cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.765+0000 7fbce9d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587d06e0 0x55e7587cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46716/0 (socket says 172.21.15.57:46716) 2023-11-28T12:04:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.765+0000 7fbce9d9b700 1 -- 172.21.15.57:0/2571670023 learned_addr learned my addr 172.21.15.57:0/2571670023 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.765+0000 7fbcea59c700 1 --2- 172.21.15.57:0/2571670023 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587cf630 0x55e7587ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.765+0000 7fbcead9d700 1 --2- 172.21.15.57:0/2571670023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 0x55e7587cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.766+0000 7fbce9d9b700 1 -- 172.21.15.57:0/2571670023 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587cf630 msgr2=0x55e7587ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.766+0000 7fbce9d9b700 1 --2- 172.21.15.57:0/2571670023 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587cf630 0x55e7587ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.766+0000 7fbce9d9b700 1 -- 172.21.15.57:0/2571670023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 msgr2=0x55e7587cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.766+0000 7fbce9d9b700 1 --2- 172.21.15.57:0/2571670023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 0x55e7587cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.766+0000 7fbce9d9b700 1 -- 172.21.15.57:0/2571670023 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7586e7330 con 0x55e7587d06e0 2023-11-28T12:04:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.766+0000 7fbce9d9b700 1 --2- 172.21.15.57:0/2571670023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587d06e0 0x55e7587cb2d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4004720 tx=0x7fbcd4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60f940d99467c57 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.766+0000 7fbce0ff9700 1 -- 172.21.15.57:0/2571670023 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd402c040 con 0x55e7587d06e0 2023-11-28T12:04:01.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.766+0000 7fbce0ff9700 1 -- 172.21.15.57:0/2571670023 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd402d040 con 0x55e7587d06e0 2023-11-28T12:04:01.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.766+0000 7fbce0ff9700 1 -- 172.21.15.57:0/2571670023 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4020070 con 0x55e7587d06e0 2023-11-28T12:04:01.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.767+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2571670023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587d06e0 msgr2=0x55e7587cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.767+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2571670023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587d06e0 0x55e7587cb2d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4004720 tx=0x7fbcd4010000 comp rx=0 tx=0).stop 2023-11-28T12:04:01.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.767+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2571670023 shutdown_connections 2023-11-28T12:04:01.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.767+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2571670023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587f90f0 0x55e7587cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.767+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2571670023 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587cf630 0x55e7587ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.767+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2571670023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587d06e0 0x55e7587cb2d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.767+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2571670023 >> 172.21.15.57:0/2571670023 conn(0x55e7587c3d70 msgr2=0x55e7587f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.767+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2571670023 shutdown_connections 2023-11-28T12:04:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.767+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2571670023 wait complete. 2023-11-28T12:04:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.767+0000 7fbcfcc69b80 1 -- start start 2023-11-28T12:04:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.767+0000 7fbcfcc69b80 1 Processor -- start 2023-11-28T12:04:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.768+0000 7fbcfcc69b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587cf630 0x55e7587cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.768+0000 7fbcfcc69b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587d06e0 0x55e7587cb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.768+0000 7fbcfcc69b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587f90f0 0x55e7587e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.768+0000 7fbcfcc69b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7587fbd90 con 0x55e7587f90f0 2023-11-28T12:04:01.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.768+0000 7fbcfcc69b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7588157b0 con 0x55e7587cf630 2023-11-28T12:04:01.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.768+0000 7fbcfcc69b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7588155a0 con 0x55e7587d06e0 2023-11-28T12:04:01.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcfcc69b80 10 client.0 _ll_register_callbacks cb 0x55e7586be450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:04:01.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcead9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587f90f0 0x55e7587e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcead9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587f90f0 0x55e7587e8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46728/0 (socket says 172.21.15.57:46728) 2023-11-28T12:04:01.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcead9d700 1 -- 172.21.15.57:0/2140106977 learned_addr learned my addr 172.21.15.57:0/2140106977 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:01.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcea59c700 1 --2- 172.21.15.57:0/2140106977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587cf630 0x55e7587cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbce9d9b700 1 --2- 172.21.15.57:0/2140106977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587d06e0 0x55e7587cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcead9d700 1 -- 172.21.15.57:0/2140106977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587d06e0 msgr2=0x55e7587cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcead9d700 1 --2- 172.21.15.57:0/2140106977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587d06e0 0x55e7587cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcead9d700 1 -- 172.21.15.57:0/2140106977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587cf630 msgr2=0x55e7587cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:01.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcead9d700 1 --2- 172.21.15.57:0/2140106977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587cf630 0x55e7587cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcead9d700 1 -- 172.21.15.57:0/2140106977 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec0ce560 con 0x55e7587f90f0 2023-11-28T12:04:01.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcead9d700 1 --2- 172.21.15.57:0/2140106977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587f90f0 0x55e7587e8db0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55e7587cfa20 tx=0x7fbce4015690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337c843a19a89ee0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:01.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcc97fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:04:01.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcc97fa700 1 -- 172.21.15.57:0/2140106977 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce401b070 con 0x55e7587f90f0 2023-11-28T12:04:01.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcc97fa700 1 -- 172.21.15.57:0/2140106977 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbce4015ca0 con 0x55e7587f90f0 2023-11-28T12:04:01.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.769+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2140106977 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcdc059040 con 0x55e7587f90f0 2023-11-28T12:04:01.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcc97fa700 1 -- 172.21.15.57:0/2140106977 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbce4015e60 con 0x55e7587f90f0 2023-11-28T12:04:01.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcfcc69b80 10 client.5982 fetch_fsmap learned FSMap version 24 2023-11-28T12:04:01.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2140106977 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e758575220 con 0x55e7587f90f0 2023-11-28T12:04:01.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcc97fa700 1 -- 172.21.15.57:0/2140106977 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc059040 con 0x55e7587f90f0 2023-11-28T12:04:01.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcc97fa700 1 -- 172.21.15.57:0/2140106977 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fbce4012050 con 0x55e7587f90f0 2023-11-28T12:04:01.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcfcc69b80 10 client.5982 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T12:04:01.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcfcc69b80 10 client.5982 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T12:04:01.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcfcc69b80 20 client.5982 populate_metadata read hostname 'smithi057' 2023-11-28T12:04:01.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2140106977 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x55e7587fcbd0 0x55e7587e92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:01.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcfcc69b80 4 client.5982 mds_command: new command op to 5739 tid=0[{"prefix": "scrub status"}] 2023-11-28T12:04:01.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2140106977 --> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e7585a5090 con 0x55e7587fcbd0 2023-11-28T12:04:01.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcea59c700 1 --2- 172.21.15.57:0/2140106977 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x55e7587fcbd0 0x55e7587e92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:01.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcc97fa700 10 client.5982 ms_handle_connect on v2:172.21.15.181:6834/3437917192 2023-11-28T12:04:01.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.770+0000 7fbcea59c700 1 --2- 172.21.15.57:0/2140106977 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x55e7587fcbd0 0x55e7587e92f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bf6292cac3a5cc1 server_cookie=464dda12a919cf8f in_seq=0 out_seq=0 2023-11-28T12:04:01.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcc97fa700 1 -- 172.21.15.57:0/2140106977 <== mds.0 v2:172.21.15.181:6834/3437917192 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55e7585a5090 con 0x55e7587fcbd0 2023-11-28T12:04:01.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcc97fa700 10 client.5982 handle_command_reply: tid=0 2023-11-28T12:04:01.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcfcc69b80 1 client.5982 shutdown 2023-11-28T12:04:01.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2140106977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587f90f0 msgr2=0x55e7587e8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2140106977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587f90f0 0x55e7587e8db0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55e7587cfa20 tx=0x7fbce4015690 comp rx=0 tx=0).stop 2023-11-28T12:04:01.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2140106977 shutdown_connections 2023-11-28T12:04:01.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2140106977 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x55e7587fcbd0 0x55e7587e92f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2140106977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7587f90f0 0x55e7587e8db0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2140106977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7587cf630 0x55e7587cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcfcc69b80 1 --2- 172.21.15.57:0/2140106977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55e7587d06e0 0x55e7587cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcc97fa700 0 client.5982 ms_handle_reset on v2:172.21.15.181:6834/3437917192 2023-11-28T12:04:01.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2140106977 >> 172.21.15.57:0/2140106977 conn(0x55e7587c3d70 msgr2=0x55e7587ed900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2140106977 shutdown_connections 2023-11-28T12:04:01.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcfcc69b80 1 -- 172.21.15.57:0/2140106977 wait complete. 2023-11-28T12:04:01.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.771+0000 7fbcfcc69b80 20 client.5982 trim_cache size 0 max 16384 2023-11-28T12:04:01.785 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:04:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T12:04:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-11-28T12:04:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "a699a63e-8a31-4605-984d-41dc222169f9": { 2023-11-28T12:04:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-11-28T12:04:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "a699a63e-8a31-4605-984d-41dc222169f9", 2023-11-28T12:04:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,repair" 2023-11-28T12:04:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:01.787 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:04:01.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.772+0000 7fbcf0aca700 1 -- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbccc0682f0 msgr2=0x7fbccc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.772+0000 7fbcf0aca700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbccc0682f0 0x7fbccc06a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4000c00 tx=0x7fbcd4008030 comp rx=0 tx=0).stop 2023-11-28T12:04:01.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.772+0000 7fbcf0aca700 1 -- 172.21.15.57:0/3702669081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcec06a7b0 msgr2=0x7fbcec0c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:01.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.772+0000 7fbcf0aca700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcec06a7b0 0x7fbcec0c6ca0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00e810 tx=0x7fbcdc00eb20 comp rx=0 tx=0).stop 2023-11-28T12:04:01.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.772+0000 7fbcf0aca700 1 -- 172.21.15.57:0/3702669081 shutdown_connections 2023-11-28T12:04:01.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.772+0000 7fbcf0aca700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbccc0682f0 0x7fbccc06a7a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.772+0000 7fbcf0aca700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcec15a4d0 0x7fbcec0c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.772+0000 7fbcf0aca700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcec06a7b0 0x7fbcec0c6ca0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.772+0000 7fbcf0aca700 1 --2- 172.21.15.57:0/3702669081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcec0cdd50 0x7fbcec0c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:01.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.772+0000 7fbcf0aca700 1 -- 172.21.15.57:0/3702669081 >> 172.21.15.57:0/3702669081 conn(0x7fbcec06c010 msgr2=0x7fbcec155050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:01.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.772+0000 7fbcf0aca700 1 -- 172.21.15.57:0/3702669081 shutdown_connections 2023-11-28T12:04:01.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:01.772+0000 7fbcf0aca700 1 -- 172.21.15.57:0/3702669081 wait complete. 2023-11-28T12:04:01.791 INFO:tasks.cephfs.filesystem:scrub status for tag:a699a63e-8a31-4605-984d-41dc222169f9 - {'path': '/', 'tag': 'a699a63e-8a31-4605-984d-41dc222169f9', 'options': 'recursive,repair'} 2023-11-28T12:04:02.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:02.763+0000 7fbb42ffd700 1 -- 172.21.15.57:0/2453931605 <== mon.0 v2:172.21.15.57:3300/0 7 ==== log(7 entries from seq 1 at 2023-11-28T12:04:01.573760+0000) v1 ==== 1229+0+0 (secure 0 0 0) 0x7fbb48010380 con 0x7fbb540cf1c0 2023-11-28T12:04:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: cluster: 2023-11-28T12:04:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: id: 0309b83f-6570-4eee-96fa-279822cd7766 2023-11-28T12:04:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: health: HEALTH_OK 2023-11-28T12:04:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: services: 2023-11-28T12:04:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-28T12:04:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: mgr: x(active, since 2m), standbys: y 2023-11-28T12:04:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: mds: 1/1 daemons up, 3 standby 2023-11-28T12:04:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2023-11-28T12:04:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: data: 2023-11-28T12:04:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: volumes: 1/1 healthy 2023-11-28T12:04:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: pools: 4 pools, 105 pgs 2023-11-28T12:04:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: objects: 24 objects, 579 KiB 2023-11-28T12:04:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: usage: 72 MiB used, 720 GiB / 720 GiB avail 2023-11-28T12:04:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: pgs: 105 active+clean 2023-11-28T12:04:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: io: 2023-11-28T12:04:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: client: 1.2 KiB/s wr, 0 op/s rd, 3 op/s wr 2023-11-28T12:04:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:02.768 INFO:teuthology.orchestra.run.smithi057.stdout:2023-11-28T12:04:01.573760+0000 mds.b [INF] scrub queued for path: / 2023-11-28T12:04:02.768 INFO:teuthology.orchestra.run.smithi057.stdout:2023-11-28T12:04:01.573764+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2023-11-28T12:04:02.768 INFO:teuthology.orchestra.run.smithi057.stdout:2023-11-28T12:04:01.573768+0000 mds.b [INF] scrub summary: active paths [/] 2023-11-28T12:04:02.768 INFO:teuthology.orchestra.run.smithi057.stdout:2023-11-28T12:04:01.574668+0000 mds.b [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2023-11-28T12:04:02.768 INFO:teuthology.orchestra.run.smithi057.stdout:2023-11-28T12:04:01.574675+0000 mds.b [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2023-11-28T12:04:02.768 INFO:teuthology.orchestra.run.smithi057.stdout:2023-11-28T12:04:01.574936+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2023-11-28T12:04:02.769 INFO:teuthology.orchestra.run.smithi057.stdout:2023-11-28T12:04:01.984079+0000 mds.b [INF] scrub summary: idle 2023-11-28T12:04:16.489 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:04:16.492 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:04:31.793 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T12:04:31.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.937+0000 7fcba1043700 1 Processor -- start 2023-11-28T12:04:31.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.937+0000 7fcba1043700 1 -- start start 2023-11-28T12:04:31.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.938+0000 7fcba1043700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb9c15a4d0 0x7fcb9c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.938+0000 7fcba1043700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcb9c06a7b0 0x7fcb9c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.938+0000 7fcba1043700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb9c15e0d0 con 0x7fcb9c0cdd50 2023-11-28T12:04:31.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.938+0000 7fcba1043700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9c06ac30 con 0x7fcb9c06a7b0 2023-11-28T12:04:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.938+0000 7fcba1043700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9c0cc020 con 0x7fcb9c15a4d0 2023-11-28T12:04:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.938+0000 7fcb9a59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcb9c06a7b0 0x7fcb9c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.938+0000 7fcb9a59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcb9c06a7b0 0x7fcb9c0cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47798/0 (socket says 172.21.15.57:47798) 2023-11-28T12:04:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.938+0000 7fcb9a59c700 1 -- 172.21.15.57:0/1583670310 learned_addr learned my addr 172.21.15.57:0/1583670310 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.938+0000 7fcb9b59e700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcb9c0cdd50 0x7fcb9c0b1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:51374/0 (socket says 172.21.15.57:51374) 2023-11-28T12:04:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.939+0000 7fcb9ad9d700 1 --2- 172.21.15.57:0/1583670310 >> v2:172.21.15.181:3300/0 conn(0x7fcb9c15a4d0 0x7fcb9c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.939+0000 7fcb9a59c700 1 -- 172.21.15.57:0/1583670310 >> v2:172.21.15.181:3300/0 conn(0x7fcb9c15a4d0 msgr2=0x7fcb9c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.939+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/1583670310 >> v2:172.21.15.181:3300/0 conn(0x7fcb9c15a4d0 0x7fcb9c06a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.939+0000 7fcb9a59c700 1 -- 172.21.15.57:0/1583670310 >> v1:172.21.15.57:6789/0 conn(0x7fcb9c0cdd50 legacy=0x7fcb9c0b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.939+0000 7fcb99d9b700 1 -- 172.21.15.57:0/1583670310 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052193598 0 0) 0x7fcb9c15e0d0 con 0x7fcb9c0cdd50 2023-11-28T12:04:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.939+0000 7fcb9a59c700 1 -- 172.21.15.57:0/1583670310 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c0ce560 con 0x7fcb9c06a7b0 2023-11-28T12:04:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.939+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/1583670310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb9c06a7b0 0x7fcb9c0cd810 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c06b140 tx=0x7fcb90009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42611f1ce2ffde6c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.939+0000 7fcb99d9b700 1 -- 172.21.15.57:0/1583670310 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9000b870 con 0x7fcb9c06a7b0 2023-11-28T12:04:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.939+0000 7fcb99d9b700 1 -- 172.21.15.57:0/1583670310 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9000b9d0 con 0x7fcb9c06a7b0 2023-11-28T12:04:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.939+0000 7fcb99d9b700 1 -- 172.21.15.57:0/1583670310 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9000bd60 con 0x7fcb9c06a7b0 2023-11-28T12:04:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.940+0000 7fcba1043700 1 -- 172.21.15.57:0/1583670310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb9c06a7b0 msgr2=0x7fcb9c0cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.940+0000 7fcba1043700 1 --2- 172.21.15.57:0/1583670310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb9c06a7b0 0x7fcb9c0cd810 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c06b140 tx=0x7fcb90009bd0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.940+0000 7fcba1043700 1 -- 172.21.15.57:0/1583670310 shutdown_connections 2023-11-28T12:04:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.940+0000 7fcba1043700 1 --2- 172.21.15.57:0/1583670310 >> v2:172.21.15.181:3300/0 conn(0x7fcb9c15a4d0 0x7fcb9c06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.940+0000 7fcba1043700 1 --2- 172.21.15.57:0/1583670310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb9c06a7b0 0x7fcb9c0cd810 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.940+0000 7fcba1043700 1 -- 172.21.15.57:0/1583670310 >> 172.21.15.57:0/1583670310 conn(0x7fcb9c06c010 msgr2=0x7fcb9c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.940+0000 7fcba1043700 1 -- 172.21.15.57:0/1583670310 shutdown_connections 2023-11-28T12:04:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.940+0000 7fcba1043700 1 -- 172.21.15.57:0/1583670310 wait complete. 2023-11-28T12:04:31.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.941+0000 7fcba1043700 1 Processor -- start 2023-11-28T12:04:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.941+0000 7fcba1043700 1 -- start start 2023-11-28T12:04:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.942+0000 7fcba1043700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb9c06a7b0 0x7fcb9c1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.942+0000 7fcba1043700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb9c0cdd50 0x7fcb9c15d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.942+0000 7fcba1043700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb9c15a4d0 0x7fcb9c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.942+0000 7fcba1043700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c1f4730 con 0x7fcb9c0cdd50 2023-11-28T12:04:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.942+0000 7fcba1043700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c1f3b20 con 0x7fcb9c06a7b0 2023-11-28T12:04:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.942+0000 7fcba1043700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c1f3e00 con 0x7fcb9c15a4d0 2023-11-28T12:04:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.942+0000 7fcb9a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb9c0cdd50 0x7fcb9c15d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.942+0000 7fcb9a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb9c0cdd50 0x7fcb9c15d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47810/0 (socket says 172.21.15.57:47810) 2023-11-28T12:04:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.942+0000 7fcb9a59c700 1 -- 172.21.15.57:0/3278347708 learned_addr learned my addr 172.21.15.57:0/3278347708 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.942+0000 7fcb9ad9d700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb9c06a7b0 0x7fcb9c1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.943+0000 7fcb9b59e700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb9c15a4d0 0x7fcb9c0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.943+0000 7fcb9a59c700 1 -- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb9c15a4d0 msgr2=0x7fcb9c0b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:31.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.943+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb9c15a4d0 0x7fcb9c0b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.943+0000 7fcb9a59c700 1 -- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb9c06a7b0 msgr2=0x7fcb9c1514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:31.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.943+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb9c06a7b0 0x7fcb9c1514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.943+0000 7fcb9a59c700 1 -- 172.21.15.57:0/3278347708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb90009550 con 0x7fcb9c0cdd50 2023-11-28T12:04:31.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.943+0000 7fcb9ad9d700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb9c06a7b0 0x7fcb9c1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:31.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.943+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb9c0cdd50 0x7fcb9c15d330 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009c10 tx=0x7fcb90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:31.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.943+0000 7fcb837fe700 1 -- 172.21.15.57:0/3278347708 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90004310 con 0x7fcb9c0cdd50 2023-11-28T12:04:31.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.943+0000 7fcb837fe700 1 -- 172.21.15.57:0/3278347708 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9001f070 con 0x7fcb9c0cdd50 2023-11-28T12:04:31.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.943+0000 7fcba1043700 1 -- 172.21.15.57:0/3278347708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb9c15e600 con 0x7fcb9c0cdd50 2023-11-28T12:04:31.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.943+0000 7fcb837fe700 1 -- 172.21.15.57:0/3278347708 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb900036a0 con 0x7fcb9c0cdd50 2023-11-28T12:04:31.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.943+0000 7fcba1043700 1 -- 172.21.15.57:0/3278347708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9c15e320 con 0x7fcb9c0cdd50 2023-11-28T12:04:31.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.945+0000 7fcb837fe700 1 -- 172.21.15.57:0/3278347708 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcb9002a020 con 0x7fcb9c0cdd50 2023-11-28T12:04:31.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.945+0000 7fcb837fe700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcb880682f0 0x7fcb8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.945+0000 7fcb837fe700 1 -- 172.21.15.57:0/3278347708 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb90087310 con 0x7fcb9c0cdd50 2023-11-28T12:04:31.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.946+0000 7fcb9ad9d700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcb880682f0 0x7fcb8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.946+0000 7fcb9ad9d700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcb880682f0 0x7fcb8806a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0b1fa0 tx=0x7fcb8c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:31.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.947+0000 7fcbad1e2b80 1 Processor -- start 2023-11-28T12:04:31.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.948+0000 7fcbad1e2b80 1 -- start start 2023-11-28T12:04:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.948+0000 7fcbad1e2b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 0x561df23fa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.948+0000 7fcbad1e2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24006e0 0x561df24294c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.948+0000 7fcbad1e2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df23ff630 0x561df2445a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.948+0000 7fcbad1e2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df237be30 con 0x561df24006e0 2023-11-28T12:04:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.948+0000 7fcbad1e2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df22e8190 con 0x561df23ff630 2023-11-28T12:04:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.948+0000 7fcbad1e2b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x561df2356e00 con 0x561df24290f0 2023-11-28T12:04:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb9a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24006e0 0x561df24294c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb9a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24006e0 0x561df24294c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47818/0 (socket says 172.21.15.57:47818) 2023-11-28T12:04:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb9a59c700 1 -- 172.21.15.57:0/1747530172 learned_addr learned my addr 172.21.15.57:0/1747530172 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb9b59e700 1 --2- 172.21.15.57:0/1747530172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df23ff630 0x561df2445a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb9ad9d700 1 --2- 172.21.15.57:0/1747530172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 0x561df23fa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb9a59c700 1 -- 172.21.15.57:0/1747530172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 msgr2=0x561df23fa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/1747530172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 0x561df23fa780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb9a59c700 1 -- 172.21.15.57:0/1747530172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df23ff630 msgr2=0x561df2445a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/1747530172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df23ff630 0x561df2445a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb9a59c700 1 -- 172.21.15.57:0/1747530172 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df21447e0 con 0x561df24006e0 2023-11-28T12:04:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/1747530172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24006e0 0x561df24294c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009fd0 tx=0x7fcb90057290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2c9c4684ddee70 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb80ff9700 1 -- 172.21.15.57:0/1747530172 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90056d20 con 0x561df24006e0 2023-11-28T12:04:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb80ff9700 1 -- 172.21.15.57:0/1747530172 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb90056e80 con 0x561df24006e0 2023-11-28T12:04:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.949+0000 7fcb80ff9700 1 -- 172.21.15.57:0/1747530172 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90056ff0 con 0x561df24006e0 2023-11-28T12:04:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.950+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/1747530172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24006e0 msgr2=0x561df24294c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.950+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/1747530172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24006e0 0x561df24294c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009fd0 tx=0x7fcb90057290 comp rx=0 tx=0).stop 2023-11-28T12:04:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.950+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/1747530172 shutdown_connections 2023-11-28T12:04:31.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.950+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/1747530172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df23ff630 0x561df2445a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.950+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/1747530172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 0x561df23fa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.950+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/1747530172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24006e0 0x561df24294c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.950+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/1747530172 >> 172.21.15.57:0/1747530172 conn(0x561df23f3d70 msgr2=0x561df23fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:31.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.950+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/1747530172 shutdown_connections 2023-11-28T12:04:31.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.950+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/1747530172 wait complete. 2023-11-28T12:04:31.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.951+0000 7fcbad1e2b80 1 -- start start 2023-11-28T12:04:31.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.951+0000 7fcbad1e2b80 1 Processor -- start 2023-11-28T12:04:31.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcbad1e2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 0x561df241bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcbad1e2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 0x561df241f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcbad1e2b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 0x561df241fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcbad1e2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df2350170 con 0x561df23ff630 2023-11-28T12:04:31.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcbad1e2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df21453e0 con 0x561df24006e0 2023-11-28T12:04:31.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcbad1e2b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x561df22e8390 con 0x561df24290f0 2023-11-28T12:04:31.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcbad1e2b80 10 client.0 _ll_register_callbacks cb 0x561df22ee450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:04:31.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb9ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 0x561df241bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb9ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 0x561df241bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47830/0 (socket says 172.21.15.57:47830) 2023-11-28T12:04:31.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb9ad9d700 1 -- 172.21.15.57:0/2466151305 learned_addr learned my addr 172.21.15.57:0/2466151305 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:31.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/2466151305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 0x561df241f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb9b59e700 1 --2- 172.21.15.57:0/2466151305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 0x561df241fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb9ad9d700 1 -- 172.21.15.57:0/2466151305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 msgr2=0x561df241fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:31.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb9ad9d700 1 --2- 172.21.15.57:0/2466151305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 0x561df241fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb9ad9d700 1 -- 172.21.15.57:0/2466151305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 msgr2=0x561df241f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:31.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb9ad9d700 1 --2- 172.21.15.57:0/2466151305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 0x561df241f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb9ad9d700 1 -- 172.21.15.57:0/2466151305 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9c15e320 con 0x561df23ff630 2023-11-28T12:04:31.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb9ad9d700 1 --2- 172.21.15.57:0/2466151305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 0x561df241bba0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00fef0 tx=0x7fcb8c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcef88d4b0455b90 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:31.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:04:31.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb79ffb700 1 -- 172.21.15.57:0/2466151305 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00a6a0 con 0x561df23ff630 2023-11-28T12:04:31.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcb79ffb700 1 -- 172.21.15.57:0/2466151305 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb8c014de0 con 0x561df23ff630 2023-11-28T12:04:31.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.952+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/2466151305 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561df2354080 con 0x561df23ff630 2023-11-28T12:04:31.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.953+0000 7fcb79ffb700 1 -- 172.21.15.57:0/2466151305 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb8c004020 con 0x561df23ff630 2023-11-28T12:04:31.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.953+0000 7fcbad1e2b80 10 client.5997 fetch_fsmap learned FSMap version 24 2023-11-28T12:04:31.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.953+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/2466151305 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x561df22310f0 con 0x561df23ff630 2023-11-28T12:04:31.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.953+0000 7fcb79ffb700 1 -- 172.21.15.57:0/2466151305 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561df2354080 con 0x561df23ff630 2023-11-28T12:04:31.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.953+0000 7fcb79ffb700 1 -- 172.21.15.57:0/2466151305 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fcb8c004180 con 0x561df23ff630 2023-11-28T12:04:31.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.953+0000 7fcbad1e2b80 10 client.5997 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T12:04:31.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.953+0000 7fcbad1e2b80 10 client.5997 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T12:04:31.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.953+0000 7fcbad1e2b80 20 client.5997 populate_metadata read hostname 'smithi057' 2023-11-28T12:04:31.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.953+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/2466151305 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x561df2446880 0x561df2446c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.953+0000 7fcbad1e2b80 4 client.5997 mds_command: new command op to 5739 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T12:04:31.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.953+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/2466151305 --> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561df219ff40 con 0x561df2446880 2023-11-28T12:04:31.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.953+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/2466151305 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x561df2446880 0x561df2446c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.954+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/2466151305 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x561df2446880 0x561df2446c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34d7ff374e09e60c server_cookie=f04a6726884e4d3e in_seq=0 out_seq=0 2023-11-28T12:04:31.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.954+0000 7fcb79ffb700 10 client.5997 ms_handle_connect on v2:172.21.15.181:6834/3437917192 2023-11-28T12:04:31.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.954+0000 7fcb79ffb700 1 -- 172.21.15.57:0/2466151305 <== mds.0 v2:172.21.15.181:6834/3437917192 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x561df219ff40 con 0x561df2446880 2023-11-28T12:04:31.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.954+0000 7fcb79ffb700 10 client.5997 handle_command_reply: tid=0 2023-11-28T12:04:31.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.954+0000 7fcbad1e2b80 1 client.5997 shutdown 2023-11-28T12:04:31.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.955+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/2466151305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 msgr2=0x561df241bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:31.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.955+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/2466151305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 0x561df241bba0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00fef0 tx=0x7fcb8c0047c0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.955+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/2466151305 shutdown_connections 2023-11-28T12:04:31.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.955+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/2466151305 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x561df2446880 0x561df2446c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.955+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/2466151305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 0x561df241fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.955+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/2466151305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 0x561df241bba0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.955+0000 7fcb79ffb700 0 client.5997 ms_handle_reset on v2:172.21.15.181:6834/3437917192 2023-11-28T12:04:31.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.955+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/2466151305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 0x561df241f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.955+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/2466151305 >> 172.21.15.57:0/2466151305 conn(0x561df23f3d70 msgr2=0x561df24294c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:31.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.955+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/2466151305 shutdown_connections 2023-11-28T12:04:31.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.955+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/2466151305 wait complete. 2023-11-28T12:04:31.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.955+0000 7fcbad1e2b80 20 client.5997 trim_cache size 0 max 16384 2023-11-28T12:04:31.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.972+0000 7fcbad1e2b80 1 Processor -- start 2023-11-28T12:04:31.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.972+0000 7fcbad1e2b80 1 -- start start 2023-11-28T12:04:31.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.973+0000 7fcbad1e2b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df23ff630 0x561df23fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.973+0000 7fcbad1e2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 0x561df24415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.973+0000 7fcbad1e2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24290f0 0x561df2441b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.973+0000 7fcbad1e2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df2426130 con 0x561df24290f0 2023-11-28T12:04:31.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.973+0000 7fcbad1e2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df241b340 con 0x561df24006e0 2023-11-28T12:04:31.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.973+0000 7fcbad1e2b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x561df23f3b00 con 0x561df23ff630 2023-11-28T12:04:31.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.973+0000 7fcb9b59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24290f0 0x561df2441b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.973+0000 7fcb9b59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24290f0 0x561df2441b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47836/0 (socket says 172.21.15.57:47836) 2023-11-28T12:04:31.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.973+0000 7fcb9b59e700 1 -- 172.21.15.57:0/229758003 learned_addr learned my addr 172.21.15.57:0/229758003 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:31.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.974+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/229758003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 0x561df24415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.974+0000 7fcb9ad9d700 1 --2- 172.21.15.57:0/229758003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df23ff630 0x561df23fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.974+0000 7fcb9b59e700 1 -- 172.21.15.57:0/229758003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df23ff630 msgr2=0x561df23fe7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:31.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.974+0000 7fcb9b59e700 1 --2- 172.21.15.57:0/229758003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df23ff630 0x561df23fe7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.974+0000 7fcb9b59e700 1 -- 172.21.15.57:0/229758003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 msgr2=0x561df24415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:31.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.974+0000 7fcb9b59e700 1 --2- 172.21.15.57:0/229758003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 0x561df24415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.974+0000 7fcb9b59e700 1 -- 172.21.15.57:0/229758003 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df2317330 con 0x561df24290f0 2023-11-28T12:04:31.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.974+0000 7fcb9b59e700 1 --2- 172.21.15.57:0/229758003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24290f0 0x561df2441b10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcb940012e0 tx=0x7fcb94007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cb48941ae9723 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:31.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.974+0000 7fcb817fa700 1 -- 172.21.15.57:0/229758003 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94007a90 con 0x561df24290f0 2023-11-28T12:04:31.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.974+0000 7fcb817fa700 1 -- 172.21.15.57:0/229758003 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb94016450 con 0x561df24290f0 2023-11-28T12:04:31.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.974+0000 7fcb817fa700 1 -- 172.21.15.57:0/229758003 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94016800 con 0x561df24290f0 2023-11-28T12:04:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.974+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/229758003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24290f0 msgr2=0x561df2441b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.974+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/229758003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24290f0 0x561df2441b10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcb940012e0 tx=0x7fcb94007330 comp rx=0 tx=0).stop 2023-11-28T12:04:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.975+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/229758003 shutdown_connections 2023-11-28T12:04:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.975+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/229758003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df24290f0 0x561df2441b10 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.975+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/229758003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df23ff630 0x561df23fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.975+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/229758003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 0x561df24415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.975+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/229758003 >> 172.21.15.57:0/229758003 conn(0x561df23f3d70 msgr2=0x561df241b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.975+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/229758003 shutdown_connections 2023-11-28T12:04:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.975+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/229758003 wait complete. 2023-11-28T12:04:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.975+0000 7fcbad1e2b80 1 -- start start 2023-11-28T12:04:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.975+0000 7fcbad1e2b80 1 Processor -- start 2023-11-28T12:04:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.976+0000 7fcbad1e2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 0x561df2444780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.976+0000 7fcbad1e2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 0x561df2496560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.976+0000 7fcbad1e2b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 0x561df242c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.976+0000 7fcbad1e2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df2496aa0 con 0x561df23ff630 2023-11-28T12:04:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.976+0000 7fcbad1e2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df24ae950 con 0x561df24006e0 2023-11-28T12:04:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.976+0000 7fcbad1e2b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x561df23f82f0 con 0x561df24290f0 2023-11-28T12:04:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.976+0000 7fcbad1e2b80 10 client.0 _ll_register_callbacks cb 0x561df22ee450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:04:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.976+0000 7fcb9ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 0x561df2444780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.976+0000 7fcb9ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 0x561df2444780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47840/0 (socket says 172.21.15.57:47840) 2023-11-28T12:04:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.976+0000 7fcb9ad9d700 1 -- 172.21.15.57:0/1228454851 learned_addr learned my addr 172.21.15.57:0/1228454851 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.976+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/1228454851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 0x561df2496560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.976+0000 7fcb9b59e700 1 --2- 172.21.15.57:0/1228454851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 0x561df242c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcb9ad9d700 1 -- 172.21.15.57:0/1228454851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 msgr2=0x561df242c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcb9ad9d700 1 --2- 172.21.15.57:0/1228454851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 0x561df242c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcb9ad9d700 1 -- 172.21.15.57:0/1228454851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 msgr2=0x561df2496560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcb9ad9d700 1 --2- 172.21.15.57:0/1228454851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 0x561df2496560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcb9ad9d700 1 -- 172.21.15.57:0/1228454851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9400f040 con 0x561df23ff630 2023-11-28T12:04:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcb9ad9d700 1 --2- 172.21.15.57:0/1228454851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 0x561df2444780 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c004620 tx=0x7fcb8c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0203799e5c7a8f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcb79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:04:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcb79ffb700 1 -- 172.21.15.57:0/1228454851 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c037070 con 0x561df23ff630 2023-11-28T12:04:31.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcb79ffb700 1 -- 172.21.15.57:0/1228454851 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb8c00a6a0 con 0x561df23ff630 2023-11-28T12:04:31.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/1228454851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb90056e80 con 0x561df23ff630 2023-11-28T12:04:31.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcb79ffb700 1 -- 172.21.15.57:0/1228454851 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb8c005a20 con 0x561df23ff630 2023-11-28T12:04:31.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcbad1e2b80 10 client.6003 fetch_fsmap learned FSMap version 24 2023-11-28T12:04:31.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/1228454851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x561df21a5220 con 0x561df23ff630 2023-11-28T12:04:31.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcb79ffb700 1 -- 172.21.15.57:0/1228454851 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb90056e80 con 0x561df23ff630 2023-11-28T12:04:31.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcb79ffb700 1 -- 172.21.15.57:0/1228454851 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fcb8c005b80 con 0x561df23ff630 2023-11-28T12:04:31.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcbad1e2b80 10 client.6003 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T12:04:31.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcbad1e2b80 10 client.6003 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T12:04:31.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcbad1e2b80 20 client.6003 populate_metadata read hostname 'smithi057' 2023-11-28T12:04:31.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/1228454851 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x561df242c880 0x561df24456c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcbad1e2b80 4 client.6003 mds_command: new command op to 5739 tid=0[{"prefix": "scrub status"}] 2023-11-28T12:04:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.977+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/1228454851 --> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561df21d5090 con 0x561df242c880 2023-11-28T12:04:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/1228454851 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x561df242c880 0x561df24456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcb9a59c700 1 --2- 172.21.15.57:0/1228454851 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x561df242c880 0x561df24456c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba0b2bc57ea3839d server_cookie=e267a081f91aa5f3 in_seq=0 out_seq=0 2023-11-28T12:04:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcb79ffb700 10 client.6003 ms_handle_connect on v2:172.21.15.181:6834/3437917192 2023-11-28T12:04:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcb79ffb700 1 -- 172.21.15.57:0/1228454851 <== mds.0 v2:172.21.15.181:6834/3437917192 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561df21d5090 con 0x561df242c880 2023-11-28T12:04:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcb79ffb700 10 client.6003 handle_command_reply: tid=0 2023-11-28T12:04:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcbad1e2b80 1 client.6003 shutdown 2023-11-28T12:04:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/1228454851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 msgr2=0x561df2444780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/1228454851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 0x561df2444780 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c004620 tx=0x7fcb8c014000 comp rx=0 tx=0).stop 2023-11-28T12:04:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/1228454851 shutdown_connections 2023-11-28T12:04:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/1228454851 >> [v2:172.21.15.181:6834/3437917192,v1:172.21.15.181:6836/3437917192] conn(0x561df242c880 0x561df24456c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/1228454851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x561df24290f0 0x561df242c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcb79ffb700 0 client.6003 ms_handle_reset on v2:172.21.15.181:6834/3437917192 2023-11-28T12:04:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/1228454851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561df23ff630 0x561df2444780 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcbad1e2b80 1 --2- 172.21.15.57:0/1228454851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561df24006e0 0x561df2496560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.978+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/1228454851 >> 172.21.15.57:0/1228454851 conn(0x561df23f3d70 msgr2=0x561df23fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/1228454851 shutdown_connections 2023-11-28T12:04:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcbad1e2b80 1 -- 172.21.15.57:0/1228454851 wait complete. 2023-11-28T12:04:31.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcbad1e2b80 20 client.6003 trim_cache size 0 max 16384 2023-11-28T12:04:31.986 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:04:31.987 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2023-11-28T12:04:31.987 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2023-11-28T12:04:31.987 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:04:31.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcba1043700 1 -- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcb880682f0 msgr2=0x7fcb8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:31.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcba1043700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcb880682f0 0x7fcb8806a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0b1fa0 tx=0x7fcb8c008040 comp rx=0 tx=0).stop 2023-11-28T12:04:31.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcba1043700 1 -- 172.21.15.57:0/3278347708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb9c0cdd50 msgr2=0x7fcb9c15d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:31.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcba1043700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb9c0cdd50 0x7fcb9c15d330 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009c10 tx=0x7fcb90004750 comp rx=0 tx=0).stop 2023-11-28T12:04:31.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcba1043700 1 -- 172.21.15.57:0/3278347708 shutdown_connections 2023-11-28T12:04:31.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcba1043700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcb880682f0 0x7fcb8806a7a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcba1043700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb9c15a4d0 0x7fcb9c0b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcba1043700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb9c06a7b0 0x7fcb9c1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcba1043700 1 --2- 172.21.15.57:0/3278347708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb9c0cdd50 0x7fcb9c15d330 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:31.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcba1043700 1 -- 172.21.15.57:0/3278347708 >> 172.21.15.57:0/3278347708 conn(0x7fcb9c06c010 msgr2=0x7fcb9c0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:31.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcba1043700 1 -- 172.21.15.57:0/3278347708 shutdown_connections 2023-11-28T12:04:31.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:31.979+0000 7fcba1043700 1 -- 172.21.15.57:0/3278347708 wait complete. 2023-11-28T12:04:31.993 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T12:04:32.094 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:04:32.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.232+0000 7effec81d700 1 Processor -- start 2023-11-28T12:04:32.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.232+0000 7effec81d700 1 -- start start 2023-11-28T12:04:32.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.233+0000 7effec81d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe80ce040 0x7effe815d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.233+0000 7effec81d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7effe80b1770 0x7effe80b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.233+0000 7effec81d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe813fc90 con 0x7effe8142db0 2023-11-28T12:04:32.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.233+0000 7effec81d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe80b2130 con 0x7effe80b1770 2023-11-28T12:04:32.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.233+0000 7effec81d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe813f480 con 0x7effe80ce040 2023-11-28T12:04:32.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.233+0000 7effe6d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7effe80b1770 0x7effe80b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.233+0000 7effe659c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe80ce040 0x7effe815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.233+0000 7effe659c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe80ce040 0x7effe815d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:48122/0 (socket says 172.21.15.57:48122) 2023-11-28T12:04:32.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.233+0000 7effe6d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7effe80b1770 0x7effe80b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:42240/0 (socket says 172.21.15.57:42240) 2023-11-28T12:04:32.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.233+0000 7effe5d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7effe8142db0 0x7effe813ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:39172/0 (socket says 172.21.15.57:39172) 2023-11-28T12:04:32.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.233+0000 7effe659c700 1 -- 172.21.15.57:0/2762614227 learned_addr learned my addr 172.21.15.57:0/2762614227 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:32.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.234+0000 7effe659c700 1 -- 172.21.15.57:0/2762614227 >> v2:172.21.15.181:3300/0 conn(0x7effe80b1770 msgr2=0x7effe80b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:32.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.234+0000 7effe659c700 1 --2- 172.21.15.57:0/2762614227 >> v2:172.21.15.181:3300/0 conn(0x7effe80b1770 0x7effe80b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.234+0000 7effe659c700 1 -- 172.21.15.57:0/2762614227 >> v1:172.21.15.181:6789/0 conn(0x7effe8142db0 legacy=0x7effe813ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:32.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.234+0000 7effe659c700 1 -- 172.21.15.57:0/2762614227 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe813f6d0 con 0x7effe80ce040 2023-11-28T12:04:32.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.234+0000 7effe659c700 1 --2- 172.21.15.57:0/2762614227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe80ce040 0x7effe815d900 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7effd80098f0 tx=0x7effd8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2d604fc566dbd2f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:32.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.234+0000 7effe559a700 1 -- 172.21.15.57:0/2762614227 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8011830 con 0x7effe80ce040 2023-11-28T12:04:32.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.234+0000 7effe559a700 1 -- 172.21.15.57:0/2762614227 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd8011990 con 0x7effe80ce040 2023-11-28T12:04:32.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.234+0000 7effe559a700 1 -- 172.21.15.57:0/2762614227 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8011d20 con 0x7effe80ce040 2023-11-28T12:04:32.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.234+0000 7effec81d700 1 -- 172.21.15.57:0/2762614227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe80ce040 msgr2=0x7effe815d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:32.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.234+0000 7effec81d700 1 --2- 172.21.15.57:0/2762614227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe80ce040 0x7effe815d900 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7effd80098f0 tx=0x7effd8009c00 comp rx=0 tx=0).stop 2023-11-28T12:04:32.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.235+0000 7effec81d700 1 -- 172.21.15.57:0/2762614227 shutdown_connections 2023-11-28T12:04:32.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.235+0000 7effec81d700 1 --2- 172.21.15.57:0/2762614227 >> v2:172.21.15.181:3300/0 conn(0x7effe80b1770 0x7effe80b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.235+0000 7effec81d700 1 --2- 172.21.15.57:0/2762614227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe80ce040 0x7effe815d900 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.235+0000 7effec81d700 1 -- 172.21.15.57:0/2762614227 >> 172.21.15.57:0/2762614227 conn(0x7effe806c010 msgr2=0x7effe80cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.235+0000 7effec81d700 1 -- 172.21.15.57:0/2762614227 shutdown_connections 2023-11-28T12:04:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.235+0000 7effec81d700 1 -- 172.21.15.57:0/2762614227 wait complete. 2023-11-28T12:04:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.236+0000 7effec81d700 1 Processor -- start 2023-11-28T12:04:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.236+0000 7effec81d700 1 -- start start 2023-11-28T12:04:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effec81d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe80b1770 0x7effe80c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effec81d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe80ce040 0x7effe81505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effec81d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe8142db0 0x7effe80713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effec81d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe80b4a80 con 0x7effe8142db0 2023-11-28T12:04:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effec81d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe80c53d0 con 0x7effe80ce040 2023-11-28T12:04:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effec81d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe81e3c40 con 0x7effe80b1770 2023-11-28T12:04:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effe6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe8142db0 0x7effe80713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effe6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe8142db0 0x7effe80713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47854/0 (socket says 172.21.15.57:47854) 2023-11-28T12:04:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effe6d9d700 1 -- 172.21.15.57:0/6941769 learned_addr learned my addr 172.21.15.57:0/6941769 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effe659c700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe80b1770 0x7effe80c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effe5d9b700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe80ce040 0x7effe81505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effe659c700 1 -- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe80ce040 msgr2=0x7effe81505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effe659c700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe80ce040 0x7effe81505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effe659c700 1 -- 172.21.15.57:0/6941769 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe8142db0 msgr2=0x7effe80713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effe659c700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe8142db0 0x7effe80713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.237+0000 7effe659c700 1 -- 172.21.15.57:0/6941769 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd8009550 con 0x7effe80b1770 2023-11-28T12:04:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.238+0000 7effe659c700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe80b1770 0x7effe80c4e90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7effe806c6c0 tx=0x7effd8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.238+0000 7effd6ffd700 1 -- 172.21.15.57:0/6941769 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8004400 con 0x7effe80b1770 2023-11-28T12:04:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.238+0000 7effd6ffd700 1 -- 172.21.15.57:0/6941769 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd801f070 con 0x7effe80b1770 2023-11-28T12:04:32.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.238+0000 7effec81d700 1 -- 172.21.15.57:0/6941769 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe815d100 con 0x7effe80b1770 2023-11-28T12:04:32.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.238+0000 7effd6ffd700 1 -- 172.21.15.57:0/6941769 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd80038e0 con 0x7effe80b1770 2023-11-28T12:04:32.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.238+0000 7effec81d700 1 -- 172.21.15.57:0/6941769 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe813f9b0 con 0x7effe80b1770 2023-11-28T12:04:32.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.239+0000 7effec81d700 1 -- 172.21.15.57:0/6941769 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe815d3e0 con 0x7effe80b1770 2023-11-28T12:04:32.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.240+0000 7effd6ffd700 1 -- 172.21.15.57:0/6941769 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7effd802a020 con 0x7effe80b1770 2023-11-28T12:04:32.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.240+0000 7effd6ffd700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7effc80682f0 0x7effc806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.240+0000 7effd6ffd700 1 -- 172.21.15.57:0/6941769 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effd80872d0 con 0x7effe80b1770 2023-11-28T12:04:32.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.240+0000 7effe5d9b700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7effc80682f0 0x7effc806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.241+0000 7effe5d9b700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7effc80682f0 0x7effc806a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7effe813ea40 tx=0x7effdc007ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:32.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.245+0000 7effd6ffd700 1 -- 172.21.15.57:0/6941769 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7effd8053ad0 con 0x7effe80b1770 2023-11-28T12:04:32.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.493+0000 7effec81d700 1 -- 172.21.15.57:0/6941769 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effe80743f0 con 0x7effe80b1770 2023-11-28T12:04:32.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.494+0000 7effd6ffd700 1 -- 172.21.15.57:0/6941769 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7effd80570f0 con 0x7effe80b1770 2023-11-28T12:04:32.496 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:32.496 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1773283092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1773283092},{"type":"v1","addr":"172.21.15.181:6837","nonce":1773283092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/3718600227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3718600227},{"type":"v1","addr":"172.21.15.57:6834","nonce":3718600227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/762261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":762261630},{"type":"v1","addr":"172.21.15.57:6835","nonce":762261630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:03:47.263787+0000","modified":"2023-11-28T12:03:50.285215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3437917192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3437917192},{"type":"v1","addr":"172.21.15.181:6836","nonce":3437917192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T12:04:32.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.497+0000 7effec81d700 1 -- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7effc80682f0 msgr2=0x7effc806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:32.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.497+0000 7effec81d700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7effc80682f0 0x7effc806a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7effe813ea40 tx=0x7effdc007ed0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.497+0000 7effec81d700 1 -- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe80b1770 msgr2=0x7effe80c4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:32.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.497+0000 7effec81d700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe80b1770 0x7effe80c4e90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7effe806c6c0 tx=0x7effd8004750 comp rx=0 tx=0).stop 2023-11-28T12:04:32.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.497+0000 7effec81d700 1 -- 172.21.15.57:0/6941769 shutdown_connections 2023-11-28T12:04:32.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.497+0000 7effec81d700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7effc80682f0 0x7effc806a7a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.497+0000 7effec81d700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effe8142db0 0x7effe80713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.497+0000 7effec81d700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effe80b1770 0x7effe80c4e90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.497+0000 7effec81d700 1 --2- 172.21.15.57:0/6941769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe80ce040 0x7effe81505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.497+0000 7effec81d700 1 -- 172.21.15.57:0/6941769 >> 172.21.15.57:0/6941769 conn(0x7effe806c010 msgr2=0x7effe806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:32.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.497+0000 7effec81d700 1 -- 172.21.15.57:0/6941769 shutdown_connections 2023-11-28T12:04:32.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.497+0000 7effec81d700 1 -- 172.21.15.57:0/6941769 wait complete. 2023-11-28T12:04:32.499 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2023-11-28T12:04:32.511 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2023-11-28T12:04:32.670 INFO:teuthology.orchestra.run.smithi181.stdout:{"message":"","return_code":0} 2023-11-28T12:04:32.670 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T12:04:32.671 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T12:04:32.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.723+0000 7f6cf7bf70c0 1 Processor -- start 2023-11-28T12:04:32.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.723+0000 7f6cf7bf70c0 1 -- start start 2023-11-28T12:04:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6cf7bf70c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5587584a52f0 0x5587584a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6cf7bf70c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5587584ef310 0x5587584d54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6cf7bf70c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587585784a0 0x5587584d5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6cf7bf70c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875844e000 con 0x5587584ef310 2023-11-28T12:04:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6cf7bf70c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587583e0fe0 con 0x5587585784a0 2023-11-28T12:04:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6cf7bf70c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x558758575590 con 0x5587584a52f0 2023-11-28T12:04:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6ced4e2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5587584ef310 0x5587584d54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6ced4e2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5587584ef310 0x5587584d54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47860/0 (socket says 172.21.15.57:47860) 2023-11-28T12:04:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6ced4e2700 1 -- 172.21.15.57:0/3230585613 learned_addr learned my addr 172.21.15.57:0/3230585613 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6cee4e4700 1 --2- 172.21.15.57:0/3230585613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587585784a0 0x5587584d5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6cedce3700 1 --2- 172.21.15.57:0/3230585613 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5587584a52f0 0x5587584a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6ced4e2700 1 -- 172.21.15.57:0/3230585613 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5587584a52f0 msgr2=0x5587584a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6ced4e2700 1 --2- 172.21.15.57:0/3230585613 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5587584a52f0 0x5587584a7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6ced4e2700 1 -- 172.21.15.57:0/3230585613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587585784a0 msgr2=0x5587584d5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6ced4e2700 1 --2- 172.21.15.57:0/3230585613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587585784a0 0x5587584d5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6ced4e2700 1 -- 172.21.15.57:0/3230585613 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd8005d20 con 0x5587584ef310 2023-11-28T12:04:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.724+0000 7f6ced4e2700 1 --2- 172.21.15.57:0/3230585613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5587584ef310 0x5587584d54f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8000c00 tx=0x7f6cd800d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732392387c1cec2e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.725+0000 7f6ce7fff700 1 -- 172.21.15.57:0/3230585613 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8013870 con 0x5587584ef310 2023-11-28T12:04:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.725+0000 7f6ce7fff700 1 -- 172.21.15.57:0/3230585613 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd80139f0 con 0x5587584ef310 2023-11-28T12:04:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.725+0000 7f6ce7fff700 1 -- 172.21.15.57:0/3230585613 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8013da0 con 0x5587584ef310 2023-11-28T12:04:32.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.725+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3230585613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5587584ef310 msgr2=0x5587584d54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.725+0000 7f6cf7bf70c0 1 --2- 172.21.15.57:0/3230585613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5587584ef310 0x5587584d54f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8000c00 tx=0x7f6cd800d000 comp rx=0 tx=0).stop 2023-11-28T12:04:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.725+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3230585613 shutdown_connections 2023-11-28T12:04:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.725+0000 7f6cf7bf70c0 1 --2- 172.21.15.57:0/3230585613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587585784a0 0x5587584d5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.725+0000 7f6cf7bf70c0 1 --2- 172.21.15.57:0/3230585613 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5587584a52f0 0x5587584a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.725+0000 7f6cf7bf70c0 1 --2- 172.21.15.57:0/3230585613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5587584ef310 0x5587584d54f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.725+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3230585613 >> 172.21.15.57:0/3230585613 conn(0x5587584ee070 msgr2=0x5587584ee440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.725+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3230585613 shutdown_connections 2023-11-28T12:04:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.725+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3230585613 wait complete. 2023-11-28T12:04:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.726+0000 7f6cf7bf70c0 1 Processor -- start 2023-11-28T12:04:32.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.726+0000 7f6cf7bf70c0 1 -- start start 2023-11-28T12:04:32.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cf7bf70c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587584a52f0 0x55875856a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cf7bf70c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5587584ef310 0x5587584d6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cf7bf70c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5587585784a0 0x5587584d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cf7bf70c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875856a9f0 con 0x5587585784a0 2023-11-28T12:04:32.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cf7bf70c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558758575040 con 0x5587584a52f0 2023-11-28T12:04:32.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cf7bf70c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x558758580cc0 con 0x5587584ef310 2023-11-28T12:04:32.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cee4e4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5587585784a0 0x5587584d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cee4e4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5587585784a0 0x5587584d6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47864/0 (socket says 172.21.15.57:47864) 2023-11-28T12:04:32.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cee4e4700 1 -- 172.21.15.57:0/3030162597 learned_addr learned my addr 172.21.15.57:0/3030162597 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:32.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cedce3700 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587584a52f0 0x55875856a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6ced4e2700 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5587584ef310 0x5587584d6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cee4e4700 1 -- 172.21.15.57:0/3030162597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5587584ef310 msgr2=0x5587584d6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:32.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cee4e4700 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5587584ef310 0x5587584d6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cee4e4700 1 -- 172.21.15.57:0/3030162597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587584a52f0 msgr2=0x55875856a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:32.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cee4e4700 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587584a52f0 0x55875856a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cee4e4700 1 -- 172.21.15.57:0/3030162597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587583f9120 con 0x5587585784a0 2023-11-28T12:04:32.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.727+0000 7f6cee4e4700 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5587585784a0 0x5587584d6a80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000e4b0 tx=0x7f6ce000e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:32.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.728+0000 7f6ce6ffd700 1 -- 172.21.15.57:0/3030162597 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce000eed0 con 0x5587585784a0 2023-11-28T12:04:32.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.728+0000 7f6ce6ffd700 1 -- 172.21.15.57:0/3030162597 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce0011350 con 0x5587585784a0 2023-11-28T12:04:32.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.728+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3030162597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5587583ec830 con 0x5587585784a0 2023-11-28T12:04:32.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.728+0000 7f6ce6ffd700 1 -- 172.21.15.57:0/3030162597 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0011700 con 0x5587585784a0 2023-11-28T12:04:32.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.728+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3030162597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587583f9360 con 0x5587585784a0 2023-11-28T12:04:32.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.729+0000 7f6ce6ffd700 1 -- 172.21.15.57:0/3030162597 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6ce001a030 con 0x5587585784a0 2023-11-28T12:04:32.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.730+0000 7f6ce6ffd700 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6cd0066c10 0x7f6cd00690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.730+0000 7f6ce6ffd700 1 -- 172.21.15.57:0/3030162597 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6ce0083ba0 con 0x5587585784a0 2023-11-28T12:04:32.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.730+0000 7f6cedce3700 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6cd0066c10 0x7f6cd00690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.730+0000 7f6cf7bf70c0 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x5587585764b0 0x5587584d1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:32.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.730+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3030162597 --> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49) v8 -- 0x5587584d1540 con 0x5587585764b0 2023-11-28T12:04:32.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.730+0000 7f6ced4e2700 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x5587585764b0 0x5587584d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:32.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.730+0000 7f6cedce3700 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6cd0066c10 0x7f6cd00690c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x5587584d52e0 tx=0x7f6ce8001720 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:32.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.730+0000 7f6ced4e2700 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x5587585764b0 0x5587584d1000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:32.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.731+0000 7f6ced4e2700 1 -- 172.21.15.57:0/3030162597 <== osd.1 v2:172.21.15.57:6808/120755 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f6cd801e440 con 0x5587585764b0 2023-11-28T12:04:32.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.731+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3030162597 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x5587585764b0 msgr2=0x5587584d1000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:32.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.731+0000 7f6cf7bf70c0 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x5587585764b0 0x5587584d1000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.731+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3030162597 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6cd0066c10 msgr2=0x7f6cd00690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:32.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.731+0000 7f6cf7bf70c0 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6cd0066c10 0x7f6cd00690c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x5587584d52e0 tx=0x7f6ce8001720 comp rx=0 tx=0).stop 2023-11-28T12:04:32.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.731+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3030162597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5587585784a0 msgr2=0x5587584d6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:32.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.731+0000 7f6cf7bf70c0 1 --2- 172.21.15.57:0/3030162597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5587585784a0 0x5587584d6a80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000e4b0 tx=0x7f6ce000e7c0 comp rx=0 tx=0).stop 2023-11-28T12:04:32.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.731+0000 7f6cee4e4700 1 -- 172.21.15.57:0/3030162597 reap_dead start 2023-11-28T12:04:32.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.731+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3030162597 shutdown_connections 2023-11-28T12:04:32.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.731+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3030162597 >> 172.21.15.57:0/3030162597 conn(0x5587584ee070 msgr2=0x5587585774c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:32.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.731+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3030162597 shutdown_connections 2023-11-28T12:04:32.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:32.731+0000 7f6cf7bf70c0 1 -- 172.21.15.57:0/3030162597 wait complete. 2023-11-28T12:04:32.738 INFO:teuthology.orchestra.run.smithi057.stdout:Yalphaparent_a 2023-11-28T12:04:32.738 DEBUG:teuthology.orchestra.run.smithi057:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T12:04:32.802 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:04:32.802 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511627777, 2023-11-28T12:04:32.802 INFO:teuthology.orchestra.run.smithi057.stdout: "ancestors": [ 2023-11-28T12:04:32.802 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:32.802 INFO:teuthology.orchestra.run.smithi057.stdout: "dirino": 1099511627776, 2023-11-28T12:04:32.803 INFO:teuthology.orchestra.run.smithi057.stdout: "dname": "alpha", 2023-11-28T12:04:32.803 INFO:teuthology.orchestra.run.smithi057.stdout: "version": 4 2023-11-28T12:04:32.803 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:32.803 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:32.803 INFO:teuthology.orchestra.run.smithi057.stdout: "dirino": 1, 2023-11-28T12:04:32.803 INFO:teuthology.orchestra.run.smithi057.stdout: "dname": "parent_a", 2023-11-28T12:04:32.803 INFO:teuthology.orchestra.run.smithi057.stdout: "version": 7 2023-11-28T12:04:32.803 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:32.803 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-11-28T12:04:32.803 INFO:teuthology.orchestra.run.smithi057.stdout: "pool": 8, 2023-11-28T12:04:32.803 INFO:teuthology.orchestra.run.smithi057.stdout: "old_pools": [] 2023-11-28T12:04:32.803 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:04:32.804 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:32.805 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T12:04:32.805 DEBUG:teuthology.orchestra.run.smithi057:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T12:04:32.897 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T12:04:32.942 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T12:04:32.942 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:04:32.942 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:04:32.966 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:04:33.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.147+0000 7f2186aed700 1 Processor -- start 2023-11-28T12:04:33.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.148+0000 7f2186aed700 1 -- start start 2023-11-28T12:04:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.148+0000 7f2186aed700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2180160e70 0x7f21800b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.148+0000 7f2186aed700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218006e900 con 0x7f21800d0c60 2023-11-28T12:04:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.148+0000 7f2186aed700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2180160390 con 0x7f218006e400 2023-11-28T12:04:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.148+0000 7f2186aed700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2180160670 con 0x7f2180160e70 2023-11-28T12:04:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.149+0000 7f2184889700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f21800d0c60 0x7f21800d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:51398/0 (socket says 172.21.15.57:51398) 2023-11-28T12:04:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.149+0000 7f2184889700 1 -- 172.21.15.57:0/390155776 learned_addr learned my addr 172.21.15.57:0/390155776 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.149+0000 7f218508a700 1 --2- 172.21.15.57:0/390155776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2180160e70 0x7f21800b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.149+0000 7f217f7fe700 1 -- 172.21.15.57:0/390155776 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813321146 0 0) 0x7f218006e900 con 0x7f21800d0c60 2023-11-28T12:04:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.149+0000 7f217f7fe700 1 -- 172.21.15.57:0/390155776 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21700035b0 con 0x7f21800d0c60 2023-11-28T12:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.149+0000 7f217f7fe700 1 -- 172.21.15.57:0/390155776 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390098835 0 0) 0x7f2180160390 con 0x7f218006e400 2023-11-28T12:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.149+0000 7f217f7fe700 1 -- 172.21.15.57:0/390155776 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f218006e900 con 0x7f218006e400 2023-11-28T12:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.149+0000 7f218508a700 1 -- 172.21.15.57:0/390155776 >> v1:172.21.15.181:6789/0 conn(0x7f218006e400 legacy=0x7f218015c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.149+0000 7f218508a700 1 -- 172.21.15.57:0/390155776 >> v1:172.21.15.57:6789/0 conn(0x7f21800d0c60 legacy=0x7f21800d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.149+0000 7f218508a700 1 -- 172.21.15.57:0/390155776 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2180160950 con 0x7f2180160e70 2023-11-28T12:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.149+0000 7f218508a700 1 --2- 172.21.15.57:0/390155776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2180160e70 0x7f21800b5be0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2178009dc0 tx=0x7f217800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3360c826e0668846 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.150+0000 7f217f7fe700 1 -- 172.21.15.57:0/390155776 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217800e070 con 0x7f2180160e70 2023-11-28T12:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.150+0000 7f217f7fe700 1 -- 172.21.15.57:0/390155776 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217800bb20 con 0x7f2180160e70 2023-11-28T12:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.150+0000 7f217f7fe700 1 -- 172.21.15.57:0/390155776 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217800be90 con 0x7f2180160e70 2023-11-28T12:04:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.150+0000 7f2186aed700 1 -- 172.21.15.57:0/390155776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2180160e70 msgr2=0x7f21800b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.150+0000 7f2186aed700 1 --2- 172.21.15.57:0/390155776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2180160e70 0x7f21800b5be0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2178009dc0 tx=0x7f217800b330 comp rx=0 tx=0).stop 2023-11-28T12:04:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.150+0000 7f2186aed700 1 -- 172.21.15.57:0/390155776 shutdown_connections 2023-11-28T12:04:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.150+0000 7f2186aed700 1 --2- 172.21.15.57:0/390155776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2180160e70 0x7f21800b5be0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.150+0000 7f2186aed700 1 -- 172.21.15.57:0/390155776 >> 172.21.15.57:0/390155776 conn(0x7f218006a270 msgr2=0x7f218006a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.150+0000 7f2186aed700 1 -- 172.21.15.57:0/390155776 shutdown_connections 2023-11-28T12:04:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.150+0000 7f2186aed700 1 -- 172.21.15.57:0/390155776 wait complete. 2023-11-28T12:04:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.151+0000 7f2186aed700 1 Processor -- start 2023-11-28T12:04:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.152+0000 7f2186aed700 1 -- start start 2023-11-28T12:04:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.152+0000 7f2186aed700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f218006e400 0x7f2180157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.152+0000 7f2186aed700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21800d0c60 0x7f2180151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.152+0000 7f2186aed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2180160e70 0x7f21800b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.152+0000 7f2186aed700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180158610 con 0x7f21800d0c60 2023-11-28T12:04:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.152+0000 7f2186aed700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21801529f0 con 0x7f2180160e70 2023-11-28T12:04:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.152+0000 7f2186aed700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21801583e0 con 0x7f218006e400 2023-11-28T12:04:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f217ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21800d0c60 0x7f2180151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f217ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21800d0c60 0x7f2180151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47874/0 (socket says 172.21.15.57:47874) 2023-11-28T12:04:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f217ffff700 1 -- 172.21.15.57:0/2689205095 learned_addr learned my addr 172.21.15.57:0/2689205095 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f2184889700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f218006e400 0x7f2180157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f218508a700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2180160e70 0x7f21800b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f217ffff700 1 -- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f218006e400 msgr2=0x7f2180157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f217ffff700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f218006e400 0x7f2180157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f217ffff700 1 -- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2180160e70 msgr2=0x7f21800b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f217ffff700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2180160e70 0x7f21800b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f217ffff700 1 -- 172.21.15.57:0/2689205095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2178009a70 con 0x7f21800d0c60 2023-11-28T12:04:33.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f217ffff700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21800d0c60 0x7f2180151950 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f216c00c4c0 tx=0x7f216c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:33.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f217d7fa700 1 -- 172.21.15.57:0/2689205095 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c0052d0 con 0x7f21800d0c60 2023-11-28T12:04:33.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f217d7fa700 1 -- 172.21.15.57:0/2689205095 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c00ce90 con 0x7f21800d0c60 2023-11-28T12:04:33.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f217d7fa700 1 -- 172.21.15.57:0/2689205095 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c003680 con 0x7f21800d0c60 2023-11-28T12:04:33.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f2186aed700 1 -- 172.21.15.57:0/2689205095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218015d080 con 0x7f21800d0c60 2023-11-28T12:04:33.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.153+0000 7f2186aed700 1 -- 172.21.15.57:0/2689205095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21800d0740 con 0x7f21800d0c60 2023-11-28T12:04:33.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.154+0000 7f2186aed700 1 -- 172.21.15.57:0/2689205095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218015d210 con 0x7f21800d0c60 2023-11-28T12:04:33.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.155+0000 7f217d7fa700 1 -- 172.21.15.57:0/2689205095 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f216c019030 con 0x7f21800d0c60 2023-11-28T12:04:33.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.155+0000 7f217d7fa700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f21640682f0 0x7f216406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.156+0000 7f217d7fa700 1 -- 172.21.15.57:0/2689205095 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f216c084ef0 con 0x7f21800d0c60 2023-11-28T12:04:33.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.156+0000 7f2184889700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f21640682f0 0x7f216406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.156+0000 7f2184889700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f21640682f0 0x7f216406a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2174007fd0 tx=0x7f2174007dc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:33.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.160+0000 7f217d7fa700 1 -- 172.21.15.57:0/2689205095 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f216c051750 con 0x7f21800d0c60 2023-11-28T12:04:33.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.363+0000 7f2186aed700 1 -- 172.21.15.57:0/2689205095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21800743f0 con 0x7f21800d0c60 2023-11-28T12:04:33.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.364+0000 7f217d7fa700 1 -- 172.21.15.57:0/2689205095 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+17921 (secure 0 0 0) 0x7f216c017090 con 0x7f21800d0c60 2023-11-28T12:04:33.365 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:33.366 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":49,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:03:48.265874+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:03:45.520415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T12:03:46.232534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:04:33.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.366+0000 7f2186aed700 1 -- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f21640682f0 msgr2=0x7f216406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.366+0000 7f2186aed700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f21640682f0 0x7f216406a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2174007fd0 tx=0x7f2174007dc0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.366+0000 7f2186aed700 1 -- 172.21.15.57:0/2689205095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21800d0c60 msgr2=0x7f2180151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.366+0000 7f2186aed700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21800d0c60 0x7f2180151950 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f216c00c4c0 tx=0x7f216c00c7d0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.367+0000 7f2186aed700 1 -- 172.21.15.57:0/2689205095 shutdown_connections 2023-11-28T12:04:33.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.367+0000 7f2186aed700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f21640682f0 0x7f216406a7a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.367+0000 7f2186aed700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2180160e70 0x7f21800b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.367+0000 7f2186aed700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f218006e400 0x7f2180157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.367+0000 7f2186aed700 1 --2- 172.21.15.57:0/2689205095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21800d0c60 0x7f2180151950 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.367+0000 7f2186aed700 1 -- 172.21.15.57:0/2689205095 >> 172.21.15.57:0/2689205095 conn(0x7f218006a270 msgr2=0x7f218015d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:33.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.367+0000 7f2186aed700 1 -- 172.21.15.57:0/2689205095 shutdown_connections 2023-11-28T12:04:33.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.367+0000 7f2186aed700 1 -- 172.21.15.57:0/2689205095 wait complete. 2023-11-28T12:04:33.381 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:03:45.520415+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:03:46.232534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:04:33.382 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:04:33.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.519+0000 7f07d28ea700 1 Processor -- start 2023-11-28T12:04:33.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.519+0000 7f07d28ea700 1 -- start start 2023-11-28T12:04:33.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07d28ea700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07cc15c810 0x7f07cc160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07d28ea700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f07cc0718e0 0x7f07cc06c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07d28ea700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07cc06cdc0 0x7f07cc06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07d28ea700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07cc15cbe0 con 0x7f07cc0718e0 2023-11-28T12:04:33.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07d28ea700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07cc1607a0 con 0x7f07cc15c810 2023-11-28T12:04:33.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07d28ea700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc160a50 con 0x7f07cc06cdc0 2023-11-28T12:04:33.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07cb7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f07cc0718e0 0x7f07cc06c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07cb7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f07cc0718e0 0x7f07cc06c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47876/0 (socket says 172.21.15.57:47876) 2023-11-28T12:04:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07cb7fe700 1 -- 172.21.15.57:0/3893674760 learned_addr learned my addr 172.21.15.57:0/3893674760 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07cbfff700 1 --2- 172.21.15.57:0/3893674760 >> v2:172.21.15.181:3300/0 conn(0x7f07cc15c810 0x7f07cc160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07d0e87700 1 --2- 172.21.15.57:0/3893674760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07cc06cdc0 0x7f07cc06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07cb7fe700 1 -- 172.21.15.57:0/3893674760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07cc06cdc0 msgr2=0x7f07cc06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07cb7fe700 1 --2- 172.21.15.57:0/3893674760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07cc06cdc0 0x7f07cc06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07cb7fe700 1 -- 172.21.15.57:0/3893674760 >> v2:172.21.15.181:3300/0 conn(0x7f07cc15c810 msgr2=0x7f07cc160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07cb7fe700 1 --2- 172.21.15.57:0/3893674760 >> v2:172.21.15.181:3300/0 conn(0x7f07cc15c810 0x7f07cc160260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07cb7fe700 1 -- 172.21.15.57:0/3893674760 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc160d30 con 0x7f07cc0718e0 2023-11-28T12:04:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07cb7fe700 1 --2- 172.21.15.57:0/3893674760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07cc0718e0 0x7f07cc06c880 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0b27c0 tx=0x7f07c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4e9b77fe49b320 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:33.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.520+0000 7f07caffd700 1 -- 172.21.15.57:0/3893674760 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0011830 con 0x7f07cc0718e0 2023-11-28T12:04:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.521+0000 7f07caffd700 1 -- 172.21.15.57:0/3893674760 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c0011990 con 0x7f07cc0718e0 2023-11-28T12:04:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.521+0000 7f07caffd700 1 -- 172.21.15.57:0/3893674760 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0011d20 con 0x7f07cc0718e0 2023-11-28T12:04:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.521+0000 7f07d28ea700 1 -- 172.21.15.57:0/3893674760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07cc0718e0 msgr2=0x7f07cc06c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.521+0000 7f07d28ea700 1 --2- 172.21.15.57:0/3893674760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07cc0718e0 0x7f07cc06c880 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0b27c0 tx=0x7f07c0009bd0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.521+0000 7f07d28ea700 1 -- 172.21.15.57:0/3893674760 shutdown_connections 2023-11-28T12:04:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.521+0000 7f07d28ea700 1 --2- 172.21.15.57:0/3893674760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07cc06cdc0 0x7f07cc06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.521+0000 7f07d28ea700 1 --2- 172.21.15.57:0/3893674760 >> v2:172.21.15.181:3300/0 conn(0x7f07cc15c810 0x7f07cc160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.521+0000 7f07d28ea700 1 --2- 172.21.15.57:0/3893674760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07cc0718e0 0x7f07cc06c880 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.521+0000 7f07d28ea700 1 -- 172.21.15.57:0/3893674760 >> 172.21.15.57:0/3893674760 conn(0x7f07cc06c010 msgr2=0x7f07cc0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.521+0000 7f07d28ea700 1 -- 172.21.15.57:0/3893674760 shutdown_connections 2023-11-28T12:04:33.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.521+0000 7f07d28ea700 1 -- 172.21.15.57:0/3893674760 wait complete. 2023-11-28T12:04:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.523+0000 7f07d28ea700 1 Processor -- start 2023-11-28T12:04:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.523+0000 7f07d28ea700 1 -- start start 2023-11-28T12:04:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.524+0000 7f07d28ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07cc06cdc0 0x7f07cc0b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.524+0000 7f07d28ea700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07cc0718e0 0x7f07cc1f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.524+0000 7f07d28ea700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07cc15c810 0x7f07cc1f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.524+0000 7f07d28ea700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc148d00 con 0x7f07cc15c810 2023-11-28T12:04:33.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.524+0000 7f07d28ea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc1492c0 con 0x7f07cc06cdc0 2023-11-28T12:04:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.524+0000 7f07d28ea700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc15bbd0 con 0x7f07cc0718e0 2023-11-28T12:04:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.524+0000 7f07d0e87700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07cc15c810 0x7f07cc1f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.524+0000 7f07d0e87700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07cc15c810 0x7f07cc1f59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47890/0 (socket says 172.21.15.57:47890) 2023-11-28T12:04:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.524+0000 7f07d0e87700 1 -- 172.21.15.57:0/517135817 learned_addr learned my addr 172.21.15.57:0/517135817 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.524+0000 7f07cb7fe700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07cc0718e0 0x7f07cc1f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.524+0000 7f07cbfff700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07cc06cdc0 0x7f07cc0b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.525+0000 7f07d0e87700 1 -- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07cc0718e0 msgr2=0x7f07cc1f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.525+0000 7f07d0e87700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07cc0718e0 0x7f07cc1f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.525+0000 7f07d0e87700 1 -- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07cc06cdc0 msgr2=0x7f07cc0b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.525+0000 7f07d0e87700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07cc06cdc0 0x7f07cc0b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.525+0000 7f07d0e87700 1 -- 172.21.15.57:0/517135817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c0009550 con 0x7f07cc15c810 2023-11-28T12:04:33.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.525+0000 7f07cbfff700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07cc06cdc0 0x7f07cc0b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.525+0000 7f07d0e87700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07cc15c810 0x7f07cc1f59a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f07c400ef90 tx=0x7f07c4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.525+0000 7f07c8ff9700 1 -- 172.21.15.57:0/517135817 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c4009b20 con 0x7f07cc15c810 2023-11-28T12:04:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.525+0000 7f07c8ff9700 1 -- 172.21.15.57:0/517135817 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c400f040 con 0x7f07cc15c810 2023-11-28T12:04:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.525+0000 7f07d28ea700 1 -- 172.21.15.57:0/517135817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07cc0ceb20 con 0x7f07cc15c810 2023-11-28T12:04:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.525+0000 7f07c8ff9700 1 -- 172.21.15.57:0/517135817 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c400cd20 con 0x7f07cc15c810 2023-11-28T12:04:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.525+0000 7f07d28ea700 1 -- 172.21.15.57:0/517135817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07cc161010 con 0x7f07cc15c810 2023-11-28T12:04:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.526+0000 7f07d28ea700 1 -- 172.21.15.57:0/517135817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07cc0ce740 con 0x7f07cc15c810 2023-11-28T12:04:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.527+0000 7f07c8ff9700 1 -- 172.21.15.57:0/517135817 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f07c4014480 con 0x7f07cc15c810 2023-11-28T12:04:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.527+0000 7f07c8ff9700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f07b80682f0 0x7f07b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.527+0000 7f07c8ff9700 1 -- 172.21.15.57:0/517135817 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f07c4086950 con 0x7f07cc15c810 2023-11-28T12:04:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.527+0000 7f07cbfff700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f07b80682f0 0x7f07b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.528+0000 7f07cbfff700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f07b80682f0 0x7f07b806a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009bb0 tx=0x7f07bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:33.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.531+0000 7f07c8ff9700 1 -- 172.21.15.57:0/517135817 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f07c4053060 con 0x7f07cc15c810 2023-11-28T12:04:33.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.729+0000 7f07d28ea700 1 -- 172.21.15.57:0/517135817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f07cc0cc130 con 0x7f07cc15c810 2023-11-28T12:04:33.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.730+0000 7f07c8ff9700 1 -- 172.21.15.57:0/517135817 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f07c40180a0 con 0x7f07cc15c810 2023-11-28T12:04:33.731 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:04:33.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.732+0000 7f07d28ea700 1 -- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f07b80682f0 msgr2=0x7f07b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.732+0000 7f07d28ea700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f07b80682f0 0x7f07b806a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009bb0 tx=0x7f07bc008040 comp rx=0 tx=0).stop 2023-11-28T12:04:33.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.732+0000 7f07d28ea700 1 -- 172.21.15.57:0/517135817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07cc15c810 msgr2=0x7f07cc1f59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.732+0000 7f07d28ea700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07cc15c810 0x7f07cc1f59a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f07c400ef90 tx=0x7f07c4009f60 comp rx=0 tx=0).stop 2023-11-28T12:04:33.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.732+0000 7f07d28ea700 1 -- 172.21.15.57:0/517135817 shutdown_connections 2023-11-28T12:04:33.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.732+0000 7f07d28ea700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f07b80682f0 0x7f07b806a7a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.732+0000 7f07d28ea700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07cc15c810 0x7f07cc1f59a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.732+0000 7f07d28ea700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07cc06cdc0 0x7f07cc0b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.732+0000 7f07d28ea700 1 --2- 172.21.15.57:0/517135817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07cc0718e0 0x7f07cc1f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.732+0000 7f07d28ea700 1 -- 172.21.15.57:0/517135817 >> 172.21.15.57:0/517135817 conn(0x7f07cc06c010 msgr2=0x7f07cc0adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:33.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.732+0000 7f07d28ea700 1 -- 172.21.15.57:0/517135817 shutdown_connections 2023-11-28T12:04:33.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.732+0000 7f07d28ea700 1 -- 172.21.15.57:0/517135817 wait complete. 2023-11-28T12:04:33.747 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:04:33.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.887+0000 7f7b5a671700 1 Processor -- start 2023-11-28T12:04:33.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.887+0000 7f7b5a671700 1 -- start start 2023-11-28T12:04:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.888+0000 7f7b5a671700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7b5406a980 0x7f7b540ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.888+0000 7f7b5a671700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7b54071890 0x7f7b54071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.888+0000 7f7b5a671700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5415cc10 con 0x7f7b540cf710 2023-11-28T12:04:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.888+0000 7f7b5a671700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5406c9b0 con 0x7f7b54071890 2023-11-28T12:04:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.888+0000 7f7b5a671700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5406cc90 con 0x7f7b5406a980 2023-11-28T12:04:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.888+0000 7f7b58c0e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7b54071890 0x7f7b54071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.888+0000 7f7b58c0e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7b54071890 0x7f7b54071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47892/0 (socket says 172.21.15.57:47892) 2023-11-28T12:04:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.888+0000 7f7b58c0e700 1 -- 172.21.15.57:0/1999597100 learned_addr learned my addr 172.21.15.57:0/1999597100 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.888+0000 7f7b537fe700 1 --2- 172.21.15.57:0/1999597100 >> v2:172.21.15.181:3300/0 conn(0x7f7b5406a980 0x7f7b540ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.888+0000 7f7b58c0e700 1 -- 172.21.15.57:0/1999597100 >> v2:172.21.15.181:3300/0 conn(0x7f7b5406a980 msgr2=0x7f7b540ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.888+0000 7f7b58c0e700 1 --2- 172.21.15.57:0/1999597100 >> v2:172.21.15.181:3300/0 conn(0x7f7b5406a980 0x7f7b540ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.888+0000 7f7b58c0e700 1 -- 172.21.15.57:0/1999597100 >> v1:172.21.15.181:6789/0 conn(0x7f7b540cf710 legacy=0x7f7b5406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.888+0000 7f7b58c0e700 1 -- 172.21.15.57:0/1999597100 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5406cf70 con 0x7f7b54071890 2023-11-28T12:04:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.889+0000 7f7b58c0e700 1 --2- 172.21.15.57:0/1999597100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b54071890 0x7f7b54071220 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7b540af5e0 tx=0x7f7b4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557865c5d0e31b81 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.889+0000 7f7b52ffd700 1 -- 172.21.15.57:0/1999597100 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c00e070 con 0x7f7b54071890 2023-11-28T12:04:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.889+0000 7f7b52ffd700 1 -- 172.21.15.57:0/1999597100 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c00b9c0 con 0x7f7b54071890 2023-11-28T12:04:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.889+0000 7f7b52ffd700 1 -- 172.21.15.57:0/1999597100 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c00bd70 con 0x7f7b54071890 2023-11-28T12:04:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.889+0000 7f7b5a671700 1 -- 172.21.15.57:0/1999597100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b54071890 msgr2=0x7f7b54071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.889+0000 7f7b5a671700 1 --2- 172.21.15.57:0/1999597100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b54071890 0x7f7b54071220 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7b540af5e0 tx=0x7f7b4c00b330 comp rx=0 tx=0).stop 2023-11-28T12:04:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.889+0000 7f7b5a671700 1 -- 172.21.15.57:0/1999597100 shutdown_connections 2023-11-28T12:04:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.889+0000 7f7b5a671700 1 --2- 172.21.15.57:0/1999597100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b54071890 0x7f7b54071220 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.889+0000 7f7b5a671700 1 --2- 172.21.15.57:0/1999597100 >> v2:172.21.15.181:3300/0 conn(0x7f7b5406a980 0x7f7b540ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.889+0000 7f7b5a671700 1 -- 172.21.15.57:0/1999597100 >> 172.21.15.57:0/1999597100 conn(0x7f7b5406c010 msgr2=0x7f7b540cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.889+0000 7f7b5a671700 1 -- 172.21.15.57:0/1999597100 shutdown_connections 2023-11-28T12:04:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.890+0000 7f7b5a671700 1 -- 172.21.15.57:0/1999597100 wait complete. 2023-11-28T12:04:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.891+0000 7f7b5a671700 1 Processor -- start 2023-11-28T12:04:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.891+0000 7f7b5a671700 1 -- start start 2023-11-28T12:04:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b5a671700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b5406a980 0x7f7b54151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b5a671700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b54071890 0x7f7b5414fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b5a671700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b540cf710 0x7f7b54155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b5a671700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b54150360 con 0x7f7b540cf710 2023-11-28T12:04:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b5a671700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5415d250 con 0x7f7b5406a980 2023-11-28T12:04:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b5a671700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5415d530 con 0x7f7b54071890 2023-11-28T12:04:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b58c0e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b540cf710 0x7f7b54155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b58c0e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b540cf710 0x7f7b54155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47906/0 (socket says 172.21.15.57:47906) 2023-11-28T12:04:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b58c0e700 1 -- 172.21.15.57:0/3734341167 learned_addr learned my addr 172.21.15.57:0/3734341167 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b53fff700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b5406a980 0x7f7b54151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b537fe700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b54071890 0x7f7b5414fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b58c0e700 1 -- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b54071890 msgr2=0x7f7b5414fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b58c0e700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b54071890 0x7f7b5414fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b58c0e700 1 -- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b5406a980 msgr2=0x7f7b54151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b58c0e700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b5406a980 0x7f7b54151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b58c0e700 1 -- 172.21.15.57:0/3734341167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c009b30 con 0x7f7b540cf710 2023-11-28T12:04:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.892+0000 7f7b58c0e700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b540cf710 0x7f7b54155730 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7b54067450 tx=0x7f7b4c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.893+0000 7f7b50ff9700 1 -- 172.21.15.57:0/3734341167 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c004880 con 0x7f7b540cf710 2023-11-28T12:04:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.893+0000 7f7b50ff9700 1 -- 172.21.15.57:0/3734341167 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c0099b0 con 0x7f7b540cf710 2023-11-28T12:04:33.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.893+0000 7f7b5a671700 1 -- 172.21.15.57:0/3734341167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5406d530 con 0x7f7b540cf710 2023-11-28T12:04:33.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.893+0000 7f7b50ff9700 1 -- 172.21.15.57:0/3734341167 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c01c680 con 0x7f7b540cf710 2023-11-28T12:04:33.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.893+0000 7f7b5a671700 1 -- 172.21.15.57:0/3734341167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5406d250 con 0x7f7b540cf710 2023-11-28T12:04:33.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.894+0000 7f7b5a671700 1 -- 172.21.15.57:0/3734341167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b54160480 con 0x7f7b540cf710 2023-11-28T12:04:33.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.894+0000 7f7b50ff9700 1 -- 172.21.15.57:0/3734341167 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7b4c022030 con 0x7f7b540cf710 2023-11-28T12:04:33.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.894+0000 7f7b50ff9700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b400682f0 0x7f7b4006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:33.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.895+0000 7f7b50ff9700 1 -- 172.21.15.57:0/3734341167 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7b4c087450 con 0x7f7b540cf710 2023-11-28T12:04:33.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.895+0000 7f7b53fff700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b400682f0 0x7f7b4006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:33.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.895+0000 7f7b53fff700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b400682f0 0x7f7b4006a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7b44007fd0 tx=0x7f7b44009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:33.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:33.899+0000 7f7b50ff9700 1 -- 172.21.15.57:0/3734341167 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7b4c053c70 con 0x7f7b540cf710 2023-11-28T12:04:34.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.098+0000 7f7b5a671700 1 -- 172.21.15.57:0/3734341167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b540743f0 con 0x7f7b540cf710 2023-11-28T12:04:34.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.098+0000 7f7b50ff9700 1 -- 172.21.15.57:0/3734341167 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b4c057290 con 0x7f7b540cf710 2023-11-28T12:04:34.100 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:04:34.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.101+0000 7f7b5a671700 1 -- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b400682f0 msgr2=0x7f7b4006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.101+0000 7f7b5a671700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b400682f0 0x7f7b4006a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7b44007fd0 tx=0x7f7b44009040 comp rx=0 tx=0).stop 2023-11-28T12:04:34.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.101+0000 7f7b5a671700 1 -- 172.21.15.57:0/3734341167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b540cf710 msgr2=0x7f7b54155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.101+0000 7f7b5a671700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b540cf710 0x7f7b54155730 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7b54067450 tx=0x7f7b4c004ad0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.101+0000 7f7b5a671700 1 -- 172.21.15.57:0/3734341167 shutdown_connections 2023-11-28T12:04:34.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.101+0000 7f7b5a671700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b400682f0 0x7f7b4006a7a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.101+0000 7f7b5a671700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b540cf710 0x7f7b54155730 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.101+0000 7f7b5a671700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b5406a980 0x7f7b54151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.101+0000 7f7b5a671700 1 --2- 172.21.15.57:0/3734341167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b54071890 0x7f7b5414fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.101+0000 7f7b5a671700 1 -- 172.21.15.57:0/3734341167 >> 172.21.15.57:0/3734341167 conn(0x7f7b5406c010 msgr2=0x7f7b540b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:34.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.101+0000 7f7b5a671700 1 -- 172.21.15.57:0/3734341167 shutdown_connections 2023-11-28T12:04:34.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.101+0000 7f7b5a671700 1 -- 172.21.15.57:0/3734341167 wait complete. 2023-11-28T12:04:34.115 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:04:34.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.252+0000 7f7b37923700 1 Processor -- start 2023-11-28T12:04:34.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.252+0000 7f7b37923700 1 -- start start 2023-11-28T12:04:34.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.253+0000 7f7b37923700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7b30160300 0x7f7b3015d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.253+0000 7f7b37923700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3015ceb0 con 0x7f7b300cec00 2023-11-28T12:04:34.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.253+0000 7f7b37923700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b301608c0 con 0x7f7b3015d880 2023-11-28T12:04:34.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.253+0000 7f7b37923700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b30160ba0 con 0x7f7b30160300 2023-11-28T12:04:34.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.253+0000 7f7b34ebe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7b30160300 0x7f7b3015d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.253+0000 7f7b356bf700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f7b300cec00 0x7f7b30065fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:51402/0 (socket says 172.21.15.57:51402) 2023-11-28T12:04:34.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.253+0000 7f7b356bf700 1 -- 172.21.15.57:0/3029600126 learned_addr learned my addr 172.21.15.57:0/3029600126 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:34.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.254+0000 7f7b34ebe700 1 -- 172.21.15.57:0/3029600126 >> v1:172.21.15.181:6789/0 conn(0x7f7b3015d880 legacy=0x7f7b300cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.254+0000 7f7b34ebe700 1 -- 172.21.15.57:0/3029600126 >> v1:172.21.15.57:6789/0 conn(0x7f7b300cec00 legacy=0x7f7b30065fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.254+0000 7f7b34ebe700 1 -- 172.21.15.57:0/3029600126 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b30160e80 con 0x7f7b30160300 2023-11-28T12:04:34.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.254+0000 7f7b34ebe700 1 --2- 172.21.15.57:0/3029600126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b30160300 0x7f7b3015d340 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009840 tx=0x7f7b2c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47cc2a663aef27d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:34.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.254+0000 7f7b23fff700 1 -- 172.21.15.57:0/3029600126 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c011830 con 0x7f7b30160300 2023-11-28T12:04:34.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.254+0000 7f7b23fff700 1 -- 172.21.15.57:0/3029600126 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c011990 con 0x7f7b30160300 2023-11-28T12:04:34.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.254+0000 7f7b23fff700 1 -- 172.21.15.57:0/3029600126 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c011d20 con 0x7f7b30160300 2023-11-28T12:04:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.255+0000 7f7b37923700 1 -- 172.21.15.57:0/3029600126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b30160300 msgr2=0x7f7b3015d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.255+0000 7f7b37923700 1 --2- 172.21.15.57:0/3029600126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b30160300 0x7f7b3015d340 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009840 tx=0x7f7b2c009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.255+0000 7f7b37923700 1 -- 172.21.15.57:0/3029600126 shutdown_connections 2023-11-28T12:04:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.255+0000 7f7b37923700 1 --2- 172.21.15.57:0/3029600126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b30160300 0x7f7b3015d340 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.255+0000 7f7b37923700 1 -- 172.21.15.57:0/3029600126 >> 172.21.15.57:0/3029600126 conn(0x7f7b300cf400 msgr2=0x7f7b300cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:34.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.255+0000 7f7b37923700 1 -- 172.21.15.57:0/3029600126 shutdown_connections 2023-11-28T12:04:34.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.255+0000 7f7b37923700 1 -- 172.21.15.57:0/3029600126 wait complete. 2023-11-28T12:04:34.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.256+0000 7f7b37923700 1 Processor -- start 2023-11-28T12:04:34.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.256+0000 7f7b37923700 1 -- start start 2023-11-28T12:04:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b37923700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b300cec00 0x7f7b301566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b37923700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b3015d880 0x7f7b300b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b37923700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b30160300 0x7f7b30155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b37923700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300b3c50 con 0x7f7b300cec00 2023-11-28T12:04:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b37923700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300b31f0 con 0x7f7b3015d880 2023-11-28T12:04:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b37923700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300c64d0 con 0x7f7b30160300 2023-11-28T12:04:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b356bf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b300cec00 0x7f7b301566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b356bf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b300cec00 0x7f7b301566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47916/0 (socket says 172.21.15.57:47916) 2023-11-28T12:04:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b356bf700 1 -- 172.21.15.57:0/2756296741 learned_addr learned my addr 172.21.15.57:0/2756296741 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b34ebe700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b3015d880 0x7f7b300b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b35ec0700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b30160300 0x7f7b30155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b356bf700 1 -- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b30160300 msgr2=0x7f7b30155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b356bf700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b30160300 0x7f7b30155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b356bf700 1 -- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b3015d880 msgr2=0x7f7b300b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b356bf700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b3015d880 0x7f7b300b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b356bf700 1 -- 172.21.15.57:0/2756296741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c0094a0 con 0x7f7b300cec00 2023-11-28T12:04:34.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.257+0000 7f7b34ebe700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b3015d880 0x7f7b300b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:34.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.258+0000 7f7b356bf700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b300cec00 0x7f7b301566f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400c4b0 tx=0x7f7b2400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:34.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.258+0000 7f7b21ffb700 1 -- 172.21.15.57:0/2756296741 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24005d70 con 0x7f7b300cec00 2023-11-28T12:04:34.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.258+0000 7f7b21ffb700 1 -- 172.21.15.57:0/2756296741 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b240070d0 con 0x7f7b300cec00 2023-11-28T12:04:34.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.258+0000 7f7b37923700 1 -- 172.21.15.57:0/2756296741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b300cefd0 con 0x7f7b300cec00 2023-11-28T12:04:34.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.258+0000 7f7b21ffb700 1 -- 172.21.15.57:0/2756296741 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24007440 con 0x7f7b300cec00 2023-11-28T12:04:34.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.258+0000 7f7b37923700 1 -- 172.21.15.57:0/2756296741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b30161160 con 0x7f7b300cec00 2023-11-28T12:04:34.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.259+0000 7f7b37923700 1 -- 172.21.15.57:0/2756296741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b300cf160 con 0x7f7b300cec00 2023-11-28T12:04:34.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.259+0000 7f7b21ffb700 1 -- 172.21.15.57:0/2756296741 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7b24019030 con 0x7f7b300cec00 2023-11-28T12:04:34.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.260+0000 7f7b21ffb700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b140682f0 0x7f7b1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.260+0000 7f7b21ffb700 1 -- 172.21.15.57:0/2756296741 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7b24084e70 con 0x7f7b300cec00 2023-11-28T12:04:34.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.260+0000 7f7b34ebe700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b140682f0 0x7f7b1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.260+0000 7f7b34ebe700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b140682f0 0x7f7b1406a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c000c00 tx=0x7f7b2c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:34.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.264+0000 7f7b21ffb700 1 -- 172.21.15.57:0/2756296741 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7b2404b5e0 con 0x7f7b300cec00 2023-11-28T12:04:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.459+0000 7f7b37923700 1 -- 172.21.15.57:0/2756296741 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7b300b3d90 con 0x7f7b300cec00 2023-11-28T12:04:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.460+0000 7f7b21ffb700 1 -- 172.21.15.57:0/2756296741 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7b24017020 con 0x7f7b300cec00 2023-11-28T12:04:34.461 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:04:34.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.462+0000 7f7b37923700 1 -- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b140682f0 msgr2=0x7f7b1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.462+0000 7f7b37923700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b140682f0 0x7f7b1406a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c000c00 tx=0x7f7b2c008030 comp rx=0 tx=0).stop 2023-11-28T12:04:34.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.462+0000 7f7b37923700 1 -- 172.21.15.57:0/2756296741 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b300cec00 msgr2=0x7f7b301566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.462+0000 7f7b37923700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b300cec00 0x7f7b301566f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400c4b0 tx=0x7f7b2400c870 comp rx=0 tx=0).stop 2023-11-28T12:04:34.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.462+0000 7f7b37923700 1 -- 172.21.15.57:0/2756296741 shutdown_connections 2023-11-28T12:04:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.462+0000 7f7b37923700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b140682f0 0x7f7b1406a7a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.462+0000 7f7b37923700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b30160300 0x7f7b30155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.462+0000 7f7b37923700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b300cec00 0x7f7b301566f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.462+0000 7f7b37923700 1 --2- 172.21.15.57:0/2756296741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b3015d880 0x7f7b300b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.462+0000 7f7b37923700 1 -- 172.21.15.57:0/2756296741 >> 172.21.15.57:0/2756296741 conn(0x7f7b300cf400 msgr2=0x7f7b300b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.462+0000 7f7b37923700 1 -- 172.21.15.57:0/2756296741 shutdown_connections 2023-11-28T12:04:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.462+0000 7f7b37923700 1 -- 172.21.15.57:0/2756296741 wait complete. 2023-11-28T12:04:34.476 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:04:34.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.613+0000 7f875960f700 1 Processor -- start 2023-11-28T12:04:34.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.613+0000 7f875960f700 1 -- start start 2023-11-28T12:04:34.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.614+0000 7f875960f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87540b2ee0 0x7f87540d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.614+0000 7f875960f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87540cc200 0x7f87540ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.614+0000 7f875960f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875415a9c0 con 0x7f875415a5f0 2023-11-28T12:04:34.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.614+0000 7f875960f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87540c7b80 con 0x7f87540b2ee0 2023-11-28T12:04:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.614+0000 7f875960f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87540cc990 con 0x7f87540cc200 2023-11-28T12:04:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f87527fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87540b2ee0 0x7f87540d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f87537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87540cc200 0x7f87540ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f87527fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87540b2ee0 0x7f87540d0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:58630/0 (socket says 172.21.15.57:58630) 2023-11-28T12:04:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f87527fc700 1 -- 172.21.15.57:0/3107173863 learned_addr learned my addr 172.21.15.57:0/3107173863 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f87537fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87540cc200 0x7f87540ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:48542/0 (socket says 172.21.15.57:48542) 2023-11-28T12:04:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f8752ffd700 1 --1- 172.21.15.57:0/3107173863 >> v1:172.21.15.181:6789/0 conn(0x7f875415a5f0 0x7f87540b27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:44126/0 (socket says 172.21.15.57:44126) 2023-11-28T12:04:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f87527fc700 1 -- 172.21.15.57:0/3107173863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87540cc200 msgr2=0x7f87540ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f87527fc700 1 --2- 172.21.15.57:0/3107173863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87540cc200 0x7f87540ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f87527fc700 1 -- 172.21.15.57:0/3107173863 >> v1:172.21.15.181:6789/0 conn(0x7f875415a5f0 legacy=0x7f87540b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f87527fc700 1 -- 172.21.15.57:0/3107173863 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540ccc70 con 0x7f87540b2ee0 2023-11-28T12:04:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f87527fc700 1 --2- 172.21.15.57:0/3107173863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87540b2ee0 0x7f87540d0ad0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f873c0098f0 tx=0x7f873c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61746e2340056b53 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:34.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f8751ffb700 1 -- 172.21.15.57:0/3107173863 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c011830 con 0x7f87540b2ee0 2023-11-28T12:04:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f8751ffb700 1 -- 172.21.15.57:0/3107173863 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873c011990 con 0x7f87540b2ee0 2023-11-28T12:04:34.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.615+0000 7f8751ffb700 1 -- 172.21.15.57:0/3107173863 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c011d20 con 0x7f87540b2ee0 2023-11-28T12:04:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.616+0000 7f875960f700 1 -- 172.21.15.57:0/3107173863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87540b2ee0 msgr2=0x7f87540d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.616+0000 7f875960f700 1 --2- 172.21.15.57:0/3107173863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87540b2ee0 0x7f87540d0ad0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f873c0098f0 tx=0x7f873c009c00 comp rx=0 tx=0).stop 2023-11-28T12:04:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.616+0000 7f875960f700 1 -- 172.21.15.57:0/3107173863 shutdown_connections 2023-11-28T12:04:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.616+0000 7f875960f700 1 --2- 172.21.15.57:0/3107173863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87540cc200 0x7f87540ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.616+0000 7f875960f700 1 --2- 172.21.15.57:0/3107173863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87540b2ee0 0x7f87540d0ad0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.616+0000 7f875960f700 1 -- 172.21.15.57:0/3107173863 >> 172.21.15.57:0/3107173863 conn(0x7f875406ea80 msgr2=0x7f875406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.616+0000 7f875960f700 1 -- 172.21.15.57:0/3107173863 shutdown_connections 2023-11-28T12:04:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.616+0000 7f875960f700 1 -- 172.21.15.57:0/3107173863 wait complete. 2023-11-28T12:04:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.618+0000 7f875960f700 1 Processor -- start 2023-11-28T12:04:34.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.618+0000 7f875960f700 1 -- start start 2023-11-28T12:04:34.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.618+0000 7f875960f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87540b2ee0 0x7f87540b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.618+0000 7f875960f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87540cc200 0x7f87540b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.618+0000 7f875960f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f875415a5f0 0x7f875406b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.618+0000 7f875960f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8754157d70 con 0x7f87540cc200 2023-11-28T12:04:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.618+0000 7f875960f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8754157a90 con 0x7f87540b2ee0 2023-11-28T12:04:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.618+0000 7f875960f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87540b3cb0 con 0x7f875415a5f0 2023-11-28T12:04:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f87527fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87540cc200 0x7f87540b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f87527fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87540cc200 0x7f87540b6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48054/0 (socket says 172.21.15.57:48054) 2023-11-28T12:04:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f87527fc700 1 -- 172.21.15.57:0/2913899920 learned_addr learned my addr 172.21.15.57:0/2913899920 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f87537fe700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f875415a5f0 0x7f875406b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f8752ffd700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87540b2ee0 0x7f87540b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f87527fc700 1 -- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f875415a5f0 msgr2=0x7f875406b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f87527fc700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f875415a5f0 0x7f875406b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f87527fc700 1 -- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87540b2ee0 msgr2=0x7f87540b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f87527fc700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87540b2ee0 0x7f87540b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f87527fc700 1 -- 172.21.15.57:0/2913899920 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c009550 con 0x7f87540cc200 2023-11-28T12:04:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f87527fc700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87540cc200 0x7f87540b6370 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f873c009c40 tx=0x7f873c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f874b7fe700 1 -- 172.21.15.57:0/2913899920 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c004510 con 0x7f87540cc200 2023-11-28T12:04:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f874b7fe700 1 -- 172.21.15.57:0/2913899920 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873c01e070 con 0x7f87540cc200 2023-11-28T12:04:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f875960f700 1 -- 172.21.15.57:0/2913899920 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87540cd230 con 0x7f87540cc200 2023-11-28T12:04:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f874b7fe700 1 -- 172.21.15.57:0/2913899920 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c0038e0 con 0x7f87540cc200 2023-11-28T12:04:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.619+0000 7f875960f700 1 -- 172.21.15.57:0/2913899920 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87540ccf50 con 0x7f87540cc200 2023-11-28T12:04:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.620+0000 7f875960f700 1 -- 172.21.15.57:0/2913899920 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87540cd510 con 0x7f87540cc200 2023-11-28T12:04:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.621+0000 7f874b7fe700 1 -- 172.21.15.57:0/2913899920 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f873c00f050 con 0x7f87540cc200 2023-11-28T12:04:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.621+0000 7f874b7fe700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f87340682f0 0x7f873406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.622+0000 7f874b7fe700 1 -- 172.21.15.57:0/2913899920 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f873c086b60 con 0x7f87540cc200 2023-11-28T12:04:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.622+0000 7f8752ffd700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f87340682f0 0x7f873406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.622+0000 7f8752ffd700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f87340682f0 0x7f873406a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8744000f60 tx=0x7f8744007ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:34.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.626+0000 7f874b7fe700 1 -- 172.21.15.57:0/2913899920 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f873c053300 con 0x7f87540cc200 2023-11-28T12:04:34.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.830+0000 7f875960f700 1 -- 172.21.15.57:0/2913899920 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87540cec30 con 0x7f87540cc200 2023-11-28T12:04:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.831+0000 7f874b7fe700 1 -- 172.21.15.57:0/2913899920 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f873c0200a0 con 0x7f87540cc200 2023-11-28T12:04:34.832 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:04:34.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.833+0000 7f875960f700 1 -- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f87340682f0 msgr2=0x7f873406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.833+0000 7f875960f700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f87340682f0 0x7f873406a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8744000f60 tx=0x7f8744007ed0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.833+0000 7f875960f700 1 -- 172.21.15.57:0/2913899920 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87540cc200 msgr2=0x7f87540b6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.833+0000 7f875960f700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87540cc200 0x7f87540b6370 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f873c009c40 tx=0x7f873c004750 comp rx=0 tx=0).stop 2023-11-28T12:04:34.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.833+0000 7f875960f700 1 -- 172.21.15.57:0/2913899920 shutdown_connections 2023-11-28T12:04:34.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.833+0000 7f875960f700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f87340682f0 0x7f873406a7a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.833+0000 7f875960f700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f875415a5f0 0x7f875406b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.833+0000 7f875960f700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87540b2ee0 0x7f87540b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.833+0000 7f875960f700 1 --2- 172.21.15.57:0/2913899920 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87540cc200 0x7f87540b6370 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.833+0000 7f875960f700 1 -- 172.21.15.57:0/2913899920 >> 172.21.15.57:0/2913899920 conn(0x7f875406ea80 msgr2=0x7f87540cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:34.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.833+0000 7f875960f700 1 -- 172.21.15.57:0/2913899920 shutdown_connections 2023-11-28T12:04:34.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.833+0000 7f875960f700 1 -- 172.21.15.57:0/2913899920 wait complete. 2023-11-28T12:04:34.849 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T12:04:34.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.983+0000 7ff6fe329700 1 Processor -- start 2023-11-28T12:04:34.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.983+0000 7ff6fe329700 1 -- start start 2023-11-28T12:04:34.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.984+0000 7ff6fe329700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f815c810 0x7ff6f8160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.984+0000 7ff6fe329700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f81607a0 con 0x7ff6f80718e0 2023-11-28T12:04:34.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.984+0000 7ff6fe329700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f8160a50 con 0x7ff6f806a270 2023-11-28T12:04:34.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.984+0000 7ff6fe329700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f815cbe0 con 0x7ff6f815c810 2023-11-28T12:04:34.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.984+0000 7ff6f77fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff6f80718e0 0x7ff6f806c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40718/0 (socket says 172.21.15.57:40718) 2023-11-28T12:04:34.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.984+0000 7ff6f77fe700 1 -- 172.21.15.57:0/2048022954 learned_addr learned my addr 172.21.15.57:0/2048022954 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.984+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988805090 0 0) 0x7ff6f81607a0 con 0x7ff6f80718e0 2023-11-28T12:04:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.984+0000 7ff6f7fff700 1 --2- 172.21.15.57:0/2048022954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f815c810 0x7ff6f8160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6e00035b0 con 0x7ff6f80718e0 2023-11-28T12:04:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 309485044 0 0) 0x7ff6e00035b0 con 0x7ff6f80718e0 2023-11-28T12:04:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f815c810 msgr2=0x7ff6f8160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 --2- 172.21.15.57:0/2048022954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f815c810 0x7ff6f8160260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 >> v1:172.21.15.181:6789/0 conn(0x7ff6f806a270 legacy=0x7ff6f80cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f8160d30 con 0x7ff6f80718e0 2023-11-28T12:04:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7ff6ec001670 con 0x7ff6f80718e0 2023-11-28T12:04:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 >> v1:172.21.15.57:6789/0 conn(0x7ff6f80718e0 legacy=0x7ff6f806c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 --2- 172.21.15.57:0/2048022954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6e0003a30 0x7ff6e0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 --2- 172.21.15.57:0/2048022954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6e0004850 0x7ff6e0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 --2- 172.21.15.57:0/2048022954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f815c810 0x7ff6e0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f815cbe0 con 0x7ff6e0004850 2023-11-28T12:04:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e000bac0 con 0x7ff6e0003a30 2023-11-28T12:04:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e000bc50 con 0x7ff6f815c810 2023-11-28T12:04:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f77fe700 1 --2- 172.21.15.57:0/2048022954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6e0004850 0x7ff6e0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f7fff700 1 --2- 172.21.15.57:0/2048022954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6e0003a30 0x7ff6e0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6fc8c6700 1 --2- 172.21.15.57:0/2048022954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f815c810 0x7ff6e0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f77fe700 1 -- 172.21.15.57:0/2048022954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f815c810 msgr2=0x7ff6e0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f77fe700 1 --2- 172.21.15.57:0/2048022954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f815c810 0x7ff6e0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f77fe700 1 -- 172.21.15.57:0/2048022954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6e0003a30 msgr2=0x7ff6e0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f77fe700 1 --2- 172.21.15.57:0/2048022954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6e0003a30 0x7ff6e0003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6f77fe700 1 -- 172.21.15.57:0/2048022954 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e00035b0 con 0x7ff6e0004850 2023-11-28T12:04:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.985+0000 7ff6fc8c6700 1 --2- 172.21.15.57:0/2048022954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f815c810 0x7ff6e0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.986+0000 7ff6f77fe700 1 --2- 172.21.15.57:0/2048022954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6e0004850 0x7ff6e0004cc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00c240 tx=0x7ff6e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbc58a345d06996 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.986+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec007c10 con 0x7ff6e0004850 2023-11-28T12:04:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.986+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ec00ce00 con 0x7ff6e0004850 2023-11-28T12:04:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.986+0000 7ff6f6ffd700 1 -- 172.21.15.57:0/2048022954 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec005ba0 con 0x7ff6e0004850 2023-11-28T12:04:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.986+0000 7ff6fe329700 1 -- 172.21.15.57:0/2048022954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6e0004850 msgr2=0x7ff6e0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.986+0000 7ff6fe329700 1 --2- 172.21.15.57:0/2048022954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6e0004850 0x7ff6e0004cc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00c240 tx=0x7ff6e0005880 comp rx=0 tx=0).stop 2023-11-28T12:04:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.986+0000 7ff6fc8c6700 1 -- 172.21.15.57:0/2048022954 reap_dead start 2023-11-28T12:04:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.986+0000 7ff6fe329700 1 -- 172.21.15.57:0/2048022954 shutdown_connections 2023-11-28T12:04:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.986+0000 7ff6fe329700 1 -- 172.21.15.57:0/2048022954 >> 172.21.15.57:0/2048022954 conn(0x7ff6f806c010 msgr2=0x7ff6f80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.986+0000 7ff6fe329700 1 -- 172.21.15.57:0/2048022954 shutdown_connections 2023-11-28T12:04:34.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.986+0000 7ff6fe329700 1 -- 172.21.15.57:0/2048022954 wait complete. 2023-11-28T12:04:34.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.988+0000 7ff6fe329700 1 Processor -- start 2023-11-28T12:04:34.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.988+0000 7ff6fe329700 1 -- start start 2023-11-28T12:04:34.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6fe329700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6e0004850 0x7ff6f8156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6fe329700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f8157320 0x7ff6f8157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6fe329700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6f815f2b0 0x7ff6f815f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6fe329700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f815fc60 con 0x7ff6e0004850 2023-11-28T12:04:34.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6fe329700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f815ddd0 con 0x7ff6f815f2b0 2023-11-28T12:04:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6fe329700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f815e0b0 con 0x7ff6f8157320 2023-11-28T12:04:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6f7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6e0004850 0x7ff6f8156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6f7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6e0004850 0x7ff6f8156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48068/0 (socket says 172.21.15.57:48068) 2023-11-28T12:04:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6f7fff700 1 -- 172.21.15.57:0/948155282 learned_addr learned my addr 172.21.15.57:0/948155282 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6f77fe700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f8157320 0x7ff6f8157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6fc8c6700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6f815f2b0 0x7ff6f815f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6f7fff700 1 -- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f8157320 msgr2=0x7ff6f8157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6f7fff700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f8157320 0x7ff6f8157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6f7fff700 1 -- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6f815f2b0 msgr2=0x7ff6f815f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6f7fff700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6f815f2b0 0x7ff6f815f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6f7fff700 1 -- 172.21.15.57:0/948155282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f80ceb20 con 0x7ff6e0004850 2023-11-28T12:04:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6fc8c6700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6f815f2b0 0x7ff6f815f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6f7fff700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6e0004850 0x7ff6f8156a60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8011aa0 tx=0x7ff6e8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6f4ff9700 1 -- 172.21.15.57:0/948155282 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e800f940 con 0x7ff6e0004850 2023-11-28T12:04:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.989+0000 7ff6f4ff9700 1 -- 172.21.15.57:0/948155282 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e800faa0 con 0x7ff6e0004850 2023-11-28T12:04:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.990+0000 7ff6f4ff9700 1 -- 172.21.15.57:0/948155282 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e800fe50 con 0x7ff6e0004850 2023-11-28T12:04:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.990+0000 7ff6fe329700 1 -- 172.21.15.57:0/948155282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f8161010 con 0x7ff6e0004850 2023-11-28T12:04:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.990+0000 7ff6fe329700 1 -- 172.21.15.57:0/948155282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f806cf90 con 0x7ff6e0004850 2023-11-28T12:04:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.990+0000 7ff6fe329700 1 -- 172.21.15.57:0/948155282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f80743f0 con 0x7ff6e0004850 2023-11-28T12:04:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.991+0000 7ff6f4ff9700 1 -- 172.21.15.57:0/948155282 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff6e8014440 con 0x7ff6e0004850 2023-11-28T12:04:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.991+0000 7ff6f4ff9700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff6e40682f0 0x7ff6e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.992+0000 7ff6f4ff9700 1 -- 172.21.15.57:0/948155282 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff6e8086680 con 0x7ff6e0004850 2023-11-28T12:04:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.992+0000 7ff6f77fe700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff6e40682f0 0x7ff6e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.992+0000 7ff6f77fe700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff6e40682f0 0x7ff6e406a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec001930 tx=0x7ff6ec007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:34.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:34.996+0000 7ff6f4ff9700 1 -- 172.21.15.57:0/948155282 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff6e8052ec0 con 0x7ff6e0004850 2023-11-28T12:04:35.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.212+0000 7ff6fe329700 1 -- 172.21.15.57:0/948155282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff6f80caa10 con 0x7ff6e0004850 2023-11-28T12:04:35.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.212+0000 7ff6f4ff9700 1 -- 172.21.15.57:0/948155282 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff6e80564e0 con 0x7ff6e0004850 2023-11-28T12:04:35.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.215+0000 7ff6fe329700 1 -- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff6e40682f0 msgr2=0x7ff6e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:35.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.215+0000 7ff6fe329700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff6e40682f0 0x7ff6e406a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec001930 tx=0x7ff6ec007000 comp rx=0 tx=0).stop 2023-11-28T12:04:35.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.215+0000 7ff6fe329700 1 -- 172.21.15.57:0/948155282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6e0004850 msgr2=0x7ff6f8156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:35.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.215+0000 7ff6fe329700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6e0004850 0x7ff6f8156a60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8011aa0 tx=0x7ff6e8011db0 comp rx=0 tx=0).stop 2023-11-28T12:04:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.215+0000 7ff6fe329700 1 -- 172.21.15.57:0/948155282 shutdown_connections 2023-11-28T12:04:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.215+0000 7ff6fe329700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff6e40682f0 0x7ff6e406a7a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.215+0000 7ff6fe329700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6f815f2b0 0x7ff6f815f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.215+0000 7ff6fe329700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6e0004850 0x7ff6f8156a60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.215+0000 7ff6fe329700 1 --2- 172.21.15.57:0/948155282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6f8157320 0x7ff6f8157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.215+0000 7ff6fe329700 1 -- 172.21.15.57:0/948155282 >> 172.21.15.57:0/948155282 conn(0x7ff6f806c010 msgr2=0x7ff6f80acb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.215+0000 7ff6fe329700 1 -- 172.21.15.57:0/948155282 shutdown_connections 2023-11-28T12:04:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.215+0000 7ff6fe329700 1 -- 172.21.15.57:0/948155282 wait complete. 2023-11-28T12:04:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:listed 0 entries 2023-11-28T12:04:35.229 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2023-11-28T12:04:35.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:04:35.230 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T12:04:35.266 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[141720]: fuse finished with error 0 and tester_r 0 2023-11-28T12:04:35.267 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:04:35.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:04:35.268 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T12:04:35.385 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:04:35.385 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:04:35.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251064k,nr_inodes=4062766,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14954 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253752k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T12:04:35.406 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:04:35.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:04:35.406 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T12:04:35.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:04:35.465 INFO:teuthology.orchestra.run.smithi181.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T12:04:35.466 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:04:35.466 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:04:35.489 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16254224k,nr_inodes=4063556,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15749 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3254388k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T12:04:35.490 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:04:35.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:04:35.490 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T12:04:35.553 INFO:teuthology.orchestra.run.smithi057.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T12:04:35.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:04:35.554 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:04:35.554 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:04:35.575 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251064k,nr_inodes=4062766,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14954 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253752k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T12:04:35.575 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:04:35.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:04:35.576 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T12:04:35.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:04:35.636 INFO:teuthology.orchestra.run.smithi181.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T12:04:35.636 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:04:35.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.779+0000 7fa25c61a700 1 Processor -- start 2023-11-28T12:04:35.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.779+0000 7fa25c61a700 1 -- start start 2023-11-28T12:04:35.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa25c61a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa25406a270 0x7fa25406a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:35.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa25c61a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa25415cc80 0x7fa25406de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:35.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa25c61a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25406ab80 con 0x7fa25406e3a0 2023-11-28T12:04:35.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa25c61a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2540b2490 con 0x7fa25406a270 2023-11-28T12:04:35.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa25c61a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2540b1810 con 0x7fa25415cc80 2023-11-28T12:04:35.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa25a3b6700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa25406a270 0x7fa25406a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:35.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa25a3b6700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa25406a270 0x7fa25406a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48076/0 (socket says 172.21.15.57:48076) 2023-11-28T12:04:35.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa25a3b6700 1 -- 172.21.15.57:0/2513701244 learned_addr learned my addr 172.21.15.57:0/2513701244 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:35.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa259bb5700 1 --2- 172.21.15.57:0/2513701244 >> v2:172.21.15.181:3300/0 conn(0x7fa25415cc80 0x7fa25406de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:35.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa25a3b6700 1 -- 172.21.15.57:0/2513701244 >> v2:172.21.15.181:3300/0 conn(0x7fa25415cc80 msgr2=0x7fa25406de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:35.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa25a3b6700 1 --2- 172.21.15.57:0/2513701244 >> v2:172.21.15.181:3300/0 conn(0x7fa25415cc80 0x7fa25406de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa25a3b6700 1 -- 172.21.15.57:0/2513701244 >> v1:172.21.15.181:6789/0 conn(0x7fa25406e3a0 legacy=0x7fa25406ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa25a3b6700 1 -- 172.21.15.57:0/2513701244 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2540b1af0 con 0x7fa25406a270 2023-11-28T12:04:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa2593b4700 1 -- 172.21.15.57:0/2513701244 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520197191 0 0) 0x7fa25406ab80 con 0x7fa25406e3a0 2023-11-28T12:04:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.781+0000 7fa25a3b6700 1 --2- 172.21.15.57:0/2513701244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25406a270 0x7fa25406a640 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa25406c6c0 tx=0x7fa250009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6cfcdde6e86aa9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.782+0000 7fa2593b4700 1 -- 172.21.15.57:0/2513701244 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250011830 con 0x7fa25406a270 2023-11-28T12:04:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.782+0000 7fa2593b4700 1 -- 172.21.15.57:0/2513701244 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa250011990 con 0x7fa25406a270 2023-11-28T12:04:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.782+0000 7fa2593b4700 1 -- 172.21.15.57:0/2513701244 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250011d20 con 0x7fa25406a270 2023-11-28T12:04:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.782+0000 7fa25c61a700 1 -- 172.21.15.57:0/2513701244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25406a270 msgr2=0x7fa25406a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.782+0000 7fa25c61a700 1 --2- 172.21.15.57:0/2513701244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25406a270 0x7fa25406a640 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa25406c6c0 tx=0x7fa250009bd0 comp rx=0 tx=0).stop 2023-11-28T12:04:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.782+0000 7fa25c61a700 1 -- 172.21.15.57:0/2513701244 shutdown_connections 2023-11-28T12:04:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.782+0000 7fa25c61a700 1 --2- 172.21.15.57:0/2513701244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25406a270 0x7fa25406a640 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.782+0000 7fa25c61a700 1 --2- 172.21.15.57:0/2513701244 >> v2:172.21.15.181:3300/0 conn(0x7fa25415cc80 0x7fa25406de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.782+0000 7fa25c61a700 1 -- 172.21.15.57:0/2513701244 >> 172.21.15.57:0/2513701244 conn(0x7fa25406c010 msgr2=0x7fa2540cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.782+0000 7fa25c61a700 1 -- 172.21.15.57:0/2513701244 shutdown_connections 2023-11-28T12:04:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.782+0000 7fa25c61a700 1 -- 172.21.15.57:0/2513701244 wait complete. 2023-11-28T12:04:35.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.784+0000 7fa25c61a700 1 Processor -- start 2023-11-28T12:04:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.784+0000 7fa25c61a700 1 -- start start 2023-11-28T12:04:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa25c61a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa25406a270 0x7fa2541e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa25c61a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25406e3a0 0x7fa2540c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa25c61a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa25415cc80 0x7fa2540b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa25c61a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2540b5110 con 0x7fa25406e3a0 2023-11-28T12:04:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa25c61a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2540b4e30 con 0x7fa25415cc80 2023-11-28T12:04:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa25c61a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2541e5090 con 0x7fa25406a270 2023-11-28T12:04:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa259bb5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25406e3a0 0x7fa2540c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa259bb5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25406e3a0 0x7fa2540c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48090/0 (socket says 172.21.15.57:48090) 2023-11-28T12:04:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa25a3b6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa25406a270 0x7fa2541e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa259bb5700 1 -- 172.21.15.57:0/1757978687 learned_addr learned my addr 172.21.15.57:0/1757978687 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa25abb7700 1 --2- 172.21.15.57:0/1757978687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa25415cc80 0x7fa2540b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa259bb5700 1 -- 172.21.15.57:0/1757978687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa25406a270 msgr2=0x7fa2541e5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa259bb5700 1 --2- 172.21.15.57:0/1757978687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa25406a270 0x7fa2541e5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa259bb5700 1 -- 172.21.15.57:0/1757978687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa25415cc80 msgr2=0x7fa2540b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa259bb5700 1 --2- 172.21.15.57:0/1757978687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa25415cc80 0x7fa2540b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa259bb5700 1 -- 172.21.15.57:0/1757978687 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa250009550 con 0x7fa25406e3a0 2023-11-28T12:04:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa259bb5700 1 --2- 172.21.15.57:0/1757978687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25406e3a0 0x7fa2540c5da0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa24400e9d0 tx=0x7fa24400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.785+0000 7fa24affd700 1 -- 172.21.15.57:0/1757978687 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244009730 con 0x7fa25406e3a0 2023-11-28T12:04:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.786+0000 7fa24affd700 1 -- 172.21.15.57:0/1757978687 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24400ee70 con 0x7fa25406e3a0 2023-11-28T12:04:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.786+0000 7fa25c61a700 1 -- 172.21.15.57:0/1757978687 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2540b20b0 con 0x7fa25406e3a0 2023-11-28T12:04:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.786+0000 7fa24affd700 1 -- 172.21.15.57:0/1757978687 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24400ca70 con 0x7fa25406e3a0 2023-11-28T12:04:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.786+0000 7fa25c61a700 1 -- 172.21.15.57:0/1757978687 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2540b1dd0 con 0x7fa25406e3a0 2023-11-28T12:04:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.787+0000 7fa25c61a700 1 -- 172.21.15.57:0/1757978687 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25406e7c0 con 0x7fa25406e3a0 2023-11-28T12:04:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.787+0000 7fa24affd700 1 -- 172.21.15.57:0/1757978687 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa244019030 con 0x7fa25406e3a0 2023-11-28T12:04:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.788+0000 7fa24affd700 1 --2- 172.21.15.57:0/1757978687 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2380682f0 0x7fa23806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.788+0000 7fa24affd700 1 -- 172.21.15.57:0/1757978687 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa244085c00 con 0x7fa25406e3a0 2023-11-28T12:04:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.788+0000 7fa25a3b6700 1 --2- 172.21.15.57:0/1757978687 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2380682f0 0x7fa23806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.788+0000 7fa25a3b6700 1 --2- 172.21.15.57:0/1757978687 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2380682f0 0x7fa23806a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa250014040 tx=0x7fa250008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:35.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:35.793+0000 7fa24affd700 1 -- 172.21.15.57:0/1757978687 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa24408b020 con 0x7fa25406e3a0 2023-11-28T12:04:36.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.042+0000 7fa25c61a700 1 -- 172.21.15.57:0/1757978687 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2540743f0 con 0x7fa25406e3a0 2023-11-28T12:04:36.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.042+0000 7fa24affd700 1 -- 172.21.15.57:0/1757978687 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fa244017090 con 0x7fa25406e3a0 2023-11-28T12:04:36.043 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:36.044 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1773283092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1773283092},{"type":"v1","addr":"172.21.15.181:6837","nonce":1773283092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/3718600227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3718600227},{"type":"v1","addr":"172.21.15.57:6834","nonce":3718600227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/762261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":762261630},{"type":"v1","addr":"172.21.15.57:6835","nonce":762261630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:03:47.263787+0000","modified":"2023-11-28T12:03:50.285215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3437917192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3437917192},{"type":"v1","addr":"172.21.15.181:6836","nonce":3437917192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T12:04:36.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.045+0000 7fa25c61a700 1 -- 172.21.15.57:0/1757978687 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2380682f0 msgr2=0x7fa23806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.045+0000 7fa25c61a700 1 --2- 172.21.15.57:0/1757978687 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2380682f0 0x7fa23806a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa250014040 tx=0x7fa250008030 comp rx=0 tx=0).stop 2023-11-28T12:04:36.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.045+0000 7fa25c61a700 1 -- 172.21.15.57:0/1757978687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25406e3a0 msgr2=0x7fa2540c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.045+0000 7fa25c61a700 1 --2- 172.21.15.57:0/1757978687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25406e3a0 0x7fa2540c5da0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa24400e9d0 tx=0x7fa24400ece0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.045+0000 7fa25c61a700 1 -- 172.21.15.57:0/1757978687 shutdown_connections 2023-11-28T12:04:36.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.045+0000 7fa25c61a700 1 --2- 172.21.15.57:0/1757978687 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa2380682f0 0x7fa23806a7a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.045+0000 7fa25c61a700 1 --2- 172.21.15.57:0/1757978687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa25415cc80 0x7fa2540b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.045+0000 7fa25c61a700 1 --2- 172.21.15.57:0/1757978687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa25406a270 0x7fa2541e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.045+0000 7fa25c61a700 1 --2- 172.21.15.57:0/1757978687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25406e3a0 0x7fa2540c5da0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.045+0000 7fa25c61a700 1 -- 172.21.15.57:0/1757978687 >> 172.21.15.57:0/1757978687 conn(0x7fa25406c010 msgr2=0x7fa2540adb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:36.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.045+0000 7fa25c61a700 1 -- 172.21.15.57:0/1757978687 shutdown_connections 2023-11-28T12:04:36.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.045+0000 7fa25c61a700 1 -- 172.21.15.57:0/1757978687 wait complete. 2023-11-28T12:04:36.048 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2023-11-28T12:04:36.060 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:04:36.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.194+0000 7fe936a70700 1 Processor -- start 2023-11-28T12:04:36.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.194+0000 7fe936a70700 1 -- start start 2023-11-28T12:04:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe936a70700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe93006e400 0x7fe93006e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe936a70700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe9300d15f0 0x7fe93015b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe936a70700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe93015af30 con 0x7fe93015e010 2023-11-28T12:04:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe936a70700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe93015a9d0 con 0x7fe9300d15f0 2023-11-28T12:04:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe936a70700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9301584e0 con 0x7fe93006e400 2023-11-28T12:04:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe92ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe9300d15f0 0x7fe93015b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe92ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe9300d15f0 0x7fe93015b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48106/0 (socket says 172.21.15.57:48106) 2023-11-28T12:04:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe92ffff700 1 -- 172.21.15.57:0/1718851016 learned_addr learned my addr 172.21.15.57:0/1718851016 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe93480c700 1 --2- 172.21.15.57:0/1718851016 >> v2:172.21.15.181:3300/0 conn(0x7fe93006e400 0x7fe93006e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe92ffff700 1 -- 172.21.15.57:0/1718851016 >> v2:172.21.15.181:3300/0 conn(0x7fe93006e400 msgr2=0x7fe93006e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe92ffff700 1 --2- 172.21.15.57:0/1718851016 >> v2:172.21.15.181:3300/0 conn(0x7fe93006e400 0x7fe93006e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe92ffff700 1 -- 172.21.15.57:0/1718851016 >> v1:172.21.15.181:6789/0 conn(0x7fe93015e010 legacy=0x7fe9300ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe92ffff700 1 -- 172.21.15.57:0/1718851016 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93015ef40 con 0x7fe9300d15f0 2023-11-28T12:04:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.195+0000 7fe92ffff700 1 --2- 172.21.15.57:0/1718851016 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9300d15f0 0x7fe93015b3c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe9300cd900 tx=0x7fe91c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9abc1fc6fed03b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.196+0000 7fe92f7fe700 1 -- 172.21.15.57:0/1718851016 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c011830 con 0x7fe9300d15f0 2023-11-28T12:04:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.196+0000 7fe92f7fe700 1 -- 172.21.15.57:0/1718851016 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c011990 con 0x7fe9300d15f0 2023-11-28T12:04:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.196+0000 7fe92f7fe700 1 -- 172.21.15.57:0/1718851016 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c011d20 con 0x7fe9300d15f0 2023-11-28T12:04:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.196+0000 7fe936a70700 1 -- 172.21.15.57:0/1718851016 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9300d15f0 msgr2=0x7fe93015b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.196+0000 7fe936a70700 1 --2- 172.21.15.57:0/1718851016 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9300d15f0 0x7fe93015b3c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe9300cd900 tx=0x7fe91c009bd0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.196+0000 7fe936a70700 1 -- 172.21.15.57:0/1718851016 shutdown_connections 2023-11-28T12:04:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.196+0000 7fe936a70700 1 --2- 172.21.15.57:0/1718851016 >> v2:172.21.15.181:3300/0 conn(0x7fe93006e400 0x7fe93006e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.196+0000 7fe936a70700 1 --2- 172.21.15.57:0/1718851016 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9300d15f0 0x7fe93015b3c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.196+0000 7fe936a70700 1 -- 172.21.15.57:0/1718851016 >> 172.21.15.57:0/1718851016 conn(0x7fe93006a270 msgr2=0x7fe93006a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.196+0000 7fe936a70700 1 -- 172.21.15.57:0/1718851016 shutdown_connections 2023-11-28T12:04:36.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.196+0000 7fe936a70700 1 -- 172.21.15.57:0/1718851016 wait complete. 2023-11-28T12:04:36.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.197+0000 7fe936a70700 1 Processor -- start 2023-11-28T12:04:36.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.198+0000 7fe936a70700 1 -- start start 2023-11-28T12:04:36.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.198+0000 7fe936a70700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe93006e400 0x7fe9301417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.198+0000 7fe936a70700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9300d15f0 0x7fe93006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.198+0000 7fe936a70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe93015e010 0x7fe9300b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.198+0000 7fe936a70700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe930141d20 con 0x7fe9300d15f0 2023-11-28T12:04:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.198+0000 7fe936a70700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9300cf6f0 con 0x7fe93015e010 2023-11-28T12:04:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.198+0000 7fe936a70700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9300cf9d0 con 0x7fe93006e400 2023-11-28T12:04:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe92ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9300d15f0 0x7fe93006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe92ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9300d15f0 0x7fe93006bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48116/0 (socket says 172.21.15.57:48116) 2023-11-28T12:04:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe92ffff700 1 -- 172.21.15.57:0/2660151506 learned_addr learned my addr 172.21.15.57:0/2660151506 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe93480c700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe93006e400 0x7fe9301417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe93500d700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe93015e010 0x7fe9300b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe92ffff700 1 -- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe93006e400 msgr2=0x7fe9301417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe92ffff700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe93006e400 0x7fe9301417e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe92ffff700 1 -- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe93015e010 msgr2=0x7fe9300b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe92ffff700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe93015e010 0x7fe9300b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe92ffff700 1 -- 172.21.15.57:0/2660151506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91c009550 con 0x7fe9300d15f0 2023-11-28T12:04:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe93500d700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe93015e010 0x7fe9300b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe92ffff700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9300d15f0 0x7fe93006bda0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe91c000c00 tx=0x7fe91c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe92d7fa700 1 -- 172.21.15.57:0/2660151506 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c0045b0 con 0x7fe9300d15f0 2023-11-28T12:04:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe92d7fa700 1 -- 172.21.15.57:0/2660151506 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c01f070 con 0x7fe9300d15f0 2023-11-28T12:04:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe936a70700 1 -- 172.21.15.57:0/2660151506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93015e980 con 0x7fe9300d15f0 2023-11-28T12:04:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe92d7fa700 1 -- 172.21.15.57:0/2660151506 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c003900 con 0x7fe9300d15f0 2023-11-28T12:04:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.199+0000 7fe936a70700 1 -- 172.21.15.57:0/2660151506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93006ed10 con 0x7fe9300d15f0 2023-11-28T12:04:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.200+0000 7fe936a70700 1 -- 172.21.15.57:0/2660151506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe93015ec60 con 0x7fe9300d15f0 2023-11-28T12:04:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.201+0000 7fe92d7fa700 1 -- 172.21.15.57:0/2660151506 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe91c02a020 con 0x7fe9300d15f0 2023-11-28T12:04:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.201+0000 7fe92d7fa700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe9140682f0 0x7fe91406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.201+0000 7fe92d7fa700 1 -- 172.21.15.57:0/2660151506 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe91c087400 con 0x7fe9300d15f0 2023-11-28T12:04:36.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.201+0000 7fe93480c700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe9140682f0 0x7fe91406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.202+0000 7fe93480c700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe9140682f0 0x7fe91406a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe924009b70 tx=0x7fe924008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:36.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.206+0000 7fe92d7fa700 1 -- 172.21.15.57:0/2660151506 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe91c053b30 con 0x7fe9300d15f0 2023-11-28T12:04:36.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.407+0000 7fe936a70700 1 -- 172.21.15.57:0/2660151506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9300743f0 con 0x7fe9300d15f0 2023-11-28T12:04:36.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.408+0000 7fe92d7fa700 1 -- 172.21.15.57:0/2660151506 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+17961 (secure 0 0 0) 0x7fe91c057150 con 0x7fe9300d15f0 2023-11-28T12:04:36.410 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:36.410 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":51,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:04:34.775114+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:03:45.520415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T12:03:46.232534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2d","mappings":[{"from":0,"to":6}]},{"pgid":"8.37","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:04:36.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.411+0000 7fe936a70700 1 -- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe9140682f0 msgr2=0x7fe91406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.411+0000 7fe936a70700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe9140682f0 0x7fe91406a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe924009b70 tx=0x7fe924008040 comp rx=0 tx=0).stop 2023-11-28T12:04:36.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.411+0000 7fe936a70700 1 -- 172.21.15.57:0/2660151506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9300d15f0 msgr2=0x7fe93006bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.411+0000 7fe936a70700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9300d15f0 0x7fe93006bda0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe91c000c00 tx=0x7fe91c004750 comp rx=0 tx=0).stop 2023-11-28T12:04:36.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.411+0000 7fe936a70700 1 -- 172.21.15.57:0/2660151506 shutdown_connections 2023-11-28T12:04:36.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.411+0000 7fe936a70700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe9140682f0 0x7fe91406a7a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.411+0000 7fe936a70700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe93015e010 0x7fe9300b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.411+0000 7fe936a70700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe93006e400 0x7fe9301417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.411+0000 7fe936a70700 1 --2- 172.21.15.57:0/2660151506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9300d15f0 0x7fe93006bda0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.411+0000 7fe936a70700 1 -- 172.21.15.57:0/2660151506 >> 172.21.15.57:0/2660151506 conn(0x7fe93006a270 msgr2=0x7fe930159c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:36.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.411+0000 7fe936a70700 1 -- 172.21.15.57:0/2660151506 shutdown_connections 2023-11-28T12:04:36.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.411+0000 7fe936a70700 1 -- 172.21.15.57:0/2660151506 wait complete. 2023-11-28T12:04:36.425 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:03:45.520415+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:03:46.232534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:04:36.426 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:04:36.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.564+0000 7f5d0d94e700 1 Processor -- start 2023-11-28T12:04:36.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.565+0000 7f5d0d94e700 1 -- start start 2023-11-28T12:04:36.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.565+0000 7f5d0d94e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5d0815c810 0x7f5d08160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.565+0000 7f5d0d94e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d080718e0 0x7f5d0806c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.565+0000 7f5d0d94e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d081607a0 con 0x7f5d0806cdc0 2023-11-28T12:04:36.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.565+0000 7f5d0d94e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0815cbe0 con 0x7f5d0815c810 2023-11-28T12:04:36.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.565+0000 7f5d0d94e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d08160a50 con 0x7f5d080718e0 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.566+0000 7f5d06ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5d0815c810 0x7f5d08160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.566+0000 7f5d06ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5d0815c810 0x7f5d08160260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48118/0 (socket says 172.21.15.57:48118) 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.566+0000 7f5d06ffd700 1 -- 172.21.15.57:0/1025553621 learned_addr learned my addr 172.21.15.57:0/1025553621 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.566+0000 7f5cfe7fc700 1 --2- 172.21.15.57:0/1025553621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d080718e0 0x7f5d0806c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.566+0000 7f5d06ffd700 1 -- 172.21.15.57:0/1025553621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d080718e0 msgr2=0x7f5d0806c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.566+0000 7f5d06ffd700 1 --2- 172.21.15.57:0/1025553621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d080718e0 0x7f5d0806c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.566+0000 7f5d06ffd700 1 -- 172.21.15.57:0/1025553621 >> v1:172.21.15.57:6789/0 conn(0x7f5d0806cdc0 legacy=0x7f5d0806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.566+0000 7f5d067fc700 1 -- 172.21.15.57:0/1025553621 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609432908 0 0) 0x7f5d081607a0 con 0x7f5d0806cdc0 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.566+0000 7f5d06ffd700 1 -- 172.21.15.57:0/1025553621 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d08160d30 con 0x7f5d0815c810 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.566+0000 7f5d06ffd700 1 --2- 172.21.15.57:0/1025553621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d0815c810 0x7f5d08160260 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80098f0 tx=0x7f5cf8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1904a10099ba26f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.566+0000 7f5d067fc700 1 -- 172.21.15.57:0/1025553621 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8011830 con 0x7f5d0815c810 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.566+0000 7f5d067fc700 1 -- 172.21.15.57:0/1025553621 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf8011990 con 0x7f5d0815c810 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.566+0000 7f5d067fc700 1 -- 172.21.15.57:0/1025553621 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8011d20 con 0x7f5d0815c810 2023-11-28T12:04:36.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.567+0000 7f5d0d94e700 1 -- 172.21.15.57:0/1025553621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d0815c810 msgr2=0x7f5d08160260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.567+0000 7f5d0d94e700 1 --2- 172.21.15.57:0/1025553621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d0815c810 0x7f5d08160260 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80098f0 tx=0x7f5cf8009c00 comp rx=0 tx=0).stop 2023-11-28T12:04:36.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.567+0000 7f5d0d94e700 1 -- 172.21.15.57:0/1025553621 shutdown_connections 2023-11-28T12:04:36.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.567+0000 7f5d0d94e700 1 --2- 172.21.15.57:0/1025553621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d0815c810 0x7f5d08160260 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.567+0000 7f5d0d94e700 1 --2- 172.21.15.57:0/1025553621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d080718e0 0x7f5d0806c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.567+0000 7f5d0d94e700 1 -- 172.21.15.57:0/1025553621 >> 172.21.15.57:0/1025553621 conn(0x7f5d0806c010 msgr2=0x7f5d080cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:36.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.567+0000 7f5d0d94e700 1 -- 172.21.15.57:0/1025553621 shutdown_connections 2023-11-28T12:04:36.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.567+0000 7f5d0d94e700 1 -- 172.21.15.57:0/1025553621 wait complete. 2023-11-28T12:04:36.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.568+0000 7f5d0d94e700 1 Processor -- start 2023-11-28T12:04:36.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.569+0000 7f5d0d94e700 1 -- start start 2023-11-28T12:04:36.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.569+0000 7f5d0d94e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d0806cdc0 0x7f5d08151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.569+0000 7f5d0d94e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d080718e0 0x7f5d080b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.569+0000 7f5d0d94e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0815c810 0x7f5d0815de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.569+0000 7f5d0d94e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080c5480 con 0x7f5d0806cdc0 2023-11-28T12:04:36.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.569+0000 7f5d0d94e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080c4610 con 0x7f5d0815c810 2023-11-28T12:04:36.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.569+0000 7f5d0d94e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080c48f0 con 0x7f5d080718e0 2023-11-28T12:04:36.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.569+0000 7f5d06ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d0806cdc0 0x7f5d08151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d06ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d0806cdc0 0x7f5d08151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48126/0 (socket says 172.21.15.57:48126) 2023-11-28T12:04:36.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d06ffd700 1 -- 172.21.15.57:0/41823297 learned_addr learned my addr 172.21.15.57:0/41823297 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:36.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5cfe7fc700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d080718e0 0x7f5d080b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d077fe700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0815c810 0x7f5d0815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d06ffd700 1 -- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d080718e0 msgr2=0x7f5d080b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d06ffd700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d080718e0 0x7f5d080b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d06ffd700 1 -- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0815c810 msgr2=0x7f5d0815de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d06ffd700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0815c810 0x7f5d0815de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d06ffd700 1 -- 172.21.15.57:0/41823297 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf8009550 con 0x7f5d0806cdc0 2023-11-28T12:04:36.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d077fe700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0815c810 0x7f5d0815de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:36.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d06ffd700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d0806cdc0 0x7f5d08151c40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009c40 tx=0x7f5cf8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:36.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d04ff9700 1 -- 172.21.15.57:0/41823297 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80045d0 con 0x7f5d0806cdc0 2023-11-28T12:04:36.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d04ff9700 1 -- 172.21.15.57:0/41823297 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf801e070 con 0x7f5d0806cdc0 2023-11-28T12:04:36.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d04ff9700 1 -- 172.21.15.57:0/41823297 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80038e0 con 0x7f5d0806cdc0 2023-11-28T12:04:36.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d0d94e700 1 -- 172.21.15.57:0/41823297 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d080ceb20 con 0x7f5d0806cdc0 2023-11-28T12:04:36.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.570+0000 7f5d0d94e700 1 -- 172.21.15.57:0/41823297 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d08161010 con 0x7f5d0806cdc0 2023-11-28T12:04:36.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.571+0000 7f5d0d94e700 1 -- 172.21.15.57:0/41823297 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d080ce740 con 0x7f5d0806cdc0 2023-11-28T12:04:36.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.572+0000 7f5d04ff9700 1 -- 172.21.15.57:0/41823297 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5cf800f050 con 0x7f5d0806cdc0 2023-11-28T12:04:36.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.572+0000 7f5d04ff9700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5ce80682f0 0x7f5ce806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.572+0000 7f5d04ff9700 1 -- 172.21.15.57:0/41823297 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5cf8086c50 con 0x7f5d0806cdc0 2023-11-28T12:04:36.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.572+0000 7f5cfe7fc700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5ce80682f0 0x7f5ce806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.573+0000 7f5cfe7fc700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5ce80682f0 0x7f5ce806a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009b70 tx=0x7f5cf0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:36.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.577+0000 7f5d04ff9700 1 -- 172.21.15.57:0/41823297 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5cf8053470 con 0x7f5d0806cdc0 2023-11-28T12:04:36.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.775+0000 7f5d0d94e700 1 -- 172.21.15.57:0/41823297 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d080743f0 con 0x7f5d0806cdc0 2023-11-28T12:04:36.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.776+0000 7f5d04ff9700 1 -- 172.21.15.57:0/41823297 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5cf80150a0 con 0x7f5d0806cdc0 2023-11-28T12:04:36.777 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:04:36.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.778+0000 7f5d0d94e700 1 -- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5ce80682f0 msgr2=0x7f5ce806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.778+0000 7f5d0d94e700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5ce80682f0 0x7f5ce806a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009b70 tx=0x7f5cf0008040 comp rx=0 tx=0).stop 2023-11-28T12:04:36.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.778+0000 7f5d0d94e700 1 -- 172.21.15.57:0/41823297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d0806cdc0 msgr2=0x7f5d08151c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.778+0000 7f5d0d94e700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d0806cdc0 0x7f5d08151c40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009c40 tx=0x7f5cf8004750 comp rx=0 tx=0).stop 2023-11-28T12:04:36.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.778+0000 7f5d0d94e700 1 -- 172.21.15.57:0/41823297 shutdown_connections 2023-11-28T12:04:36.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.778+0000 7f5d0d94e700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5ce80682f0 0x7f5ce806a7a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.778+0000 7f5d0d94e700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0815c810 0x7f5d0815de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.778+0000 7f5d0d94e700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d0806cdc0 0x7f5d08151c40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.778+0000 7f5d0d94e700 1 --2- 172.21.15.57:0/41823297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d080718e0 0x7f5d080b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.778+0000 7f5d0d94e700 1 -- 172.21.15.57:0/41823297 >> 172.21.15.57:0/41823297 conn(0x7f5d0806c010 msgr2=0x7f5d080cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:36.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.778+0000 7f5d0d94e700 1 -- 172.21.15.57:0/41823297 shutdown_connections 2023-11-28T12:04:36.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.778+0000 7f5d0d94e700 1 -- 172.21.15.57:0/41823297 wait complete. 2023-11-28T12:04:36.794 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:04:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.933+0000 7fd69df6e700 1 Processor -- start 2023-11-28T12:04:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.933+0000 7fd69df6e700 1 -- start start 2023-11-28T12:04:36.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd69df6e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd69815fe50 0x7fd69806d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd69df6e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6980b27d0 0x7fd6980b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd69df6e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd698161370 con 0x7fd6980b3b80 2023-11-28T12:04:36.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd69df6e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd698160220 con 0x7fd69815fe50 2023-11-28T12:04:36.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd69df6e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6980cf700 con 0x7fd6980b27d0 2023-11-28T12:04:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd6977fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd69815fe50 0x7fd69806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd6977fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd69815fe50 0x7fd69806d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48140/0 (socket says 172.21.15.57:48140) 2023-11-28T12:04:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd6977fe700 1 -- 172.21.15.57:0/1914736415 learned_addr learned my addr 172.21.15.57:0/1914736415 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd697fff700 1 --2- 172.21.15.57:0/1914736415 >> v2:172.21.15.181:3300/0 conn(0x7fd6980b27d0 0x7fd6980b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd6977fe700 1 -- 172.21.15.57:0/1914736415 >> v2:172.21.15.181:3300/0 conn(0x7fd6980b27d0 msgr2=0x7fd6980b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd6977fe700 1 --2- 172.21.15.57:0/1914736415 >> v2:172.21.15.181:3300/0 conn(0x7fd6980b27d0 0x7fd6980b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd6977fe700 1 -- 172.21.15.57:0/1914736415 >> v1:172.21.15.181:6789/0 conn(0x7fd6980b3b80 legacy=0x7fd698160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd6977fe700 1 -- 172.21.15.57:0/1914736415 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6980b3170 con 0x7fd69815fe50 2023-11-28T12:04:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.934+0000 7fd697fff700 1 --2- 172.21.15.57:0/1914736415 >> v2:172.21.15.181:3300/0 conn(0x7fd6980b27d0 0x7fd6980b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.935+0000 7fd6977fe700 1 --2- 172.21.15.57:0/1914736415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd69815fe50 0x7fd69806d170 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd698071b30 tx=0x7fd688009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baaf600c34daf5cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.935+0000 7fd6967fc700 1 -- 172.21.15.57:0/1914736415 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800b870 con 0x7fd69815fe50 2023-11-28T12:04:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.935+0000 7fd6967fc700 1 -- 172.21.15.57:0/1914736415 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68800b9d0 con 0x7fd69815fe50 2023-11-28T12:04:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.935+0000 7fd6967fc700 1 -- 172.21.15.57:0/1914736415 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800bd60 con 0x7fd69815fe50 2023-11-28T12:04:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.935+0000 7fd69df6e700 1 -- 172.21.15.57:0/1914736415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd69815fe50 msgr2=0x7fd69806d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.935+0000 7fd69df6e700 1 --2- 172.21.15.57:0/1914736415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd69815fe50 0x7fd69806d170 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd698071b30 tx=0x7fd688009bd0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.935+0000 7fd69df6e700 1 -- 172.21.15.57:0/1914736415 shutdown_connections 2023-11-28T12:04:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.935+0000 7fd69df6e700 1 --2- 172.21.15.57:0/1914736415 >> v2:172.21.15.181:3300/0 conn(0x7fd6980b27d0 0x7fd6980b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.935+0000 7fd69df6e700 1 --2- 172.21.15.57:0/1914736415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd69815fe50 0x7fd69806d170 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.935+0000 7fd69df6e700 1 -- 172.21.15.57:0/1914736415 >> 172.21.15.57:0/1914736415 conn(0x7fd69806ea80 msgr2=0x7fd69806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.935+0000 7fd69df6e700 1 -- 172.21.15.57:0/1914736415 shutdown_connections 2023-11-28T12:04:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.935+0000 7fd69df6e700 1 -- 172.21.15.57:0/1914736415 wait complete. 2023-11-28T12:04:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.937+0000 7fd69df6e700 1 Processor -- start 2023-11-28T12:04:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.937+0000 7fd69df6e700 1 -- start start 2023-11-28T12:04:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.938+0000 7fd69df6e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6980b27d0 0x7fd698155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.938+0000 7fd69df6e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6980b3b80 0x7fd6980c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.938+0000 7fd69df6e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd69815fe50 0x7fd69815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.938+0000 7fd69df6e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980ca170 con 0x7fd6980b3b80 2023-11-28T12:04:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.938+0000 7fd69df6e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980c8ec0 con 0x7fd6980b27d0 2023-11-28T12:04:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.938+0000 7fd69df6e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980c88b0 con 0x7fd69815fe50 2023-11-28T12:04:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.938+0000 7fd696ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6980b3b80 0x7fd6980c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.938+0000 7fd696ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6980b3b80 0x7fd6980c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48148/0 (socket says 172.21.15.57:48148) 2023-11-28T12:04:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.938+0000 7fd696ffd700 1 -- 172.21.15.57:0/2845694737 learned_addr learned my addr 172.21.15.57:0/2845694737 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd697fff700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd69815fe50 0x7fd69815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd6977fe700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6980b27d0 0x7fd698155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd696ffd700 1 -- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd69815fe50 msgr2=0x7fd69815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd696ffd700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd69815fe50 0x7fd69815aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd696ffd700 1 -- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6980b27d0 msgr2=0x7fd698155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd696ffd700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6980b27d0 0x7fd698155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd696ffd700 1 -- 172.21.15.57:0/2845694737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd688009550 con 0x7fd6980b3b80 2023-11-28T12:04:36.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd6977fe700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6980b27d0 0x7fd698155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:36.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd696ffd700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6980b3b80 0x7fd6980c9c30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c450 tx=0x7fd68000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:36.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd68ffff700 1 -- 172.21.15.57:0/2845694737 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6800052d0 con 0x7fd6980b3b80 2023-11-28T12:04:36.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd68ffff700 1 -- 172.21.15.57:0/2845694737 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd680005430 con 0x7fd6980b3b80 2023-11-28T12:04:36.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd68ffff700 1 -- 172.21.15.57:0/2845694737 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6800038c0 con 0x7fd6980b3b80 2023-11-28T12:04:36.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd69df6e700 1 -- 172.21.15.57:0/2845694737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6980cedd0 con 0x7fd6980b3b80 2023-11-28T12:04:36.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.939+0000 7fd69df6e700 1 -- 172.21.15.57:0/2845694737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6980b3fd0 con 0x7fd6980b3b80 2023-11-28T12:04:36.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.940+0000 7fd69df6e700 1 -- 172.21.15.57:0/2845694737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6980cf0b0 con 0x7fd6980b3b80 2023-11-28T12:04:36.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.941+0000 7fd68ffff700 1 -- 172.21.15.57:0/2845694737 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd680019030 con 0x7fd6980b3b80 2023-11-28T12:04:36.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.941+0000 7fd68ffff700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd6780682f0 0x7fd67806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:36.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.941+0000 7fd68ffff700 1 -- 172.21.15.57:0/2845694737 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd680084f00 con 0x7fd6980b3b80 2023-11-28T12:04:36.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.941+0000 7fd6977fe700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd6780682f0 0x7fd67806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:36.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.942+0000 7fd6977fe700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd6780682f0 0x7fd67806a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd688009130 tx=0x7fd688008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:36.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:36.946+0000 7fd68ffff700 1 -- 172.21.15.57:0/2845694737 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd680051690 con 0x7fd6980b3b80 2023-11-28T12:04:37.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.145+0000 7fd69df6e700 1 -- 172.21.15.57:0/2845694737 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6980743f0 con 0x7fd6980b3b80 2023-11-28T12:04:37.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.145+0000 7fd68ffff700 1 -- 172.21.15.57:0/2845694737 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6800170a0 con 0x7fd6980b3b80 2023-11-28T12:04:37.147 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:04:37.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.148+0000 7fd69df6e700 1 -- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd6780682f0 msgr2=0x7fd67806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.148+0000 7fd69df6e700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd6780682f0 0x7fd67806a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd688009130 tx=0x7fd688008030 comp rx=0 tx=0).stop 2023-11-28T12:04:37.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.148+0000 7fd69df6e700 1 -- 172.21.15.57:0/2845694737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6980b3b80 msgr2=0x7fd6980c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.148+0000 7fd69df6e700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6980b3b80 0x7fd6980c9c30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c450 tx=0x7fd68000c810 comp rx=0 tx=0).stop 2023-11-28T12:04:37.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.148+0000 7fd69df6e700 1 -- 172.21.15.57:0/2845694737 shutdown_connections 2023-11-28T12:04:37.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.148+0000 7fd69df6e700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd6780682f0 0x7fd67806a7a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.148+0000 7fd69df6e700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd69815fe50 0x7fd69815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.148+0000 7fd69df6e700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6980b27d0 0x7fd698155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.148+0000 7fd69df6e700 1 --2- 172.21.15.57:0/2845694737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6980b3b80 0x7fd6980c9c30 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.148+0000 7fd69df6e700 1 -- 172.21.15.57:0/2845694737 >> 172.21.15.57:0/2845694737 conn(0x7fd69806ea80 msgr2=0x7fd69815de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:37.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.148+0000 7fd69df6e700 1 -- 172.21.15.57:0/2845694737 shutdown_connections 2023-11-28T12:04:37.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.148+0000 7fd69df6e700 1 -- 172.21.15.57:0/2845694737 wait complete. 2023-11-28T12:04:37.162 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:04:37.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.300+0000 7ff85e51f700 1 Processor -- start 2023-11-28T12:04:37.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.300+0000 7ff85e51f700 1 -- start start 2023-11-28T12:04:37.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff85e51f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff858071890 0x7ff8580ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:37.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff85e51f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85815cc10 con 0x7ff85806a980 2023-11-28T12:04:37.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff85e51f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85806c9b0 con 0x7ff8580cf710 2023-11-28T12:04:37.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff85e51f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85806cc90 con 0x7ff858071890 2023-11-28T12:04:37.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff8577fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff85806a980 0x7ff858160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40746/0 (socket says 172.21.15.57:40746) 2023-11-28T12:04:37.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff8577fe700 1 -- 172.21.15.57:0/3742902414 learned_addr learned my addr 172.21.15.57:0/3742902414 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:37.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff85cabc700 1 --2- 172.21.15.57:0/3742902414 >> v2:172.21.15.181:3300/0 conn(0x7ff858071890 0x7ff8580ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:37.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff856ffd700 1 -- 172.21.15.57:0/3742902414 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426378093 0 0) 0x7ff85815cc10 con 0x7ff85806a980 2023-11-28T12:04:37.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff856ffd700 1 -- 172.21.15.57:0/3742902414 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8400035b0 con 0x7ff85806a980 2023-11-28T12:04:37.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff856ffd700 1 -- 172.21.15.57:0/3742902414 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471567488 0 0) 0x7ff85806c9b0 con 0x7ff8580cf710 2023-11-28T12:04:37.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff856ffd700 1 -- 172.21.15.57:0/3742902414 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff85815cc10 con 0x7ff8580cf710 2023-11-28T12:04:37.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff85cabc700 1 -- 172.21.15.57:0/3742902414 >> v1:172.21.15.181:6789/0 conn(0x7ff8580cf710 legacy=0x7ff85806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff85cabc700 1 -- 172.21.15.57:0/3742902414 >> v1:172.21.15.57:6789/0 conn(0x7ff85806a980 legacy=0x7ff858160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff85cabc700 1 -- 172.21.15.57:0/3742902414 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85806cf70 con 0x7ff858071890 2023-11-28T12:04:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.301+0000 7ff85cabc700 1 --2- 172.21.15.57:0/3742902414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff858071890 0x7ff8580ce740 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff850009d90 tx=0x7ff85000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=570c06bb50ed4496 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.302+0000 7ff856ffd700 1 -- 172.21.15.57:0/3742902414 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85000e070 con 0x7ff858071890 2023-11-28T12:04:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.302+0000 7ff856ffd700 1 -- 172.21.15.57:0/3742902414 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85000bb20 con 0x7ff858071890 2023-11-28T12:04:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.302+0000 7ff856ffd700 1 -- 172.21.15.57:0/3742902414 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85000be90 con 0x7ff858071890 2023-11-28T12:04:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.302+0000 7ff85e51f700 1 -- 172.21.15.57:0/3742902414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff858071890 msgr2=0x7ff8580ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.302+0000 7ff85e51f700 1 --2- 172.21.15.57:0/3742902414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff858071890 0x7ff8580ce740 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff850009d90 tx=0x7ff85000b330 comp rx=0 tx=0).stop 2023-11-28T12:04:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.302+0000 7ff85e51f700 1 -- 172.21.15.57:0/3742902414 shutdown_connections 2023-11-28T12:04:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.302+0000 7ff85e51f700 1 --2- 172.21.15.57:0/3742902414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff858071890 0x7ff8580ce740 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.302+0000 7ff85e51f700 1 -- 172.21.15.57:0/3742902414 >> 172.21.15.57:0/3742902414 conn(0x7ff85806c010 msgr2=0x7ff8580cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.302+0000 7ff85e51f700 1 -- 172.21.15.57:0/3742902414 shutdown_connections 2023-11-28T12:04:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.302+0000 7ff85e51f700 1 -- 172.21.15.57:0/3742902414 wait complete. 2023-11-28T12:04:37.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.304+0000 7ff85e51f700 1 Processor -- start 2023-11-28T12:04:37.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.304+0000 7ff85e51f700 1 -- start start 2023-11-28T12:04:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85e51f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff85806a980 0x7ff8580b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85e51f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff858071890 0x7ff85814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85e51f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8580cf710 0x7ff8580c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85e51f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8580b5200 con 0x7ff8580cf710 2023-11-28T12:04:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85e51f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8580b2ba0 con 0x7ff85806a980 2023-11-28T12:04:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85e51f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8580b2e80 con 0x7ff858071890 2023-11-28T12:04:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85cabc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8580cf710 0x7ff8580c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85cabc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8580cf710 0x7ff8580c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48156/0 (socket says 172.21.15.57:48156) 2023-11-28T12:04:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85cabc700 1 -- 172.21.15.57:0/350853952 learned_addr learned my addr 172.21.15.57:0/350853952 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff857fff700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff85806a980 0x7ff8580b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff8577fe700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff858071890 0x7ff85814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85cabc700 1 -- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff858071890 msgr2=0x7ff85814cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85cabc700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff858071890 0x7ff85814cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85cabc700 1 -- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff85806a980 msgr2=0x7ff8580b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85cabc700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff85806a980 0x7ff8580b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85cabc700 1 -- 172.21.15.57:0/350853952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff850009a70 con 0x7ff8580cf710 2023-11-28T12:04:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff8577fe700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff858071890 0x7ff85814cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff85cabc700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8580cf710 0x7ff8580c55d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff858067450 tx=0x7ff850004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.305+0000 7ff854ff9700 1 -- 172.21.15.57:0/350853952 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850004970 con 0x7ff8580cf710 2023-11-28T12:04:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.306+0000 7ff854ff9700 1 -- 172.21.15.57:0/350853952 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff850020070 con 0x7ff8580cf710 2023-11-28T12:04:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.306+0000 7ff85e51f700 1 -- 172.21.15.57:0/350853952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff85806d530 con 0x7ff8580cf710 2023-11-28T12:04:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.306+0000 7ff854ff9700 1 -- 172.21.15.57:0/350853952 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8500276a0 con 0x7ff8580cf710 2023-11-28T12:04:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.306+0000 7ff85e51f700 1 -- 172.21.15.57:0/350853952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85806d250 con 0x7ff8580cf710 2023-11-28T12:04:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.307+0000 7ff85e51f700 1 -- 172.21.15.57:0/350853952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8580743f0 con 0x7ff8580cf710 2023-11-28T12:04:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.307+0000 7ff854ff9700 1 -- 172.21.15.57:0/350853952 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff85002e070 con 0x7ff8580cf710 2023-11-28T12:04:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.307+0000 7ff854ff9700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff8440682f0 0x7ff84406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.307+0000 7ff854ff9700 1 -- 172.21.15.57:0/350853952 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff850022070 con 0x7ff8580cf710 2023-11-28T12:04:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.308+0000 7ff857fff700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff8440682f0 0x7ff84406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.308+0000 7ff857fff700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff8440682f0 0x7ff84406a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff8480011e0 tx=0x7ff848009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.312+0000 7ff854ff9700 1 -- 172.21.15.57:0/350853952 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff850054140 con 0x7ff8580cf710 2023-11-28T12:04:37.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.508+0000 7ff85e51f700 1 -- 172.21.15.57:0/350853952 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8580711a0 con 0x7ff8580cf710 2023-11-28T12:04:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.508+0000 7ff854ff9700 1 -- 172.21.15.57:0/350853952 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff850057760 con 0x7ff8580cf710 2023-11-28T12:04:37.510 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:04:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.511+0000 7ff85e51f700 1 -- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff8440682f0 msgr2=0x7ff84406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.511+0000 7ff85e51f700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff8440682f0 0x7ff84406a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff8480011e0 tx=0x7ff848009040 comp rx=0 tx=0).stop 2023-11-28T12:04:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.511+0000 7ff85e51f700 1 -- 172.21.15.57:0/350853952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8580cf710 msgr2=0x7ff8580c55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.511+0000 7ff85e51f700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8580cf710 0x7ff8580c55d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff858067450 tx=0x7ff850004ad0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.511+0000 7ff85e51f700 1 -- 172.21.15.57:0/350853952 shutdown_connections 2023-11-28T12:04:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.511+0000 7ff85e51f700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff8440682f0 0x7ff84406a7a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.511+0000 7ff85e51f700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8580cf710 0x7ff8580c55d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.511+0000 7ff85e51f700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff85806a980 0x7ff8580b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.511+0000 7ff85e51f700 1 --2- 172.21.15.57:0/350853952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff858071890 0x7ff85814cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.511+0000 7ff85e51f700 1 -- 172.21.15.57:0/350853952 >> 172.21.15.57:0/350853952 conn(0x7ff85806c010 msgr2=0x7ff85806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:37.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.511+0000 7ff85e51f700 1 -- 172.21.15.57:0/350853952 shutdown_connections 2023-11-28T12:04:37.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.511+0000 7ff85e51f700 1 -- 172.21.15.57:0/350853952 wait complete. 2023-11-28T12:04:37.525 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:04:37.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.662+0000 7f4eeba27700 1 Processor -- start 2023-11-28T12:04:37.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.662+0000 7f4eeba27700 1 -- start start 2023-11-28T12:04:37.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.663+0000 7f4eeba27700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ee406a980 0x7f4ee40cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:37.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.663+0000 7f4eeba27700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee40cdd50 0x7f4ee40cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:37.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.663+0000 7f4eeba27700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee415e0d0 con 0x7f4ee415a4d0 2023-11-28T12:04:37.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.663+0000 7f4eeba27700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee40ce560 con 0x7f4ee406a980 2023-11-28T12:04:37.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.663+0000 7f4eeba27700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee415e320 con 0x7f4ee40cdd50 2023-11-28T12:04:37.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.663+0000 7f4ee97c3700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4ee415a4d0 0x7f4ee406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40756/0 (socket says 172.21.15.57:40756) 2023-11-28T12:04:37.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.663+0000 7f4ee97c3700 1 -- 172.21.15.57:0/234001327 learned_addr learned my addr 172.21.15.57:0/234001327 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:37.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.663+0000 7f4ee9fc4700 1 --2- 172.21.15.57:0/234001327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee40cdd50 0x7f4ee40cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:37.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.664+0000 7f4ee8fc2700 1 --2- 172.21.15.57:0/234001327 >> v2:172.21.15.181:3300/0 conn(0x7f4ee406a980 0x7f4ee40cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:37.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.664+0000 7f4ed7fff700 1 -- 172.21.15.57:0/234001327 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933709915 0 0) 0x7f4ee415e0d0 con 0x7f4ee415a4d0 2023-11-28T12:04:37.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.664+0000 7f4ed7fff700 1 -- 172.21.15.57:0/234001327 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed00035b0 con 0x7f4ee415a4d0 2023-11-28T12:04:37.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.664+0000 7f4ee8fc2700 1 -- 172.21.15.57:0/234001327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee40cdd50 msgr2=0x7f4ee40cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:37.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.664+0000 7f4ee8fc2700 1 --2- 172.21.15.57:0/234001327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee40cdd50 0x7f4ee40cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.664+0000 7f4ee8fc2700 1 -- 172.21.15.57:0/234001327 >> v1:172.21.15.57:6789/0 conn(0x7f4ee415a4d0 legacy=0x7f4ee406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.664+0000 7f4ee8fc2700 1 -- 172.21.15.57:0/234001327 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee415e600 con 0x7f4ee406a980 2023-11-28T12:04:37.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.664+0000 7f4ee9fc4700 1 --2- 172.21.15.57:0/234001327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee40cdd50 0x7f4ee40cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:37.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.664+0000 7f4ee8fc2700 1 --2- 172.21.15.57:0/234001327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee406a980 0x7f4ee40cd810 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009840 tx=0x7f4ee0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ea70e6eb8af154f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:37.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.664+0000 7f4ed7fff700 1 -- 172.21.15.57:0/234001327 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0011830 con 0x7f4ee406a980 2023-11-28T12:04:37.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.664+0000 7f4ed7fff700 1 -- 172.21.15.57:0/234001327 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee0011990 con 0x7f4ee406a980 2023-11-28T12:04:37.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.664+0000 7f4ed7fff700 1 -- 172.21.15.57:0/234001327 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0011d20 con 0x7f4ee406a980 2023-11-28T12:04:37.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.665+0000 7f4eeba27700 1 -- 172.21.15.57:0/234001327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee406a980 msgr2=0x7f4ee40cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.665+0000 7f4eeba27700 1 --2- 172.21.15.57:0/234001327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee406a980 0x7f4ee40cd810 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009840 tx=0x7f4ee0009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:37.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.665+0000 7f4eeba27700 1 -- 172.21.15.57:0/234001327 shutdown_connections 2023-11-28T12:04:37.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.665+0000 7f4eeba27700 1 --2- 172.21.15.57:0/234001327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee40cdd50 0x7f4ee40cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.665+0000 7f4eeba27700 1 --2- 172.21.15.57:0/234001327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee406a980 0x7f4ee40cd810 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.665+0000 7f4eeba27700 1 -- 172.21.15.57:0/234001327 >> 172.21.15.57:0/234001327 conn(0x7f4ee406c010 msgr2=0x7f4ee40cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:37.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.665+0000 7f4eeba27700 1 -- 172.21.15.57:0/234001327 shutdown_connections 2023-11-28T12:04:37.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.665+0000 7f4eeba27700 1 -- 172.21.15.57:0/234001327 wait complete. 2023-11-28T12:04:37.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.666+0000 7f4eeba27700 1 Processor -- start 2023-11-28T12:04:37.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.666+0000 7f4eeba27700 1 -- start start 2023-11-28T12:04:37.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.667+0000 7f4eeba27700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee406a980 0x7f4ee40ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:37.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.667+0000 7f4eeba27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee40cdd50 0x7f4ee40ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:37.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.667+0000 7f4eeba27700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee415a4d0 0x7f4ee40ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:37.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.667+0000 7f4eeba27700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40ada20 con 0x7f4ee406a980 2023-11-28T12:04:37.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.667+0000 7f4eeba27700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40ad460 con 0x7f4ee40cdd50 2023-11-28T12:04:37.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.667+0000 7f4eeba27700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40ad740 con 0x7f4ee415a4d0 2023-11-28T12:04:37.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.667+0000 7f4ee97c3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee406a980 0x7f4ee40ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:37.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.667+0000 7f4ee97c3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee406a980 0x7f4ee40ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48166/0 (socket says 172.21.15.57:48166) 2023-11-28T12:04:37.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.667+0000 7f4ee97c3700 1 -- 172.21.15.57:0/1129248076 learned_addr learned my addr 172.21.15.57:0/1129248076 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:37.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.667+0000 7f4ee8fc2700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee40cdd50 0x7f4ee40ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:37.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.667+0000 7f4ee97c3700 1 -- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee415a4d0 msgr2=0x7f4ee40ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.667+0000 7f4ee9fc4700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee415a4d0 0x7f4ee40ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:37.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.668+0000 7f4ee97c3700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee415a4d0 0x7f4ee40ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.668+0000 7f4ee97c3700 1 -- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee40cdd50 msgr2=0x7f4ee40ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.668+0000 7f4ee97c3700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee40cdd50 0x7f4ee40ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.668+0000 7f4ee97c3700 1 -- 172.21.15.57:0/1129248076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc009a70 con 0x7f4ee406a980 2023-11-28T12:04:37.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.668+0000 7f4ee8fc2700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee40cdd50 0x7f4ee40ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:37.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.668+0000 7f4ee9fc4700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee415a4d0 0x7f4ee40ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:37.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.668+0000 7f4ee97c3700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee406a980 0x7f4ee40ae190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8001eb0 tx=0x7f4ed800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:37.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.668+0000 7f4ed5ffb700 1 -- 172.21.15.57:0/1129248076 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8005d70 con 0x7f4ee406a980 2023-11-28T12:04:37.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.668+0000 7f4ed5ffb700 1 -- 172.21.15.57:0/1129248076 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed80070d0 con 0x7f4ee406a980 2023-11-28T12:04:37.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.668+0000 7f4ed5ffb700 1 -- 172.21.15.57:0/1129248076 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8007460 con 0x7f4ee406a980 2023-11-28T12:04:37.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.668+0000 7f4eeba27700 1 -- 172.21.15.57:0/1129248076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee00094a0 con 0x7f4ee406a980 2023-11-28T12:04:37.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.668+0000 7f4eeba27700 1 -- 172.21.15.57:0/1129248076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee40ce1b0 con 0x7f4ee406a980 2023-11-28T12:04:37.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.669+0000 7f4eeba27700 1 -- 172.21.15.57:0/1129248076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee415e8e0 con 0x7f4ee406a980 2023-11-28T12:04:37.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.670+0000 7f4ed5ffb700 1 -- 172.21.15.57:0/1129248076 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4ed8019030 con 0x7f4ee406a980 2023-11-28T12:04:37.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.670+0000 7f4ed5ffb700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4ec80682f0 0x7f4ec806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:37.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.670+0000 7f4ed5ffb700 1 -- 172.21.15.57:0/1129248076 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4ed80850d0 con 0x7f4ee406a980 2023-11-28T12:04:37.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.670+0000 7f4ee8fc2700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4ec80682f0 0x7f4ec806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:37.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.671+0000 7f4ee8fc2700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4ec80682f0 0x7f4ec806a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40cd350 tx=0x7f4ee0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:37.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.674+0000 7f4ed5ffb700 1 -- 172.21.15.57:0/1129248076 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4ed80518f0 con 0x7f4ee406a980 2023-11-28T12:04:37.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.874+0000 7f4eeba27700 1 -- 172.21.15.57:0/1129248076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ee415a8a0 con 0x7f4ee406a980 2023-11-28T12:04:37.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.875+0000 7f4ed5ffb700 1 -- 172.21.15.57:0/1129248076 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ed8054f10 con 0x7f4ee406a980 2023-11-28T12:04:37.876 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:04:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.877+0000 7f4eeba27700 1 -- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4ec80682f0 msgr2=0x7f4ec806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.877+0000 7f4eeba27700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4ec80682f0 0x7f4ec806a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40cd350 tx=0x7f4ee0008030 comp rx=0 tx=0).stop 2023-11-28T12:04:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.877+0000 7f4eeba27700 1 -- 172.21.15.57:0/1129248076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee406a980 msgr2=0x7f4ee40ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.877+0000 7f4eeba27700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee406a980 0x7f4ee40ae190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8001eb0 tx=0x7f4ed800c860 comp rx=0 tx=0).stop 2023-11-28T12:04:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.877+0000 7f4eeba27700 1 -- 172.21.15.57:0/1129248076 shutdown_connections 2023-11-28T12:04:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.877+0000 7f4eeba27700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4ec80682f0 0x7f4ec806a7a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.877+0000 7f4eeba27700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee415a4d0 0x7f4ee40ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.877+0000 7f4eeba27700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee406a980 0x7f4ee40ae190 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.877+0000 7f4eeba27700 1 --2- 172.21.15.57:0/1129248076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee40cdd50 0x7f4ee40ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.877+0000 7f4eeba27700 1 -- 172.21.15.57:0/1129248076 >> 172.21.15.57:0/1129248076 conn(0x7f4ee406c010 msgr2=0x7f4ee406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:37.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.877+0000 7f4eeba27700 1 -- 172.21.15.57:0/1129248076 shutdown_connections 2023-11-28T12:04:37.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:37.877+0000 7f4eeba27700 1 -- 172.21.15.57:0/1129248076 wait complete. 2023-11-28T12:04:37.892 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:04:38.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.026+0000 7f74e52e6700 1 Processor -- start 2023-11-28T12:04:38.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.026+0000 7f74e52e6700 1 -- start start 2023-11-28T12:04:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74e52e6700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f74e00d0c60 0x7f74e00b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74e52e6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74e006e400 0x7f74e00d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74e52e6700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e00d1570 con 0x7f74e0160e70 2023-11-28T12:04:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74e52e6700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e006e900 con 0x7f74e00d0c60 2023-11-28T12:04:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74e52e6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e0160390 con 0x7f74e006e400 2023-11-28T12:04:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74deffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f74e00d0c60 0x7f74e00b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74df7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f74e0160e70 0x7f74e006f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40766/0 (socket says 172.21.15.57:40766) 2023-11-28T12:04:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74df7fe700 1 -- 172.21.15.57:0/3413349946 learned_addr learned my addr 172.21.15.57:0/3413349946 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74de7fc700 1 --2- 172.21.15.57:0/3413349946 >> v2:172.21.15.181:3300/0 conn(0x7f74e006e400 0x7f74e00d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74ddffb700 1 -- 172.21.15.57:0/3413349946 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589059120 0 0) 0x7f74e00d1570 con 0x7f74e0160e70 2023-11-28T12:04:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74ddffb700 1 -- 172.21.15.57:0/3413349946 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74cc0035b0 con 0x7f74e0160e70 2023-11-28T12:04:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74deffd700 1 -- 172.21.15.57:0/3413349946 >> v2:172.21.15.181:3300/0 conn(0x7f74e006e400 msgr2=0x7f74e00d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74deffd700 1 --2- 172.21.15.57:0/3413349946 >> v2:172.21.15.181:3300/0 conn(0x7f74e006e400 0x7f74e00d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74deffd700 1 -- 172.21.15.57:0/3413349946 >> v1:172.21.15.57:6789/0 conn(0x7f74e0160e70 legacy=0x7f74e006f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.027+0000 7f74deffd700 1 -- 172.21.15.57:0/3413349946 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e0160670 con 0x7f74e00d0c60 2023-11-28T12:04:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.028+0000 7f74deffd700 1 --2- 172.21.15.57:0/3413349946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e00d0c60 0x7f74e00b5be0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f74e0071a60 tx=0x7f74d0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38d78118b59130c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.028+0000 7f74ddffb700 1 -- 172.21.15.57:0/3413349946 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d0011830 con 0x7f74e00d0c60 2023-11-28T12:04:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.028+0000 7f74ddffb700 1 -- 172.21.15.57:0/3413349946 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d0011990 con 0x7f74e00d0c60 2023-11-28T12:04:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.028+0000 7f74ddffb700 1 -- 172.21.15.57:0/3413349946 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d0011d20 con 0x7f74e00d0c60 2023-11-28T12:04:38.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.028+0000 7f74e52e6700 1 -- 172.21.15.57:0/3413349946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e00d0c60 msgr2=0x7f74e00b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.028+0000 7f74e52e6700 1 --2- 172.21.15.57:0/3413349946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e00d0c60 0x7f74e00b5be0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f74e0071a60 tx=0x7f74d0009b20 comp rx=0 tx=0).stop 2023-11-28T12:04:38.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.028+0000 7f74e52e6700 1 -- 172.21.15.57:0/3413349946 shutdown_connections 2023-11-28T12:04:38.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.028+0000 7f74e52e6700 1 --2- 172.21.15.57:0/3413349946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e00d0c60 0x7f74e00b5be0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.028+0000 7f74e52e6700 1 --2- 172.21.15.57:0/3413349946 >> v2:172.21.15.181:3300/0 conn(0x7f74e006e400 0x7f74e00d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.028+0000 7f74e52e6700 1 -- 172.21.15.57:0/3413349946 >> 172.21.15.57:0/3413349946 conn(0x7f74e006a270 msgr2=0x7f74e006a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:38.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.028+0000 7f74e52e6700 1 -- 172.21.15.57:0/3413349946 shutdown_connections 2023-11-28T12:04:38.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.028+0000 7f74e52e6700 1 -- 172.21.15.57:0/3413349946 wait complete. 2023-11-28T12:04:38.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.030+0000 7f74e52e6700 1 Processor -- start 2023-11-28T12:04:38.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.030+0000 7f74e52e6700 1 -- start start 2023-11-28T12:04:38.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74e52e6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e006e400 0x7f74e006b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74e52e6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e00d0c60 0x7f74e01453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74e52e6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e0160e70 0x7f74e0145930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74e52e6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00cc100 con 0x7f74e00d0c60 2023-11-28T12:04:38.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74e52e6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00a8560 con 0x7f74e0160e70 2023-11-28T12:04:38.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74e52e6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00cb480 con 0x7f74e006e400 2023-11-28T12:04:38.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74de7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e00d0c60 0x7f74e01453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74de7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e00d0c60 0x7f74e01453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48182/0 (socket says 172.21.15.57:48182) 2023-11-28T12:04:38.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74de7fc700 1 -- 172.21.15.57:0/3102927699 learned_addr learned my addr 172.21.15.57:0/3102927699 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:38.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74df7fe700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e0160e70 0x7f74e0145930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74deffd700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e006e400 0x7f74e006b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74de7fc700 1 -- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e006e400 msgr2=0x7f74e006b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74de7fc700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e006e400 0x7f74e006b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74de7fc700 1 -- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e0160e70 msgr2=0x7f74e0145930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74de7fc700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e0160e70 0x7f74e0145930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74de7fc700 1 -- 172.21.15.57:0/3102927699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d00094a0 con 0x7f74e00d0c60 2023-11-28T12:04:38.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74df7fe700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e0160e70 0x7f74e0145930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:38.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74deffd700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e006e400 0x7f74e006b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:38.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.031+0000 7f74de7fc700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e00d0c60 0x7f74e01453f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e840 tx=0x7f74c800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:38.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.032+0000 7f74d77fe700 1 -- 172.21.15.57:0/3102927699 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c8009590 con 0x7f74e00d0c60 2023-11-28T12:04:38.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.032+0000 7f74d77fe700 1 -- 172.21.15.57:0/3102927699 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c8013680 con 0x7f74e00d0c60 2023-11-28T12:04:38.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.032+0000 7f74e52e6700 1 -- 172.21.15.57:0/3102927699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e00d0740 con 0x7f74e00d0c60 2023-11-28T12:04:38.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.032+0000 7f74d77fe700 1 -- 172.21.15.57:0/3102927699 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c8013a30 con 0x7f74e00d0c60 2023-11-28T12:04:38.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.032+0000 7f74e52e6700 1 -- 172.21.15.57:0/3102927699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e0160950 con 0x7f74e00d0c60 2023-11-28T12:04:38.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.033+0000 7f74e52e6700 1 -- 172.21.15.57:0/3102927699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e015d980 con 0x7f74e00d0c60 2023-11-28T12:04:38.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.033+0000 7f74d77fe700 1 -- 172.21.15.57:0/3102927699 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f74c8019030 con 0x7f74e00d0c60 2023-11-28T12:04:38.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.033+0000 7f74d77fe700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f74c00682f0 0x7f74c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.034+0000 7f74d77fe700 1 -- 172.21.15.57:0/3102927699 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f74c8085b70 con 0x7f74e00d0c60 2023-11-28T12:04:38.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.034+0000 7f74deffd700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f74c00682f0 0x7f74c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.034+0000 7f74deffd700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f74c00682f0 0x7f74c006a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f74d00090b0 tx=0x7f74d0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:38.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.038+0000 7f74d77fe700 1 -- 172.21.15.57:0/3102927699 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f74c8052390 con 0x7f74e00d0c60 2023-11-28T12:04:38.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.289+0000 7f74e52e6700 1 -- 172.21.15.57:0/3102927699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74e00cc240 con 0x7f74e00d0c60 2023-11-28T12:04:38.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.289+0000 7f74d77fe700 1 -- 172.21.15.57:0/3102927699 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f74e00cc240 con 0x7f74e00d0c60 2023-11-28T12:04:38.290 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:38.291 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1773283092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1773283092},{"type":"v1","addr":"172.21.15.181:6837","nonce":1773283092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/3718600227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3718600227},{"type":"v1","addr":"172.21.15.57:6834","nonce":3718600227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/762261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":762261630},{"type":"v1","addr":"172.21.15.57:6835","nonce":762261630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:03:47.263787+0000","modified":"2023-11-28T12:03:50.285215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3437917192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3437917192},{"type":"v1","addr":"172.21.15.181:6836","nonce":3437917192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T12:04:38.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.291+0000 7f74e52e6700 1 -- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f74c00682f0 msgr2=0x7f74c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.291+0000 7f74e52e6700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f74c00682f0 0x7f74c006a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f74d00090b0 tx=0x7f74d0008030 comp rx=0 tx=0).stop 2023-11-28T12:04:38.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.291+0000 7f74e52e6700 1 -- 172.21.15.57:0/3102927699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e00d0c60 msgr2=0x7f74e01453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.291+0000 7f74e52e6700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e00d0c60 0x7f74e01453f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e840 tx=0x7f74c800ec00 comp rx=0 tx=0).stop 2023-11-28T12:04:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.292+0000 7f74e52e6700 1 -- 172.21.15.57:0/3102927699 shutdown_connections 2023-11-28T12:04:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.292+0000 7f74e52e6700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f74c00682f0 0x7f74c006a7a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.292+0000 7f74e52e6700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e0160e70 0x7f74e0145930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.292+0000 7f74e52e6700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74e006e400 0x7f74e006b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.292+0000 7f74e52e6700 1 --2- 172.21.15.57:0/3102927699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e00d0c60 0x7f74e01453f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.292+0000 7f74e52e6700 1 -- 172.21.15.57:0/3102927699 >> 172.21.15.57:0/3102927699 conn(0x7f74e006a270 msgr2=0x7f74e00b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:38.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.292+0000 7f74e52e6700 1 -- 172.21.15.57:0/3102927699 shutdown_connections 2023-11-28T12:04:38.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.292+0000 7f74e52e6700 1 -- 172.21.15.57:0/3102927699 wait complete. 2023-11-28T12:04:38.296 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2023-11-28T12:04:38.306 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:04:38.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.441+0000 7fa28027d700 1 Processor -- start 2023-11-28T12:04:38.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.441+0000 7fa28027d700 1 -- start start 2023-11-28T12:04:38.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.442+0000 7fa28027d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa278163430 0x7fa278165810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.442+0000 7fa28027d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa278151a50 con 0x7fa27806a270 2023-11-28T12:04:38.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.442+0000 7fa28027d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27815b440 con 0x7fa278070d90 2023-11-28T12:04:38.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.442+0000 7fa28027d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27815f0c0 con 0x7fa278163430 2023-11-28T12:04:38.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.442+0000 7fa27e019700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa27806a270 0x7fa27806a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40784/0 (socket says 172.21.15.57:40784) 2023-11-28T12:04:38.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.442+0000 7fa27e019700 1 -- 172.21.15.57:0/1625951566 learned_addr learned my addr 172.21.15.57:0/1625951566 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:38.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.442+0000 7fa27e81a700 1 --2- 172.21.15.57:0/1625951566 >> v2:172.21.15.181:3300/0 conn(0x7fa278163430 0x7fa278165810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.442+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325573207 0 0) 0x7fa278151a50 con 0x7fa27806a270 2023-11-28T12:04:38.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.442+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2640035b0 con 0x7fa27806a270 2023-11-28T12:04:38.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1021674915 0 0) 0x7fa2640035b0 con 0x7fa27806a270 2023-11-28T12:04:38.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 >> v2:172.21.15.181:3300/0 conn(0x7fa278163430 msgr2=0x7fa278165810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:38.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d017700 1 --2- 172.21.15.57:0/1625951566 >> v2:172.21.15.181:3300/0 conn(0x7fa278163430 0x7fa278165810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 >> v1:172.21.15.181:6789/0 conn(0x7fa278070d90 legacy=0x7fa2780711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa278071ac0 con 0x7fa27806a270 2023-11-28T12:04:38.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fa274001670 con 0x7fa27806a270 2023-11-28T12:04:38.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 >> v1:172.21.15.57:6789/0 conn(0x7fa27806a270 legacy=0x7fa27806a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d017700 1 --2- 172.21.15.57:0/1625951566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa264003a00 0x7fa264003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d017700 1 --2- 172.21.15.57:0/1625951566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2640043f0 0x7fa2640048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d017700 1 --2- 172.21.15.57:0/1625951566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2640050b0 0x7fa26400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27815f0c0 con 0x7fa264003a00 2023-11-28T12:04:38.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26400c000 con 0x7fa2640050b0 2023-11-28T12:04:38.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26400c1c0 con 0x7fa2640043f0 2023-11-28T12:04:38.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27e81a700 1 --2- 172.21.15.57:0/1625951566 >> v2:172.21.15.181:3300/0 conn(0x7fa278163430 0x7fa278165810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:38.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27e019700 1 --2- 172.21.15.57:0/1625951566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa264003a00 0x7fa264003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d818700 1 --2- 172.21.15.57:0/1625951566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2640043f0 0x7fa2640048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27e81a700 1 --2- 172.21.15.57:0/1625951566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2640050b0 0x7fa26400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27e019700 1 -- 172.21.15.57:0/1625951566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2640043f0 msgr2=0x7fa2640048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:38.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27e019700 1 --2- 172.21.15.57:0/1625951566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2640043f0 0x7fa2640048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27e019700 1 -- 172.21.15.57:0/1625951566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2640050b0 msgr2=0x7fa26400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:38.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27e019700 1 --2- 172.21.15.57:0/1625951566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2640050b0 0x7fa26400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27e019700 1 -- 172.21.15.57:0/1625951566 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2640035b0 con 0x7fa264003a00 2023-11-28T12:04:38.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27e81a700 1 -- 172.21.15.57:0/1625951566 reap_dead start 2023-11-28T12:04:38.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27d818700 1 --2- 172.21.15.57:0/1625951566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2640043f0 0x7fa2640048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:38.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.443+0000 7fa27e019700 1 --2- 172.21.15.57:0/1625951566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa264003a00 0x7fa264003dd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa27400c2f0 tx=0x7fa264005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5901f6adbbcdba4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:38.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.444+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27400cdf0 con 0x7fa264003a00 2023-11-28T12:04:38.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.444+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa274005960 con 0x7fa264003a00 2023-11-28T12:04:38.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.444+0000 7fa27d017700 1 -- 172.21.15.57:0/1625951566 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274005d10 con 0x7fa264003a00 2023-11-28T12:04:38.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.444+0000 7fa28027d700 1 -- 172.21.15.57:0/1625951566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa264003a00 msgr2=0x7fa264003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.444+0000 7fa28027d700 1 --2- 172.21.15.57:0/1625951566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa264003a00 0x7fa264003dd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa27400c2f0 tx=0x7fa264005880 comp rx=0 tx=0).stop 2023-11-28T12:04:38.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.444+0000 7fa28027d700 1 -- 172.21.15.57:0/1625951566 shutdown_connections 2023-11-28T12:04:38.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.444+0000 7fa28027d700 1 --2- 172.21.15.57:0/1625951566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa264003a00 0x7fa264003dd0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.444+0000 7fa28027d700 1 -- 172.21.15.57:0/1625951566 >> 172.21.15.57:0/1625951566 conn(0x7fa27806c010 msgr2=0x7fa2780cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:38.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.444+0000 7fa28027d700 1 -- 172.21.15.57:0/1625951566 shutdown_connections 2023-11-28T12:04:38.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.444+0000 7fa28027d700 1 -- 172.21.15.57:0/1625951566 wait complete. 2023-11-28T12:04:38.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.446+0000 7fa28027d700 1 Processor -- start 2023-11-28T12:04:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.446+0000 7fa28027d700 1 -- start start 2023-11-28T12:04:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.447+0000 7fa28027d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa264003a00 0x7fa2780c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.447+0000 7fa28027d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa27814bc90 0x7fa2780c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.447+0000 7fa28027d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27814b0f0 0x7fa27814b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.447+0000 7fa28027d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27814bad0 con 0x7fa27814bc90 2023-11-28T12:04:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.447+0000 7fa28027d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2780c9ef0 con 0x7fa264003a00 2023-11-28T12:04:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.447+0000 7fa28027d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2780c9930 con 0x7fa27814b0f0 2023-11-28T12:04:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.447+0000 7fa27d818700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa27814bc90 0x7fa2780c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.447+0000 7fa27d818700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa27814bc90 0x7fa2780c74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48206/0 (socket says 172.21.15.57:48206) 2023-11-28T12:04:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.447+0000 7fa27d818700 1 -- 172.21.15.57:0/2184759315 learned_addr learned my addr 172.21.15.57:0/2184759315 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.447+0000 7fa27e019700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa264003a00 0x7fa2780c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.447+0000 7fa27e81a700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27814b0f0 0x7fa27814b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.448+0000 7fa27d818700 1 -- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27814b0f0 msgr2=0x7fa27814b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.448+0000 7fa27d818700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27814b0f0 0x7fa27814b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.448+0000 7fa27d818700 1 -- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa264003a00 msgr2=0x7fa2780c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.448+0000 7fa27d818700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa264003a00 0x7fa2780c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.448+0000 7fa27d818700 1 -- 172.21.15.57:0/2184759315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27815b0d0 con 0x7fa27814bc90 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.448+0000 7fa27e019700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa264003a00 0x7fa2780c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.448+0000 7fa27d818700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa27814bc90 0x7fa2780c74d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00e580 tx=0x7fa26c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.448+0000 7fa26affd700 1 -- 172.21.15.57:0/2184759315 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c0052e0 con 0x7fa27814bc90 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.448+0000 7fa26affd700 1 -- 172.21.15.57:0/2184759315 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c005440 con 0x7fa27814bc90 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.448+0000 7fa28027d700 1 -- 172.21.15.57:0/2184759315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27815adf0 con 0x7fa27814bc90 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.448+0000 7fa26affd700 1 -- 172.21.15.57:0/2184759315 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c0038c0 con 0x7fa27814bc90 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.448+0000 7fa28027d700 1 -- 172.21.15.57:0/2184759315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27815aac0 con 0x7fa27814bc90 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.449+0000 7fa28027d700 1 -- 172.21.15.57:0/2184759315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2780743f0 con 0x7fa27814bc90 2023-11-28T12:04:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.449+0000 7fa26affd700 1 -- 172.21.15.57:0/2184759315 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa26c01b030 con 0x7fa27814bc90 2023-11-28T12:04:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.450+0000 7fa26affd700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa25c0682f0 0x7fa25c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.450+0000 7fa26affd700 1 -- 172.21.15.57:0/2184759315 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa26c085240 con 0x7fa27814bc90 2023-11-28T12:04:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.450+0000 7fa27e019700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa25c0682f0 0x7fa25c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.451+0000 7fa27e019700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa25c0682f0 0x7fa25c06a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa274015040 tx=0x7fa274007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.454+0000 7fa26affd700 1 -- 172.21.15.57:0/2184759315 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa26c04d5e0 con 0x7fa27814bc90 2023-11-28T12:04:38.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.653+0000 7fa28027d700 1 -- 172.21.15.57:0/2184759315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2780cd520 con 0x7fa27814bc90 2023-11-28T12:04:38.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.654+0000 7fa26affd700 1 -- 172.21.15.57:0/2184759315 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+17961 (secure 0 0 0) 0x7fa26c051b10 con 0x7fa27814bc90 2023-11-28T12:04:38.655 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:38.656 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":51,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:04:34.775114+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:03:45.520415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T12:03:46.232534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2d","mappings":[{"from":0,"to":6}]},{"pgid":"8.37","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:04:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.656+0000 7fa28027d700 1 -- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa25c0682f0 msgr2=0x7fa25c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.656+0000 7fa28027d700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa25c0682f0 0x7fa25c06a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa274015040 tx=0x7fa274007000 comp rx=0 tx=0).stop 2023-11-28T12:04:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.656+0000 7fa28027d700 1 -- 172.21.15.57:0/2184759315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa27814bc90 msgr2=0x7fa2780c74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.656+0000 7fa28027d700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa27814bc90 0x7fa2780c74d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00e580 tx=0x7fa26c00e940 comp rx=0 tx=0).stop 2023-11-28T12:04:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.656+0000 7fa28027d700 1 -- 172.21.15.57:0/2184759315 shutdown_connections 2023-11-28T12:04:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.656+0000 7fa28027d700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa25c0682f0 0x7fa25c06a7a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.656+0000 7fa28027d700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa27814b0f0 0x7fa27814b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.656+0000 7fa28027d700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa264003a00 0x7fa2780c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.656+0000 7fa28027d700 1 --2- 172.21.15.57:0/2184759315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa27814bc90 0x7fa2780c74d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.656+0000 7fa28027d700 1 -- 172.21.15.57:0/2184759315 >> 172.21.15.57:0/2184759315 conn(0x7fa27806c010 msgr2=0x7fa2780b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.656+0000 7fa28027d700 1 -- 172.21.15.57:0/2184759315 shutdown_connections 2023-11-28T12:04:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.656+0000 7fa28027d700 1 -- 172.21.15.57:0/2184759315 wait complete. 2023-11-28T12:04:38.671 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T12:04:38.672 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:04:38.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.807+0000 7fcaccdee700 1 Processor -- start 2023-11-28T12:04:38.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.807+0000 7fcaccdee700 1 -- start start 2023-11-28T12:04:38.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcaccdee700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcac806e400 0x7fcac80b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcaccdee700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcac8160e70 0x7fcac815c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcaccdee700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac806e900 con 0x7fcac80d0c60 2023-11-28T12:04:38.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcaccdee700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac8160390 con 0x7fcac806e400 2023-11-28T12:04:38.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcaccdee700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac8160670 con 0x7fcac8160e70 2023-11-28T12:04:38.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcac5d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcac806e400 0x7fcac80b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcac659c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcac80d0c60 0x7fcac80d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40790/0 (socket says 172.21.15.57:40790) 2023-11-28T12:04:38.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcac659c700 1 -- 172.21.15.57:0/3338481528 learned_addr learned my addr 172.21.15.57:0/3338481528 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:38.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcac5d9b700 1 -- 172.21.15.57:0/3338481528 >> v2:172.21.15.181:3300/0 conn(0x7fcac8160e70 msgr2=0x7fcac815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:38.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcac5d9b700 1 --2- 172.21.15.57:0/3338481528 >> v2:172.21.15.181:3300/0 conn(0x7fcac8160e70 0x7fcac815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcac5d9b700 1 -- 172.21.15.57:0/3338481528 >> v1:172.21.15.57:6789/0 conn(0x7fcac80d0c60 legacy=0x7fcac80d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcac5d9b700 1 -- 172.21.15.57:0/3338481528 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac8160950 con 0x7fcac806e400 2023-11-28T12:04:38.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcac559a700 1 -- 172.21.15.57:0/3338481528 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782832267 0 0) 0x7fcac806e900 con 0x7fcac80d0c60 2023-11-28T12:04:38.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.808+0000 7fcac5d9b700 1 --2- 172.21.15.57:0/3338481528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcac806e400 0x7fcac80b5be0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcac8068250 tx=0x7fcabc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81f27302a12f62b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:38.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.809+0000 7fcac559a700 1 -- 172.21.15.57:0/3338481528 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc011830 con 0x7fcac806e400 2023-11-28T12:04:38.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.809+0000 7fcac559a700 1 -- 172.21.15.57:0/3338481528 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcabc011990 con 0x7fcac806e400 2023-11-28T12:04:38.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.809+0000 7fcac559a700 1 -- 172.21.15.57:0/3338481528 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc011d20 con 0x7fcac806e400 2023-11-28T12:04:38.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.809+0000 7fcaccdee700 1 -- 172.21.15.57:0/3338481528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcac806e400 msgr2=0x7fcac80b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.809+0000 7fcaccdee700 1 --2- 172.21.15.57:0/3338481528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcac806e400 0x7fcac80b5be0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcac8068250 tx=0x7fcabc009b20 comp rx=0 tx=0).stop 2023-11-28T12:04:38.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.809+0000 7fcaccdee700 1 -- 172.21.15.57:0/3338481528 shutdown_connections 2023-11-28T12:04:38.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.809+0000 7fcaccdee700 1 --2- 172.21.15.57:0/3338481528 >> v2:172.21.15.181:3300/0 conn(0x7fcac8160e70 0x7fcac815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.809+0000 7fcaccdee700 1 --2- 172.21.15.57:0/3338481528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcac806e400 0x7fcac80b5be0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.809+0000 7fcaccdee700 1 -- 172.21.15.57:0/3338481528 >> 172.21.15.57:0/3338481528 conn(0x7fcac806a270 msgr2=0x7fcac806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:38.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.809+0000 7fcaccdee700 1 -- 172.21.15.57:0/3338481528 shutdown_connections 2023-11-28T12:04:38.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.809+0000 7fcaccdee700 1 -- 172.21.15.57:0/3338481528 wait complete. 2023-11-28T12:04:38.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.811+0000 7fcaccdee700 1 Processor -- start 2023-11-28T12:04:38.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.811+0000 7fcaccdee700 1 -- start start 2023-11-28T12:04:38.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.811+0000 7fcaccdee700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcac806e400 0x7fcac81599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.811+0000 7fcaccdee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcac80d0c60 0x7fcac814be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.811+0000 7fcaccdee700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcac8160e70 0x7fcac80b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.811+0000 7fcaccdee700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80cef90 con 0x7fcac8160e70 2023-11-28T12:04:38.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.811+0000 7fcaccdee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac814c380 con 0x7fcac80d0c60 2023-11-28T12:04:38.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.811+0000 7fcaccdee700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80ce310 con 0x7fcac806e400 2023-11-28T12:04:38.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcac6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcac8160e70 0x7fcac80b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcac6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcac8160e70 0x7fcac80b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48216/0 (socket says 172.21.15.57:48216) 2023-11-28T12:04:38.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcac6d9d700 1 -- 172.21.15.57:0/2732730950 learned_addr learned my addr 172.21.15.57:0/2732730950 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:38.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcac659c700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcac806e400 0x7fcac81599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcac5d9b700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcac80d0c60 0x7fcac814be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcac6d9d700 1 -- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcac806e400 msgr2=0x7fcac81599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcac6d9d700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcac806e400 0x7fcac81599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcac6d9d700 1 -- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcac80d0c60 msgr2=0x7fcac814be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:38.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcac6d9d700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcac80d0c60 0x7fcac814be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:38.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcac6d9d700 1 -- 172.21.15.57:0/2732730950 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcabc0094a0 con 0x7fcac8160e70 2023-11-28T12:04:38.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcac6d9d700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcac8160e70 0x7fcac80b2aa0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcac000b9e0 tx=0x7fcac000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:38.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcab6ffd700 1 -- 172.21.15.57:0/2732730950 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac000d8e0 con 0x7fcac8160e70 2023-11-28T12:04:38.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcab6ffd700 1 -- 172.21.15.57:0/2732730950 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac000da60 con 0x7fcac8160e70 2023-11-28T12:04:38.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcaccdee700 1 -- 172.21.15.57:0/2732730950 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac815ceb0 con 0x7fcac8160e70 2023-11-28T12:04:38.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcab6ffd700 1 -- 172.21.15.57:0/2732730950 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac000de10 con 0x7fcac8160e70 2023-11-28T12:04:38.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.812+0000 7fcaccdee700 1 -- 172.21.15.57:0/2732730950 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac80d0740 con 0x7fcac8160e70 2023-11-28T12:04:38.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.813+0000 7fcaccdee700 1 -- 172.21.15.57:0/2732730950 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac815d040 con 0x7fcac8160e70 2023-11-28T12:04:38.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.814+0000 7fcab6ffd700 1 -- 172.21.15.57:0/2732730950 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcac0015420 con 0x7fcac8160e70 2023-11-28T12:04:38.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.814+0000 7fcab6ffd700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcaa80682f0 0x7fcaa806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:38.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.814+0000 7fcab6ffd700 1 -- 172.21.15.57:0/2732730950 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fcac0011070 con 0x7fcac8160e70 2023-11-28T12:04:38.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.815+0000 7fcac659c700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcaa80682f0 0x7fcaa806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:38.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.815+0000 7fcac659c700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcaa80682f0 0x7fcaa806a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcab8007fd0 tx=0x7fcab8007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:38.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:38.819+0000 7fcab6ffd700 1 -- 172.21.15.57:0/2732730950 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcac004d5e0 con 0x7fcac8160e70 2023-11-28T12:04:39.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.071+0000 7fcaccdee700 1 -- 172.21.15.57:0/2732730950 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcac80cf0d0 con 0x7fcac8160e70 2023-11-28T12:04:39.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.071+0000 7fcab6ffd700 1 -- 172.21.15.57:0/2732730950 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fcac80cf0d0 con 0x7fcac8160e70 2023-11-28T12:04:39.072 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:39.072 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1773283092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1773283092},{"type":"v1","addr":"172.21.15.181:6837","nonce":1773283092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/3718600227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3718600227},{"type":"v1","addr":"172.21.15.57:6834","nonce":3718600227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/762261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":762261630},{"type":"v1","addr":"172.21.15.57:6835","nonce":762261630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:03:47.263787+0000","modified":"2023-11-28T12:03:50.285215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3437917192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3437917192},{"type":"v1","addr":"172.21.15.181:6836","nonce":3437917192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T12:04:39.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.073+0000 7fcaccdee700 1 -- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcaa80682f0 msgr2=0x7fcaa806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.073+0000 7fcaccdee700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcaa80682f0 0x7fcaa806a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcab8007fd0 tx=0x7fcab8007de0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.073+0000 7fcaccdee700 1 -- 172.21.15.57:0/2732730950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcac8160e70 msgr2=0x7fcac80b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.073+0000 7fcaccdee700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcac8160e70 0x7fcac80b2aa0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcac000b9e0 tx=0x7fcac000bcf0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.074+0000 7fcaccdee700 1 -- 172.21.15.57:0/2732730950 shutdown_connections 2023-11-28T12:04:39.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.074+0000 7fcaccdee700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcaa80682f0 0x7fcaa806a7a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.074+0000 7fcaccdee700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcac8160e70 0x7fcac80b2aa0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.074+0000 7fcaccdee700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcac806e400 0x7fcac81599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.074+0000 7fcaccdee700 1 --2- 172.21.15.57:0/2732730950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcac80d0c60 0x7fcac814be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.074+0000 7fcaccdee700 1 -- 172.21.15.57:0/2732730950 >> 172.21.15.57:0/2732730950 conn(0x7fcac806a270 msgr2=0x7fcac806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:39.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.074+0000 7fcaccdee700 1 -- 172.21.15.57:0/2732730950 shutdown_connections 2023-11-28T12:04:39.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.074+0000 7fcaccdee700 1 -- 172.21.15.57:0/2732730950 wait complete. 2023-11-28T12:04:39.076 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2023-11-28T12:04:39.088 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:04:39.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.223+0000 7f6e3c426700 1 Processor -- start 2023-11-28T12:04:39.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.223+0000 7f6e3c426700 1 -- start start 2023-11-28T12:04:39.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.224+0000 7f6e3c426700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e340d0c60 0x7f6e340b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:39.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.224+0000 7f6e3c426700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e3406e400 0x7f6e340d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:39.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.224+0000 7f6e3c426700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f6e34160e70 0x7f6e3406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:39.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.224+0000 7f6e3c426700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e340d1570 con 0x7f6e34160e70 2023-11-28T12:04:39.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.224+0000 7f6e3c426700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e3406e900 con 0x7f6e340d0c60 2023-11-28T12:04:39.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.224+0000 7f6e3c426700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34160390 con 0x7f6e3406e400 2023-11-28T12:04:39.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.224+0000 7f6e3a9c3700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f6e34160e70 0x7f6e3406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:39.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.224+0000 7f6e3a9c3700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f6e34160e70 0x7f6e3406f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48226/0 (socket says 172.21.15.57:48226) 2023-11-28T12:04:39.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.224+0000 7f6e3a9c3700 1 -- 172.21.15.57:0/940968110 learned_addr learned my addr 172.21.15.57:0/940968110 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:39.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.225+0000 7f6e3a1c2700 1 --2- 172.21.15.57:0/940968110 >> v2:172.21.15.181:3300/0 conn(0x7f6e340d0c60 0x7f6e340b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:39.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.225+0000 7f6e399c1700 1 --2- 172.21.15.57:0/940968110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e3406e400 0x7f6e340d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:39.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.225+0000 7f6e3a9c3700 1 -- 172.21.15.57:0/940968110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e3406e400 msgr2=0x7f6e340d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:39.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.225+0000 7f6e3a9c3700 1 --2- 172.21.15.57:0/940968110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e3406e400 0x7f6e340d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.225+0000 7f6e3a9c3700 1 -- 172.21.15.57:0/940968110 >> v2:172.21.15.181:3300/0 conn(0x7f6e340d0c60 msgr2=0x7f6e340b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:39.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.225+0000 7f6e3a9c3700 1 --2- 172.21.15.57:0/940968110 >> v2:172.21.15.181:3300/0 conn(0x7f6e340d0c60 0x7f6e340b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.225+0000 7f6e3a9c3700 1 -- 172.21.15.57:0/940968110 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e34160670 con 0x7f6e34160e70 2023-11-28T12:04:39.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.225+0000 7f6e3a1c2700 1 --2- 172.21.15.57:0/940968110 >> v2:172.21.15.181:3300/0 conn(0x7f6e340d0c60 0x7f6e340b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:39.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.225+0000 7f6e3a9c3700 1 --2- 172.21.15.57:0/940968110 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e34160e70 0x7f6e3406f6b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6e34067340 tx=0x7f6e2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cf907c3cf081f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:39.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.225+0000 7f6e391c0700 1 -- 172.21.15.57:0/940968110 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c00e070 con 0x7f6e34160e70 2023-11-28T12:04:39.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.225+0000 7f6e391c0700 1 -- 172.21.15.57:0/940968110 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c00b9c0 con 0x7f6e34160e70 2023-11-28T12:04:39.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.225+0000 7f6e391c0700 1 -- 172.21.15.57:0/940968110 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c00bd70 con 0x7f6e34160e70 2023-11-28T12:04:39.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.226+0000 7f6e3c426700 1 -- 172.21.15.57:0/940968110 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e34160e70 msgr2=0x7f6e3406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.226+0000 7f6e3c426700 1 --2- 172.21.15.57:0/940968110 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e34160e70 0x7f6e3406f6b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6e34067340 tx=0x7f6e2c00b330 comp rx=0 tx=0).stop 2023-11-28T12:04:39.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.226+0000 7f6e3c426700 1 -- 172.21.15.57:0/940968110 shutdown_connections 2023-11-28T12:04:39.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.226+0000 7f6e3c426700 1 --2- 172.21.15.57:0/940968110 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e34160e70 0x7f6e3406f6b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.226+0000 7f6e3c426700 1 --2- 172.21.15.57:0/940968110 >> v2:172.21.15.181:3300/0 conn(0x7f6e340d0c60 0x7f6e340b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.226+0000 7f6e3c426700 1 --2- 172.21.15.57:0/940968110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e3406e400 0x7f6e340d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.226+0000 7f6e3c426700 1 -- 172.21.15.57:0/940968110 >> 172.21.15.57:0/940968110 conn(0x7f6e3406a270 msgr2=0x7f6e3406a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:39.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.226+0000 7f6e3c426700 1 -- 172.21.15.57:0/940968110 shutdown_connections 2023-11-28T12:04:39.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.226+0000 7f6e3c426700 1 -- 172.21.15.57:0/940968110 wait complete. 2023-11-28T12:04:39.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.227+0000 7f6e3c426700 1 Processor -- start 2023-11-28T12:04:39.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.227+0000 7f6e3c426700 1 -- start start 2023-11-28T12:04:39.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3c426700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e3406e400 0x7f6e3415c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:39.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3c426700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e340d0c60 0x7f6e34145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:39.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3c426700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e34160e70 0x7f6e34155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:39.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3c426700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34152380 con 0x7f6e3406e400 2023-11-28T12:04:39.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3c426700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340b1c40 con 0x7f6e340d0c60 2023-11-28T12:04:39.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3c426700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340b1f20 con 0x7f6e34160e70 2023-11-28T12:04:39.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3a1c2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e3406e400 0x7f6e3415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:39.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3a1c2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e3406e400 0x7f6e3415c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48234/0 (socket says 172.21.15.57:48234) 2023-11-28T12:04:39.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3a1c2700 1 -- 172.21.15.57:0/3527379796 learned_addr learned my addr 172.21.15.57:0/3527379796 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:39.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e399c1700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e340d0c60 0x7f6e34145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:39.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3a9c3700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e34160e70 0x7f6e34155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:39.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3a1c2700 1 -- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e34160e70 msgr2=0x7f6e34155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3a1c2700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e34160e70 0x7f6e34155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3a1c2700 1 -- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e340d0c60 msgr2=0x7f6e34145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3a1c2700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e340d0c60 0x7f6e34145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3a1c2700 1 -- 172.21.15.57:0/3527379796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c009b30 con 0x7f6e3406e400 2023-11-28T12:04:39.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.228+0000 7f6e3a9c3700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e34160e70 0x7f6e34155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:39.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.229+0000 7f6e3a1c2700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e3406e400 0x7f6e3415c970 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000e810 tx=0x7f6e3000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:39.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.229+0000 7f6e26ffd700 1 -- 172.21.15.57:0/3527379796 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30004230 con 0x7f6e3406e400 2023-11-28T12:04:39.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.229+0000 7f6e26ffd700 1 -- 172.21.15.57:0/3527379796 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e300043b0 con 0x7f6e3406e400 2023-11-28T12:04:39.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.229+0000 7f6e3c426700 1 -- 172.21.15.57:0/3527379796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e340d0740 con 0x7f6e3406e400 2023-11-28T12:04:39.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.229+0000 7f6e26ffd700 1 -- 172.21.15.57:0/3527379796 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e300038e0 con 0x7f6e3406e400 2023-11-28T12:04:39.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.229+0000 7f6e3c426700 1 -- 172.21.15.57:0/3527379796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e34160950 con 0x7f6e3406e400 2023-11-28T12:04:39.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.230+0000 7f6e3c426700 1 -- 172.21.15.57:0/3527379796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3406fbf0 con 0x7f6e3406e400 2023-11-28T12:04:39.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.230+0000 7f6e26ffd700 1 -- 172.21.15.57:0/3527379796 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6e30022070 con 0x7f6e3406e400 2023-11-28T12:04:39.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.231+0000 7f6e26ffd700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6e180682f0 0x7f6e1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:39.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.231+0000 7f6e26ffd700 1 -- 172.21.15.57:0/3527379796 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6e30085760 con 0x7f6e3406e400 2023-11-28T12:04:39.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.231+0000 7f6e399c1700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6e180682f0 0x7f6e1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:39.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.232+0000 7f6e399c1700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6e180682f0 0x7f6e1806a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6e34156250 tx=0x7f6e28008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:39.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.235+0000 7f6e26ffd700 1 -- 172.21.15.57:0/3527379796 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6e30051f80 con 0x7f6e3406e400 2023-11-28T12:04:39.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.486+0000 7f6e3c426700 1 -- 172.21.15.57:0/3527379796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e340743f0 con 0x7f6e3406e400 2023-11-28T12:04:39.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.487+0000 7f6e26ffd700 1 -- 172.21.15.57:0/3527379796 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f6e3000f040 con 0x7f6e3406e400 2023-11-28T12:04:39.488 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:39.488 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1773283092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1773283092},{"type":"v1","addr":"172.21.15.181:6837","nonce":1773283092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/3718600227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3718600227},{"type":"v1","addr":"172.21.15.57:6834","nonce":3718600227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/762261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":762261630},{"type":"v1","addr":"172.21.15.57:6835","nonce":762261630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:03:47.263787+0000","modified":"2023-11-28T12:03:50.285215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3437917192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3437917192},{"type":"v1","addr":"172.21.15.181:6836","nonce":3437917192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T12:04:39.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.489+0000 7f6e3c426700 1 -- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6e180682f0 msgr2=0x7f6e1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.489+0000 7f6e3c426700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6e180682f0 0x7f6e1806a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6e34156250 tx=0x7f6e28008040 comp rx=0 tx=0).stop 2023-11-28T12:04:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.489+0000 7f6e3c426700 1 -- 172.21.15.57:0/3527379796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e3406e400 msgr2=0x7f6e3415c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.489+0000 7f6e3c426700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e3406e400 0x7f6e3415c970 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000e810 tx=0x7f6e3000eb20 comp rx=0 tx=0).stop 2023-11-28T12:04:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.489+0000 7f6e3c426700 1 -- 172.21.15.57:0/3527379796 shutdown_connections 2023-11-28T12:04:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.489+0000 7f6e3c426700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6e180682f0 0x7f6e1806a7a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.489+0000 7f6e3c426700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e34160e70 0x7f6e34155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.489+0000 7f6e3c426700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e3406e400 0x7f6e3415c970 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.489+0000 7f6e3c426700 1 --2- 172.21.15.57:0/3527379796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e340d0c60 0x7f6e34145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.489+0000 7f6e3c426700 1 -- 172.21.15.57:0/3527379796 >> 172.21.15.57:0/3527379796 conn(0x7f6e3406a270 msgr2=0x7f6e34152ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:39.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.489+0000 7f6e3c426700 1 -- 172.21.15.57:0/3527379796 shutdown_connections 2023-11-28T12:04:39.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.489+0000 7f6e3c426700 1 -- 172.21.15.57:0/3527379796 wait complete. 2023-11-28T12:04:39.492 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2023-11-28T12:04:39.503 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:04:39.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.638+0000 7fe14140a700 1 Processor -- start 2023-11-28T12:04:39.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.638+0000 7fe14140a700 1 -- start start 2023-11-28T12:04:39.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.639+0000 7fe14140a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe13c06ab10 0x7fe13c06aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:39.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.639+0000 7fe14140a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13c0d1ee0 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.639+0000 7fe14140a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13c161120 con 0x7fe13c0d0450 2023-11-28T12:04:39.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.639+0000 7fe14140a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13c06b420 con 0x7fe13c06ab10 2023-11-28T12:04:39.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.639+0000 7fe13affd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe13c06ab10 0x7fe13c06aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:39.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.639+0000 7fe13a7fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe13c0cd5a0 0x7fe13c160300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40804/0 (socket says 172.21.15.57:40804) 2023-11-28T12:04:39.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.639+0000 7fe13a7fc700 1 -- 172.21.15.57:0/3293272900 learned_addr learned my addr 172.21.15.57:0/3293272900 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:39.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.639+0000 7fe13affd700 1 -- 172.21.15.57:0/3293272900 >> v1:172.21.15.181:6789/0 conn(0x7fe13c0d0450 legacy=0x7fe13c160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.639+0000 7fe13affd700 1 -- 172.21.15.57:0/3293272900 >> v1:172.21.15.57:6789/0 conn(0x7fe13c0cd5a0 legacy=0x7fe13c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.639+0000 7fe13affd700 1 -- 172.21.15.57:0/3293272900 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c0cdb10 con 0x7fe13c06ab10 2023-11-28T12:04:39.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.639+0000 7fe139ffb700 1 -- 172.21.15.57:0/3293272900 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183643761 0 0) 0x7fe13c0d1ee0 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.639+0000 7fe13affd700 1 --2- 172.21.15.57:0/3293272900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe13c06ab10 0x7fe13c06aee0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009840 tx=0x7fe12c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c170b66317ecb887 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:39.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.640+0000 7fe139ffb700 1 -- 172.21.15.57:0/3293272900 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c011830 con 0x7fe13c06ab10 2023-11-28T12:04:39.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.640+0000 7fe139ffb700 1 -- 172.21.15.57:0/3293272900 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12c011990 con 0x7fe13c06ab10 2023-11-28T12:04:39.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.640+0000 7fe139ffb700 1 -- 172.21.15.57:0/3293272900 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c011d20 con 0x7fe13c06ab10 2023-11-28T12:04:39.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.640+0000 7fe14140a700 1 -- 172.21.15.57:0/3293272900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe13c06ab10 msgr2=0x7fe13c06aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.640+0000 7fe14140a700 1 --2- 172.21.15.57:0/3293272900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe13c06ab10 0x7fe13c06aee0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009840 tx=0x7fe12c009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:39.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.640+0000 7fe14140a700 1 -- 172.21.15.57:0/3293272900 shutdown_connections 2023-11-28T12:04:39.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.640+0000 7fe14140a700 1 --2- 172.21.15.57:0/3293272900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe13c06ab10 0x7fe13c06aee0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.640+0000 7fe14140a700 1 -- 172.21.15.57:0/3293272900 >> 172.21.15.57:0/3293272900 conn(0x7fe13c06c520 msgr2=0x7fe13c06c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:39.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.640+0000 7fe14140a700 1 -- 172.21.15.57:0/3293272900 shutdown_connections 2023-11-28T12:04:39.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.640+0000 7fe14140a700 1 -- 172.21.15.57:0/3293272900 wait complete. 2023-11-28T12:04:39.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.642+0000 7fe14140a700 1 Processor -- start 2023-11-28T12:04:39.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.642+0000 7fe14140a700 1 -- start start 2023-11-28T12:04:39.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.642+0000 7fe14140a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe13c06ab10 0x7fe13c1513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:39.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.642+0000 7fe14140a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe13c0cd5a0 0x7fe13c0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:39.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.642+0000 7fe14140a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c0d0450 0x7fe13c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:39.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.642+0000 7fe14140a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c1543e0 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe14140a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c151910 con 0x7fe13c0d0450 2023-11-28T12:04:39.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe14140a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c158920 con 0x7fe13c06ab10 2023-11-28T12:04:39.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe13a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe13c0cd5a0 0x7fe13c0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:39.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe13a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe13c0cd5a0 0x7fe13c0d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48254/0 (socket says 172.21.15.57:48254) 2023-11-28T12:04:39.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe13a7fc700 1 -- 172.21.15.57:0/2511397949 learned_addr learned my addr 172.21.15.57:0/2511397949 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:39.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe13affd700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe13c06ab10 0x7fe13c1513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:39.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe13b7fe700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c0d0450 0x7fe13c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:39.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe13a7fc700 1 -- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe13c06ab10 msgr2=0x7fe13c1513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe13a7fc700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe13c06ab10 0x7fe13c1513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe13a7fc700 1 -- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c0d0450 msgr2=0x7fe13c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe13a7fc700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c0d0450 0x7fe13c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe13a7fc700 1 -- 172.21.15.57:0/2511397949 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c0094a0 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe13a7fc700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe13c0cd5a0 0x7fe13c0d0b20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe130001eb0 tx=0x7fe13000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe1237fe700 1 -- 172.21.15.57:0/2511397949 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130004c80 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe1237fe700 1 -- 172.21.15.57:0/2511397949 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe130004de0 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe14140a700 1 -- 172.21.15.57:0/2511397949 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c0d1b70 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.643+0000 7fe1237fe700 1 -- 172.21.15.57:0/2511397949 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130007310 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.644+0000 7fe14140a700 1 -- 172.21.15.57:0/2511397949 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13c0d1890 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.644+0000 7fe14140a700 1 -- 172.21.15.57:0/2511397949 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13c15cad0 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.645+0000 7fe1237fe700 1 -- 172.21.15.57:0/2511397949 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe13001a020 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.646+0000 7fe1237fe700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1280682f0 0x7fe12806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:39.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.646+0000 7fe1237fe700 1 -- 172.21.15.57:0/2511397949 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe130085ee0 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.646+0000 7fe13affd700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1280682f0 0x7fe12806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:39.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.647+0000 7fe13affd700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1280682f0 0x7fe12806a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe12c0090b0 tx=0x7fe12c01c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:39.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.650+0000 7fe1237fe700 1 -- 172.21.15.57:0/2511397949 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe130052610 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.850+0000 7fe14140a700 1 -- 172.21.15.57:0/2511397949 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe13c0743f0 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.852+0000 7fe1237fe700 1 -- 172.21.15.57:0/2511397949 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+17961 (secure 0 0 0) 0x7fe130055c30 con 0x7fe13c0cd5a0 2023-11-28T12:04:39.853 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:39.853 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":51,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:04:34.775114+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:03:45.520415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T12:03:46.232534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2d","mappings":[{"from":0,"to":6}]},{"pgid":"8.37","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:04:39.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.854+0000 7fe14140a700 1 -- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1280682f0 msgr2=0x7fe12806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.854+0000 7fe14140a700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1280682f0 0x7fe12806a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe12c0090b0 tx=0x7fe12c01c040 comp rx=0 tx=0).stop 2023-11-28T12:04:39.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.854+0000 7fe14140a700 1 -- 172.21.15.57:0/2511397949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe13c0cd5a0 msgr2=0x7fe13c0d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:39.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.854+0000 7fe14140a700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe13c0cd5a0 0x7fe13c0d0b20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe130001eb0 tx=0x7fe13000c800 comp rx=0 tx=0).stop 2023-11-28T12:04:39.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.854+0000 7fe14140a700 1 -- 172.21.15.57:0/2511397949 shutdown_connections 2023-11-28T12:04:39.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.854+0000 7fe14140a700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1280682f0 0x7fe12806a7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.854+0000 7fe14140a700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c0d0450 0x7fe13c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.854+0000 7fe14140a700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe13c06ab10 0x7fe13c1513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.854+0000 7fe14140a700 1 --2- 172.21.15.57:0/2511397949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe13c0cd5a0 0x7fe13c0d0b20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:39.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.854+0000 7fe14140a700 1 -- 172.21.15.57:0/2511397949 >> 172.21.15.57:0/2511397949 conn(0x7fe13c06c520 msgr2=0x7fe13c0b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:39.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.854+0000 7fe14140a700 1 -- 172.21.15.57:0/2511397949 shutdown_connections 2023-11-28T12:04:39.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:39.854+0000 7fe14140a700 1 -- 172.21.15.57:0/2511397949 wait complete. 2023-11-28T12:04:39.869 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T12:04:40.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.005+0000 7f2fa072d700 1 Processor -- start 2023-11-28T12:04:40.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.005+0000 7f2fa072d700 1 -- start start 2023-11-28T12:04:40.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.006+0000 7f2fa072d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f980cf710 0x7f2f980ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.006+0000 7f2fa072d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f9815cc10 con 0x7f2f9806a980 2023-11-28T12:04:40.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.006+0000 7f2fa072d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f9806c9b0 con 0x7f2f98071890 2023-11-28T12:04:40.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.006+0000 7f2fa072d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9806cc90 con 0x7f2f980cf710 2023-11-28T12:04:40.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.006+0000 7f2f9ecca700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2f9806a980 0x7f2f98160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40810/0 (socket says 172.21.15.57:40810) 2023-11-28T12:04:40.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.006+0000 7f2f9ecca700 1 -- 172.21.15.57:0/3514545856 learned_addr learned my addr 172.21.15.57:0/3514545856 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:40.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.006+0000 7f2f9e4c9700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f980cf710 0x7f2f980ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.006+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082067667 0 0) 0x7f2f9815cc10 con 0x7f2f9806a980 2023-11-28T12:04:40.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f840035b0 con 0x7f2f9806a980 2023-11-28T12:04:40.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360752750 0 0) 0x7f2f9806c9b0 con 0x7f2f98071890 2023-11-28T12:04:40.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f9815cc10 con 0x7f2f98071890 2023-11-28T12:04:40.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 452273639 0 0) 0x7f2f840035b0 con 0x7f2f9806a980 2023-11-28T12:04:40.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f980cf710 msgr2=0x7f2f980ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:40.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f980cf710 0x7f2f980ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 >> v1:172.21.15.181:6789/0 conn(0x7f2f98071890 legacy=0x7f2f9806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:40.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9806cf70 con 0x7f2f9806a980 2023-11-28T12:04:40.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f2f900019f0 con 0x7f2f9806a980 2023-11-28T12:04:40.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 >> v1:172.21.15.57:6789/0 conn(0x7f2f9806a980 legacy=0x7f2f98160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:40.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f84003bc0 0x7f2f84003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f84004850 0x7f2f84004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f84005200 0x7f2f8400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9806cc90 con 0x7f2f84004850 2023-11-28T12:04:40.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8400c0e0 con 0x7f2f84005200 2023-11-28T12:04:40.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8400c270 con 0x7f2f84003bc0 2023-11-28T12:04:40.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9e4c9700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f980cf710 0x7f2f980ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:40.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9dcc8700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f84004850 0x7f2f84004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9ecca700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f84005200 0x7f2f8400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9e4c9700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f84003bc0 0x7f2f84003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9dcc8700 1 -- 172.21.15.57:0/3514545856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f84003bc0 msgr2=0x7f2f84003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:40.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9dcc8700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f84003bc0 0x7f2f84003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9dcc8700 1 -- 172.21.15.57:0/3514545856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f84005200 msgr2=0x7f2f8400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:40.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9dcc8700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f84005200 0x7f2f8400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9dcc8700 1 -- 172.21.15.57:0/3514545856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f840035b0 con 0x7f2f84004850 2023-11-28T12:04:40.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.007+0000 7f2f9e4c9700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f84003bc0 0x7f2f84003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:40.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.008+0000 7f2f9dcc8700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f84004850 0x7f2f84004cc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2f88001930 tx=0x7f2f84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eec41ab15e681eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:40.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.008+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f880052d0 con 0x7f2f84004850 2023-11-28T12:04:40.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.008+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f88005430 con 0x7f2f84004850 2023-11-28T12:04:40.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.008+0000 7f2f9d4c7700 1 -- 172.21.15.57:0/3514545856 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f880038c0 con 0x7f2f84004850 2023-11-28T12:04:40.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.008+0000 7f2fa072d700 1 -- 172.21.15.57:0/3514545856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f84004850 msgr2=0x7f2f84004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:40.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.008+0000 7f2fa072d700 1 --2- 172.21.15.57:0/3514545856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f84004850 0x7f2f84004cc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2f88001930 tx=0x7f2f84005880 comp rx=0 tx=0).stop 2023-11-28T12:04:40.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.008+0000 7f2f9ecca700 1 -- 172.21.15.57:0/3514545856 reap_dead start 2023-11-28T12:04:40.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.008+0000 7f2fa072d700 1 -- 172.21.15.57:0/3514545856 shutdown_connections 2023-11-28T12:04:40.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.008+0000 7f2fa072d700 1 -- 172.21.15.57:0/3514545856 >> 172.21.15.57:0/3514545856 conn(0x7f2f9806c010 msgr2=0x7f2f980cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:40.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.008+0000 7f2fa072d700 1 -- 172.21.15.57:0/3514545856 shutdown_connections 2023-11-28T12:04:40.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.008+0000 7f2fa072d700 1 -- 172.21.15.57:0/3514545856 wait complete. 2023-11-28T12:04:40.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.010+0000 7f2fa072d700 1 Processor -- start 2023-11-28T12:04:40.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.010+0000 7f2fa072d700 1 -- start start 2023-11-28T12:04:40.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.010+0000 7f2fa072d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f84004850 0x7f2f98158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.010+0000 7f2fa072d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f9814d0b0 0x7f2f980b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.010+0000 7f2fa072d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f980b4f50 0x7f2f98156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.010+0000 7f2fa072d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f98158d50 con 0x7f2f84004850 2023-11-28T12:04:40.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.010+0000 7f2fa072d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980b53f0 con 0x7f2f9814d0b0 2023-11-28T12:04:40.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.010+0000 7f2fa072d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f98158380 con 0x7f2f980b4f50 2023-11-28T12:04:40.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f9e4c9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f84004850 0x7f2f98158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f9e4c9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f84004850 0x7f2f98158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48272/0 (socket says 172.21.15.57:48272) 2023-11-28T12:04:40.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f9e4c9700 1 -- 172.21.15.57:0/3442119076 learned_addr learned my addr 172.21.15.57:0/3442119076 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:40.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f9dcc8700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f9814d0b0 0x7f2f980b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f9ecca700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f980b4f50 0x7f2f98156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f9e4c9700 1 -- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f980b4f50 msgr2=0x7f2f98156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:40.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f9e4c9700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f980b4f50 0x7f2f98156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f9e4c9700 1 -- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f9814d0b0 msgr2=0x7f2f980b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:40.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f9e4c9700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f9814d0b0 0x7f2f980b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f9e4c9700 1 -- 172.21.15.57:0/3442119076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9806d530 con 0x7f2f84004850 2023-11-28T12:04:40.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f9e4c9700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f84004850 0x7f2f98158810 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2f940109e0 tx=0x7f2f94010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:40.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f8effd700 1 -- 172.21.15.57:0/3442119076 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94010e80 con 0x7f2f84004850 2023-11-28T12:04:40.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f8effd700 1 -- 172.21.15.57:0/3442119076 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9400e870 con 0x7f2f84004850 2023-11-28T12:04:40.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2fa072d700 1 -- 172.21.15.57:0/3442119076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9806d250 con 0x7f2f84004850 2023-11-28T12:04:40.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2f8effd700 1 -- 172.21.15.57:0/3442119076 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9400ebe0 con 0x7f2f84004850 2023-11-28T12:04:40.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.011+0000 7f2fa072d700 1 -- 172.21.15.57:0/3442119076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f980ce740 con 0x7f2f84004850 2023-11-28T12:04:40.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.012+0000 7f2fa072d700 1 -- 172.21.15.57:0/3442119076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f980743f0 con 0x7f2f84004850 2023-11-28T12:04:40.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.013+0000 7f2f8effd700 1 -- 172.21.15.57:0/3442119076 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2f9401c030 con 0x7f2f84004850 2023-11-28T12:04:40.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.013+0000 7f2f8effd700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2f7c0682f0 0x7f2f7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.013+0000 7f2f8effd700 1 -- 172.21.15.57:0/3442119076 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2f94085e10 con 0x7f2f84004850 2023-11-28T12:04:40.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.014+0000 7f2f9dcc8700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2f7c0682f0 0x7f2f7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.014+0000 7f2f9dcc8700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2f7c0682f0 0x7f2f7c06a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005040 tx=0x7f2f88009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:40.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.018+0000 7f2f8effd700 1 -- 172.21.15.57:0/3442119076 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2f940525d0 con 0x7f2f84004850 2023-11-28T12:04:40.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.261+0000 7f2fa072d700 1 -- 172.21.15.57:0/3442119076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2f980b5b50 con 0x7f2f84004850 2023-11-28T12:04:40.801 INFO:tasks.ceph.mon.a.smithi057.stderr:2023-11-28T12:04:40.800+0000 7fb777740700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T12:04:40.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.806+0000 7f2f8effd700 1 -- 172.21.15.57:0/3442119076 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2f94055bf0 con 0x7f2f84004850 2023-11-28T12:04:40.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.809+0000 7f2fa072d700 1 -- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2f7c0682f0 msgr2=0x7f2f7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.809+0000 7f2fa072d700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2f7c0682f0 0x7f2f7c06a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005040 tx=0x7f2f88009030 comp rx=0 tx=0).stop 2023-11-28T12:04:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.809+0000 7f2fa072d700 1 -- 172.21.15.57:0/3442119076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f84004850 msgr2=0x7f2f98158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.809+0000 7f2fa072d700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f84004850 0x7f2f98158810 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2f940109e0 tx=0x7f2f94010cf0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.809+0000 7f2fa072d700 1 -- 172.21.15.57:0/3442119076 shutdown_connections 2023-11-28T12:04:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.809+0000 7f2fa072d700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2f7c0682f0 0x7f2f7c06a7a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.809+0000 7f2fa072d700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f980b4f50 0x7f2f98156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.809+0000 7f2fa072d700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f84004850 0x7f2f98158810 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.809+0000 7f2fa072d700 1 --2- 172.21.15.57:0/3442119076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f9814d0b0 0x7f2f980b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.809+0000 7f2fa072d700 1 -- 172.21.15.57:0/3442119076 >> 172.21.15.57:0/3442119076 conn(0x7f2f9806c010 msgr2=0x7f2f980b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.809+0000 7f2fa072d700 1 -- 172.21.15.57:0/3442119076 shutdown_connections 2023-11-28T12:04:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.809+0000 7f2fa072d700 1 -- 172.21.15.57:0/3442119076 wait complete. 2023-11-28T12:04:40.812 INFO:tasks.ceph.mds.b.smithi181.stderr: -4618> 2023-11-28T12:03:52.820+0000 7f4db2f28700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T12:04:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T12:04:40.814 INFO:tasks.ceph.mds.b.smithi181.stderr: -3000> 2023-11-28T12:04:01.573+0000 7f4daf721700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5913=0-4194304@1} caps={5913=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5913 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x56429dce3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T12:04:40.826 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T12:04:40.844 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-11-28T12:04:40.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.965+0000 7f285f3a0700 1 Processor -- start 2023-11-28T12:04:40.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.966+0000 7f285f3a0700 1 -- start start 2023-11-28T12:04:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.966+0000 7f285f3a0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f285815a5f0 0x7f28580d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.966+0000 7f285f3a0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f28580cc200 0x7f28580b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.966+0000 7f285f3a0700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285815a9c0 con 0x7f28580b2d10 2023-11-28T12:04:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.966+0000 7f285f3a0700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28580b3190 con 0x7f28580cc200 2023-11-28T12:04:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.966+0000 7f285f3a0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28580c7b80 con 0x7f285815a5f0 2023-11-28T12:04:40.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.967+0000 7f285c93b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f28580cc200 0x7f28580b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.967+0000 7f285d93d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f28580b2d10 0x7f285815e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40822/0 (socket says 172.21.15.57:40822) 2023-11-28T12:04:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.967+0000 7f285d93d700 1 -- 172.21.15.57:0/1553959815 learned_addr learned my addr 172.21.15.57:0/1553959815 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.967+0000 7f285d13c700 1 --2- 172.21.15.57:0/1553959815 >> v2:172.21.15.181:3300/0 conn(0x7f285815a5f0 0x7f28580d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.967+0000 7f285c93b700 1 -- 172.21.15.57:0/1553959815 >> v2:172.21.15.181:3300/0 conn(0x7f285815a5f0 msgr2=0x7f28580d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.967+0000 7f285c93b700 1 --2- 172.21.15.57:0/1553959815 >> v2:172.21.15.181:3300/0 conn(0x7f285815a5f0 0x7f28580d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.967+0000 7f285c93b700 1 -- 172.21.15.57:0/1553959815 >> v1:172.21.15.57:6789/0 conn(0x7f28580b2d10 legacy=0x7f285815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.967+0000 7f285c93b700 1 -- 172.21.15.57:0/1553959815 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28580cc990 con 0x7f28580cc200 2023-11-28T12:04:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.967+0000 7f284bfff700 1 -- 172.21.15.57:0/1553959815 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302449228 0 0) 0x7f285815a9c0 con 0x7f28580b2d10 2023-11-28T12:04:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.967+0000 7f285c93b700 1 --2- 172.21.15.57:0/1553959815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28580cc200 0x7f28580b27d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f28580d1d40 tx=0x7f284c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c24c05b7126a4d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.967+0000 7f284bfff700 1 -- 172.21.15.57:0/1553959815 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c011830 con 0x7f28580cc200 2023-11-28T12:04:40.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.967+0000 7f284bfff700 1 -- 172.21.15.57:0/1553959815 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c011990 con 0x7f28580cc200 2023-11-28T12:04:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.967+0000 7f284bfff700 1 -- 172.21.15.57:0/1553959815 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c011d20 con 0x7f28580cc200 2023-11-28T12:04:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.968+0000 7f285f3a0700 1 -- 172.21.15.57:0/1553959815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28580cc200 msgr2=0x7f28580b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.968+0000 7f285f3a0700 1 --2- 172.21.15.57:0/1553959815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28580cc200 0x7f28580b27d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f28580d1d40 tx=0x7f284c009b20 comp rx=0 tx=0).stop 2023-11-28T12:04:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.968+0000 7f285f3a0700 1 -- 172.21.15.57:0/1553959815 shutdown_connections 2023-11-28T12:04:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.968+0000 7f285f3a0700 1 --2- 172.21.15.57:0/1553959815 >> v2:172.21.15.181:3300/0 conn(0x7f285815a5f0 0x7f28580d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.968+0000 7f285f3a0700 1 --2- 172.21.15.57:0/1553959815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28580cc200 0x7f28580b27d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.968+0000 7f285f3a0700 1 -- 172.21.15.57:0/1553959815 >> 172.21.15.57:0/1553959815 conn(0x7f285806ea80 msgr2=0x7f285806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.968+0000 7f285f3a0700 1 -- 172.21.15.57:0/1553959815 shutdown_connections 2023-11-28T12:04:40.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.968+0000 7f285f3a0700 1 -- 172.21.15.57:0/1553959815 wait complete. 2023-11-28T12:04:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.969+0000 7f285f3a0700 1 Processor -- start 2023-11-28T12:04:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.969+0000 7f285f3a0700 1 -- start start 2023-11-28T12:04:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.970+0000 7f285f3a0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28580b2d10 0x7f28580b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.970+0000 7f285f3a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28580cc200 0x7f285814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.970+0000 7f285f3a0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f285815a5f0 0x7f28580b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.970+0000 7f285f3a0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28580b5510 con 0x7f28580b2d10 2023-11-28T12:04:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.970+0000 7f285f3a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28580b2630 con 0x7f28580cc200 2023-11-28T12:04:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.970+0000 7f285f3a0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28580b2910 con 0x7f285815a5f0 2023-11-28T12:04:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.970+0000 7f285d13c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28580b2d10 0x7f28580b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.970+0000 7f285d13c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28580b2d10 0x7f28580b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48294/0 (socket says 172.21.15.57:48294) 2023-11-28T12:04:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.970+0000 7f285d13c700 1 -- 172.21.15.57:0/3822728228 learned_addr learned my addr 172.21.15.57:0/3822728228 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.970+0000 7f285d93d700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f285815a5f0 0x7f28580b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.970+0000 7f285c93b700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28580cc200 0x7f285814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.971+0000 7f285d13c700 1 -- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f285815a5f0 msgr2=0x7f28580b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:40.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.971+0000 7f285d13c700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f285815a5f0 0x7f28580b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.971+0000 7f285d13c700 1 -- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28580cc200 msgr2=0x7f285814d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:40.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.971+0000 7f285d13c700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28580cc200 0x7f285814d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:40.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.971+0000 7f285d13c700 1 -- 172.21.15.57:0/3822728228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c0094a0 con 0x7f28580b2d10 2023-11-28T12:04:40.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.971+0000 7f285d13c700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28580b2d10 0x7f28580b1170 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f285400e810 tx=0x7f285400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:40.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.971+0000 7f2849ffb700 1 -- 172.21.15.57:0/3822728228 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854013610 con 0x7f28580b2d10 2023-11-28T12:04:40.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.971+0000 7f2849ffb700 1 -- 172.21.15.57:0/3822728228 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2854013770 con 0x7f28580b2d10 2023-11-28T12:04:40.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.971+0000 7f285f3a0700 1 -- 172.21.15.57:0/3822728228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28580ccf50 con 0x7f28580b2d10 2023-11-28T12:04:40.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.971+0000 7f2849ffb700 1 -- 172.21.15.57:0/3822728228 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854013b00 con 0x7f28580b2d10 2023-11-28T12:04:40.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.971+0000 7f285f3a0700 1 -- 172.21.15.57:0/3822728228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28580ccc70 con 0x7f28580b2d10 2023-11-28T12:04:40.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.972+0000 7f285f3a0700 1 -- 172.21.15.57:0/3822728228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28580cd230 con 0x7f28580b2d10 2023-11-28T12:04:40.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.973+0000 7f2849ffb700 1 -- 172.21.15.57:0/3822728228 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2854023070 con 0x7f28580b2d10 2023-11-28T12:04:40.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.973+0000 7f2849ffb700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f283c0682f0 0x7f283c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:40.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.973+0000 7f2849ffb700 1 -- 172.21.15.57:0/3822728228 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f2854085aa0 con 0x7f28580b2d10 2023-11-28T12:04:40.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.973+0000 7f285c93b700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f283c0682f0 0x7f283c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:40.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.974+0000 7f285c93b700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f283c0682f0 0x7f283c06a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f284c0090b0 tx=0x7f284c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:40.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:40.978+0000 7f2849ffb700 1 -- 172.21.15.57:0/3822728228 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f28540521f0 con 0x7f28580b2d10 2023-11-28T12:04:41.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.219+0000 7f285f3a0700 1 -- 172.21.15.57:0/3822728228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f28580b5d70 con 0x7f28580b2d10 2023-11-28T12:04:41.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.809+0000 7f2849ffb700 1 -- 172.21.15.57:0/3822728228 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2854055810 con 0x7f28580b2d10 2023-11-28T12:04:41.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.812+0000 7f285f3a0700 1 -- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f283c0682f0 msgr2=0x7f283c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:41.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.812+0000 7f285f3a0700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f283c0682f0 0x7f283c06a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f284c0090b0 tx=0x7f284c008030 comp rx=0 tx=0).stop 2023-11-28T12:04:41.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.812+0000 7f285f3a0700 1 -- 172.21.15.57:0/3822728228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28580b2d10 msgr2=0x7f28580b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:41.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.812+0000 7f285f3a0700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28580b2d10 0x7f28580b1170 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f285400e810 tx=0x7f285400ebd0 comp rx=0 tx=0).stop 2023-11-28T12:04:41.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.812+0000 7f285f3a0700 1 -- 172.21.15.57:0/3822728228 shutdown_connections 2023-11-28T12:04:41.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.812+0000 7f285f3a0700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f283c0682f0 0x7f283c06a7a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:41.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.812+0000 7f285f3a0700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f285815a5f0 0x7f28580b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:41.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.812+0000 7f285f3a0700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28580b2d10 0x7f28580b1170 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:41.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.812+0000 7f285f3a0700 1 --2- 172.21.15.57:0/3822728228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28580cc200 0x7f285814d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:41.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.812+0000 7f285f3a0700 1 -- 172.21.15.57:0/3822728228 >> 172.21.15.57:0/3822728228 conn(0x7f285806ea80 msgr2=0x7f28580d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:41.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.812+0000 7f285f3a0700 1 -- 172.21.15.57:0/3822728228 shutdown_connections 2023-11-28T12:04:41.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.812+0000 7f285f3a0700 1 -- 172.21.15.57:0/3822728228 wait complete. 2023-11-28T12:04:41.829 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T12:04:41.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.971+0000 7ff04ead0700 1 Processor -- start 2023-11-28T12:04:41.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.971+0000 7ff04ead0700 1 -- start start 2023-11-28T12:04:41.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.972+0000 7ff04ead0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0480ae7a0 0x7ff048160920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:41.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.972+0000 7ff04ead0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff048160e60 0x7ff0480ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:41.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.972+0000 7ff04ead0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0480cb420 con 0x7ff04815c620 2023-11-28T12:04:41.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.972+0000 7ff04ead0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff048066620 con 0x7ff048160e60 2023-11-28T12:04:41.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.972+0000 7ff04ead0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480cb700 con 0x7ff0480ae7a0 2023-11-28T12:04:41.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff04d06d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff048160e60 0x7ff0480ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:41.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff047fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0480ae7a0 0x7ff048160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:41.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff04d06d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff048160e60 0x7ff0480ccf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:58878/0 (socket says 172.21.15.57:58878) 2023-11-28T12:04:41.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff047fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0480ae7a0 0x7ff048160920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:48744/0 (socket says 172.21.15.57:48744) 2023-11-28T12:04:41.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff04d06d700 1 -- 172.21.15.57:0/593744173 learned_addr learned my addr 172.21.15.57:0/593744173 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:41.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff0477fe700 1 -- 172.21.15.57:0/593744173 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449744516 0 0) 0x7ff0480cb420 con 0x7ff04815c620 2023-11-28T12:04:41.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff0477fe700 1 -- 172.21.15.57:0/593744173 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0340035b0 con 0x7ff04815c620 2023-11-28T12:04:41.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff047fff700 1 -- 172.21.15.57:0/593744173 >> v2:172.21.15.181:3300/0 conn(0x7ff048160e60 msgr2=0x7ff0480ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:41.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff047fff700 1 --2- 172.21.15.57:0/593744173 >> v2:172.21.15.181:3300/0 conn(0x7ff048160e60 0x7ff0480ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:41.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff047fff700 1 -- 172.21.15.57:0/593744173 >> v1:172.21.15.181:6789/0 conn(0x7ff04815c620 legacy=0x7ff0480cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:41.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff047fff700 1 -- 172.21.15.57:0/593744173 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0480cb9e0 con 0x7ff0480ae7a0 2023-11-28T12:04:41.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff04d06d700 1 --2- 172.21.15.57:0/593744173 >> v2:172.21.15.181:3300/0 conn(0x7ff048160e60 0x7ff0480ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:41.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff047fff700 1 --2- 172.21.15.57:0/593744173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0480ae7a0 0x7ff048160920 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff030009840 tx=0x7ff030009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1da155781f9bc02 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:41.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff0477fe700 1 -- 172.21.15.57:0/593744173 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030011830 con 0x7ff0480ae7a0 2023-11-28T12:04:41.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff0477fe700 1 -- 172.21.15.57:0/593744173 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff030011990 con 0x7ff0480ae7a0 2023-11-28T12:04:41.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.973+0000 7ff0477fe700 1 -- 172.21.15.57:0/593744173 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030011d20 con 0x7ff0480ae7a0 2023-11-28T12:04:41.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.974+0000 7ff04ead0700 1 -- 172.21.15.57:0/593744173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0480ae7a0 msgr2=0x7ff048160920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:41.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.974+0000 7ff04ead0700 1 --2- 172.21.15.57:0/593744173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0480ae7a0 0x7ff048160920 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff030009840 tx=0x7ff030009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:41.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.974+0000 7ff04ead0700 1 -- 172.21.15.57:0/593744173 shutdown_connections 2023-11-28T12:04:41.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.974+0000 7ff04ead0700 1 --2- 172.21.15.57:0/593744173 >> v2:172.21.15.181:3300/0 conn(0x7ff048160e60 0x7ff0480ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:41.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.974+0000 7ff04ead0700 1 --2- 172.21.15.57:0/593744173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0480ae7a0 0x7ff048160920 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:41.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.974+0000 7ff04ead0700 1 -- 172.21.15.57:0/593744173 >> 172.21.15.57:0/593744173 conn(0x7ff0480cf370 msgr2=0x7ff0480cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:41.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.974+0000 7ff04ead0700 1 -- 172.21.15.57:0/593744173 shutdown_connections 2023-11-28T12:04:41.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.974+0000 7ff04ead0700 1 -- 172.21.15.57:0/593744173 wait complete. 2023-11-28T12:04:41.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.975+0000 7ff04ead0700 1 Processor -- start 2023-11-28T12:04:41.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.975+0000 7ff04ead0700 1 -- start start 2023-11-28T12:04:41.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.976+0000 7ff04ead0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0480ae7a0 0x7ff04814b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:41.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.976+0000 7ff04ead0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff04815c620 0x7ff04815f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:41.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.976+0000 7ff04ead0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff048160e60 0x7ff0480c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:41.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.976+0000 7ff04ead0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04815f010 con 0x7ff0480ae7a0 2023-11-28T12:04:41.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.976+0000 7ff04ead0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04815e400 con 0x7ff04815c620 2023-11-28T12:04:41.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.976+0000 7ff04ead0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04815e6e0 con 0x7ff048160e60 2023-11-28T12:04:41.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.976+0000 7ff04c86c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0480ae7a0 0x7ff04814b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:41.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff04c86c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0480ae7a0 0x7ff04814b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48308/0 (socket says 172.21.15.57:48308) 2023-11-28T12:04:41.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff04c86c700 1 -- 172.21.15.57:0/2535064422 learned_addr learned my addr 172.21.15.57:0/2535064422 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:41.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff047fff700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff04815c620 0x7ff04815f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:41.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff04d06d700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff048160e60 0x7ff0480c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:41.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff04c86c700 1 -- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff048160e60 msgr2=0x7ff0480c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:41.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff04c86c700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff048160e60 0x7ff0480c7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:41.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff04c86c700 1 -- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff04815c620 msgr2=0x7ff04815f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff04c86c700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff04815c620 0x7ff04815f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff04c86c700 1 -- 172.21.15.57:0/2535064422 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff040009b30 con 0x7ff0480ae7a0 2023-11-28T12:04:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff04d06d700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff048160e60 0x7ff0480c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff04c86c700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0480ae7a0 0x7ff04814b510 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff03800c520 tx=0x7ff03800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff0457fa700 1 -- 172.21.15.57:0/2535064422 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380052d0 con 0x7ff0480ae7a0 2023-11-28T12:04:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff0457fa700 1 -- 172.21.15.57:0/2535064422 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff038005430 con 0x7ff0480ae7a0 2023-11-28T12:04:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff04ead0700 1 -- 172.21.15.57:0/2535064422 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0300094a0 con 0x7ff0480ae7a0 2023-11-28T12:04:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff0457fa700 1 -- 172.21.15.57:0/2535064422 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380038a0 con 0x7ff0480ae7a0 2023-11-28T12:04:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.977+0000 7ff04ead0700 1 -- 172.21.15.57:0/2535064422 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04815c9f0 con 0x7ff0480ae7a0 2023-11-28T12:04:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.978+0000 7ff04ead0700 1 -- 172.21.15.57:0/2535064422 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff048160400 con 0x7ff0480ae7a0 2023-11-28T12:04:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.979+0000 7ff0457fa700 1 -- 172.21.15.57:0/2535064422 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff038019030 con 0x7ff0480ae7a0 2023-11-28T12:04:41.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.979+0000 7ff0457fa700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0280682f0 0x7ff02806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:41.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.979+0000 7ff0457fa700 1 -- 172.21.15.57:0/2535064422 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6206+0+0 (secure 0 0 0) 0x7ff038085090 con 0x7ff0480ae7a0 2023-11-28T12:04:41.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.979+0000 7ff047fff700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0280682f0 0x7ff02806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:41.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.980+0000 7ff047fff700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0280682f0 0x7ff02806a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff030000c00 tx=0x7ff030008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:41.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:41.984+0000 7ff0457fa700 1 -- 172.21.15.57:0/2535064422 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff038051860 con 0x7ff0480ae7a0 2023-11-28T12:04:42.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.182+0000 7ff04ead0700 1 -- 172.21.15.57:0/2535064422 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff04815cb80 con 0x7ff0480ae7a0 2023-11-28T12:04:42.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.868+0000 7ff0457fa700 1 -- 172.21.15.57:0/2535064422 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff038054e80 con 0x7ff0480ae7a0 2023-11-28T12:04:42.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.872+0000 7ff04ead0700 1 -- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0280682f0 msgr2=0x7ff02806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:42.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.872+0000 7ff04ead0700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0280682f0 0x7ff02806a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff030000c00 tx=0x7ff030008030 comp rx=0 tx=0).stop 2023-11-28T12:04:42.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.872+0000 7ff04ead0700 1 -- 172.21.15.57:0/2535064422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0480ae7a0 msgr2=0x7ff04814b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:42.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.872+0000 7ff04ead0700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0480ae7a0 0x7ff04814b510 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff03800c520 tx=0x7ff03800c830 comp rx=0 tx=0).stop 2023-11-28T12:04:42.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.872+0000 7ff04ead0700 1 -- 172.21.15.57:0/2535064422 shutdown_connections 2023-11-28T12:04:42.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.872+0000 7ff04ead0700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0280682f0 0x7ff02806a7a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:42.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.872+0000 7ff04ead0700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff048160e60 0x7ff0480c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:42.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.872+0000 7ff04ead0700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0480ae7a0 0x7ff04814b510 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:42.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.872+0000 7ff04ead0700 1 --2- 172.21.15.57:0/2535064422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff04815c620 0x7ff04815f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:42.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.872+0000 7ff04ead0700 1 -- 172.21.15.57:0/2535064422 >> 172.21.15.57:0/2535064422 conn(0x7ff0480cf370 msgr2=0x7ff0480adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:42.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.872+0000 7ff04ead0700 1 -- 172.21.15.57:0/2535064422 shutdown_connections 2023-11-28T12:04:42.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:42.872+0000 7ff04ead0700 1 -- 172.21.15.57:0/2535064422 wait complete. 2023-11-28T12:04:42.875 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' removed 2023-11-28T12:04:42.889 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T12:04:43.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.033+0000 7f29eb3fd700 1 Processor -- start 2023-11-28T12:04:43.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.033+0000 7f29eb3fd700 1 -- start start 2023-11-28T12:04:43.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29eb3fd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e415a4a0 0x7f29e40ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:43.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29eb3fd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f29e40af4e0 0x7f29e40ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:43.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29eb3fd700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e40af050 con 0x7f29e40adec0 2023-11-28T12:04:43.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29eb3fd700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e415a870 con 0x7f29e40af4e0 2023-11-28T12:04:43.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29eb3fd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40ae440 con 0x7f29e415a4a0 2023-11-28T12:04:43.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29e8998700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f29e40af4e0 0x7f29e40ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:43.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29e999a700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f29e40adec0 0x7f29e415df60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40836/0 (socket says 172.21.15.57:40836) 2023-11-28T12:04:43.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29e8998700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f29e40af4e0 0x7f29e40ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48322/0 (socket says 172.21.15.57:48322) 2023-11-28T12:04:43.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29e999a700 1 -- 172.21.15.57:0/503714400 learned_addr learned my addr 172.21.15.57:0/503714400 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:43.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29e9199700 1 --2- 172.21.15.57:0/503714400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e415a4a0 0x7f29e40ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:43.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29e8998700 1 -- 172.21.15.57:0/503714400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e415a4a0 msgr2=0x7f29e40ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:43.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29e8998700 1 --2- 172.21.15.57:0/503714400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e415a4a0 0x7f29e40ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:43.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29e8998700 1 -- 172.21.15.57:0/503714400 >> v1:172.21.15.57:6789/0 conn(0x7f29e40adec0 legacy=0x7f29e415df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:43.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29e8998700 1 -- 172.21.15.57:0/503714400 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e40ae720 con 0x7f29e40af4e0 2023-11-28T12:04:43.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.034+0000 7f29dbfff700 1 -- 172.21.15.57:0/503714400 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499729318 0 0) 0x7f29e40af050 con 0x7f29e40adec0 2023-11-28T12:04:43.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.035+0000 7f29e8998700 1 --2- 172.21.15.57:0/503714400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29e40af4e0 0x7f29e40ce920 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009840 tx=0x7f29d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec07f95f4eed76b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:43.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.035+0000 7f29dbfff700 1 -- 172.21.15.57:0/503714400 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4011830 con 0x7f29e40af4e0 2023-11-28T12:04:43.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.035+0000 7f29dbfff700 1 -- 172.21.15.57:0/503714400 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d4011990 con 0x7f29e40af4e0 2023-11-28T12:04:43.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.035+0000 7f29dbfff700 1 -- 172.21.15.57:0/503714400 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4011d20 con 0x7f29e40af4e0 2023-11-28T12:04:43.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.035+0000 7f29eb3fd700 1 -- 172.21.15.57:0/503714400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29e40af4e0 msgr2=0x7f29e40ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:43.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.035+0000 7f29eb3fd700 1 --2- 172.21.15.57:0/503714400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29e40af4e0 0x7f29e40ce920 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009840 tx=0x7f29d4009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:43.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.035+0000 7f29eb3fd700 1 -- 172.21.15.57:0/503714400 shutdown_connections 2023-11-28T12:04:43.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.035+0000 7f29eb3fd700 1 --2- 172.21.15.57:0/503714400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e415a4a0 0x7f29e40ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:43.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.035+0000 7f29eb3fd700 1 --2- 172.21.15.57:0/503714400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29e40af4e0 0x7f29e40ce920 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:43.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.035+0000 7f29eb3fd700 1 -- 172.21.15.57:0/503714400 >> 172.21.15.57:0/503714400 conn(0x7f29e406ab90 msgr2=0x7f29e4099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:43.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.036+0000 7f29eb3fd700 1 -- 172.21.15.57:0/503714400 shutdown_connections 2023-11-28T12:04:43.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.036+0000 7f29eb3fd700 1 -- 172.21.15.57:0/503714400 wait complete. 2023-11-28T12:04:43.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.037+0000 7f29eb3fd700 1 Processor -- start 2023-11-28T12:04:43.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.037+0000 7f29eb3fd700 1 -- start start 2023-11-28T12:04:43.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.038+0000 7f29eb3fd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e40adec0 0x7f29e40c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:43.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.038+0000 7f29eb3fd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29e40af4e0 0x7f29e40ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:43.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.038+0000 7f29eb3fd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29e415a4a0 0x7f29e4158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:43.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.038+0000 7f29eb3fd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40c4f10 con 0x7f29e40af4e0 2023-11-28T12:04:43.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.038+0000 7f29eb3fd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40c6e10 con 0x7f29e415a4a0 2023-11-28T12:04:43.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.038+0000 7f29eb3fd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40cacc0 con 0x7f29e40adec0 2023-11-28T12:04:43.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.038+0000 7f29e8998700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29e40af4e0 0x7f29e40ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:43.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.038+0000 7f29e8998700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29e40af4e0 0x7f29e40ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48330/0 (socket says 172.21.15.57:48330) 2023-11-28T12:04:43.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.038+0000 7f29e8998700 1 -- 172.21.15.57:0/2040591854 learned_addr learned my addr 172.21.15.57:0/2040591854 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:43.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29e9199700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e40adec0 0x7f29e40c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:43.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29e999a700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29e415a4a0 0x7f29e4158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:43.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29e8998700 1 -- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e40adec0 msgr2=0x7f29e40c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:43.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29e8998700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e40adec0 0x7f29e40c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:43.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29e8998700 1 -- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29e415a4a0 msgr2=0x7f29e4158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:43.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29e8998700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29e415a4a0 0x7f29e4158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:43.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29e8998700 1 -- 172.21.15.57:0/2040591854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d40094a0 con 0x7f29e40af4e0 2023-11-28T12:04:43.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29e9199700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e40adec0 0x7f29e40c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:43.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29e8998700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29e40af4e0 0x7f29e40ca780 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f29e40ce8d0 tx=0x7f29d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:43.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29d9ffb700 1 -- 172.21.15.57:0/2040591854 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d40038a0 con 0x7f29e40af4e0 2023-11-28T12:04:43.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29d9ffb700 1 -- 172.21.15.57:0/2040591854 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d401f070 con 0x7f29e40af4e0 2023-11-28T12:04:43.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29eb3fd700 1 -- 172.21.15.57:0/2040591854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e40aece0 con 0x7f29e40af4e0 2023-11-28T12:04:43.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29d9ffb700 1 -- 172.21.15.57:0/2040591854 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4003c40 con 0x7f29e40af4e0 2023-11-28T12:04:43.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.039+0000 7f29eb3fd700 1 -- 172.21.15.57:0/2040591854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e40aea00 con 0x7f29e40af4e0 2023-11-28T12:04:43.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.040+0000 7f29eb3fd700 1 -- 172.21.15.57:0/2040591854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e40d0bd0 con 0x7f29e40af4e0 2023-11-28T12:04:43.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.041+0000 7f29d9ffb700 1 -- 172.21.15.57:0/2040591854 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f29d400f050 con 0x7f29e40af4e0 2023-11-28T12:04:43.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.041+0000 7f29d9ffb700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f29c80682f0 0x7f29c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:43.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.041+0000 7f29d9ffb700 1 -- 172.21.15.57:0/2040591854 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f29d4021070 con 0x7f29e40af4e0 2023-11-28T12:04:43.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.042+0000 7f29e9199700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f29c80682f0 0x7f29c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:43.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.042+0000 7f29e9199700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f29c80682f0 0x7f29c806a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f29e0006fd0 tx=0x7f29e0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:43.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.046+0000 7f29d9ffb700 1 -- 172.21.15.57:0/2040591854 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f29d4053f70 con 0x7f29e40af4e0 2023-11-28T12:04:43.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.245+0000 7f29eb3fd700 1 -- 172.21.15.57:0/2040591854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f29e40c7f60 con 0x7f29e40af4e0 2023-11-28T12:04:43.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.876+0000 7f29d9ffb700 1 -- 172.21.15.57:0/2040591854 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) v1 ==== 169+0+0 (secure 0 0 0) 0x7f29e40c7f60 con 0x7f29e40af4e0 2023-11-28T12:04:43.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.881+0000 7f29eb3fd700 1 -- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f29c80682f0 msgr2=0x7f29c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:43.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.881+0000 7f29eb3fd700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f29c80682f0 0x7f29c806a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f29e0006fd0 tx=0x7f29e0008040 comp rx=0 tx=0).stop 2023-11-28T12:04:43.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.881+0000 7f29eb3fd700 1 -- 172.21.15.57:0/2040591854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29e40af4e0 msgr2=0x7f29e40ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:43.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.881+0000 7f29eb3fd700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29e40af4e0 0x7f29e40ca780 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f29e40ce8d0 tx=0x7f29d4004750 comp rx=0 tx=0).stop 2023-11-28T12:04:43.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.881+0000 7f29eb3fd700 1 -- 172.21.15.57:0/2040591854 shutdown_connections 2023-11-28T12:04:43.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.881+0000 7f29eb3fd700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f29c80682f0 0x7f29c806a7a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:43.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.881+0000 7f29eb3fd700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29e415a4a0 0x7f29e4158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:43.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.881+0000 7f29eb3fd700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e40adec0 0x7f29e40c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:43.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.881+0000 7f29eb3fd700 1 --2- 172.21.15.57:0/2040591854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29e40af4e0 0x7f29e40ca780 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:43.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.881+0000 7f29eb3fd700 1 -- 172.21.15.57:0/2040591854 >> 172.21.15.57:0/2040591854 conn(0x7f29e406ab90 msgr2=0x7f29e406f510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:43.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.881+0000 7f29eb3fd700 1 -- 172.21.15.57:0/2040591854 shutdown_connections 2023-11-28T12:04:43.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:43.882+0000 7f29eb3fd700 1 -- 172.21.15.57:0/2040591854 wait complete. 2023-11-28T12:04:43.884 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' removed 2023-11-28T12:04:43.899 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T12:04:44.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.038+0000 7faa76596700 1 Processor -- start 2023-11-28T12:04:44.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.038+0000 7faa76596700 1 -- start start 2023-11-28T12:04:44.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.039+0000 7faa76596700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7faa7006d170 0x7faa700b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:44.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.039+0000 7faa76596700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa700b27d0 0x7faa700b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.039+0000 7faa76596700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa70161370 con 0x7faa7015fe50 2023-11-28T12:04:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.039+0000 7faa76596700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7006d5c0 con 0x7faa7006d170 2023-11-28T12:04:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.039+0000 7faa76596700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa70160220 con 0x7faa700b27d0 2023-11-28T12:04:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.039+0000 7faa6f7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7faa7006d170 0x7faa700b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.039+0000 7faa6f7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7faa7006d170 0x7faa700b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48342/0 (socket says 172.21.15.57:48342) 2023-11-28T12:04:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.039+0000 7faa6f7fe700 1 -- 172.21.15.57:0/267386864 learned_addr learned my addr 172.21.15.57:0/267386864 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.040+0000 7faa74b33700 1 --2- 172.21.15.57:0/267386864 >> v2:172.21.15.181:3300/0 conn(0x7faa700b27d0 0x7faa700b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.040+0000 7faa6f7fe700 1 -- 172.21.15.57:0/267386864 >> v2:172.21.15.181:3300/0 conn(0x7faa700b27d0 msgr2=0x7faa700b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.040+0000 7faa6f7fe700 1 --2- 172.21.15.57:0/267386864 >> v2:172.21.15.181:3300/0 conn(0x7faa700b27d0 0x7faa700b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.040+0000 7faa6f7fe700 1 -- 172.21.15.57:0/267386864 >> v1:172.21.15.181:6789/0 conn(0x7faa7015fe50 legacy=0x7faa70160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:44.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.040+0000 7faa6f7fe700 1 -- 172.21.15.57:0/267386864 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa700b3170 con 0x7faa7006d170 2023-11-28T12:04:44.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.040+0000 7faa74b33700 1 --2- 172.21.15.57:0/267386864 >> v2:172.21.15.181:3300/0 conn(0x7faa700b27d0 0x7faa700b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:44.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.040+0000 7faa6f7fe700 1 --2- 172.21.15.57:0/267386864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7006d170 0x7faa700b3b80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7faa70071b30 tx=0x7faa58009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3daadcde4296d39 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:44.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.040+0000 7faa6effd700 1 -- 172.21.15.57:0/267386864 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58004230 con 0x7faa7006d170 2023-11-28T12:04:44.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.040+0000 7faa6effd700 1 -- 172.21.15.57:0/267386864 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58004390 con 0x7faa7006d170 2023-11-28T12:04:44.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.040+0000 7faa6effd700 1 -- 172.21.15.57:0/267386864 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa580036a0 con 0x7faa7006d170 2023-11-28T12:04:44.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.040+0000 7faa76596700 1 -- 172.21.15.57:0/267386864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7006d170 msgr2=0x7faa700b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:44.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.040+0000 7faa76596700 1 --2- 172.21.15.57:0/267386864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7006d170 0x7faa700b3b80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7faa70071b30 tx=0x7faa58009c00 comp rx=0 tx=0).stop 2023-11-28T12:04:44.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.041+0000 7faa76596700 1 -- 172.21.15.57:0/267386864 shutdown_connections 2023-11-28T12:04:44.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.041+0000 7faa76596700 1 --2- 172.21.15.57:0/267386864 >> v2:172.21.15.181:3300/0 conn(0x7faa700b27d0 0x7faa700b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.041+0000 7faa76596700 1 --2- 172.21.15.57:0/267386864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7006d170 0x7faa700b3b80 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.041+0000 7faa76596700 1 -- 172.21.15.57:0/267386864 >> 172.21.15.57:0/267386864 conn(0x7faa7006ea80 msgr2=0x7faa7006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:44.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.041+0000 7faa76596700 1 -- 172.21.15.57:0/267386864 shutdown_connections 2023-11-28T12:04:44.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.041+0000 7faa76596700 1 -- 172.21.15.57:0/267386864 wait complete. 2023-11-28T12:04:44.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.042+0000 7faa76596700 1 Processor -- start 2023-11-28T12:04:44.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.042+0000 7faa76596700 1 -- start start 2023-11-28T12:04:44.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa76596700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7006d170 0x7faa700c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:44.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa76596700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa700b27d0 0x7faa70152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:44.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa76596700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa7015fe50 0x7faa70153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:44.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa76596700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa700c6fd0 con 0x7faa7006d170 2023-11-28T12:04:44.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa76596700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa70153640 con 0x7faa7015fe50 2023-11-28T12:04:44.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa76596700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa700c63c0 con 0x7faa700b27d0 2023-11-28T12:04:44.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa6ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7006d170 0x7faa700c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:44.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa6ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7006d170 0x7faa700c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48346/0 (socket says 172.21.15.57:48346) 2023-11-28T12:04:44.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa6ffff700 1 -- 172.21.15.57:0/4176780582 learned_addr learned my addr 172.21.15.57:0/4176780582 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:44.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa74b33700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa7015fe50 0x7faa70153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:44.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa6f7fe700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa700b27d0 0x7faa70152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:44.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa6ffff700 1 -- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa700b27d0 msgr2=0x7faa70152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:44.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa6ffff700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa700b27d0 0x7faa70152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa6ffff700 1 -- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa7015fe50 msgr2=0x7faa70153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:44.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa6ffff700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa7015fe50 0x7faa70153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa6ffff700 1 -- 172.21.15.57:0/4176780582 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa58009550 con 0x7faa7006d170 2023-11-28T12:04:44.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.043+0000 7faa74b33700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa7015fe50 0x7faa70153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:44.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.044+0000 7faa6ffff700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7006d170 0x7faa700c97a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7faa6000c4d0 tx=0x7faa6000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:44.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.044+0000 7faa6cff9700 1 -- 172.21.15.57:0/4176780582 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60011790 con 0x7faa7006d170 2023-11-28T12:04:44.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.044+0000 7faa6cff9700 1 -- 172.21.15.57:0/4176780582 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa600118f0 con 0x7faa7006d170 2023-11-28T12:04:44.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.044+0000 7faa76596700 1 -- 172.21.15.57:0/4176780582 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa700cedd0 con 0x7faa7006d170 2023-11-28T12:04:44.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.044+0000 7faa6cff9700 1 -- 172.21.15.57:0/4176780582 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60011ca0 con 0x7faa7006d170 2023-11-28T12:04:44.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.044+0000 7faa76596700 1 -- 172.21.15.57:0/4176780582 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa700cf700 con 0x7faa7006d170 2023-11-28T12:04:44.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.045+0000 7faa76596700 1 -- 172.21.15.57:0/4176780582 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa700cf0b0 con 0x7faa7006d170 2023-11-28T12:04:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.045+0000 7faa6cff9700 1 -- 172.21.15.57:0/4176780582 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faa60010050 con 0x7faa7006d170 2023-11-28T12:04:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.046+0000 7faa6cff9700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faa500682f0 0x7faa5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.046+0000 7faa6cff9700 1 -- 172.21.15.57:0/4176780582 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7faa60085680 con 0x7faa7006d170 2023-11-28T12:04:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.046+0000 7faa6f7fe700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faa500682f0 0x7faa5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.046+0000 7faa6f7fe700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faa500682f0 0x7faa5006a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faa58009130 tx=0x7faa58008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.050+0000 7faa6cff9700 1 -- 172.21.15.57:0/4176780582 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faa600521e0 con 0x7faa7006d170 2023-11-28T12:04:44.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.258+0000 7faa76596700 1 -- 172.21.15.57:0/4176780582 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7faa700b1ee0 con 0x7faa7006d170 2023-11-28T12:04:44.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.262+0000 7faa6cff9700 1 -- 172.21.15.57:0/4176780582 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7faa60085030 con 0x7faa7006d170 2023-11-28T12:04:44.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.264+0000 7faa76596700 1 -- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faa500682f0 msgr2=0x7faa5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:44.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.264+0000 7faa76596700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faa500682f0 0x7faa5006a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faa58009130 tx=0x7faa58008030 comp rx=0 tx=0).stop 2023-11-28T12:04:44.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.264+0000 7faa76596700 1 -- 172.21.15.57:0/4176780582 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7006d170 msgr2=0x7faa700c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:44.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.264+0000 7faa76596700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7006d170 0x7faa700c97a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7faa6000c4d0 tx=0x7faa6000c890 comp rx=0 tx=0).stop 2023-11-28T12:04:44.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.264+0000 7faa76596700 1 -- 172.21.15.57:0/4176780582 shutdown_connections 2023-11-28T12:04:44.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.264+0000 7faa76596700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7faa500682f0 0x7faa5006a7a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.264+0000 7faa76596700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa7015fe50 0x7faa70153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.264+0000 7faa76596700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7006d170 0x7faa700c97a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.264+0000 7faa76596700 1 --2- 172.21.15.57:0/4176780582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa700b27d0 0x7faa70152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.264+0000 7faa76596700 1 -- 172.21.15.57:0/4176780582 >> 172.21.15.57:0/4176780582 conn(0x7faa7006ea80 msgr2=0x7faa700cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:44.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.264+0000 7faa76596700 1 -- 172.21.15.57:0/4176780582 shutdown_connections 2023-11-28T12:04:44.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.264+0000 7faa76596700 1 -- 172.21.15.57:0/4176780582 wait complete. 2023-11-28T12:04:44.279 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T12:04:44.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.415+0000 7f208e1f0700 1 Processor -- start 2023-11-28T12:04:44.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.415+0000 7f208e1f0700 1 -- start start 2023-11-28T12:04:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.416+0000 7f208e1f0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f208806ab10 0x7f208806aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.416+0000 7f208e1f0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20880cd5a0 0x7f2088160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.416+0000 7f208e1f0700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20880d1ee0 con 0x7f20880b22a0 2023-11-28T12:04:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.416+0000 7f208e1f0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208806b420 con 0x7f208806ab10 2023-11-28T12:04:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.416+0000 7f208e1f0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20880cdb10 con 0x7f20880cd5a0 2023-11-28T12:04:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.416+0000 7f2087fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f20880b22a0 0x7f2088160840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:40840/0 (socket says 172.21.15.57:40840) 2023-11-28T12:04:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.416+0000 7f2087fff700 1 -- 172.21.15.57:0/704323424 learned_addr learned my addr 172.21.15.57:0/704323424 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.416+0000 7f20877fe700 1 --2- 172.21.15.57:0/704323424 >> v2:172.21.15.181:3300/0 conn(0x7f208806ab10 0x7f208806aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.417+0000 7f2086ffd700 1 --2- 172.21.15.57:0/704323424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20880cd5a0 0x7f2088160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.417+0000 7f20867fc700 1 -- 172.21.15.57:0/704323424 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992739688 0 0) 0x7f20880d1ee0 con 0x7f20880b22a0 2023-11-28T12:04:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.417+0000 7f20867fc700 1 -- 172.21.15.57:0/704323424 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20700035b0 con 0x7f20880b22a0 2023-11-28T12:04:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.417+0000 7f20877fe700 1 -- 172.21.15.57:0/704323424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20880cd5a0 msgr2=0x7f2088160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.417+0000 7f20877fe700 1 --2- 172.21.15.57:0/704323424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20880cd5a0 0x7f2088160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.417+0000 7f20877fe700 1 -- 172.21.15.57:0/704323424 >> v1:172.21.15.57:6789/0 conn(0x7f20880b22a0 legacy=0x7f2088160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.417+0000 7f20877fe700 1 -- 172.21.15.57:0/704323424 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20880d1890 con 0x7f208806ab10 2023-11-28T12:04:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.417+0000 7f20867fc700 1 -- 172.21.15.57:0/704323424 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2257822889 0 0) 0x7f20700035b0 con 0x7f20880b22a0 2023-11-28T12:04:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.417+0000 7f2086ffd700 1 --2- 172.21.15.57:0/704323424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20880cd5a0 0x7f2088160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.417+0000 7f20877fe700 1 --2- 172.21.15.57:0/704323424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f208806ab10 0x7f208806aee0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2078009840 tx=0x7f2078009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b840d62f705b026b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.417+0000 7f20867fc700 1 -- 172.21.15.57:0/704323424 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800019f0 con 0x7f208806ab10 2023-11-28T12:04:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.417+0000 7f20867fc700 1 -- 172.21.15.57:0/704323424 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2078004030 con 0x7f208806ab10 2023-11-28T12:04:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.417+0000 7f20867fc700 1 -- 172.21.15.57:0/704323424 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20780043c0 con 0x7f208806ab10 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.418+0000 7f208e1f0700 1 -- 172.21.15.57:0/704323424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f208806ab10 msgr2=0x7f208806aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.418+0000 7f208e1f0700 1 --2- 172.21.15.57:0/704323424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f208806ab10 0x7f208806aee0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2078009840 tx=0x7f2078009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.418+0000 7f208e1f0700 1 -- 172.21.15.57:0/704323424 shutdown_connections 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.418+0000 7f208e1f0700 1 --2- 172.21.15.57:0/704323424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f208806ab10 0x7f208806aee0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.418+0000 7f208e1f0700 1 --2- 172.21.15.57:0/704323424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20880cd5a0 0x7f2088160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.418+0000 7f208e1f0700 1 -- 172.21.15.57:0/704323424 >> 172.21.15.57:0/704323424 conn(0x7f208806c520 msgr2=0x7f208806c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.418+0000 7f208e1f0700 1 -- 172.21.15.57:0/704323424 shutdown_connections 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.418+0000 7f208e1f0700 1 -- 172.21.15.57:0/704323424 wait complete. 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.419+0000 7f208e1f0700 1 Processor -- start 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.419+0000 7f208e1f0700 1 -- start start 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.420+0000 7f208e1f0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f208806ab10 0x7f20881551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.420+0000 7f208e1f0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20880b22a0 0x7f20880cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.420+0000 7f208e1f0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20880cd5a0 0x7f20880c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.420+0000 7f208e1f0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2088159a30 con 0x7f20880b22a0 2023-11-28T12:04:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.420+0000 7f208e1f0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20880ccb10 con 0x7f208806ab10 2023-11-28T12:04:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.420+0000 7f208e1f0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20880cc270 con 0x7f20880cd5a0 2023-11-28T12:04:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.420+0000 7f2086ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20880b22a0 0x7f20880cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.420+0000 7f2086ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20880b22a0 0x7f20880cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48358/0 (socket says 172.21.15.57:48358) 2023-11-28T12:04:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.420+0000 7f2086ffd700 1 -- 172.21.15.57:0/1976932862 learned_addr learned my addr 172.21.15.57:0/1976932862 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.421+0000 7f20877fe700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f208806ab10 0x7f20881551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.421+0000 7f2087fff700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20880cd5a0 0x7f20880c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.421+0000 7f2086ffd700 1 -- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20880cd5a0 msgr2=0x7f20880c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.421+0000 7f2086ffd700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20880cd5a0 0x7f20880c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.421+0000 7f2086ffd700 1 -- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f208806ab10 msgr2=0x7f20881551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.421+0000 7f2086ffd700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f208806ab10 0x7f20881551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.421+0000 7f2086ffd700 1 -- 172.21.15.57:0/1976932862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c0094a0 con 0x7f20880b22a0 2023-11-28T12:04:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.421+0000 7f2086ffd700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20880b22a0 0x7f20880cba10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f207c00ebe0 tx=0x7f207c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.421+0000 7f206ffff700 1 -- 172.21.15.57:0/1976932862 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c00c850 con 0x7f20880b22a0 2023-11-28T12:04:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.421+0000 7f206ffff700 1 -- 172.21.15.57:0/1976932862 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c00c9b0 con 0x7f20880b22a0 2023-11-28T12:04:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.421+0000 7f208e1f0700 1 -- 172.21.15.57:0/1976932862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20780094a0 con 0x7f20880b22a0 2023-11-28T12:04:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.421+0000 7f206ffff700 1 -- 172.21.15.57:0/1976932862 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c00cd40 con 0x7f20880b22a0 2023-11-28T12:04:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.421+0000 7f208e1f0700 1 -- 172.21.15.57:0/1976932862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208815cad0 con 0x7f20880b22a0 2023-11-28T12:04:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.422+0000 7f208e1f0700 1 -- 172.21.15.57:0/1976932862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20880d1b70 con 0x7f20880b22a0 2023-11-28T12:04:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.423+0000 7f206ffff700 1 -- 172.21.15.57:0/1976932862 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f207c019030 con 0x7f20880b22a0 2023-11-28T12:04:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.423+0000 7f206ffff700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20740682f0 0x7f207406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.423+0000 7f206ffff700 1 -- 172.21.15.57:0/1976932862 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f207c0858f0 con 0x7f20880b22a0 2023-11-28T12:04:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.424+0000 7f20877fe700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20740682f0 0x7f207406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:44.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.424+0000 7f20877fe700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20740682f0 0x7f207406a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f20780090b0 tx=0x7f2078008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:44.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.428+0000 7f206ffff700 1 -- 172.21.15.57:0/1976932862 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f207c052450 con 0x7f20880b22a0 2023-11-28T12:04:44.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.710+0000 7f208e1f0700 1 -- 172.21.15.57:0/1976932862 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f20880b1ec0 con 0x7f20880b22a0 2023-11-28T12:04:44.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.873+0000 7f206ffff700 1 -- 172.21.15.57:0/1976932862 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v165) v1 ==== 149+0+0 (secure 0 0 0) 0x7f207c0170d0 con 0x7f20880b22a0 2023-11-28T12:04:44.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.875+0000 7f208e1f0700 1 -- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20740682f0 msgr2=0x7f207406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.875+0000 7f208e1f0700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20740682f0 0x7f207406a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f20780090b0 tx=0x7f2078008030 comp rx=0 tx=0).stop 2023-11-28T12:04:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.875+0000 7f208e1f0700 1 -- 172.21.15.57:0/1976932862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20880b22a0 msgr2=0x7f20880cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.875+0000 7f208e1f0700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20880b22a0 0x7f20880cba10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f207c00ebe0 tx=0x7f207c00eef0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.875+0000 7f208e1f0700 1 -- 172.21.15.57:0/1976932862 shutdown_connections 2023-11-28T12:04:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.875+0000 7f208e1f0700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20740682f0 0x7f207406a7a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.875+0000 7f208e1f0700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20880cd5a0 0x7f20880c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.875+0000 7f208e1f0700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f208806ab10 0x7f20881551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.875+0000 7f208e1f0700 1 --2- 172.21.15.57:0/1976932862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20880b22a0 0x7f20880cba10 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.875+0000 7f208e1f0700 1 -- 172.21.15.57:0/1976932862 >> 172.21.15.57:0/1976932862 conn(0x7f208806c520 msgr2=0x7f20880b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.875+0000 7f208e1f0700 1 -- 172.21.15.57:0/1976932862 shutdown_connections 2023-11-28T12:04:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:44.875+0000 7f208e1f0700 1 -- 172.21.15.57:0/1976932862 wait complete. 2023-11-28T12:04:44.891 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2023-11-28T12:04:44.892 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2023-11-28T12:04:44.892 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T12:04:44.892 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T12:04:45.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.029+0000 7fdb215ce700 1 Processor -- start 2023-11-28T12:04:45.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.029+0000 7fdb215ce700 1 -- start start 2023-11-28T12:04:45.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.030+0000 7fdb215ce700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdb1c06ab10 0x7fdb1c06aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:45.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.030+0000 7fdb215ce700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb1c0cd5a0 0x7fdb1c160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:45.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.030+0000 7fdb215ce700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdb1c0b22a0 0x7fdb1c160840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:45.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.030+0000 7fdb215ce700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb1c0d1ee0 con 0x7fdb1c06ab10 2023-11-28T12:04:45.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.030+0000 7fdb215ce700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb1c06b420 con 0x7fdb1c0b22a0 2023-11-28T12:04:45.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb215ce700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c0cdb10 con 0x7fdb1c0cd5a0 2023-11-28T12:04:45.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb1affd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdb1c06ab10 0x7fdb1c06aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:45.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb1affd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdb1c06ab10 0x7fdb1c06aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54530/0 (socket says 172.21.15.57:54530) 2023-11-28T12:04:45.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb1affd700 1 -- 172.21.15.57:0/3100986276 learned_addr learned my addr 172.21.15.57:0/3100986276 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:45.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb1b7fe700 1 --2- 172.21.15.57:0/3100986276 >> v2:172.21.15.181:3300/0 conn(0x7fdb1c0b22a0 0x7fdb1c160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:45.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb1a7fc700 1 --2- 172.21.15.57:0/3100986276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb1c0cd5a0 0x7fdb1c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:45.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb1affd700 1 -- 172.21.15.57:0/3100986276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb1c0cd5a0 msgr2=0x7fdb1c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:45.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb1affd700 1 --2- 172.21.15.57:0/3100986276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb1c0cd5a0 0x7fdb1c160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:45.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb1affd700 1 -- 172.21.15.57:0/3100986276 >> v2:172.21.15.181:3300/0 conn(0x7fdb1c0b22a0 msgr2=0x7fdb1c160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:45.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb1affd700 1 --2- 172.21.15.57:0/3100986276 >> v2:172.21.15.181:3300/0 conn(0x7fdb1c0b22a0 0x7fdb1c160840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:45.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb1affd700 1 -- 172.21.15.57:0/3100986276 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1c0d1890 con 0x7fdb1c06ab10 2023-11-28T12:04:45.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb1a7fc700 1 --2- 172.21.15.57:0/3100986276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb1c0cd5a0 0x7fdb1c160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:45.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb1b7fe700 1 --2- 172.21.15.57:0/3100986276 >> v2:172.21.15.181:3300/0 conn(0x7fdb1c0b22a0 0x7fdb1c160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:45.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb1affd700 1 --2- 172.21.15.57:0/3100986276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb1c06ab10 0x7fdb1c06aee0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c06a940 tx=0x7fdb0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d32339ba1279e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:45.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb19ffb700 1 -- 172.21.15.57:0/3100986276 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c00b870 con 0x7fdb1c06ab10 2023-11-28T12:04:45.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb19ffb700 1 -- 172.21.15.57:0/3100986276 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c00b9d0 con 0x7fdb1c06ab10 2023-11-28T12:04:45.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.031+0000 7fdb19ffb700 1 -- 172.21.15.57:0/3100986276 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c00bd60 con 0x7fdb1c06ab10 2023-11-28T12:04:45.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.032+0000 7fdb215ce700 1 -- 172.21.15.57:0/3100986276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb1c06ab10 msgr2=0x7fdb1c06aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:45.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.032+0000 7fdb215ce700 1 --2- 172.21.15.57:0/3100986276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb1c06ab10 0x7fdb1c06aee0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c06a940 tx=0x7fdb0c009bd0 comp rx=0 tx=0).stop 2023-11-28T12:04:45.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.032+0000 7fdb215ce700 1 -- 172.21.15.57:0/3100986276 shutdown_connections 2023-11-28T12:04:45.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.032+0000 7fdb215ce700 1 --2- 172.21.15.57:0/3100986276 >> v2:172.21.15.181:3300/0 conn(0x7fdb1c0b22a0 0x7fdb1c160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:45.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.032+0000 7fdb215ce700 1 --2- 172.21.15.57:0/3100986276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb1c06ab10 0x7fdb1c06aee0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:45.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.032+0000 7fdb215ce700 1 --2- 172.21.15.57:0/3100986276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb1c0cd5a0 0x7fdb1c160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:45.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.032+0000 7fdb215ce700 1 -- 172.21.15.57:0/3100986276 >> 172.21.15.57:0/3100986276 conn(0x7fdb1c06c520 msgr2=0x7fdb1c06c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:45.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.032+0000 7fdb215ce700 1 -- 172.21.15.57:0/3100986276 shutdown_connections 2023-11-28T12:04:45.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.032+0000 7fdb215ce700 1 -- 172.21.15.57:0/3100986276 wait complete. 2023-11-28T12:04:45.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.033+0000 7fdb215ce700 1 Processor -- start 2023-11-28T12:04:45.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.033+0000 7fdb215ce700 1 -- start start 2023-11-28T12:04:45.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.034+0000 7fdb215ce700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb1c06ab10 0x7fdb1c1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:45.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.034+0000 7fdb215ce700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb1c0b22a0 0x7fdb1c0cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:45.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.034+0000 7fdb215ce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb1c0cd5a0 0x7fdb1c1f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:45.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.034+0000 7fdb215ce700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c158cf0 con 0x7fdb1c06ab10 2023-11-28T12:04:45.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.034+0000 7fdb215ce700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c0cc8b0 con 0x7fdb1c0cd5a0 2023-11-28T12:04:45.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.034+0000 7fdb215ce700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c0ccb90 con 0x7fdb1c0b22a0 2023-11-28T12:04:45.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.034+0000 7fdb1affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb1c06ab10 0x7fdb1c1565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:45.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.034+0000 7fdb1affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb1c06ab10 0x7fdb1c1565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54536/0 (socket says 172.21.15.57:54536) 2023-11-28T12:04:45.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.034+0000 7fdb1affd700 1 -- 172.21.15.57:0/1954465627 learned_addr learned my addr 172.21.15.57:0/1954465627 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:45.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb1b7fe700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb1c0cd5a0 0x7fdb1c1f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:45.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb1a7fc700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb1c0b22a0 0x7fdb1c0cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:45.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb1affd700 1 -- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb1c0b22a0 msgr2=0x7fdb1c0cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:45.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb1affd700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb1c0b22a0 0x7fdb1c0cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:45.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb1affd700 1 -- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb1c0cd5a0 msgr2=0x7fdb1c1f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:45.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb1affd700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb1c0cd5a0 0x7fdb1c1f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:45.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb1affd700 1 -- 172.21.15.57:0/1954465627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c009550 con 0x7fdb1c06ab10 2023-11-28T12:04:45.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb1a7fc700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb1c0b22a0 0x7fdb1c0cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:45.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb1b7fe700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb1c0cd5a0 0x7fdb1c1f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:45.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb1affd700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb1c06ab10 0x7fdb1c1565b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c014040 tx=0x7fdb0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:45.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb037fe700 1 -- 172.21.15.57:0/1954465627 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c004520 con 0x7fdb1c06ab10 2023-11-28T12:04:45.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb037fe700 1 -- 172.21.15.57:0/1954465627 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c01e070 con 0x7fdb1c06ab10 2023-11-28T12:04:45.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb215ce700 1 -- 172.21.15.57:0/1954465627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1c15cad0 con 0x7fdb1c06ab10 2023-11-28T12:04:45.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb037fe700 1 -- 172.21.15.57:0/1954465627 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0038e0 con 0x7fdb1c06ab10 2023-11-28T12:04:45.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.035+0000 7fdb215ce700 1 -- 172.21.15.57:0/1954465627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1c0d1b70 con 0x7fdb1c06ab10 2023-11-28T12:04:45.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.036+0000 7fdb215ce700 1 -- 172.21.15.57:0/1954465627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1c0b1ec0 con 0x7fdb1c06ab10 2023-11-28T12:04:45.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.037+0000 7fdb037fe700 1 -- 172.21.15.57:0/1954465627 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdb0c00f050 con 0x7fdb1c06ab10 2023-11-28T12:04:45.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.037+0000 7fdb037fe700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdb080682f0 0x7fdb0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:45.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.037+0000 7fdb037fe700 1 -- 172.21.15.57:0/1954465627 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdb0c020070 con 0x7fdb1c06ab10 2023-11-28T12:04:45.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.037+0000 7fdb1a7fc700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdb080682f0 0x7fdb0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:45.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.038+0000 7fdb1a7fc700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdb080682f0 0x7fdb0806a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0cd410 tx=0x7fdb10008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:45.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.042+0000 7fdb037fe700 1 -- 172.21.15.57:0/1954465627 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdb0c053b80 con 0x7fdb1c06ab10 2023-11-28T12:04:45.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.330+0000 7fdb215ce700 1 -- 172.21.15.57:0/1954465627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fdb1c0743f0 con 0x7fdb1c06ab10 2023-11-28T12:04:45.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.875+0000 7fdb037fe700 1 -- 172.21.15.57:0/1954465627 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v166) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdb0c02a0f0 con 0x7fdb1c06ab10 2023-11-28T12:04:45.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.878+0000 7fdb215ce700 1 -- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdb080682f0 msgr2=0x7fdb0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:45.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.878+0000 7fdb215ce700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdb080682f0 0x7fdb0806a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0cd410 tx=0x7fdb10008040 comp rx=0 tx=0).stop 2023-11-28T12:04:45.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.878+0000 7fdb215ce700 1 -- 172.21.15.57:0/1954465627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb1c06ab10 msgr2=0x7fdb1c1565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:45.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.878+0000 7fdb215ce700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb1c06ab10 0x7fdb1c1565b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c014040 tx=0x7fdb0c004750 comp rx=0 tx=0).stop 2023-11-28T12:04:45.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.878+0000 7fdb215ce700 1 -- 172.21.15.57:0/1954465627 shutdown_connections 2023-11-28T12:04:45.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.878+0000 7fdb215ce700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdb080682f0 0x7fdb0806a7a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:45.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.878+0000 7fdb215ce700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb1c0cd5a0 0x7fdb1c1f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:45.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.878+0000 7fdb215ce700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb1c06ab10 0x7fdb1c1565b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:45.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.878+0000 7fdb215ce700 1 --2- 172.21.15.57:0/1954465627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb1c0b22a0 0x7fdb1c0cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:45.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.878+0000 7fdb215ce700 1 -- 172.21.15.57:0/1954465627 >> 172.21.15.57:0/1954465627 conn(0x7fdb1c06c520 msgr2=0x7fdb1c0b6450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:45.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.878+0000 7fdb215ce700 1 -- 172.21.15.57:0/1954465627 shutdown_connections 2023-11-28T12:04:45.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:45.878+0000 7fdb215ce700 1 -- 172.21.15.57:0/1954465627 wait complete. 2023-11-28T12:04:45.896 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T12:04:46.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.041+0000 7f34ddcf8700 1 Processor -- start 2023-11-28T12:04:46.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.041+0000 7f34ddcf8700 1 -- start start 2023-11-28T12:04:46.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.041+0000 7f34ddcf8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d815a4d0 0x7f34d806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.041+0000 7f34ddcf8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f34d80cdf20 0x7f34d80cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.041+0000 7f34ddcf8700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d806ac00 con 0x7f34d806a7b0 2023-11-28T12:04:46.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.041+0000 7f34ddcf8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d80ce360 con 0x7f34d80cdf20 2023-11-28T12:04:46.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.041+0000 7f34ddcf8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d815e0d0 con 0x7f34d815a4d0 2023-11-28T12:04:46.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.041+0000 7f34ceffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f34d806a7b0 0x7f34d80cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37766/0 (socket says 172.21.15.57:37766) 2023-11-28T12:04:46.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.041+0000 7f34ceffd700 1 -- 172.21.15.57:0/3595310754 learned_addr learned my addr 172.21.15.57:0/3595310754 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:46.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d7fff700 1 --2- 172.21.15.57:0/3595310754 >> v2:172.21.15.181:3300/0 conn(0x7f34d80cdf20 0x7f34d80cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d77fe700 1 --2- 172.21.15.57:0/3595310754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d815a4d0 0x7f34d806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295700879 0 0) 0x7f34c0001110 con 0x7f34d806a7b0 2023-11-28T12:04:46.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34c40035b0 con 0x7f34d806a7b0 2023-11-28T12:04:46.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4033181314 0 0) 0x7f34c40035b0 con 0x7f34d806a7b0 2023-11-28T12:04:46.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d815a4d0 msgr2=0x7f34d806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:46.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 --2- 172.21.15.57:0/3595310754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d815a4d0 0x7f34d806a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 >> v2:172.21.15.181:3300/0 conn(0x7f34d80cdf20 msgr2=0x7f34d80cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:46.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 --2- 172.21.15.57:0/3595310754 >> v2:172.21.15.181:3300/0 conn(0x7f34d80cdf20 0x7f34d80cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d80ce560 con 0x7f34d806a7b0 2023-11-28T12:04:46.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f34c0001800 con 0x7f34d806a7b0 2023-11-28T12:04:46.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 >> v1:172.21.15.57:6789/0 conn(0x7f34d806a7b0 legacy=0x7f34d80cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:46.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 --2- 172.21.15.57:0/3595310754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34c4003a00 0x7f34c4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 --2- 172.21.15.57:0/3595310754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34c4004430 0x7f34c40048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 --2- 172.21.15.57:0/3595310754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d815a4d0 0x7f34c4004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d80ce360 con 0x7f34c4003a00 2023-11-28T12:04:46.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d815e0d0 con 0x7f34c4004430 2023-11-28T12:04:46.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c400ba10 con 0x7f34d815a4d0 2023-11-28T12:04:46.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d7fff700 1 --2- 172.21.15.57:0/3595310754 >> v2:172.21.15.181:3300/0 conn(0x7f34d80cdf20 0x7f34d80cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:46.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d77fe700 1 --2- 172.21.15.57:0/3595310754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34c4003a00 0x7f34c4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34ceffd700 1 --2- 172.21.15.57:0/3595310754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34c4004430 0x7f34c40048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.042+0000 7f34d7fff700 1 --2- 172.21.15.57:0/3595310754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d815a4d0 0x7f34c4004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.043+0000 7f34d77fe700 1 -- 172.21.15.57:0/3595310754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d815a4d0 msgr2=0x7f34c4004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:46.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.043+0000 7f34d77fe700 1 --2- 172.21.15.57:0/3595310754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d815a4d0 0x7f34c4004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.043+0000 7f34d77fe700 1 -- 172.21.15.57:0/3595310754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34c4004430 msgr2=0x7f34c40048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:46.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.043+0000 7f34d77fe700 1 --2- 172.21.15.57:0/3595310754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34c4004430 0x7f34c40048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.043+0000 7f34d77fe700 1 -- 172.21.15.57:0/3595310754 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c40035b0 con 0x7f34c4003a00 2023-11-28T12:04:46.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.043+0000 7f34ceffd700 1 --2- 172.21.15.57:0/3595310754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34c4004430 0x7f34c40048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:46.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.043+0000 7f34d77fe700 1 --2- 172.21.15.57:0/3595310754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34c4003a00 0x7f34c4003e10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009b60 tx=0x7f34c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36676efca7b2473d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:46.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.043+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c800e6c0 con 0x7f34c4003a00 2023-11-28T12:04:46.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.043+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c800e820 con 0x7f34c4003a00 2023-11-28T12:04:46.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.043+0000 7f34d6ffd700 1 -- 172.21.15.57:0/3595310754 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c800eb90 con 0x7f34c4003a00 2023-11-28T12:04:46.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.043+0000 7f34ddcf8700 1 -- 172.21.15.57:0/3595310754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34c4003a00 msgr2=0x7f34c4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:46.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.043+0000 7f34ddcf8700 1 --2- 172.21.15.57:0/3595310754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34c4003a00 0x7f34c4003e10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009b60 tx=0x7f34c4005880 comp rx=0 tx=0).stop 2023-11-28T12:04:46.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.044+0000 7f34d7fff700 1 -- 172.21.15.57:0/3595310754 reap_dead start 2023-11-28T12:04:46.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.044+0000 7f34ddcf8700 1 -- 172.21.15.57:0/3595310754 shutdown_connections 2023-11-28T12:04:46.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.044+0000 7f34ddcf8700 1 -- 172.21.15.57:0/3595310754 >> 172.21.15.57:0/3595310754 conn(0x7f34d806c010 msgr2=0x7f34d80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:46.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.044+0000 7f34ddcf8700 1 -- 172.21.15.57:0/3595310754 shutdown_connections 2023-11-28T12:04:46.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.044+0000 7f34ddcf8700 1 -- 172.21.15.57:0/3595310754 wait complete. 2023-11-28T12:04:46.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.045+0000 7f34ddcf8700 1 Processor -- start 2023-11-28T12:04:46.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.045+0000 7f34ddcf8700 1 -- start start 2023-11-28T12:04:46.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ddcf8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34c4003a00 0x7f34d80b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ddcf8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34d80c9ae0 0x7f34d81e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ddcf8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34d81e5bf0 0x7f34d814e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ddcf8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d81e6060 con 0x7f34d80c9ae0 2023-11-28T12:04:46.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ddcf8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d81e6250 con 0x7f34d81e5bf0 2023-11-28T12:04:46.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ddcf8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d81e6530 con 0x7f34c4003a00 2023-11-28T12:04:46.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34d80c9ae0 0x7f34d81e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34d80c9ae0 0x7f34d81e56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54566/0 (socket says 172.21.15.57:54566) 2023-11-28T12:04:46.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ceffd700 1 -- 172.21.15.57:0/287346019 learned_addr learned my addr 172.21.15.57:0/287346019 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:46.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34d7fff700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34d81e5bf0 0x7f34d814e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34d77fe700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34c4003a00 0x7f34d80b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ceffd700 1 -- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34c4003a00 msgr2=0x7f34d80b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:46.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ceffd700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34c4003a00 0x7f34d80b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ceffd700 1 -- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34d81e5bf0 msgr2=0x7f34d814e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:46.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ceffd700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34d81e5bf0 0x7f34d814e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ceffd700 1 -- 172.21.15.57:0/287346019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d815e600 con 0x7f34d80c9ae0 2023-11-28T12:04:46.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34d7fff700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34d81e5bf0 0x7f34d814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:46.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ceffd700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34d80c9ae0 0x7f34d81e56b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f34c0001390 tx=0x7f34c00075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:46.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34d4ff9700 1 -- 172.21.15.57:0/287346019 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c00077d0 con 0x7f34d80c9ae0 2023-11-28T12:04:46.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34d4ff9700 1 -- 172.21.15.57:0/287346019 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c000f8a0 con 0x7f34d80c9ae0 2023-11-28T12:04:46.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ddcf8700 1 -- 172.21.15.57:0/287346019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d815e320 con 0x7f34d80c9ae0 2023-11-28T12:04:46.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34d4ff9700 1 -- 172.21.15.57:0/287346019 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c000fc50 con 0x7f34d80c9ae0 2023-11-28T12:04:46.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.047+0000 7f34ddcf8700 1 -- 172.21.15.57:0/287346019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d815e8e0 con 0x7f34d80c9ae0 2023-11-28T12:04:46.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.048+0000 7f34ddcf8700 1 -- 172.21.15.57:0/287346019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d80743f0 con 0x7f34d80c9ae0 2023-11-28T12:04:46.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.048+0000 7f34d4ff9700 1 -- 172.21.15.57:0/287346019 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f34c000fdb0 con 0x7f34d80c9ae0 2023-11-28T12:04:46.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.049+0000 7f34d4ff9700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f34b80682f0 0x7f34b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.049+0000 7f34d4ff9700 1 -- 172.21.15.57:0/287346019 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f34c0085bb0 con 0x7f34d80c9ae0 2023-11-28T12:04:46.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.049+0000 7f34d77fe700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f34b80682f0 0x7f34b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.050+0000 7f34d77fe700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f34b80682f0 0x7f34b806a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f34c80090b0 tx=0x7f34c8008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:46.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.053+0000 7f34d4ff9700 1 -- 172.21.15.57:0/287346019 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f34c0052730 con 0x7f34d80c9ae0 2023-11-28T12:04:46.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.254+0000 7f34ddcf8700 1 -- 172.21.15.57:0/287346019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f34d80cf3c0 con 0x7f34d80c9ae0 2023-11-28T12:04:46.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.259+0000 7f34d4ff9700 1 -- 172.21.15.57:0/287346019 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f34c0055d50 con 0x7f34d80c9ae0 2023-11-28T12:04:46.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.261+0000 7f34ddcf8700 1 -- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f34b80682f0 msgr2=0x7f34b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:46.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.261+0000 7f34ddcf8700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f34b80682f0 0x7f34b806a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f34c80090b0 tx=0x7f34c8008030 comp rx=0 tx=0).stop 2023-11-28T12:04:46.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.261+0000 7f34ddcf8700 1 -- 172.21.15.57:0/287346019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34d80c9ae0 msgr2=0x7f34d81e56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:46.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.261+0000 7f34ddcf8700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34d80c9ae0 0x7f34d81e56b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f34c0001390 tx=0x7f34c00075f0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.261+0000 7f34ddcf8700 1 -- 172.21.15.57:0/287346019 shutdown_connections 2023-11-28T12:04:46.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.261+0000 7f34ddcf8700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f34b80682f0 0x7f34b806a7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.261+0000 7f34ddcf8700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34d81e5bf0 0x7f34d814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.261+0000 7f34ddcf8700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34c4003a00 0x7f34d80b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.261+0000 7f34ddcf8700 1 --2- 172.21.15.57:0/287346019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34d80c9ae0 0x7f34d81e56b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.261+0000 7f34ddcf8700 1 -- 172.21.15.57:0/287346019 >> 172.21.15.57:0/287346019 conn(0x7f34d806c010 msgr2=0x7f34d80ce930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:46.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.261+0000 7f34ddcf8700 1 -- 172.21.15.57:0/287346019 shutdown_connections 2023-11-28T12:04:46.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.261+0000 7f34ddcf8700 1 -- 172.21.15.57:0/287346019 wait complete. 2023-11-28T12:04:46.276 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T12:04:46.276 DEBUG:teuthology.orchestra.run.smithi057:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T12:04:46.329 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T12:04:46.380 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T12:04:46.381 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:04:46.381 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:04:46.403 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16254224k,nr_inodes=4063556,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15749 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3254388k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T12:04:46.404 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:04:46.404 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:04:46.424 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251064k,nr_inodes=4062766,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14954 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253752k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T12:04:46.424 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:04:46.425 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:04:46.467 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16254224k,nr_inodes=4063556,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15749 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3254388k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T12:04:46.467 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:04:46.467 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:04:46.487 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251064k,nr_inodes=4062766,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14954 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253752k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T12:04:46.487 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T12:04:46.488 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:04:46.562 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:04:46.564 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:04:46.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.652+0000 7fb4a44f2700 1 Processor -- start 2023-11-28T12:04:46.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.652+0000 7fb4a44f2700 1 -- start start 2023-11-28T12:04:46.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.653+0000 7fb4a44f2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb49c06e400 0x7fb49c0b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.653+0000 7fb4a44f2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb49c160e70 0x7fb49c15c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.653+0000 7fb4a44f2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49c06e900 con 0x7fb49c0d0c60 2023-11-28T12:04:46.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.653+0000 7fb4a44f2700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49c160390 con 0x7fb49c06e400 2023-11-28T12:04:46.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.653+0000 7fb4a44f2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c160670 con 0x7fb49c160e70 2023-11-28T12:04:46.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.653+0000 7fb4a1a8d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb49c06e400 0x7fb49c0b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.653+0000 7fb4a1a8d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb49c06e400 0x7fb49c0b5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54576/0 (socket says 172.21.15.57:54576) 2023-11-28T12:04:46.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.653+0000 7fb4a1a8d700 1 -- 172.21.15.57:0/4015438328 learned_addr learned my addr 172.21.15.57:0/4015438328 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.653+0000 7fb4a2a8f700 1 --2- 172.21.15.57:0/4015438328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb49c160e70 0x7fb49c15c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.653+0000 7fb4a1a8d700 1 -- 172.21.15.57:0/4015438328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb49c160e70 msgr2=0x7fb49c15c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.653+0000 7fb4a1a8d700 1 --2- 172.21.15.57:0/4015438328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb49c160e70 0x7fb49c15c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.653+0000 7fb4a1a8d700 1 -- 172.21.15.57:0/4015438328 >> v1:172.21.15.181:6789/0 conn(0x7fb49c0d0c60 legacy=0x7fb49c0d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.653+0000 7fb4a1a8d700 1 -- 172.21.15.57:0/4015438328 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c160950 con 0x7fb49c06e400 2023-11-28T12:04:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.654+0000 7fb4a1a8d700 1 --2- 172.21.15.57:0/4015438328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb49c06e400 0x7fb49c0b5be0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb4980098f0 tx=0x7fb498009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393ad867a7c08411 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.654+0000 7fb4a128c700 1 -- 172.21.15.57:0/4015438328 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498011830 con 0x7fb49c06e400 2023-11-28T12:04:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.654+0000 7fb4a128c700 1 -- 172.21.15.57:0/4015438328 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb498011990 con 0x7fb49c06e400 2023-11-28T12:04:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.654+0000 7fb4a128c700 1 -- 172.21.15.57:0/4015438328 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498011d20 con 0x7fb49c06e400 2023-11-28T12:04:46.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.654+0000 7fb4a44f2700 1 -- 172.21.15.57:0/4015438328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb49c06e400 msgr2=0x7fb49c0b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:46.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.654+0000 7fb4a44f2700 1 --2- 172.21.15.57:0/4015438328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb49c06e400 0x7fb49c0b5be0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb4980098f0 tx=0x7fb498009c00 comp rx=0 tx=0).stop 2023-11-28T12:04:46.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.654+0000 7fb4a44f2700 1 -- 172.21.15.57:0/4015438328 shutdown_connections 2023-11-28T12:04:46.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.654+0000 7fb4a44f2700 1 --2- 172.21.15.57:0/4015438328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb49c160e70 0x7fb49c15c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.654+0000 7fb4a44f2700 1 --2- 172.21.15.57:0/4015438328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb49c06e400 0x7fb49c0b5be0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.654+0000 7fb4a44f2700 1 -- 172.21.15.57:0/4015438328 >> 172.21.15.57:0/4015438328 conn(0x7fb49c06a270 msgr2=0x7fb49c06a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:46.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.655+0000 7fb4a44f2700 1 -- 172.21.15.57:0/4015438328 shutdown_connections 2023-11-28T12:04:46.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.655+0000 7fb4a44f2700 1 -- 172.21.15.57:0/4015438328 wait complete. 2023-11-28T12:04:46.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.656+0000 7fb4a44f2700 1 Processor -- start 2023-11-28T12:04:46.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.656+0000 7fb4a44f2700 1 -- start start 2023-11-28T12:04:46.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a44f2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb49c06e400 0x7fb49c1489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a44f2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb49c0d0c60 0x7fb49c15e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a44f2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb49c160e70 0x7fb49c15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a44f2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c1550b0 con 0x7fb49c160e70 2023-11-28T12:04:46.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a44f2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c152bd0 con 0x7fb49c0d0c60 2023-11-28T12:04:46.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a44f2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c155300 con 0x7fb49c06e400 2023-11-28T12:04:46.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a2a8f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb49c160e70 0x7fb49c15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a2a8f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb49c160e70 0x7fb49c15ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54584/0 (socket says 172.21.15.57:54584) 2023-11-28T12:04:46.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a2a8f700 1 -- 172.21.15.57:0/1319184117 learned_addr learned my addr 172.21.15.57:0/1319184117 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:46.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a1a8d700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb49c0d0c60 0x7fb49c15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a228e700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb49c06e400 0x7fb49c1489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a2a8f700 1 -- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb49c06e400 msgr2=0x7fb49c1489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:46.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a2a8f700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb49c06e400 0x7fb49c1489e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a2a8f700 1 -- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb49c0d0c60 msgr2=0x7fb49c15e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:46.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a2a8f700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb49c0d0c60 0x7fb49c15e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a2a8f700 1 -- 172.21.15.57:0/1319184117 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb498009550 con 0x7fb49c160e70 2023-11-28T12:04:46.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a2a8f700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb49c160e70 0x7fb49c15ebb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb49400efc0 tx=0x7fb494009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:46.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb492ffd700 1 -- 172.21.15.57:0/1319184117 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb494009b20 con 0x7fb49c160e70 2023-11-28T12:04:46.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb492ffd700 1 -- 172.21.15.57:0/1319184117 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49400f040 con 0x7fb49c160e70 2023-11-28T12:04:46.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a44f2700 1 -- 172.21.15.57:0/1319184117 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49c15ceb0 con 0x7fb49c160e70 2023-11-28T12:04:46.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb492ffd700 1 -- 172.21.15.57:0/1319184117 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49400cc90 con 0x7fb49c160e70 2023-11-28T12:04:46.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.657+0000 7fb4a44f2700 1 -- 172.21.15.57:0/1319184117 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c0d0740 con 0x7fb49c160e70 2023-11-28T12:04:46.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.658+0000 7fb4a44f2700 1 -- 172.21.15.57:0/1319184117 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49c15d040 con 0x7fb49c160e70 2023-11-28T12:04:46.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.659+0000 7fb492ffd700 1 -- 172.21.15.57:0/1319184117 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb494014480 con 0x7fb49c160e70 2023-11-28T12:04:46.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.660+0000 7fb492ffd700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb4800682f0 0x7fb48006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:46.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.660+0000 7fb492ffd700 1 -- 172.21.15.57:0/1319184117 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb4940865b0 con 0x7fb49c160e70 2023-11-28T12:04:46.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.660+0000 7fb4a228e700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb4800682f0 0x7fb48006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:46.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.660+0000 7fb4a228e700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb4800682f0 0x7fb48006a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb48c007fd0 tx=0x7fb48c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:46.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.664+0000 7fb492ffd700 1 -- 172.21.15.57:0/1319184117 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb494052fe0 con 0x7fb49c160e70 2023-11-28T12:04:46.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.908+0000 7fb4a44f2700 1 -- 172.21.15.57:0/1319184117 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb49c0743f0 con 0x7fb49c160e70 2023-11-28T12:04:46.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.910+0000 7fb492ffd700 1 -- 172.21.15.57:0/1319184117 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fb494018090 con 0x7fb49c160e70 2023-11-28T12:04:46.910 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:46.910 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1773283092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1773283092},{"type":"v1","addr":"172.21.15.181:6837","nonce":1773283092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/3718600227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3718600227},{"type":"v1","addr":"172.21.15.57:6834","nonce":3718600227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/762261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":762261630},{"type":"v1","addr":"172.21.15.57:6835","nonce":762261630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2643817233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2643817233},{"type":"v1","addr":"172.21.15.181:6836","nonce":2643817233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-11-28T12:04:46.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.912+0000 7fb4a44f2700 1 -- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb4800682f0 msgr2=0x7fb48006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:46.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.912+0000 7fb4a44f2700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb4800682f0 0x7fb48006a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb48c007fd0 tx=0x7fb48c009040 comp rx=0 tx=0).stop 2023-11-28T12:04:46.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.912+0000 7fb4a44f2700 1 -- 172.21.15.57:0/1319184117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb49c160e70 msgr2=0x7fb49c15ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:46.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.912+0000 7fb4a44f2700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb49c160e70 0x7fb49c15ebb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb49400efc0 tx=0x7fb494009f60 comp rx=0 tx=0).stop 2023-11-28T12:04:46.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.912+0000 7fb4a44f2700 1 -- 172.21.15.57:0/1319184117 shutdown_connections 2023-11-28T12:04:46.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.912+0000 7fb4a44f2700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb4800682f0 0x7fb48006a7a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.912+0000 7fb4a44f2700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb49c160e70 0x7fb49c15ebb0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.912+0000 7fb4a44f2700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb49c06e400 0x7fb49c1489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.912+0000 7fb4a44f2700 1 --2- 172.21.15.57:0/1319184117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb49c0d0c60 0x7fb49c15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:46.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.912+0000 7fb4a44f2700 1 -- 172.21.15.57:0/1319184117 >> 172.21.15.57:0/1319184117 conn(0x7fb49c06a270 msgr2=0x7fb49c159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:46.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.912+0000 7fb4a44f2700 1 -- 172.21.15.57:0/1319184117 shutdown_connections 2023-11-28T12:04:46.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:46.912+0000 7fb4a44f2700 1 -- 172.21.15.57:0/1319184117 wait complete. 2023-11-28T12:04:46.914 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 26 2023-11-28T12:04:46.926 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T12:04:46.926 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T12:04:46.926 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T12:04:46.926 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:04:46.927 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T12:04:46.927 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T12:04:46.927 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T12:04:46.927 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:04:46.927 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T12:04:46.927 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T12:04:46.928 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T12:04:46.928 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:04:46.928 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T12:04:46.928 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T12:04:46.928 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T12:04:46.928 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:04:46.939 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T12:04:46.939 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T12:04:46.940 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T12:04:46.940 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T12:04:46.980 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T12:04:46.980 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T12:04:46.981 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T12:04:46.981 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T12:04:46.983 INFO:tasks.ceph.mds.b:Started 2023-11-28T12:04:46.984 DEBUG:teuthology.parallel:result is None 2023-11-28T12:04:46.984 INFO:tasks.ceph.mds.d:Started 2023-11-28T12:04:46.984 DEBUG:teuthology.parallel:result is None 2023-11-28T12:04:47.023 INFO:tasks.ceph.mds.a:Started 2023-11-28T12:04:47.023 DEBUG:teuthology.parallel:result is None 2023-11-28T12:04:47.024 INFO:tasks.ceph.mds.c:Started 2023-11-28T12:04:47.024 DEBUG:teuthology.parallel:result is None 2023-11-28T12:04:47.024 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T12:04:47.073 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-11-28T12:04:47.074 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-11-28T12:04:47.120 INFO:tasks.ceph.mds.a.smithi057.stdout:starting mds.a at 2023-11-28T12:04:47.121 INFO:tasks.ceph.mds.c.smithi057.stdout:starting mds.c at 2023-11-28T12:04:47.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.176+0000 7f716366b700 1 Processor -- start 2023-11-28T12:04:47.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.176+0000 7f716366b700 1 -- start start 2023-11-28T12:04:47.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.177+0000 7f716366b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c15c8c0 0x7f715c160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:47.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.177+0000 7f716366b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f715c0b5dc0 0x7f715c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:47.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.177+0000 7f716366b700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715c0d2d30 con 0x7f715c06b050 2023-11-28T12:04:47.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.177+0000 7f716366b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c06b4d0 con 0x7f715c0b5dc0 2023-11-28T12:04:47.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.177+0000 7f716366b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c0d0b40 con 0x7f715c15c8c0 2023-11-28T12:04:47.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.177+0000 7f7161c08700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f715c06b050 0x7f715c0d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37774/0 (socket says 172.21.15.57:37774) 2023-11-28T12:04:47.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.177+0000 7f7161c08700 1 -- 172.21.15.57:0/2591055282 learned_addr learned my addr 172.21.15.57:0/2591055282 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:47.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.178+0000 7f7160c06700 1 --2- 172.21.15.57:0/2591055282 >> v2:172.21.15.181:3300/0 conn(0x7f715c0b5dc0 0x7f715c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:47.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.178+0000 7f7161407700 1 --2- 172.21.15.57:0/2591055282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c15c8c0 0x7f715c160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:47.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.178+0000 7f7153fff700 1 -- 172.21.15.57:0/2591055282 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684051038 0 0) 0x7f7154001490 con 0x7f715c06b050 2023-11-28T12:04:47.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.178+0000 7f7153fff700 1 -- 172.21.15.57:0/2591055282 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71480035b0 con 0x7f715c06b050 2023-11-28T12:04:47.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.178+0000 7f7161407700 1 -- 172.21.15.57:0/2591055282 >> v2:172.21.15.181:3300/0 conn(0x7f715c0b5dc0 msgr2=0x7f715c06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:47.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.178+0000 7f7161407700 1 --2- 172.21.15.57:0/2591055282 >> v2:172.21.15.181:3300/0 conn(0x7f715c0b5dc0 0x7f715c06ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:47.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.178+0000 7f7161407700 1 -- 172.21.15.57:0/2591055282 >> v1:172.21.15.57:6789/0 conn(0x7f715c06b050 legacy=0x7f715c0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:47.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.178+0000 7f7161407700 1 -- 172.21.15.57:0/2591055282 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c1611f0 con 0x7f715c15c8c0 2023-11-28T12:04:47.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.178+0000 7f7161407700 1 --2- 172.21.15.57:0/2591055282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c15c8c0 0x7f715c160cb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7158009840 tx=0x7f7158009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed945e18990d826a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:47.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.178+0000 7f7153fff700 1 -- 172.21.15.57:0/2591055282 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7158011830 con 0x7f715c15c8c0 2023-11-28T12:04:47.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.178+0000 7f7153fff700 1 -- 172.21.15.57:0/2591055282 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7158011990 con 0x7f715c15c8c0 2023-11-28T12:04:47.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.178+0000 7f7153fff700 1 -- 172.21.15.57:0/2591055282 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7158011d20 con 0x7f715c15c8c0 2023-11-28T12:04:47.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.179+0000 7f716366b700 1 -- 172.21.15.57:0/2591055282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c15c8c0 msgr2=0x7f715c160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:47.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.179+0000 7f716366b700 1 --2- 172.21.15.57:0/2591055282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c15c8c0 0x7f715c160cb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7158009840 tx=0x7f7158009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:47.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.179+0000 7f716366b700 1 -- 172.21.15.57:0/2591055282 shutdown_connections 2023-11-28T12:04:47.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.179+0000 7f716366b700 1 --2- 172.21.15.57:0/2591055282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c15c8c0 0x7f715c160cb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:47.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.179+0000 7f716366b700 1 --2- 172.21.15.57:0/2591055282 >> v2:172.21.15.181:3300/0 conn(0x7f715c0b5dc0 0x7f715c06ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:47.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.179+0000 7f716366b700 1 -- 172.21.15.57:0/2591055282 >> 172.21.15.57:0/2591055282 conn(0x7f715c06c520 msgr2=0x7f715c06c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:47.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.179+0000 7f716366b700 1 -- 172.21.15.57:0/2591055282 shutdown_connections 2023-11-28T12:04:47.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.179+0000 7f716366b700 1 -- 172.21.15.57:0/2591055282 wait complete. 2023-11-28T12:04:47.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.180+0000 7f716366b700 1 Processor -- start 2023-11-28T12:04:47.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.180+0000 7f716366b700 1 -- start start 2023-11-28T12:04:47.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.181+0000 7f716366b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f715c06b050 0x7f715c14d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:47.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.181+0000 7f716366b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c0b5dc0 0x7f715c1504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:47.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.181+0000 7f716366b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715c15c8c0 0x7f715c0d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:47.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.181+0000 7f716366b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c14dd10 con 0x7f715c15c8c0 2023-11-28T12:04:47.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.181+0000 7f716366b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c15dc60 con 0x7f715c06b050 2023-11-28T12:04:47.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.181+0000 7f716366b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c15df40 con 0x7f715c0b5dc0 2023-11-28T12:04:47.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.181+0000 7f7161c08700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715c15c8c0 0x7f715c0d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:47.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.181+0000 7f7161c08700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715c15c8c0 0x7f715c0d2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54624/0 (socket says 172.21.15.57:54624) 2023-11-28T12:04:47.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.181+0000 7f7161c08700 1 -- 172.21.15.57:0/3396976640 learned_addr learned my addr 172.21.15.57:0/3396976640 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:47.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.181+0000 7f7161407700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f715c06b050 0x7f715c14d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:47.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.181+0000 7f7160c06700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c0b5dc0 0x7f715c1504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:47.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.182+0000 7f7161c08700 1 -- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c0b5dc0 msgr2=0x7f715c1504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:47.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.182+0000 7f7161c08700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c0b5dc0 0x7f715c1504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:47.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.182+0000 7f7161c08700 1 -- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f715c06b050 msgr2=0x7f715c14d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:47.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.182+0000 7f7161c08700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f715c06b050 0x7f715c14d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:47.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.182+0000 7f7161c08700 1 -- 172.21.15.57:0/3396976640 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71580094a0 con 0x7f715c15c8c0 2023-11-28T12:04:47.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.182+0000 7f7160c06700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c0b5dc0 0x7f715c1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:47.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.182+0000 7f7161c08700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715c15c8c0 0x7f715c0d2f00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f715400cae0 tx=0x7f715400cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:47.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.182+0000 7f7151ffb700 1 -- 172.21.15.57:0/3396976640 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400aa60 con 0x7f715c15c8c0 2023-11-28T12:04:47.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.182+0000 7f7151ffb700 1 -- 172.21.15.57:0/3396976640 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715400abc0 con 0x7f715c15c8c0 2023-11-28T12:04:47.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.182+0000 7f7151ffb700 1 -- 172.21.15.57:0/3396976640 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154012400 con 0x7f715c15c8c0 2023-11-28T12:04:47.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.182+0000 7f716366b700 1 -- 172.21.15.57:0/3396976640 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c0cd760 con 0x7f715c15c8c0 2023-11-28T12:04:47.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.182+0000 7f716366b700 1 -- 172.21.15.57:0/3396976640 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c0d2a50 con 0x7f715c15c8c0 2023-11-28T12:04:47.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.183+0000 7f716366b700 1 -- 172.21.15.57:0/3396976640 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715c0cda40 con 0x7f715c15c8c0 2023-11-28T12:04:47.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.184+0000 7f7151ffb700 1 -- 172.21.15.57:0/3396976640 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7154019030 con 0x7f715c15c8c0 2023-11-28T12:04:47.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.184+0000 7f7151ffb700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f71400682f0 0x7f714006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:47.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.184+0000 7f7151ffb700 1 -- 172.21.15.57:0/3396976640 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7154085350 con 0x7f715c15c8c0 2023-11-28T12:04:47.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.184+0000 7f7161407700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f71400682f0 0x7f714006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:47.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.185+0000 7f7161407700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f71400682f0 0x7f714006a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7158000c00 tx=0x7f7158008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:47.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.188+0000 7f7151ffb700 1 -- 172.21.15.57:0/3396976640 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7154051e20 con 0x7f715c15c8c0 2023-11-28T12:04:47.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:47.403+0000 7f716366b700 1 -- 172.21.15.57:0/3396976640 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f715c067d70 con 0x7f715c15c8c0 2023-11-28T12:04:48.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.170+0000 7f7151ffb700 1 -- 172.21.15.57:0/3396976640 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7154017080 con 0x7f715c15c8c0 2023-11-28T12:04:48.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.173+0000 7f716366b700 1 -- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f71400682f0 msgr2=0x7f714006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:48.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.173+0000 7f716366b700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f71400682f0 0x7f714006a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7158000c00 tx=0x7f7158008030 comp rx=0 tx=0).stop 2023-11-28T12:04:48.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.173+0000 7f716366b700 1 -- 172.21.15.57:0/3396976640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715c15c8c0 msgr2=0x7f715c0d2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:48.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.173+0000 7f716366b700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715c15c8c0 0x7f715c0d2f00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f715400cae0 tx=0x7f715400cdf0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.173+0000 7f716366b700 1 -- 172.21.15.57:0/3396976640 shutdown_connections 2023-11-28T12:04:48.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.173+0000 7f716366b700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f71400682f0 0x7f714006a7a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.173+0000 7f716366b700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f715c15c8c0 0x7f715c0d2f00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.173+0000 7f716366b700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f715c06b050 0x7f715c14d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.173+0000 7f716366b700 1 --2- 172.21.15.57:0/3396976640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f715c0b5dc0 0x7f715c1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.173+0000 7f716366b700 1 -- 172.21.15.57:0/3396976640 >> 172.21.15.57:0/3396976640 conn(0x7f715c06c520 msgr2=0x7f715c0b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:48.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.173+0000 7f716366b700 1 -- 172.21.15.57:0/3396976640 shutdown_connections 2023-11-28T12:04:48.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.173+0000 7f716366b700 1 -- 172.21.15.57:0/3396976640 wait complete. 2023-11-28T12:04:48.175 INFO:teuthology.orchestra.run.smithi057.stderr: removed all blocklist entries 2023-11-28T12:04:48.188 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T12:04:48.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.328+0000 7fc5deb27700 1 Processor -- start 2023-11-28T12:04:48.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.328+0000 7fc5deb27700 1 -- start start 2023-11-28T12:04:48.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.328+0000 7fc5deb27700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d806a270 0x7fc5d806a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:48.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5deb27700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5d8070d90 0x7fc5d80711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:48.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5deb27700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d8151a50 con 0x7fc5d806da40 2023-11-28T12:04:48.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5deb27700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d815f0c0 con 0x7fc5d8070d90 2023-11-28T12:04:48.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5deb27700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d815b440 con 0x7fc5d806a270 2023-11-28T12:04:48.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5dd0c4700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc5d806da40 0x7fc5d806dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37778/0 (socket says 172.21.15.57:37778) 2023-11-28T12:04:48.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5dd0c4700 1 -- 172.21.15.57:0/178588787 learned_addr learned my addr 172.21.15.57:0/178588787 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:48.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5d7fff700 1 --2- 172.21.15.57:0/178588787 >> v2:172.21.15.181:3300/0 conn(0x7fc5d8070d90 0x7fc5d80711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:48.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5dc8c3700 1 --2- 172.21.15.57:0/178588787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d806a270 0x7fc5d806a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:48.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5d77fe700 1 -- 172.21.15.57:0/178588787 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251612131 0 0) 0x7fc5d8151a50 con 0x7fc5d806da40 2023-11-28T12:04:48.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5d77fe700 1 -- 172.21.15.57:0/178588787 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5c40035b0 con 0x7fc5d806da40 2023-11-28T12:04:48.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5d7fff700 1 -- 172.21.15.57:0/178588787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d806a270 msgr2=0x7fc5d806a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:48.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5d7fff700 1 --2- 172.21.15.57:0/178588787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d806a270 0x7fc5d806a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5d7fff700 1 -- 172.21.15.57:0/178588787 >> v1:172.21.15.57:6789/0 conn(0x7fc5d806da40 legacy=0x7fc5d806dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:48.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5d7fff700 1 -- 172.21.15.57:0/178588787 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d806ab80 con 0x7fc5d8070d90 2023-11-28T12:04:48.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.329+0000 7fc5dc8c3700 1 --2- 172.21.15.57:0/178588787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d806a270 0x7fc5d806a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:48.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.330+0000 7fc5d7fff700 1 --2- 172.21.15.57:0/178588787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d8070d90 0x7fc5d80711e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009840 tx=0x7fc5c0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=586bef434aa4154f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:48.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.330+0000 7fc5d77fe700 1 -- 172.21.15.57:0/178588787 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0011830 con 0x7fc5d8070d90 2023-11-28T12:04:48.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.330+0000 7fc5d77fe700 1 -- 172.21.15.57:0/178588787 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c0011990 con 0x7fc5d8070d90 2023-11-28T12:04:48.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.330+0000 7fc5d77fe700 1 -- 172.21.15.57:0/178588787 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0011d20 con 0x7fc5d8070d90 2023-11-28T12:04:48.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.330+0000 7fc5deb27700 1 -- 172.21.15.57:0/178588787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d8070d90 msgr2=0x7fc5d80711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:48.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.330+0000 7fc5deb27700 1 --2- 172.21.15.57:0/178588787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d8070d90 0x7fc5d80711e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009840 tx=0x7fc5c0009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:48.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.330+0000 7fc5deb27700 1 -- 172.21.15.57:0/178588787 shutdown_connections 2023-11-28T12:04:48.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.330+0000 7fc5deb27700 1 --2- 172.21.15.57:0/178588787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d806a270 0x7fc5d806a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.330+0000 7fc5deb27700 1 --2- 172.21.15.57:0/178588787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d8070d90 0x7fc5d80711e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.330+0000 7fc5deb27700 1 -- 172.21.15.57:0/178588787 >> 172.21.15.57:0/178588787 conn(0x7fc5d806c010 msgr2=0x7fc5d80cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:48.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.330+0000 7fc5deb27700 1 -- 172.21.15.57:0/178588787 shutdown_connections 2023-11-28T12:04:48.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.330+0000 7fc5deb27700 1 -- 172.21.15.57:0/178588787 wait complete. 2023-11-28T12:04:48.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.332+0000 7fc5deb27700 1 Processor -- start 2023-11-28T12:04:48.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.332+0000 7fc5deb27700 1 -- start start 2023-11-28T12:04:48.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5deb27700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d806a270 0x7fc5d81f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:48.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5deb27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d806da40 0x7fc5d81e58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:48.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5deb27700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5d8070d90 0x7fc5d80b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:48.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5deb27700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d81f4710 con 0x7fc5d8070d90 2023-11-28T12:04:48.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5deb27700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80c7eb0 con 0x7fc5d806da40 2023-11-28T12:04:48.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5deb27700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d81f07c0 con 0x7fc5d806a270 2023-11-28T12:04:48.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5dd0c4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5d8070d90 0x7fc5d80b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:48.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5dd0c4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5d8070d90 0x7fc5d80b4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54636/0 (socket says 172.21.15.57:54636) 2023-11-28T12:04:48.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5dd0c4700 1 -- 172.21.15.57:0/876008477 learned_addr learned my addr 172.21.15.57:0/876008477 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:48.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5d7fff700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d806da40 0x7fc5d81e58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:48.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5dc8c3700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d806a270 0x7fc5d81f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:48.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5dd0c4700 1 -- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d806a270 msgr2=0x7fc5d81f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:48.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5dd0c4700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d806a270 0x7fc5d81f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5dd0c4700 1 -- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d806da40 msgr2=0x7fc5d81e58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:48.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5dd0c4700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d806da40 0x7fc5d81e58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5dd0c4700 1 -- 172.21.15.57:0/876008477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c80094a0 con 0x7fc5d8070d90 2023-11-28T12:04:48.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.333+0000 7fc5dd0c4700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5d8070d90 0x7fc5d80b4b30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000cb40 tx=0x7fc5d000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:48.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.334+0000 7fc5d57fa700 1 -- 172.21.15.57:0/876008477 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d000a9e0 con 0x7fc5d8070d90 2023-11-28T12:04:48.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.334+0000 7fc5d57fa700 1 -- 172.21.15.57:0/876008477 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d000ab40 con 0x7fc5d8070d90 2023-11-28T12:04:48.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.334+0000 7fc5d57fa700 1 -- 172.21.15.57:0/876008477 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0012400 con 0x7fc5d8070d90 2023-11-28T12:04:48.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.334+0000 7fc5deb27700 1 -- 172.21.15.57:0/876008477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c00094a0 con 0x7fc5d8070d90 2023-11-28T12:04:48.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.334+0000 7fc5deb27700 1 -- 172.21.15.57:0/876008477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d815adf0 con 0x7fc5d8070d90 2023-11-28T12:04:48.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.335+0000 7fc5deb27700 1 -- 172.21.15.57:0/876008477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d8071ac0 con 0x7fc5d8070d90 2023-11-28T12:04:48.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.336+0000 7fc5d57fa700 1 -- 172.21.15.57:0/876008477 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc5d0019030 con 0x7fc5d8070d90 2023-11-28T12:04:48.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.336+0000 7fc5d57fa700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5b80682f0 0x7fc5b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:48.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.337+0000 7fc5d57fa700 1 -- 172.21.15.57:0/876008477 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc5d00857b0 con 0x7fc5d8070d90 2023-11-28T12:04:48.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.337+0000 7fc5dc8c3700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5b80682f0 0x7fc5b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:48.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.337+0000 7fc5dc8c3700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5b80682f0 0x7fc5b806a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009d70 tx=0x7fc5c8009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:48.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.340+0000 7fc5d57fa700 1 -- 172.21.15.57:0/876008477 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc5d00524b0 con 0x7fc5d8070d90 2023-11-28T12:04:48.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.626+0000 7fc5deb27700 1 -- 172.21.15.57:0/876008477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc5d815b0d0 con 0x7fc5d8070d90 2023-11-28T12:04:48.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.626+0000 7fc5d57fa700 1 -- 172.21.15.57:0/876008477 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fc5d0055ad0 con 0x7fc5d8070d90 2023-11-28T12:04:48.627 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:48.627 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:04:48.627 INFO:teuthology.orchestra.run.smithi057.stdout: "auth_dump": [ 2023-11-28T12:04:48.628 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.628 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.a", 2023-11-28T12:04:48.628 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVlFqRBJBAAxYPlgmjdxnRzi0h9JBB1BQ==", 2023-11-28T12:04:48.628 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.628 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:04:48.629 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:04:48.629 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:04:48.629 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.629 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.630 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.630 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.630 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.b", 2023-11-28T12:04:48.630 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVl0d0ZMhAAykXowuVhIqRSjBr/XMvanw==", 2023-11-28T12:04:48.630 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.631 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:04:48.631 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:04:48.631 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:04:48.631 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.632 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.632 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.632 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.632 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.c", 2023-11-28T12:04:48.632 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVl4Z2qLBAACYGN7Glu6Bv2yLio50flgQ==", 2023-11-28T12:04:48.633 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.633 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:04:48.634 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:04:48.634 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:04:48.634 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.634 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.634 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.635 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.635 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.d", 2023-11-28T12:04:48.635 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVl2P5jORAAR7iZXovJiiCXuxlaaZJ75Q==", 2023-11-28T12:04:48.635 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.636 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:04:48.636 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:04:48.636 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:04:48.636 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.636 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.637 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.637 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.637 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.0", 2023-11-28T12:04:48.637 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQB31mVlUtyoIRAAVGG65BAriZ4e4QHUYCJXdg==", 2023-11-28T12:04:48.637 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.638 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:04:48.638 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:04:48.638 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.641 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.641 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.641 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.642 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.1", 2023-11-28T12:04:48.642 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQB61mVlNA98IRAArcXcx4glfEvQ1gxAXQYxHA==", 2023-11-28T12:04:48.642 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.642 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:04:48.642 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:04:48.643 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.643 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.643 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.643 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.644 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.2", 2023-11-28T12:04:48.644 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQB91mVlzthPIhAAFgsWUTI/2kmVCVMgcZrIdA==", 2023-11-28T12:04:48.644 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.644 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:04:48.644 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:04:48.645 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.645 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.645 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.645 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.646 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.3", 2023-11-28T12:04:48.646 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCA1mVlZPuzJBAAyAAUBUEitzTq7fPdxj5gzQ==", 2023-11-28T12:04:48.646 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.646 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:04:48.646 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:04:48.647 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.647 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.647 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.647 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.647 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.4", 2023-11-28T12:04:48.648 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCG1mVlWibrKhAAzuTgZ4ayPw3imRALW/Y4FQ==", 2023-11-28T12:04:48.648 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.648 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:04:48.648 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:04:48.648 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.649 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.649 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.649 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.649 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.5", 2023-11-28T12:04:48.650 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCJ1mVlnX7bKBAAYYUYhDSDTVKkrzVKw78+Ug==", 2023-11-28T12:04:48.650 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.650 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:04:48.650 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:04:48.651 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.651 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.651 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.651 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.651 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.6", 2023-11-28T12:04:48.651 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCM1mVlE7aUJxAAXM3axU5A9IEXt2/pNcvJ9Q==", 2023-11-28T12:04:48.651 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.651 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:04:48.651 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:04:48.651 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.651 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.652 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.652 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.652 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.7", 2023-11-28T12:04:48.652 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCP1mVlSApqJhAA93hHHLl+o8cKTgdCaG+0Bg==", 2023-11-28T12:04:48.652 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.652 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-11-28T12:04:48.652 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-11-28T12:04:48.652 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.652 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.652 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.652 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.653 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.0", 2023-11-28T12:04:48.653 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBx1mVlmhvFAhAAnKh5ldWFcp+P85m8HR2vug==", 2023-11-28T12:04:48.653 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.653 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:04:48.653 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow r", 2023-11-28T12:04:48.653 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T12:04:48.653 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.653 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.653 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.653 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.1", 2023-11-28T12:04:48.653 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBx1mVlMEgaChAAGCjKh3RcPIVp2UWKGqmqRA==", 2023-11-28T12:04:48.653 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.654 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:04:48.654 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-11-28T12:04:48.654 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-11-28T12:04:48.654 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-11-28T12:04:48.654 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.654 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.654 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.654 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.2", 2023-11-28T12:04:48.655 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBx1mVlqGCwBhAA6ZJBICxFbxIjYlgA80ekuA==", 2023-11-28T12:04:48.655 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.655 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:04:48.655 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-11-28T12:04:48.655 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-11-28T12:04:48.655 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-11-28T12:04:48.655 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.655 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.655 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.655 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.3", 2023-11-28T12:04:48.655 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBx1mVldQXGDxAAkY9/3er6vMwer5oWETJvfg==", 2023-11-28T12:04:48.655 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.656 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-11-28T12:04:48.656 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-11-28T12:04:48.656 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-11-28T12:04:48.656 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-11-28T12:04:48.656 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.656 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.656 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.656 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.admin", 2023-11-28T12:04:48.656 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVlzqiCAxAAIOe5RWQ+gxWKaK64A5jhlA==", 2023-11-28T12:04:48.656 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.656 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-11-28T12:04:48.656 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-11-28T12:04:48.657 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-11-28T12:04:48.657 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.657 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.657 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.657 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.657 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mgr.x", 2023-11-28T12:04:48.657 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVl+yMTIBAAUgQtxbubBwh+1DLDFEaSKQ==", 2023-11-28T12:04:48.657 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.657 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-11-28T12:04:48.658 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile mgr", 2023-11-28T12:04:48.658 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.658 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.658 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:48.658 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:48.658 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mgr.y", 2023-11-28T12:04:48.658 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQBw1mVlEPXqGxAAxVwmP5CKh4D1Hv+kYm7fxg==", 2023-11-28T12:04:48.658 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-11-28T12:04:48.658 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-11-28T12:04:48.658 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile mgr", 2023-11-28T12:04:48.659 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-11-28T12:04:48.659 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.659 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:48.659 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-11-28T12:04:48.659 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:04:48.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.628+0000 7fc5deb27700 1 -- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5b80682f0 msgr2=0x7fc5b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:48.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.628+0000 7fc5deb27700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5b80682f0 0x7fc5b806a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009d70 tx=0x7fc5c8009210 comp rx=0 tx=0).stop 2023-11-28T12:04:48.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.628+0000 7fc5deb27700 1 -- 172.21.15.57:0/876008477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5d8070d90 msgr2=0x7fc5d80b4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:48.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.628+0000 7fc5deb27700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5d8070d90 0x7fc5d80b4b30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000cb40 tx=0x7fc5d000ce50 comp rx=0 tx=0).stop 2023-11-28T12:04:48.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.628+0000 7fc5deb27700 1 -- 172.21.15.57:0/876008477 shutdown_connections 2023-11-28T12:04:48.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.628+0000 7fc5deb27700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc5b80682f0 0x7fc5b806a7a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.628+0000 7fc5deb27700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5d8070d90 0x7fc5d80b4b30 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.628+0000 7fc5deb27700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d806a270 0x7fc5d81f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.628+0000 7fc5deb27700 1 --2- 172.21.15.57:0/876008477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d806da40 0x7fc5d81e58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.628+0000 7fc5deb27700 1 -- 172.21.15.57:0/876008477 >> 172.21.15.57:0/876008477 conn(0x7fc5d806c010 msgr2=0x7fc5d806de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:48.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.628+0000 7fc5deb27700 1 -- 172.21.15.57:0/876008477 shutdown_connections 2023-11-28T12:04:48.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.628+0000 7fc5deb27700 1 -- 172.21.15.57:0/876008477 wait complete. 2023-11-28T12:04:48.664 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:04:48.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.800+0000 7fc6f5cc2700 1 Processor -- start 2023-11-28T12:04:48.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.800+0000 7fc6f5cc2700 1 -- start start 2023-11-28T12:04:48.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.801+0000 7fc6f5cc2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc6f006a270 0x7fc6f006a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:48.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.801+0000 7fc6f5cc2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f015cc80 0x7fc6f006de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:48.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.801+0000 7fc6f5cc2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f006ab80 con 0x7fc6f006e3a0 2023-11-28T12:04:48.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.801+0000 7fc6f5cc2700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f00b2490 con 0x7fc6f006a270 2023-11-28T12:04:48.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.801+0000 7fc6f5cc2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00b1810 con 0x7fc6f015cc80 2023-11-28T12:04:48.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.801+0000 7fc6ef7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc6f006a270 0x7fc6f006a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:48.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.801+0000 7fc6ef7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc6f006a270 0x7fc6f006a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54644/0 (socket says 172.21.15.57:54644) 2023-11-28T12:04:48.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.801+0000 7fc6ef7fe700 1 -- 172.21.15.57:0/2340526230 learned_addr learned my addr 172.21.15.57:0/2340526230 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:48.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.801+0000 7fc6eeffd700 1 --2- 172.21.15.57:0/2340526230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f015cc80 0x7fc6f006de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:48.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.802+0000 7fc6ef7fe700 1 -- 172.21.15.57:0/2340526230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f015cc80 msgr2=0x7fc6f006de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:48.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.802+0000 7fc6ef7fe700 1 --2- 172.21.15.57:0/2340526230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f015cc80 0x7fc6f006de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.802+0000 7fc6ef7fe700 1 -- 172.21.15.57:0/2340526230 >> v1:172.21.15.181:6789/0 conn(0x7fc6f006e3a0 legacy=0x7fc6f006ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:48.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.802+0000 7fc6ef7fe700 1 -- 172.21.15.57:0/2340526230 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f00b1af0 con 0x7fc6f006a270 2023-11-28T12:04:48.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.802+0000 7fc6ef7fe700 1 --2- 172.21.15.57:0/2340526230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f006a270 0x7fc6f006a640 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00098f0 tx=0x7fc6e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ca99838f1ff7ac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:48.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.802+0000 7fc6ee7fc700 1 -- 172.21.15.57:0/2340526230 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0011830 con 0x7fc6f006a270 2023-11-28T12:04:48.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.802+0000 7fc6ee7fc700 1 -- 172.21.15.57:0/2340526230 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e0011990 con 0x7fc6f006a270 2023-11-28T12:04:48.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.802+0000 7fc6ee7fc700 1 -- 172.21.15.57:0/2340526230 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0011d20 con 0x7fc6f006a270 2023-11-28T12:04:48.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.803+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/2340526230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f006a270 msgr2=0x7fc6f006a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:48.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.803+0000 7fc6f5cc2700 1 --2- 172.21.15.57:0/2340526230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f006a270 0x7fc6f006a640 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00098f0 tx=0x7fc6e0009c00 comp rx=0 tx=0).stop 2023-11-28T12:04:48.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.803+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/2340526230 shutdown_connections 2023-11-28T12:04:48.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.803+0000 7fc6f5cc2700 1 --2- 172.21.15.57:0/2340526230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f006a270 0x7fc6f006a640 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.803+0000 7fc6f5cc2700 1 --2- 172.21.15.57:0/2340526230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f015cc80 0x7fc6f006de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.803+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/2340526230 >> 172.21.15.57:0/2340526230 conn(0x7fc6f006c010 msgr2=0x7fc6f00cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:48.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.803+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/2340526230 shutdown_connections 2023-11-28T12:04:48.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.803+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/2340526230 wait complete. 2023-11-28T12:04:48.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.804+0000 7fc6f5cc2700 1 Processor -- start 2023-11-28T12:04:48.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.804+0000 7fc6f5cc2700 1 -- start start 2023-11-28T12:04:48.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.805+0000 7fc6f5cc2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f006a270 0x7fc6f014a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:48.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.805+0000 7fc6f5cc2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f006e3a0 0x7fc6f00b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:48.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.805+0000 7fc6f5cc2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f015cc80 0x7fc6f00b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:48.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.805+0000 7fc6f5cc2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00cbcb0 con 0x7fc6f015cc80 2023-11-28T12:04:48.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.805+0000 7fc6f5cc2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00c7380 con 0x7fc6f006e3a0 2023-11-28T12:04:48.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.805+0000 7fc6f5cc2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00b4140 con 0x7fc6f006a270 2023-11-28T12:04:48.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.805+0000 7fc6effff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f015cc80 0x7fc6f00b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:48.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.805+0000 7fc6effff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f015cc80 0x7fc6f00b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54658/0 (socket says 172.21.15.57:54658) 2023-11-28T12:04:48.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.805+0000 7fc6effff700 1 -- 172.21.15.57:0/3990583999 learned_addr learned my addr 172.21.15.57:0/3990583999 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:48.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.805+0000 7fc6eeffd700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f006e3a0 0x7fc6f00b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:48.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.805+0000 7fc6ef7fe700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f006a270 0x7fc6f014a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:48.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.806+0000 7fc6effff700 1 -- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f006a270 msgr2=0x7fc6f014a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:48.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.806+0000 7fc6effff700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f006a270 0x7fc6f014a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.806+0000 7fc6effff700 1 -- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f006e3a0 msgr2=0x7fc6f00b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:48.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.806+0000 7fc6effff700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f006e3a0 0x7fc6f00b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:48.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.806+0000 7fc6effff700 1 -- 172.21.15.57:0/3990583999 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e0009550 con 0x7fc6f015cc80 2023-11-28T12:04:48.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.806+0000 7fc6effff700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f015cc80 0x7fc6f00b0e80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800cb80 tx=0x7fc6e800ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:48.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.806+0000 7fc6e7fff700 1 -- 172.21.15.57:0/3990583999 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8003c00 con 0x7fc6f015cc80 2023-11-28T12:04:48.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.806+0000 7fc6e7fff700 1 -- 172.21.15.57:0/3990583999 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e8003d60 con 0x7fc6f015cc80 2023-11-28T12:04:48.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.806+0000 7fc6e7fff700 1 -- 172.21.15.57:0/3990583999 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e800aaa0 con 0x7fc6f015cc80 2023-11-28T12:04:48.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.806+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/3990583999 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f00b20b0 con 0x7fc6f015cc80 2023-11-28T12:04:48.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.806+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/3990583999 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f00b1dd0 con 0x7fc6f015cc80 2023-11-28T12:04:48.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.807+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/3990583999 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f006e7e0 con 0x7fc6f015cc80 2023-11-28T12:04:48.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.808+0000 7fc6e7fff700 1 -- 172.21.15.57:0/3990583999 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc6e8010050 con 0x7fc6f015cc80 2023-11-28T12:04:48.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.808+0000 7fc6e7fff700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc6d0068320 0x7fc6d006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:48.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.808+0000 7fc6e7fff700 1 -- 172.21.15.57:0/3990583999 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc6e8085790 con 0x7fc6f015cc80 2023-11-28T12:04:48.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.808+0000 7fc6ef7fe700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc6d0068320 0x7fc6d006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:48.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.809+0000 7fc6ef7fe700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc6d0068320 0x7fc6d006a7d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0014040 tx=0x7fc6e0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:48.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:48.812+0000 7fc6e7fff700 1 -- 172.21.15.57:0/3990583999 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc6e80522b0 con 0x7fc6f015cc80 2023-11-28T12:04:49.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.014+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/3990583999 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6f00c74c0 con 0x7fc6f015cc80 2023-11-28T12:04:49.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.015+0000 7fc6e7fff700 1 -- 172.21.15.57:0/3990583999 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+14209 (secure 0 0 0) 0x7fc6e80558d0 con 0x7fc6f015cc80 2023-11-28T12:04:49.015 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:49.016 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":59,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:04:48.154786+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:04:49.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.017+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc6d0068320 msgr2=0x7fc6d006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.017+0000 7fc6f5cc2700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc6d0068320 0x7fc6d006a7d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0014040 tx=0x7fc6e0008030 comp rx=0 tx=0).stop 2023-11-28T12:04:49.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.017+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/3990583999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f015cc80 msgr2=0x7fc6f00b0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.017+0000 7fc6f5cc2700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f015cc80 0x7fc6f00b0e80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800cb80 tx=0x7fc6e800ce90 comp rx=0 tx=0).stop 2023-11-28T12:04:49.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.017+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/3990583999 shutdown_connections 2023-11-28T12:04:49.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.018+0000 7fc6f5cc2700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc6d0068320 0x7fc6d006a7d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.018+0000 7fc6f5cc2700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6f015cc80 0x7fc6f00b0e80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.018+0000 7fc6f5cc2700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6f006a270 0x7fc6f014a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.018+0000 7fc6f5cc2700 1 --2- 172.21.15.57:0/3990583999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f006e3a0 0x7fc6f00b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.018+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/3990583999 >> 172.21.15.57:0/3990583999 conn(0x7fc6f006c010 msgr2=0x7fc6f0159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:49.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.018+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/3990583999 shutdown_connections 2023-11-28T12:04:49.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.018+0000 7fc6f5cc2700 1 -- 172.21.15.57:0/3990583999 wait complete. 2023-11-28T12:04:49.031 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:04:49.032 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:04:49.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.170+0000 7f2570eae700 1 Processor -- start 2023-11-28T12:04:49.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.170+0000 7f2570eae700 1 -- start start 2023-11-28T12:04:49.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.171+0000 7f2570eae700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f256c0d0ad0 0x7f256c0cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.171+0000 7f2570eae700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256c0cc740 con 0x7f256c0b2ee0 2023-11-28T12:04:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.171+0000 7f2570eae700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256c15a9c0 con 0x7f256c15a5f0 2023-11-28T12:04:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.171+0000 7f2570eae700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256c0c7b80 con 0x7f256c0d0ad0 2023-11-28T12:04:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.171+0000 7f256ad9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f256c0d0ad0 0x7f256c0cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.171+0000 7f2569d9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f256c0b2ee0 0x7f256c0ced30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37790/0 (socket says 172.21.15.57:37790) 2023-11-28T12:04:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.171+0000 7f2569d9b700 1 -- 172.21.15.57:0/2258123891 learned_addr learned my addr 172.21.15.57:0/2258123891 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.172+0000 7f256ad9d700 1 -- 172.21.15.57:0/2258123891 >> v1:172.21.15.181:6789/0 conn(0x7f256c15a5f0 legacy=0x7f256c0b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.172+0000 7f256ad9d700 1 -- 172.21.15.57:0/2258123891 >> v1:172.21.15.57:6789/0 conn(0x7f256c0b2ee0 legacy=0x7f256c0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.172+0000 7f256ad9d700 1 -- 172.21.15.57:0/2258123891 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c0cc990 con 0x7f256c0d0ad0 2023-11-28T12:04:49.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.172+0000 7f256ad9d700 1 --2- 172.21.15.57:0/2258123891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f256c0d0ad0 0x7f256c0cc200 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2564009d90 tx=0x7f256400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0dde87aae3b1c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:49.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.172+0000 7f256959a700 1 -- 172.21.15.57:0/2258123891 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256400e070 con 0x7f256c0d0ad0 2023-11-28T12:04:49.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.172+0000 7f256959a700 1 -- 172.21.15.57:0/2258123891 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256400bbd0 con 0x7f256c0d0ad0 2023-11-28T12:04:49.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.172+0000 7f256959a700 1 -- 172.21.15.57:0/2258123891 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2564012430 con 0x7f256c0d0ad0 2023-11-28T12:04:49.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.172+0000 7f2570eae700 1 -- 172.21.15.57:0/2258123891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f256c0d0ad0 msgr2=0x7f256c0cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.172+0000 7f2570eae700 1 --2- 172.21.15.57:0/2258123891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f256c0d0ad0 0x7f256c0cc200 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2564009d90 tx=0x7f256400b330 comp rx=0 tx=0).stop 2023-11-28T12:04:49.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.173+0000 7f2570eae700 1 -- 172.21.15.57:0/2258123891 shutdown_connections 2023-11-28T12:04:49.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.173+0000 7f2570eae700 1 --2- 172.21.15.57:0/2258123891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f256c0d0ad0 0x7f256c0cc200 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.173+0000 7f2570eae700 1 -- 172.21.15.57:0/2258123891 >> 172.21.15.57:0/2258123891 conn(0x7f256c06ea80 msgr2=0x7f256c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:49.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.173+0000 7f2570eae700 1 -- 172.21.15.57:0/2258123891 shutdown_connections 2023-11-28T12:04:49.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.173+0000 7f2570eae700 1 -- 172.21.15.57:0/2258123891 wait complete. 2023-11-28T12:04:49.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.174+0000 7f2570eae700 1 Processor -- start 2023-11-28T12:04:49.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.174+0000 7f2570eae700 1 -- start start 2023-11-28T12:04:49.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2570eae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f256c0b2ee0 0x7f256c14b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2570eae700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f256c0d0ad0 0x7f256c153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2570eae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f256c15a5f0 0x7f256c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2570eae700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c14b740 con 0x7f256c0d0ad0 2023-11-28T12:04:49.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2570eae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0b4ec0 con 0x7f256c15a5f0 2023-11-28T12:04:49.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2570eae700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0c8aa0 con 0x7f256c0b2ee0 2023-11-28T12:04:49.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2569d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f256c0d0ad0 0x7f256c153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2569d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f256c0d0ad0 0x7f256c153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54668/0 (socket says 172.21.15.57:54668) 2023-11-28T12:04:49.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2569d9b700 1 -- 172.21.15.57:0/4088161504 learned_addr learned my addr 172.21.15.57:0/4088161504 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:49.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f256a59c700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f256c0b2ee0 0x7f256c14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f256ad9d700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f256c15a5f0 0x7f256c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2569d9b700 1 -- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f256c0b2ee0 msgr2=0x7f256c14b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2569d9b700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f256c0b2ee0 0x7f256c14b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2569d9b700 1 -- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f256c15a5f0 msgr2=0x7f256c153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2569d9b700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f256c15a5f0 0x7f256c153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.175+0000 7f2569d9b700 1 -- 172.21.15.57:0/4088161504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2564009a70 con 0x7f256c0d0ad0 2023-11-28T12:04:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.176+0000 7f2569d9b700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f256c0d0ad0 0x7f256c153970 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f255c00c480 tx=0x7f255c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.176+0000 7f2562ffd700 1 -- 172.21.15.57:0/4088161504 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c005d70 con 0x7f256c0d0ad0 2023-11-28T12:04:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.176+0000 7f2562ffd700 1 -- 172.21.15.57:0/4088161504 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255c0070d0 con 0x7f256c0d0ad0 2023-11-28T12:04:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.176+0000 7f2570eae700 1 -- 172.21.15.57:0/4088161504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256c0ccf50 con 0x7f256c0d0ad0 2023-11-28T12:04:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.176+0000 7f2562ffd700 1 -- 172.21.15.57:0/4088161504 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c007440 con 0x7f256c0d0ad0 2023-11-28T12:04:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.176+0000 7f2570eae700 1 -- 172.21.15.57:0/4088161504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256c0ccc70 con 0x7f256c0d0ad0 2023-11-28T12:04:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.177+0000 7f2570eae700 1 -- 172.21.15.57:0/4088161504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256c0cd230 con 0x7f256c0d0ad0 2023-11-28T12:04:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.177+0000 7f2562ffd700 1 -- 172.21.15.57:0/4088161504 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f255c019030 con 0x7f256c0d0ad0 2023-11-28T12:04:49.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.178+0000 7f2562ffd700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f254c0682f0 0x7f254c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.178+0000 7f2562ffd700 1 -- 172.21.15.57:0/4088161504 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f255c084ae0 con 0x7f256c0d0ad0 2023-11-28T12:04:49.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.178+0000 7f256a59c700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f254c0682f0 0x7f254c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.178+0000 7f256a59c700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f254c0682f0 0x7f254c06a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f256c0c9b30 tx=0x7f2554009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:49.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.182+0000 7f2562ffd700 1 -- 172.21.15.57:0/4088161504 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f255c04b5e0 con 0x7f256c0d0ad0 2023-11-28T12:04:49.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.383+0000 7f2570eae700 1 -- 172.21.15.57:0/4088161504 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f256c0b2d50 con 0x7f256c0d0ad0 2023-11-28T12:04:49.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.383+0000 7f2562ffd700 1 -- 172.21.15.57:0/4088161504 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7f255c0170a0 con 0x7f256c0d0ad0 2023-11-28T12:04:49.383 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:04:49.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.385+0000 7f2570eae700 1 -- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f254c0682f0 msgr2=0x7f254c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.385+0000 7f2570eae700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f254c0682f0 0x7f254c06a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f256c0c9b30 tx=0x7f2554009040 comp rx=0 tx=0).stop 2023-11-28T12:04:49.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.385+0000 7f2570eae700 1 -- 172.21.15.57:0/4088161504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f256c0d0ad0 msgr2=0x7f256c153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.385+0000 7f2570eae700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f256c0d0ad0 0x7f256c153970 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f255c00c480 tx=0x7f255c00c840 comp rx=0 tx=0).stop 2023-11-28T12:04:49.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.385+0000 7f2570eae700 1 -- 172.21.15.57:0/4088161504 shutdown_connections 2023-11-28T12:04:49.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.385+0000 7f2570eae700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f254c0682f0 0x7f254c06a7a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.385+0000 7f2570eae700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f256c15a5f0 0x7f256c153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.385+0000 7f2570eae700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f256c0b2ee0 0x7f256c14b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.385+0000 7f2570eae700 1 --2- 172.21.15.57:0/4088161504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f256c0d0ad0 0x7f256c153970 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.385+0000 7f2570eae700 1 -- 172.21.15.57:0/4088161504 >> 172.21.15.57:0/4088161504 conn(0x7f256c06ea80 msgr2=0x7f256c0cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:49.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.385+0000 7f2570eae700 1 -- 172.21.15.57:0/4088161504 shutdown_connections 2023-11-28T12:04:49.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.385+0000 7f2570eae700 1 -- 172.21.15.57:0/4088161504 wait complete. 2023-11-28T12:04:49.399 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:04:49.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.538+0000 7f4aaf6ac700 1 Processor -- start 2023-11-28T12:04:49.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.538+0000 7f4aaf6ac700 1 -- start start 2023-11-28T12:04:49.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.539+0000 7f4aaf6ac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4aa80b2ee0 0x7f4aa80d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.539+0000 7f4aaf6ac700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa815a9c0 con 0x7f4aa815a5f0 2023-11-28T12:04:49.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.539+0000 7f4aaf6ac700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa80c7b80 con 0x7f4aa80cc200 2023-11-28T12:04:49.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.539+0000 7f4aaf6ac700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa80cc990 con 0x7f4aa80b2ee0 2023-11-28T12:04:49.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.539+0000 7f4aad448700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4aa815a5f0 0x7f4aa80b27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37800/0 (socket says 172.21.15.57:37800) 2023-11-28T12:04:49.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.539+0000 7f4aad448700 1 -- 172.21.15.57:0/3456141206 learned_addr learned my addr 172.21.15.57:0/3456141206 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:49.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.539+0000 7f4aacc47700 1 --2- 172.21.15.57:0/3456141206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4aa80b2ee0 0x7f4aa80d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.539+0000 7f4aa7fff700 1 -- 172.21.15.57:0/3456141206 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882827389 0 0) 0x7f4aa815a9c0 con 0x7f4aa815a5f0 2023-11-28T12:04:49.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.539+0000 7f4aa7fff700 1 -- 172.21.15.57:0/3456141206 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a900035b0 con 0x7f4aa815a5f0 2023-11-28T12:04:49.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.540+0000 7f4aa7fff700 1 -- 172.21.15.57:0/3456141206 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637105398 0 0) 0x7f4aa80c7b80 con 0x7f4aa80cc200 2023-11-28T12:04:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.540+0000 7f4aa7fff700 1 -- 172.21.15.57:0/3456141206 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4aa815a9c0 con 0x7f4aa80cc200 2023-11-28T12:04:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.540+0000 7f4aacc47700 1 -- 172.21.15.57:0/3456141206 >> v1:172.21.15.181:6789/0 conn(0x7f4aa80cc200 legacy=0x7f4aa80ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.540+0000 7f4aacc47700 1 -- 172.21.15.57:0/3456141206 >> v1:172.21.15.57:6789/0 conn(0x7f4aa815a5f0 legacy=0x7f4aa80b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.540+0000 7f4aacc47700 1 -- 172.21.15.57:0/3456141206 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa80ccc70 con 0x7f4aa80b2ee0 2023-11-28T12:04:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.540+0000 7f4aacc47700 1 --2- 172.21.15.57:0/3456141206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4aa80b2ee0 0x7f4aa80d0ad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009870 tx=0x7f4a9c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ca426426708ce68 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.540+0000 7f4aa7fff700 1 -- 172.21.15.57:0/3456141206 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c011830 con 0x7f4aa80b2ee0 2023-11-28T12:04:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.540+0000 7f4aa7fff700 1 -- 172.21.15.57:0/3456141206 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c011990 con 0x7f4aa80b2ee0 2023-11-28T12:04:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.540+0000 7f4aa7fff700 1 -- 172.21.15.57:0/3456141206 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c011d20 con 0x7f4aa80b2ee0 2023-11-28T12:04:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.541+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3456141206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4aa80b2ee0 msgr2=0x7f4aa80d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.541+0000 7f4aaf6ac700 1 --2- 172.21.15.57:0/3456141206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4aa80b2ee0 0x7f4aa80d0ad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009870 tx=0x7f4a9c009b80 comp rx=0 tx=0).stop 2023-11-28T12:04:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.541+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3456141206 shutdown_connections 2023-11-28T12:04:49.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.541+0000 7f4aaf6ac700 1 --2- 172.21.15.57:0/3456141206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4aa80b2ee0 0x7f4aa80d0ad0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.541+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3456141206 >> 172.21.15.57:0/3456141206 conn(0x7f4aa806ea80 msgr2=0x7f4aa806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:49.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.541+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3456141206 shutdown_connections 2023-11-28T12:04:49.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.541+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3456141206 wait complete. 2023-11-28T12:04:49.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.542+0000 7f4aaf6ac700 1 Processor -- start 2023-11-28T12:04:49.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.542+0000 7f4aaf6ac700 1 -- start start 2023-11-28T12:04:49.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aaf6ac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4aa80b2ee0 0x7f4aa814b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aaf6ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa80cc200 0x7f4aa806bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aaf6ac700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4aa815a5f0 0x7f4aa80c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aaf6ac700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa814b940 con 0x7f4aa815a5f0 2023-11-28T12:04:49.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aaf6ac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa806c8f0 con 0x7f4aa80cc200 2023-11-28T12:04:49.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aaf6ac700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa806cbd0 con 0x7f4aa80b2ee0 2023-11-28T12:04:49.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aadc49700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4aa815a5f0 0x7f4aa80c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aadc49700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4aa815a5f0 0x7f4aa80c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54676/0 (socket says 172.21.15.57:54676) 2023-11-28T12:04:49.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aadc49700 1 -- 172.21.15.57:0/3148894214 learned_addr learned my addr 172.21.15.57:0/3148894214 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:49.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aad448700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4aa80b2ee0 0x7f4aa814b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aacc47700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa80cc200 0x7f4aa806bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aadc49700 1 -- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4aa80b2ee0 msgr2=0x7f4aa814b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aadc49700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4aa80b2ee0 0x7f4aa814b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aadc49700 1 -- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa80cc200 msgr2=0x7f4aa806bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aadc49700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa80cc200 0x7f4aa806bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aadc49700 1 -- 172.21.15.57:0/3148894214 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c0094a0 con 0x7f4aa815a5f0 2023-11-28T12:04:49.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aacc47700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa80cc200 0x7f4aa806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:49.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.543+0000 7f4aadc49700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4aa815a5f0 0x7f4aa80c90b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800cbc0 tx=0x7f4a9800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:49.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.544+0000 7f4aa5ffb700 1 -- 172.21.15.57:0/3148894214 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98003c00 con 0x7f4aa815a5f0 2023-11-28T12:04:49.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.544+0000 7f4aa5ffb700 1 -- 172.21.15.57:0/3148894214 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a98003d60 con 0x7f4aa815a5f0 2023-11-28T12:04:49.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.544+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3148894214 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa80cd230 con 0x7f4aa815a5f0 2023-11-28T12:04:49.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.544+0000 7f4aa5ffb700 1 -- 172.21.15.57:0/3148894214 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9800abb0 con 0x7f4aa815a5f0 2023-11-28T12:04:49.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.544+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3148894214 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa80ccf50 con 0x7f4aa815a5f0 2023-11-28T12:04:49.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.545+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3148894214 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa80cd510 con 0x7f4aa815a5f0 2023-11-28T12:04:49.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.545+0000 7f4aa5ffb700 1 -- 172.21.15.57:0/3148894214 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4a98010050 con 0x7f4aa815a5f0 2023-11-28T12:04:49.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.546+0000 7f4aa5ffb700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4a940682f0 0x7f4a9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.546+0000 7f4aa5ffb700 1 -- 172.21.15.57:0/3148894214 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4a980858c0 con 0x7f4aa815a5f0 2023-11-28T12:04:49.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.546+0000 7f4aad448700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4a940682f0 0x7f4a9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.547+0000 7f4aad448700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4a940682f0 0x7f4a9406a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00011e0 tx=0x7f4aa0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:49.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.550+0000 7f4aa5ffb700 1 -- 172.21.15.57:0/3148894214 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4a98052470 con 0x7f4aa815a5f0 2023-11-28T12:04:49.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.750+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3148894214 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4aa80743f0 con 0x7f4aa815a5f0 2023-11-28T12:04:49.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.751+0000 7f4aa5ffb700 1 -- 172.21.15.57:0/3148894214 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4a98055a90 con 0x7f4aa815a5f0 2023-11-28T12:04:49.751 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:04:49.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.753+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4a940682f0 msgr2=0x7f4a9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.753+0000 7f4aaf6ac700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4a940682f0 0x7f4a9406a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00011e0 tx=0x7f4aa0009040 comp rx=0 tx=0).stop 2023-11-28T12:04:49.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.753+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3148894214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4aa815a5f0 msgr2=0x7f4aa80c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.753+0000 7f4aaf6ac700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4aa815a5f0 0x7f4aa80c90b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800cbc0 tx=0x7f4a9800cf80 comp rx=0 tx=0).stop 2023-11-28T12:04:49.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.753+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3148894214 shutdown_connections 2023-11-28T12:04:49.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.753+0000 7f4aaf6ac700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4a940682f0 0x7f4a9406a7a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.753+0000 7f4aaf6ac700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4aa815a5f0 0x7f4aa80c90b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.753+0000 7f4aaf6ac700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4aa80b2ee0 0x7f4aa814b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.753+0000 7f4aaf6ac700 1 --2- 172.21.15.57:0/3148894214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aa80cc200 0x7f4aa806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.753+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3148894214 >> 172.21.15.57:0/3148894214 conn(0x7f4aa806ea80 msgr2=0x7f4aa80cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:49.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.753+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3148894214 shutdown_connections 2023-11-28T12:04:49.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.753+0000 7f4aaf6ac700 1 -- 172.21.15.57:0/3148894214 wait complete. 2023-11-28T12:04:49.766 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T12:04:49.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.904+0000 7fe00d660700 1 Processor -- start 2023-11-28T12:04:49.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.904+0000 7fe00d660700 1 -- start start 2023-11-28T12:04:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.905+0000 7fe00d660700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080cd5a0 0x7fe0080cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.905+0000 7fe00d660700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe0080b22a0 0x7fe0080cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.905+0000 7fe00d660700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0080d1ee0 con 0x7fe00806ab10 2023-11-28T12:04:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.905+0000 7fe00d660700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0080cdb10 con 0x7fe0080b22a0 2023-11-28T12:04:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.905+0000 7fe00d660700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080d1890 con 0x7fe0080cd5a0 2023-11-28T12:04:49.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.905+0000 7fe0067fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080cd5a0 0x7fe0080cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.905+0000 7fe0077fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe0080b22a0 0x7fe0080cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.905+0000 7fe006ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe00806ab10 0x7fe00806aee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:54766/0 (socket says 172.21.15.57:54766) 2023-11-28T12:04:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.905+0000 7fe006ffd700 1 -- 172.21.15.57:0/3370751858 learned_addr learned my addr 172.21.15.57:0/3370751858 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.906+0000 7fe005ffb700 1 -- 172.21.15.57:0/3370751858 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388943244 0 0) 0x7fe0080d1ee0 con 0x7fe00806ab10 2023-11-28T12:04:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.906+0000 7fe005ffb700 1 -- 172.21.15.57:0/3370751858 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff00035b0 con 0x7fe00806ab10 2023-11-28T12:04:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.906+0000 7fe0067fc700 1 -- 172.21.15.57:0/3370751858 >> v2:172.21.15.181:3300/0 conn(0x7fe0080b22a0 msgr2=0x7fe0080cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.906+0000 7fe0067fc700 1 --2- 172.21.15.57:0/3370751858 >> v2:172.21.15.181:3300/0 conn(0x7fe0080b22a0 0x7fe0080cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.906+0000 7fe0067fc700 1 -- 172.21.15.57:0/3370751858 >> v1:172.21.15.181:6789/0 conn(0x7fe00806ab10 legacy=0x7fe00806aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.906+0000 7fe0067fc700 1 -- 172.21.15.57:0/3370751858 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0080d1b70 con 0x7fe0080cd5a0 2023-11-28T12:04:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.906+0000 7fe0077fe700 1 --2- 172.21.15.57:0/3370751858 >> v2:172.21.15.181:3300/0 conn(0x7fe0080b22a0 0x7fe0080cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.906+0000 7fe0067fc700 1 --2- 172.21.15.57:0/3370751858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080cd5a0 0x7fe0080cf440 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009840 tx=0x7fdffc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf295fe46e0a76d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:49.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.906+0000 7fe005ffb700 1 -- 172.21.15.57:0/3370751858 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc011830 con 0x7fe0080cd5a0 2023-11-28T12:04:49.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.906+0000 7fe005ffb700 1 -- 172.21.15.57:0/3370751858 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc011990 con 0x7fe0080cd5a0 2023-11-28T12:04:49.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.906+0000 7fe005ffb700 1 -- 172.21.15.57:0/3370751858 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc011d20 con 0x7fe0080cd5a0 2023-11-28T12:04:49.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.907+0000 7fe00d660700 1 -- 172.21.15.57:0/3370751858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080cd5a0 msgr2=0x7fe0080cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.907+0000 7fe00d660700 1 --2- 172.21.15.57:0/3370751858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080cd5a0 0x7fe0080cf440 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009840 tx=0x7fdffc009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:49.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.907+0000 7fe00d660700 1 -- 172.21.15.57:0/3370751858 shutdown_connections 2023-11-28T12:04:49.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.907+0000 7fe00d660700 1 --2- 172.21.15.57:0/3370751858 >> v2:172.21.15.181:3300/0 conn(0x7fe0080b22a0 0x7fe0080cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.907+0000 7fe00d660700 1 --2- 172.21.15.57:0/3370751858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080cd5a0 0x7fe0080cf440 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.907+0000 7fe00d660700 1 -- 172.21.15.57:0/3370751858 >> 172.21.15.57:0/3370751858 conn(0x7fe00806c520 msgr2=0x7fe00806c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:49.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.907+0000 7fe00d660700 1 -- 172.21.15.57:0/3370751858 shutdown_connections 2023-11-28T12:04:49.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.907+0000 7fe00d660700 1 -- 172.21.15.57:0/3370751858 wait complete. 2023-11-28T12:04:49.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.909+0000 7fe00d660700 1 Processor -- start 2023-11-28T12:04:49.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.909+0000 7fe00d660700 1 -- start start 2023-11-28T12:04:49.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe00d660700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe00806ab10 0x7fe0080b8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe00d660700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080b22a0 0x7fe0080c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe00d660700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0080cd5a0 0x7fe0080b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe00d660700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080c72a0 con 0x7fe00806ab10 2023-11-28T12:04:49.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe00d660700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080c6c00 con 0x7fe0080cd5a0 2023-11-28T12:04:49.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe00d660700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080b89d0 con 0x7fe0080b22a0 2023-11-28T12:04:49.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe006ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe00806ab10 0x7fe0080b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe006ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe00806ab10 0x7fe0080b8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54682/0 (socket says 172.21.15.57:54682) 2023-11-28T12:04:49.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe006ffd700 1 -- 172.21.15.57:0/937979890 learned_addr learned my addr 172.21.15.57:0/937979890 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:49.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe0067fc700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080b22a0 0x7fe0080c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe0077fe700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0080cd5a0 0x7fe0080b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe006ffd700 1 -- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080b22a0 msgr2=0x7fe0080c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe006ffd700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080b22a0 0x7fe0080c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe006ffd700 1 -- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0080cd5a0 msgr2=0x7fe0080b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:49.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe006ffd700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0080cd5a0 0x7fe0080b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:49.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe006ffd700 1 -- 172.21.15.57:0/937979890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe000009a70 con 0x7fe00806ab10 2023-11-28T12:04:49.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.910+0000 7fe006ffd700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe00806ab10 0x7fe0080b8180 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdff800c700 tx=0x7fdff800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:49.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.911+0000 7fdfef7fe700 1 -- 172.21.15.57:0/937979890 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8005d70 con 0x7fe00806ab10 2023-11-28T12:04:49.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.911+0000 7fdfef7fe700 1 -- 172.21.15.57:0/937979890 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff80070d0 con 0x7fe00806ab10 2023-11-28T12:04:49.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.911+0000 7fe00d660700 1 -- 172.21.15.57:0/937979890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdffc0094a0 con 0x7fe00806ab10 2023-11-28T12:04:49.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.911+0000 7fdfef7fe700 1 -- 172.21.15.57:0/937979890 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8007460 con 0x7fe00806ab10 2023-11-28T12:04:49.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.911+0000 7fe00d660700 1 -- 172.21.15.57:0/937979890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0080b1ec0 con 0x7fe00806ab10 2023-11-28T12:04:49.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.912+0000 7fe00d660700 1 -- 172.21.15.57:0/937979890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00815cad0 con 0x7fe00806ab10 2023-11-28T12:04:49.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.912+0000 7fdfef7fe700 1 -- 172.21.15.57:0/937979890 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdff8019030 con 0x7fe00806ab10 2023-11-28T12:04:49.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.913+0000 7fdfef7fe700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdff40682f0 0x7fdff406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:49.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.913+0000 7fdfef7fe700 1 -- 172.21.15.57:0/937979890 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdff8084c60 con 0x7fe00806ab10 2023-11-28T12:04:49.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.913+0000 7fe0067fc700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdff40682f0 0x7fdff406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:49.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.913+0000 7fe0067fc700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdff40682f0 0x7fdff406a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009fd0 tx=0x7fdffc01c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:49.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:49.917+0000 7fdfef7fe700 1 -- 172.21.15.57:0/937979890 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdff8051810 con 0x7fe00806ab10 2023-11-28T12:04:50.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.117+0000 7fe00d660700 1 -- 172.21.15.57:0/937979890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe0080b2050 con 0x7fe00806ab10 2023-11-28T12:04:50.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.118+0000 7fdfef7fe700 1 -- 172.21.15.57:0/937979890 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) v1 ==== 84+0+126 (secure 0 0 0) 0x7fdff8017090 con 0x7fe00806ab10 2023-11-28T12:04:50.118 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:50.118 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2023-11-28T12:04:50.118 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:50.119 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 1, 2023-11-28T12:04:50.119 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": "rbd" 2023-11-28T12:04:50.119 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:04:50.120 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:04:50.120 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 2, 2023-11-28T12:04:50.120 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": ".mgr" 2023-11-28T12:04:50.120 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:04:50.120 INFO:teuthology.orchestra.run.smithi057.stdout:] 2023-11-28T12:04:50.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.120+0000 7fe00d660700 1 -- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdff40682f0 msgr2=0x7fdff406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:50.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.120+0000 7fe00d660700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdff40682f0 0x7fdff406a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009fd0 tx=0x7fdffc01c040 comp rx=0 tx=0).stop 2023-11-28T12:04:50.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.120+0000 7fe00d660700 1 -- 172.21.15.57:0/937979890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe00806ab10 msgr2=0x7fe0080b8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:50.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.120+0000 7fe00d660700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe00806ab10 0x7fe0080b8180 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdff800c700 tx=0x7fdff800ca10 comp rx=0 tx=0).stop 2023-11-28T12:04:50.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.120+0000 7fe00d660700 1 -- 172.21.15.57:0/937979890 shutdown_connections 2023-11-28T12:04:50.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.120+0000 7fe00d660700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdff40682f0 0x7fdff406a7a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:50.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.120+0000 7fe00d660700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0080cd5a0 0x7fe0080b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:50.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.120+0000 7fe00d660700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe00806ab10 0x7fe0080b8180 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:50.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.120+0000 7fe00d660700 1 --2- 172.21.15.57:0/937979890 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0080b22a0 0x7fe0080c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:50.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.120+0000 7fe00d660700 1 -- 172.21.15.57:0/937979890 >> 172.21.15.57:0/937979890 conn(0x7fe00806c520 msgr2=0x7fe0080b66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:50.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.120+0000 7fe00d660700 1 -- 172.21.15.57:0/937979890 shutdown_connections 2023-11-28T12:04:50.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.120+0000 7fe00d660700 1 -- 172.21.15.57:0/937979890 wait complete. 2023-11-28T12:04:50.134 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T12:04:50.134 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T12:04:50.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.273+0000 7f73f47e9700 1 Processor -- start 2023-11-28T12:04:50.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.273+0000 7f73f47e9700 1 -- start start 2023-11-28T12:04:50.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.274+0000 7f73f47e9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec15c620 0x7f73ec0cac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:50.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.274+0000 7f73f47e9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f73ec0ccf40 0x7f73ec0cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:50.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.274+0000 7f73f47e9700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ec0cb160 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.274+0000 7f73f47e9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ec066620 con 0x7f73ec0ccf40 2023-11-28T12:04:50.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.274+0000 7f73f47e9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec0cb420 con 0x7f73ec15c620 2023-11-28T12:04:50.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.274+0000 7f73f1d84700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f73ec0ae7a0 0x7f73ec160920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37810/0 (socket says 172.21.15.57:37810) 2023-11-28T12:04:50.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.274+0000 7f73f1d84700 1 -- 172.21.15.57:0/2172979950 learned_addr learned my addr 172.21.15.57:0/2172979950 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:50.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.274+0000 7f73f2585700 1 --2- 172.21.15.57:0/2172979950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec15c620 0x7f73ec0cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:50.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.274+0000 7f73f2d86700 1 --2- 172.21.15.57:0/2172979950 >> v2:172.21.15.181:3300/0 conn(0x7f73ec0ccf40 0x7f73ec0cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:50.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.275+0000 7f73f1583700 1 -- 172.21.15.57:0/2172979950 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785522200 0 0) 0x7f73ec0cb160 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.275+0000 7f73f1583700 1 -- 172.21.15.57:0/2172979950 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73d80035b0 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.275+0000 7f73f2585700 1 -- 172.21.15.57:0/2172979950 >> v2:172.21.15.181:3300/0 conn(0x7f73ec0ccf40 msgr2=0x7f73ec0cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:50.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.275+0000 7f73f2585700 1 --2- 172.21.15.57:0/2172979950 >> v2:172.21.15.181:3300/0 conn(0x7f73ec0ccf40 0x7f73ec0cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:50.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.275+0000 7f73f2585700 1 -- 172.21.15.57:0/2172979950 >> v1:172.21.15.57:6789/0 conn(0x7f73ec0ae7a0 legacy=0x7f73ec160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:50.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.275+0000 7f73f2585700 1 -- 172.21.15.57:0/2172979950 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec0cb700 con 0x7f73ec15c620 2023-11-28T12:04:50.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.275+0000 7f73f2585700 1 --2- 172.21.15.57:0/2172979950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec15c620 0x7f73ec0cac20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009840 tx=0x7f73e8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93b795b3a4ae8373 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:50.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.275+0000 7f73f1583700 1 -- 172.21.15.57:0/2172979950 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e8011830 con 0x7f73ec15c620 2023-11-28T12:04:50.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.275+0000 7f73f1583700 1 -- 172.21.15.57:0/2172979950 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e8011990 con 0x7f73ec15c620 2023-11-28T12:04:50.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.275+0000 7f73f1583700 1 -- 172.21.15.57:0/2172979950 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e8011d20 con 0x7f73ec15c620 2023-11-28T12:04:50.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.276+0000 7f73f47e9700 1 -- 172.21.15.57:0/2172979950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec15c620 msgr2=0x7f73ec0cac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:50.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.276+0000 7f73f47e9700 1 --2- 172.21.15.57:0/2172979950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec15c620 0x7f73ec0cac20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009840 tx=0x7f73e8009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:50.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.276+0000 7f73f47e9700 1 -- 172.21.15.57:0/2172979950 shutdown_connections 2023-11-28T12:04:50.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.276+0000 7f73f47e9700 1 --2- 172.21.15.57:0/2172979950 >> v2:172.21.15.181:3300/0 conn(0x7f73ec0ccf40 0x7f73ec0cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:50.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.276+0000 7f73f47e9700 1 --2- 172.21.15.57:0/2172979950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec15c620 0x7f73ec0cac20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:50.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.276+0000 7f73f47e9700 1 -- 172.21.15.57:0/2172979950 >> 172.21.15.57:0/2172979950 conn(0x7f73ec0cf370 msgr2=0x7f73ec0cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:50.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.276+0000 7f73f47e9700 1 -- 172.21.15.57:0/2172979950 shutdown_connections 2023-11-28T12:04:50.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.276+0000 7f73f47e9700 1 -- 172.21.15.57:0/2172979950 wait complete. 2023-11-28T12:04:50.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.277+0000 7f73f47e9700 1 Processor -- start 2023-11-28T12:04:50.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.277+0000 7f73f47e9700 1 -- start start 2023-11-28T12:04:50.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f47e9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ec0ae7a0 0x7f73ec1f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:50.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f47e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ec0ccf40 0x7f73ec1f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:50.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f47e9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec15c620 0x7f73ec15f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:50.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f47e9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec1f3820 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f47e9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec0b3220 con 0x7f73ec0ccf40 2023-11-28T12:04:50.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f47e9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec1f30e0 con 0x7f73ec15c620 2023-11-28T12:04:50.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f2585700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ec0ae7a0 0x7f73ec1f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:50.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f2585700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ec0ae7a0 0x7f73ec1f32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54694/0 (socket says 172.21.15.57:54694) 2023-11-28T12:04:50.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f2585700 1 -- 172.21.15.57:0/1952099921 learned_addr learned my addr 172.21.15.57:0/1952099921 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:50.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f2d86700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec15c620 0x7f73ec15f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:50.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f1d84700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ec0ccf40 0x7f73ec1f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:50.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f2585700 1 -- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec15c620 msgr2=0x7f73ec15f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:50.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f2585700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec15c620 0x7f73ec15f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:50.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f2585700 1 -- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ec0ccf40 msgr2=0x7f73ec1f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:50.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f2585700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ec0ccf40 0x7f73ec1f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:50.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f2585700 1 -- 172.21.15.57:0/1952099921 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e80094a0 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.278+0000 7f73f2585700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ec0ae7a0 0x7f73ec1f32e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f73e80090b0 tx=0x7f73e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:50.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.279+0000 7f73deffd700 1 -- 172.21.15.57:0/1952099921 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e80036a0 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.279+0000 7f73deffd700 1 -- 172.21.15.57:0/1952099921 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e801e070 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.279+0000 7f73f47e9700 1 -- 172.21.15.57:0/1952099921 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ec160400 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.279+0000 7f73deffd700 1 -- 172.21.15.57:0/1952099921 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e8003a60 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.279+0000 7f73f47e9700 1 -- 172.21.15.57:0/1952099921 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ec0cb9e0 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.280+0000 7f73f47e9700 1 -- 172.21.15.57:0/1952099921 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ec161030 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.280+0000 7f73deffd700 1 -- 172.21.15.57:0/1952099921 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f73e800f050 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.281+0000 7f73deffd700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f73d0068320 0x7f73d006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:50.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.281+0000 7f73deffd700 1 -- 172.21.15.57:0/1952099921 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f73e8086800 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.281+0000 7f73f1d84700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f73d0068320 0x7f73d006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:50.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.281+0000 7f73f1d84700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f73d0068320 0x7f73d006a7d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f73e00011e0 tx=0x7f73e0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:50.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.285+0000 7f73deffd700 1 -- 172.21.15.57:0/1952099921 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f73e80533b0 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.491+0000 7f73f47e9700 1 -- 172.21.15.57:0/1952099921 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f73ec15a750 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.917+0000 7f73deffd700 1 -- 172.21.15.57:0/1952099921 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v60) v1 ==== 138+0+0 (secure 0 0 0) 0x7f73e80200b0 con 0x7f73ec0ae7a0 2023-11-28T12:04:50.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.922+0000 7f73f47e9700 1 -- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f73d0068320 msgr2=0x7f73d006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.922+0000 7f73f47e9700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f73d0068320 0x7f73d006a7d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f73e00011e0 tx=0x7f73e0009040 comp rx=0 tx=0).stop 2023-11-28T12:04:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.922+0000 7f73f47e9700 1 -- 172.21.15.57:0/1952099921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ec0ae7a0 msgr2=0x7f73ec1f32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.922+0000 7f73f47e9700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ec0ae7a0 0x7f73ec1f32e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f73e80090b0 tx=0x7f73e8004750 comp rx=0 tx=0).stop 2023-11-28T12:04:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.922+0000 7f73f47e9700 1 -- 172.21.15.57:0/1952099921 shutdown_connections 2023-11-28T12:04:50.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.922+0000 7f73f47e9700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f73d0068320 0x7f73d006a7d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:50.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.922+0000 7f73f47e9700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec15c620 0x7f73ec15f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:50.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.922+0000 7f73f47e9700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73ec0ae7a0 0x7f73ec1f32e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:50.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.922+0000 7f73f47e9700 1 --2- 172.21.15.57:0/1952099921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ec0ccf40 0x7f73ec1f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:50.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.922+0000 7f73f47e9700 1 -- 172.21.15.57:0/1952099921 >> 172.21.15.57:0/1952099921 conn(0x7f73ec0cf370 msgr2=0x7f73ec15a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:50.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.922+0000 7f73f47e9700 1 -- 172.21.15.57:0/1952099921 shutdown_connections 2023-11-28T12:04:50.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:50.923+0000 7f73f47e9700 1 -- 172.21.15.57:0/1952099921 wait complete. 2023-11-28T12:04:50.926 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' created 2023-11-28T12:04:50.939 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T12:04:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.086+0000 7f9a81310700 1 Processor -- start 2023-11-28T12:04:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.086+0000 7f9a81310700 1 -- start start 2023-11-28T12:04:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.086+0000 7f9a81310700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a7c15fe50 0x7f9a7c06d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.086+0000 7f9a81310700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9a7c0b27d0 0x7f9a7c0b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.086+0000 7f9a81310700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7c161370 con 0x7f9a7c0b3b80 2023-11-28T12:04:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.086+0000 7f9a81310700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a7c0b3190 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.086+0000 7f9a81310700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c160220 con 0x7f9a7c15fe50 2023-11-28T12:04:51.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.087+0000 7f9a7b7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9a7c0b27d0 0x7f9a7c0b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:51.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.087+0000 7f9a7b7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9a7c0b27d0 0x7f9a7c0b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54702/0 (socket says 172.21.15.57:54702) 2023-11-28T12:04:51.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.087+0000 7f9a7b7fe700 1 -- 172.21.15.57:0/2439834514 learned_addr learned my addr 172.21.15.57:0/2439834514 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:51.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.087+0000 7f9a7affd700 1 --2- 172.21.15.57:0/2439834514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a7c15fe50 0x7f9a7c06d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:51.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.087+0000 7f9a7b7fe700 1 -- 172.21.15.57:0/2439834514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a7c15fe50 msgr2=0x7f9a7c06d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:51.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.087+0000 7f9a7b7fe700 1 --2- 172.21.15.57:0/2439834514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a7c15fe50 0x7f9a7c06d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:51.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.087+0000 7f9a7b7fe700 1 -- 172.21.15.57:0/2439834514 >> v1:172.21.15.181:6789/0 conn(0x7f9a7c0b3b80 legacy=0x7f9a7c160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:51.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.087+0000 7f9a7b7fe700 1 -- 172.21.15.57:0/2439834514 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c0cf700 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.087+0000 7f9a7b7fe700 1 --2- 172.21.15.57:0/2439834514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a7c0b27d0 0x7f9a7c0b2c50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009e50 tx=0x7f9a7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6847babad94b1c9e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:51.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.087+0000 7f9a79ffb700 1 -- 172.21.15.57:0/2439834514 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7400e070 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.087+0000 7f9a79ffb700 1 -- 172.21.15.57:0/2439834514 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7400b9c0 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.087+0000 7f9a79ffb700 1 -- 172.21.15.57:0/2439834514 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7400bd70 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.088+0000 7f9a81310700 1 -- 172.21.15.57:0/2439834514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a7c0b27d0 msgr2=0x7f9a7c0b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.088+0000 7f9a81310700 1 --2- 172.21.15.57:0/2439834514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a7c0b27d0 0x7f9a7c0b2c50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009e50 tx=0x7f9a7400b330 comp rx=0 tx=0).stop 2023-11-28T12:04:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.088+0000 7f9a81310700 1 -- 172.21.15.57:0/2439834514 shutdown_connections 2023-11-28T12:04:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.088+0000 7f9a81310700 1 --2- 172.21.15.57:0/2439834514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a7c0b27d0 0x7f9a7c0b2c50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.088+0000 7f9a81310700 1 --2- 172.21.15.57:0/2439834514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a7c15fe50 0x7f9a7c06d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.088+0000 7f9a81310700 1 -- 172.21.15.57:0/2439834514 >> 172.21.15.57:0/2439834514 conn(0x7f9a7c06ea80 msgr2=0x7f9a7c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.088+0000 7f9a81310700 1 -- 172.21.15.57:0/2439834514 shutdown_connections 2023-11-28T12:04:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.088+0000 7f9a81310700 1 -- 172.21.15.57:0/2439834514 wait complete. 2023-11-28T12:04:51.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.090+0000 7f9a81310700 1 Processor -- start 2023-11-28T12:04:51.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.090+0000 7f9a81310700 1 -- start start 2023-11-28T12:04:51.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.090+0000 7f9a81310700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a7c0b27d0 0x7f9a7c0cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:51.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.090+0000 7f9a81310700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a7c0b3b80 0x7f9a7c0c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:51.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.090+0000 7f9a81310700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a7c15fe50 0x7f9a7c14ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:51.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.090+0000 7f9a81310700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0c8340 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.090+0000 7f9a81310700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0c8df0 con 0x7f9a7c0b3b80 2023-11-28T12:04:51.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.090+0000 7f9a81310700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0c8550 con 0x7f9a7c15fe50 2023-11-28T12:04:51.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.090+0000 7f9a7affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a7c0b27d0 0x7f9a7c0cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:51.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.090+0000 7f9a7affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a7c0b27d0 0x7f9a7c0cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54718/0 (socket says 172.21.15.57:54718) 2023-11-28T12:04:51.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.090+0000 7f9a7affd700 1 -- 172.21.15.57:0/3263961209 learned_addr learned my addr 172.21.15.57:0/3263961209 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:51.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a7a7fc700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a7c0b3b80 0x7f9a7c0c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:51.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a7b7fe700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a7c15fe50 0x7f9a7c14ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:51.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a7affd700 1 -- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a7c15fe50 msgr2=0x7f9a7c14ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:51.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a7affd700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a7c15fe50 0x7f9a7c14ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:51.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a7affd700 1 -- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a7c0b3b80 msgr2=0x7f9a7c0c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:51.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a7affd700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a7c0b3b80 0x7f9a7c0c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:51.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a7affd700 1 -- 172.21.15.57:0/3263961209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a74009b30 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a7a7fc700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a7c0b3b80 0x7f9a7c0c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:51.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a7affd700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a7c0b27d0 0x7f9a7c0cde00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00d800 tx=0x7f9a6c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:51.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a737fe700 1 -- 172.21.15.57:0/3263961209 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0125f0 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a737fe700 1 -- 172.21.15.57:0/3263961209 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c012750 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a81310700 1 -- 172.21.15.57:0/3263961209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7c0cedd0 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a737fe700 1 -- 172.21.15.57:0/3263961209 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c012b00 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.091+0000 7f9a81310700 1 -- 172.21.15.57:0/3263961209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7c0b3fd0 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.092+0000 7f9a81310700 1 -- 172.21.15.57:0/3263961209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7c0cf0b0 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.093+0000 7f9a737fe700 1 -- 172.21.15.57:0/3263961209 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9a6c012c60 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.093+0000 7f9a737fe700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9a5c068320 0x7f9a5c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:51.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.094+0000 7f9a737fe700 1 -- 172.21.15.57:0/3263961209 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f9a6c085f80 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.094+0000 7f9a7a7fc700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9a5c068320 0x7f9a5c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:51.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.094+0000 7f9a7a7fc700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9a5c068320 0x7f9a5c06a7d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0c91e0 tx=0x7f9a64009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:51.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.098+0000 7f9a737fe700 1 -- 172.21.15.57:0/3263961209 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9a6c0529c0 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.302+0000 7f9a81310700 1 -- 172.21.15.57:0/3263961209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9a7c0743f0 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.937+0000 7f9a737fe700 1 -- 172.21.15.57:0/3263961209 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v61) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9a6c055fe0 con 0x7f9a7c0b27d0 2023-11-28T12:04:51.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.947+0000 7f9a81310700 1 -- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9a5c068320 msgr2=0x7f9a5c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:51.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.947+0000 7f9a81310700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9a5c068320 0x7f9a5c06a7d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0c91e0 tx=0x7f9a64009040 comp rx=0 tx=0).stop 2023-11-28T12:04:51.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.947+0000 7f9a81310700 1 -- 172.21.15.57:0/3263961209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a7c0b27d0 msgr2=0x7f9a7c0cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:51.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.947+0000 7f9a81310700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a7c0b27d0 0x7f9a7c0cde00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00d800 tx=0x7f9a6c00dbc0 comp rx=0 tx=0).stop 2023-11-28T12:04:51.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.947+0000 7f9a81310700 1 -- 172.21.15.57:0/3263961209 shutdown_connections 2023-11-28T12:04:51.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.947+0000 7f9a81310700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9a5c068320 0x7f9a5c06a7d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:51.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.947+0000 7f9a81310700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a7c15fe50 0x7f9a7c14ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:51.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.947+0000 7f9a81310700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a7c0b27d0 0x7f9a7c0cde00 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:51.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.947+0000 7f9a81310700 1 --2- 172.21.15.57:0/3263961209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a7c0b3b80 0x7f9a7c0c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:51.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.947+0000 7f9a81310700 1 -- 172.21.15.57:0/3263961209 >> 172.21.15.57:0/3263961209 conn(0x7f9a7c06ea80 msgr2=0x7f9a7c15dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:51.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.947+0000 7f9a81310700 1 -- 172.21.15.57:0/3263961209 shutdown_connections 2023-11-28T12:04:51.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:51.947+0000 7f9a81310700 1 -- 172.21.15.57:0/3263961209 wait complete. 2023-11-28T12:04:51.949 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' created 2023-11-28T12:04:51.962 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T12:04:52.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.104+0000 7f52d1cbf700 1 Processor -- start 2023-11-28T12:04:52.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.104+0000 7f52d1cbf700 1 -- start start 2023-11-28T12:04:52.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.104+0000 7f52d1cbf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52cc15a4d0 0x7f52cc06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.104+0000 7f52d1cbf700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f52cc0cdf20 0x7f52cc0cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.104+0000 7f52d1cbf700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc06ac00 con 0x7f52cc06a7b0 2023-11-28T12:04:52.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.104+0000 7f52d1cbf700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc0ce360 con 0x7f52cc0cdf20 2023-11-28T12:04:52.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.104+0000 7f52d1cbf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc15e0d0 con 0x7f52cc15a4d0 2023-11-28T12:04:52.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52cbfff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f52cc0cdf20 0x7f52cc0cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52caffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f52cc06a7b0 0x7f52cc0cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37822/0 (socket says 172.21.15.57:37822) 2023-11-28T12:04:52.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52caffd700 1 -- 172.21.15.57:0/880011414 learned_addr learned my addr 172.21.15.57:0/880011414 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:52.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52cbfff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f52cc0cdf20 0x7f52cc0cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54722/0 (socket says 172.21.15.57:54722) 2023-11-28T12:04:52.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52cb7fe700 1 --2- 172.21.15.57:0/880011414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52cc15a4d0 0x7f52cc06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52cbfff700 1 -- 172.21.15.57:0/880011414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52cc15a4d0 msgr2=0x7f52cc06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:52.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52cbfff700 1 --2- 172.21.15.57:0/880011414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52cc15a4d0 0x7f52cc06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52cbfff700 1 -- 172.21.15.57:0/880011414 >> v1:172.21.15.57:6789/0 conn(0x7f52cc06a7b0 legacy=0x7f52cc0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52cbfff700 1 -- 172.21.15.57:0/880011414 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc0ce560 con 0x7f52cc0cdf20 2023-11-28T12:04:52.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52ca7fc700 1 -- 172.21.15.57:0/880011414 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224734006 0 0) 0x7f52c0001110 con 0x7f52cc06a7b0 2023-11-28T12:04:52.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52cbfff700 1 --2- 172.21.15.57:0/880011414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52cc0cdf20 0x7f52cc0cb010 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009d90 tx=0x7f52c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef0aab598ede298 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:52.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52ca7fc700 1 -- 172.21.15.57:0/880011414 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c400e070 con 0x7f52cc0cdf20 2023-11-28T12:04:52.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52ca7fc700 1 -- 172.21.15.57:0/880011414 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c400bbd0 con 0x7f52cc0cdf20 2023-11-28T12:04:52.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.105+0000 7f52ca7fc700 1 -- 172.21.15.57:0/880011414 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4012430 con 0x7f52cc0cdf20 2023-11-28T12:04:52.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.106+0000 7f52d1cbf700 1 -- 172.21.15.57:0/880011414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52cc0cdf20 msgr2=0x7f52cc0cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.106+0000 7f52d1cbf700 1 --2- 172.21.15.57:0/880011414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52cc0cdf20 0x7f52cc0cb010 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009d90 tx=0x7f52c400b330 comp rx=0 tx=0).stop 2023-11-28T12:04:52.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.106+0000 7f52d1cbf700 1 -- 172.21.15.57:0/880011414 shutdown_connections 2023-11-28T12:04:52.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.106+0000 7f52d1cbf700 1 --2- 172.21.15.57:0/880011414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52cc0cdf20 0x7f52cc0cb010 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.106+0000 7f52d1cbf700 1 --2- 172.21.15.57:0/880011414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52cc15a4d0 0x7f52cc06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.106+0000 7f52d1cbf700 1 -- 172.21.15.57:0/880011414 >> 172.21.15.57:0/880011414 conn(0x7f52cc06c010 msgr2=0x7f52cc0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:52.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.106+0000 7f52d1cbf700 1 -- 172.21.15.57:0/880011414 shutdown_connections 2023-11-28T12:04:52.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.106+0000 7f52d1cbf700 1 -- 172.21.15.57:0/880011414 wait complete. 2023-11-28T12:04:52.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.107+0000 7f52d1cbf700 1 Processor -- start 2023-11-28T12:04:52.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.108+0000 7f52d1cbf700 1 -- start start 2023-11-28T12:04:52.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.108+0000 7f52d1cbf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52cc06a7b0 0x7f52cc0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.108+0000 7f52d1cbf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52cc0cdf20 0x7f52cc1f1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.108+0000 7f52d1cbf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52cc15a4d0 0x7f52cc0caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.108+0000 7f52d1cbf700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0b22f0 con 0x7f52cc0cdf20 2023-11-28T12:04:52.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.108+0000 7f52d1cbf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc156960 con 0x7f52cc06a7b0 2023-11-28T12:04:52.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.108+0000 7f52d1cbf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc1577c0 con 0x7f52cc15a4d0 2023-11-28T12:04:52.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.108+0000 7f52caffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52cc0cdf20 0x7f52cc1f1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.108+0000 7f52caffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52cc0cdf20 0x7f52cc1f1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54734/0 (socket says 172.21.15.57:54734) 2023-11-28T12:04:52.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.108+0000 7f52caffd700 1 -- 172.21.15.57:0/1549494327 learned_addr learned my addr 172.21.15.57:0/1549494327 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:52.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.109+0000 7f52cbfff700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52cc15a4d0 0x7f52cc0caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.109+0000 7f52cb7fe700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52cc06a7b0 0x7f52cc0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.109+0000 7f52caffd700 1 -- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52cc15a4d0 msgr2=0x7f52cc0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.109+0000 7f52caffd700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52cc15a4d0 0x7f52cc0caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.109+0000 7f52caffd700 1 -- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52cc06a7b0 msgr2=0x7f52cc0b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.109+0000 7f52caffd700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52cc06a7b0 0x7f52cc0b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.109+0000 7f52caffd700 1 -- 172.21.15.57:0/1549494327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4009a70 con 0x7f52cc0cdf20 2023-11-28T12:04:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.109+0000 7f52caffd700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52cc0cdf20 0x7f52cc1f1410 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f52c000c760 tx=0x7f52c000ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.109+0000 7f52b3fff700 1 -- 172.21.15.57:0/1549494327 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c0004c80 con 0x7f52cc0cdf20 2023-11-28T12:04:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.109+0000 7f52b3fff700 1 -- 172.21.15.57:0/1549494327 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c0004e00 con 0x7f52cc0cdf20 2023-11-28T12:04:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.109+0000 7f52b3fff700 1 -- 172.21.15.57:0/1549494327 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c0007310 con 0x7f52cc0cdf20 2023-11-28T12:04:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.109+0000 7f52d1cbf700 1 -- 172.21.15.57:0/1549494327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52cc15e600 con 0x7f52cc0cdf20 2023-11-28T12:04:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.109+0000 7f52d1cbf700 1 -- 172.21.15.57:0/1549494327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc15e320 con 0x7f52cc0cdf20 2023-11-28T12:04:52.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.110+0000 7f52d1cbf700 1 -- 172.21.15.57:0/1549494327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52cc15e8e0 con 0x7f52cc0cdf20 2023-11-28T12:04:52.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.111+0000 7f52b3fff700 1 -- 172.21.15.57:0/1549494327 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f52c001a020 con 0x7f52cc0cdf20 2023-11-28T12:04:52.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.111+0000 7f52b3fff700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f52b80682f0 0x7f52b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.111+0000 7f52b3fff700 1 -- 172.21.15.57:0/1549494327 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f52c0085ba0 con 0x7f52cc0cdf20 2023-11-28T12:04:52.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.112+0000 7f52cb7fe700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f52b80682f0 0x7f52b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.112+0000 7f52cb7fe700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f52b80682f0 0x7f52b806a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0b37a0 tx=0x7f52bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:52.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.115+0000 7f52b3fff700 1 -- 172.21.15.57:0/1549494327 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f52c0052500 con 0x7f52cc0cdf20 2023-11-28T12:04:52.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.358+0000 7f52d1cbf700 1 -- 172.21.15.57:0/1549494327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f52cc06ac00 con 0x7f52cc0cdf20 2023-11-28T12:04:52.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.365+0000 7f52b3fff700 1 -- 172.21.15.57:0/1549494327 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v28) v1 ==== 173+0+0 (secure 0 0 0) 0x7f52cc06ac00 con 0x7f52cc0cdf20 2023-11-28T12:04:52.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.371+0000 7f52d1cbf700 1 -- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f52b80682f0 msgr2=0x7f52b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.371+0000 7f52d1cbf700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f52b80682f0 0x7f52b806a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0b37a0 tx=0x7f52bc008040 comp rx=0 tx=0).stop 2023-11-28T12:04:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.371+0000 7f52d1cbf700 1 -- 172.21.15.57:0/1549494327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52cc0cdf20 msgr2=0x7f52cc1f1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.371+0000 7f52d1cbf700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52cc0cdf20 0x7f52cc1f1410 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f52c000c760 tx=0x7f52c000ca70 comp rx=0 tx=0).stop 2023-11-28T12:04:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.372+0000 7f52d1cbf700 1 -- 172.21.15.57:0/1549494327 shutdown_connections 2023-11-28T12:04:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.372+0000 7f52d1cbf700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f52b80682f0 0x7f52b806a7a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.372+0000 7f52d1cbf700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52cc15a4d0 0x7f52cc0caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.372+0000 7f52d1cbf700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52cc06a7b0 0x7f52cc0b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.372+0000 7f52d1cbf700 1 --2- 172.21.15.57:0/1549494327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52cc0cdf20 0x7f52cc1f1410 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.372+0000 7f52d1cbf700 1 -- 172.21.15.57:0/1549494327 >> 172.21.15.57:0/1549494327 conn(0x7f52cc06c010 msgr2=0x7f52cc0cf940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:52.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.373+0000 7f52d1cbf700 1 -- 172.21.15.57:0/1549494327 shutdown_connections 2023-11-28T12:04:52.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.374+0000 7f52d1cbf700 1 -- 172.21.15.57:0/1549494327 wait complete. 2023-11-28T12:04:52.378 INFO:teuthology.orchestra.run.smithi057.stderr:new fs with metadata pool 9 and data pool 10 2023-11-28T12:04:52.399 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:04:52.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.533+0000 7f62a94e8700 1 Processor -- start 2023-11-28T12:04:52.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.533+0000 7f62a94e8700 1 -- start start 2023-11-28T12:04:52.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a94e8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a40d0c60 0x7f62a40b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a94e8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f62a406e400 0x7f62a40d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a94e8700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a40d1570 con 0x7f62a4160e70 2023-11-28T12:04:52.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a94e8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a406e900 con 0x7f62a406e400 2023-11-28T12:04:52.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a94e8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a4160390 con 0x7f62a40d0c60 2023-11-28T12:04:52.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a37fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f62a4160e70 0x7f62a406f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37834/0 (socket says 172.21.15.57:37834) 2023-11-28T12:04:52.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a37fe700 1 -- 172.21.15.57:0/4157876728 learned_addr learned my addr 172.21.15.57:0/4157876728 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:52.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a27fc700 1 --2- 172.21.15.57:0/4157876728 >> v2:172.21.15.181:3300/0 conn(0x7f62a406e400 0x7f62a40d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/4157876728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a40d0c60 0x7f62a40b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a1ffb700 1 -- 172.21.15.57:0/4157876728 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785990694 0 0) 0x7f62a40d1570 con 0x7f62a4160e70 2023-11-28T12:04:52.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a1ffb700 1 -- 172.21.15.57:0/4157876728 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f628c0035b0 con 0x7f62a4160e70 2023-11-28T12:04:52.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a1ffb700 1 -- 172.21.15.57:0/4157876728 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2191173610 0 0) 0x7f628c0035b0 con 0x7f62a4160e70 2023-11-28T12:04:52.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a27fc700 1 -- 172.21.15.57:0/4157876728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a40d0c60 msgr2=0x7f62a40b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:52.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a27fc700 1 --2- 172.21.15.57:0/4157876728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a40d0c60 0x7f62a40b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a27fc700 1 -- 172.21.15.57:0/4157876728 >> v1:172.21.15.57:6789/0 conn(0x7f62a4160e70 legacy=0x7f62a406f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a27fc700 1 -- 172.21.15.57:0/4157876728 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a4160670 con 0x7f62a406e400 2023-11-28T12:04:52.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.534+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/4157876728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a40d0c60 0x7f62a40b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:52.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.535+0000 7f62a27fc700 1 --2- 172.21.15.57:0/4157876728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a406e400 0x7f62a40d1030 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6298009840 tx=0x7f6298009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=238c491a0ff61747 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:52.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.535+0000 7f62a1ffb700 1 -- 172.21.15.57:0/4157876728 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c0019f0 con 0x7f62a406e400 2023-11-28T12:04:52.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.535+0000 7f62a1ffb700 1 -- 172.21.15.57:0/4157876728 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6298004030 con 0x7f62a406e400 2023-11-28T12:04:52.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.535+0000 7f62a1ffb700 1 -- 172.21.15.57:0/4157876728 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62980043c0 con 0x7f62a406e400 2023-11-28T12:04:52.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.535+0000 7f62a94e8700 1 -- 172.21.15.57:0/4157876728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a406e400 msgr2=0x7f62a40d1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.535+0000 7f62a94e8700 1 --2- 172.21.15.57:0/4157876728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a406e400 0x7f62a40d1030 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6298009840 tx=0x7f6298009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:52.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.535+0000 7f62a94e8700 1 -- 172.21.15.57:0/4157876728 shutdown_connections 2023-11-28T12:04:52.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.535+0000 7f62a94e8700 1 --2- 172.21.15.57:0/4157876728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a40d0c60 0x7f62a40b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.535+0000 7f62a94e8700 1 --2- 172.21.15.57:0/4157876728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a406e400 0x7f62a40d1030 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.535+0000 7f62a94e8700 1 -- 172.21.15.57:0/4157876728 >> 172.21.15.57:0/4157876728 conn(0x7f62a406a270 msgr2=0x7f62a406a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:52.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.536+0000 7f62a94e8700 1 -- 172.21.15.57:0/4157876728 shutdown_connections 2023-11-28T12:04:52.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.536+0000 7f62a94e8700 1 -- 172.21.15.57:0/4157876728 wait complete. 2023-11-28T12:04:52.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.537+0000 7f62a94e8700 1 Processor -- start 2023-11-28T12:04:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.537+0000 7f62a94e8700 1 -- start start 2023-11-28T12:04:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a94e8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a406e400 0x7f62a41498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a94e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a40d0c60 0x7f62a415f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a94e8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a4160e70 0x7f62a4151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a94e8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a41f2470 con 0x7f62a4160e70 2023-11-28T12:04:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a94e8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a41f3290 con 0x7f62a40d0c60 2023-11-28T12:04:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a94e8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a41f2680 con 0x7f62a406e400 2023-11-28T12:04:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a4160e70 0x7f62a4151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a4160e70 0x7f62a4151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54744/0 (socket says 172.21.15.57:54744) 2023-11-28T12:04:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a37fe700 1 -- 172.21.15.57:0/3474597154 learned_addr learned my addr 172.21.15.57:0/3474597154 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a27fc700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a40d0c60 0x7f62a415f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a406e400 0x7f62a41498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a37fe700 1 -- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a406e400 msgr2=0x7f62a41498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a37fe700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a406e400 0x7f62a41498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a37fe700 1 -- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a40d0c60 msgr2=0x7f62a415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a37fe700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a40d0c60 0x7f62a415f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a37fe700 1 -- 172.21.15.57:0/3474597154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62940094a0 con 0x7f62a4160e70 2023-11-28T12:04:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.538+0000 7f62a37fe700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a4160e70 0x7f62a4151cf0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f629c00cc80 tx=0x7f629c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.539+0000 7f628b7fe700 1 -- 172.21.15.57:0/3474597154 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c00ade0 con 0x7f62a4160e70 2023-11-28T12:04:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.539+0000 7f628b7fe700 1 -- 172.21.15.57:0/3474597154 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c00ce80 con 0x7f62a4160e70 2023-11-28T12:04:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.539+0000 7f62a94e8700 1 -- 172.21.15.57:0/3474597154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62980094a0 con 0x7f62a4160e70 2023-11-28T12:04:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.539+0000 7f628b7fe700 1 -- 172.21.15.57:0/3474597154 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c0072d0 con 0x7f62a4160e70 2023-11-28T12:04:52.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.539+0000 7f62a94e8700 1 -- 172.21.15.57:0/3474597154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a40d0740 con 0x7f62a4160e70 2023-11-28T12:04:52.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.540+0000 7f62a94e8700 1 -- 172.21.15.57:0/3474597154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a4160950 con 0x7f62a4160e70 2023-11-28T12:04:52.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.540+0000 7f628b7fe700 1 -- 172.21.15.57:0/3474597154 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f629c01a030 con 0x7f62a4160e70 2023-11-28T12:04:52.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.540+0000 7f628b7fe700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f62900682f0 0x7f629006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.541+0000 7f628b7fe700 1 -- 172.21.15.57:0/3474597154 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f629c085c10 con 0x7f62a4160e70 2023-11-28T12:04:52.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.541+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f62900682f0 0x7f629006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.541+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f62900682f0 0x7f629006a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6294009d00 tx=0x7f6294008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:52.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.545+0000 7f628b7fe700 1 -- 172.21.15.57:0/3474597154 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f629c052470 con 0x7f62a4160e70 2023-11-28T12:04:52.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.743+0000 7f62a94e8700 1 -- 172.21.15.57:0/3474597154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62a415d980 con 0x7f62a4160e70 2023-11-28T12:04:52.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.745+0000 7f628b7fe700 1 -- 172.21.15.57:0/3474597154 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+17876 (secure 0 0 0) 0x7f629c018070 con 0x7f62a4160e70 2023-11-28T12:04:52.746 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:52.746 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":62,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:04:52.360071+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:04:50.522537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T12:04:51.333855+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:04:52.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.747+0000 7f62a94e8700 1 -- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f62900682f0 msgr2=0x7f629006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.747+0000 7f62a94e8700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f62900682f0 0x7f629006a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6294009d00 tx=0x7f6294008040 comp rx=0 tx=0).stop 2023-11-28T12:04:52.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.747+0000 7f62a94e8700 1 -- 172.21.15.57:0/3474597154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a4160e70 msgr2=0x7f62a4151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.747+0000 7f62a94e8700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a4160e70 0x7f62a4151cf0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f629c00cc80 tx=0x7f629c00a340 comp rx=0 tx=0).stop 2023-11-28T12:04:52.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.747+0000 7f62a94e8700 1 -- 172.21.15.57:0/3474597154 shutdown_connections 2023-11-28T12:04:52.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.747+0000 7f62a94e8700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f62900682f0 0x7f629006a7a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.747+0000 7f62a94e8700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a4160e70 0x7f62a4151cf0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.747+0000 7f62a94e8700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a406e400 0x7f62a41498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.747+0000 7f62a94e8700 1 --2- 172.21.15.57:0/3474597154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a40d0c60 0x7f62a415f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.747+0000 7f62a94e8700 1 -- 172.21.15.57:0/3474597154 >> 172.21.15.57:0/3474597154 conn(0x7f62a406a270 msgr2=0x7f62a406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:52.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.747+0000 7f62a94e8700 1 -- 172.21.15.57:0/3474597154 shutdown_connections 2023-11-28T12:04:52.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.747+0000 7f62a94e8700 1 -- 172.21.15.57:0/3474597154 wait complete. 2023-11-28T12:04:52.762 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:04:52.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.897+0000 7fb89f572700 1 Processor -- start 2023-11-28T12:04:52.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.898+0000 7fb89f572700 1 -- start start 2023-11-28T12:04:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.898+0000 7fb89f572700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb89806b7b0 0x7fb898072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.898+0000 7fb89f572700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb8980ad1f0 0x7fb8980ad610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.898+0000 7fb89f572700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8980726c0 con 0x7fb8980b0880 2023-11-28T12:04:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.898+0000 7fb89f572700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb89806bc50 con 0x7fb8980ad1f0 2023-11-28T12:04:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.898+0000 7fb89f572700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8980adb50 con 0x7fb89806b7b0 2023-11-28T12:04:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb89d30e700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fb8980b0880 0x7fb8980aeb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37846/0 (socket says 172.21.15.57:37846) 2023-11-28T12:04:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb89d30e700 1 -- 172.21.15.57:0/1217688960 learned_addr learned my addr 172.21.15.57:0/1217688960 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb89db0f700 1 --2- 172.21.15.57:0/1217688960 >> v2:172.21.15.57:3300/0 conn(0x7fb8980ad1f0 0x7fb8980ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb89cb0d700 1 --2- 172.21.15.57:0/1217688960 >> v2:172.21.15.181:3300/0 conn(0x7fb89806b7b0 0x7fb898072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb88bfff700 1 -- 172.21.15.57:0/1217688960 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742166995 0 0) 0x7fb8980726c0 con 0x7fb8980b0880 2023-11-28T12:04:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb88bfff700 1 -- 172.21.15.57:0/1217688960 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8840035b0 con 0x7fb8980b0880 2023-11-28T12:04:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb89db0f700 1 -- 172.21.15.57:0/1217688960 >> v2:172.21.15.181:3300/0 conn(0x7fb89806b7b0 msgr2=0x7fb898072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb89db0f700 1 --2- 172.21.15.57:0/1217688960 >> v2:172.21.15.181:3300/0 conn(0x7fb89806b7b0 0x7fb898072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb89db0f700 1 -- 172.21.15.57:0/1217688960 >> v1:172.21.15.57:6789/0 conn(0x7fb8980b0880 legacy=0x7fb8980aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb89db0f700 1 -- 172.21.15.57:0/1217688960 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8980b43f0 con 0x7fb8980ad1f0 2023-11-28T12:04:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb89db0f700 1 --2- 172.21.15.57:0/1217688960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8980ad1f0 0x7fb8980ad610 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb89806f560 tx=0x7fb89000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70b2007ba9e7553 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb88bfff700 1 -- 172.21.15.57:0/1217688960 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89000e070 con 0x7fb8980ad1f0 2023-11-28T12:04:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb88bfff700 1 -- 172.21.15.57:0/1217688960 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89000bb20 con 0x7fb8980ad1f0 2023-11-28T12:04:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.899+0000 7fb88bfff700 1 -- 172.21.15.57:0/1217688960 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89000be90 con 0x7fb8980ad1f0 2023-11-28T12:04:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.900+0000 7fb89f572700 1 -- 172.21.15.57:0/1217688960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8980ad1f0 msgr2=0x7fb8980ad610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.900+0000 7fb89f572700 1 --2- 172.21.15.57:0/1217688960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8980ad1f0 0x7fb8980ad610 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb89806f560 tx=0x7fb89000b330 comp rx=0 tx=0).stop 2023-11-28T12:04:52.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.900+0000 7fb89f572700 1 -- 172.21.15.57:0/1217688960 shutdown_connections 2023-11-28T12:04:52.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.900+0000 7fb89f572700 1 --2- 172.21.15.57:0/1217688960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8980ad1f0 0x7fb8980ad610 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.900+0000 7fb89f572700 1 --2- 172.21.15.57:0/1217688960 >> v2:172.21.15.181:3300/0 conn(0x7fb89806b7b0 0x7fb898072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.900+0000 7fb89f572700 1 -- 172.21.15.57:0/1217688960 >> 172.21.15.57:0/1217688960 conn(0x7fb8980b2fc0 msgr2=0x7fb898071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:52.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.900+0000 7fb89f572700 1 -- 172.21.15.57:0/1217688960 shutdown_connections 2023-11-28T12:04:52.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.900+0000 7fb89f572700 1 -- 172.21.15.57:0/1217688960 wait complete. 2023-11-28T12:04:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.901+0000 7fb89f572700 1 Processor -- start 2023-11-28T12:04:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.901+0000 7fb89f572700 1 -- start start 2023-11-28T12:04:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.902+0000 7fb89f572700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb89806b7b0 0x7fb8981f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.902+0000 7fb89f572700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8980ad1f0 0x7fb898099ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.902+0000 7fb89f572700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8980b0880 0x7fb8980980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.902+0000 7fb89f572700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8981f3280 con 0x7fb8980b0880 2023-11-28T12:04:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.902+0000 7fb89f572700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89809a540 con 0x7fb8980ad1f0 2023-11-28T12:04:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.902+0000 7fb89f572700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89809a820 con 0x7fb89806b7b0 2023-11-28T12:04:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.902+0000 7fb89db0f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8980b0880 0x7fb8980980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.902+0000 7fb89db0f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8980b0880 0x7fb8980980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54766/0 (socket says 172.21.15.57:54766) 2023-11-28T12:04:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.902+0000 7fb89db0f700 1 -- 172.21.15.57:0/1331558189 learned_addr learned my addr 172.21.15.57:0/1331558189 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.902+0000 7fb89cb0d700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8980ad1f0 0x7fb898099ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.902+0000 7fb89d30e700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb89806b7b0 0x7fb8981f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.903+0000 7fb89db0f700 1 -- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb89806b7b0 msgr2=0x7fb8981f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.903+0000 7fb89db0f700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb89806b7b0 0x7fb8981f2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.903+0000 7fb89db0f700 1 -- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8980ad1f0 msgr2=0x7fb898099ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.903+0000 7fb89db0f700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8980ad1f0 0x7fb898099ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.903+0000 7fb89db0f700 1 -- 172.21.15.57:0/1331558189 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb890009a70 con 0x7fb8980b0880 2023-11-28T12:04:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.903+0000 7fb89cb0d700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8980ad1f0 0x7fb898099ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.903+0000 7fb89db0f700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8980b0880 0x7fb8980980d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb89000b370 tx=0x7fb890004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.903+0000 7fb889ffb700 1 -- 172.21.15.57:0/1331558189 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890003e30 con 0x7fb8980b0880 2023-11-28T12:04:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.903+0000 7fb889ffb700 1 -- 172.21.15.57:0/1331558189 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb890020070 con 0x7fb8980b0880 2023-11-28T12:04:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.903+0000 7fb89f572700 1 -- 172.21.15.57:0/1331558189 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8980b4d00 con 0x7fb8980b0880 2023-11-28T12:04:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.903+0000 7fb889ffb700 1 -- 172.21.15.57:0/1331558189 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8900276c0 con 0x7fb8980b0880 2023-11-28T12:04:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.903+0000 7fb89f572700 1 -- 172.21.15.57:0/1331558189 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8980b46d0 con 0x7fb8980b0880 2023-11-28T12:04:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.904+0000 7fb89f572700 1 -- 172.21.15.57:0/1331558189 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8980b4fe0 con 0x7fb8980b0880 2023-11-28T12:04:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.905+0000 7fb889ffb700 1 -- 172.21.15.57:0/1331558189 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb890027820 con 0x7fb8980b0880 2023-11-28T12:04:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.905+0000 7fb889ffb700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb87c0682f0 0x7fb87c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.905+0000 7fb889ffb700 1 -- 172.21.15.57:0/1331558189 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb890022070 con 0x7fb8980b0880 2023-11-28T12:04:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.905+0000 7fb89d30e700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb87c0682f0 0x7fb87c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.906+0000 7fb89d30e700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb87c0682f0 0x7fb87c06a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb894005910 tx=0x7fb894009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:52.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:52.910+0000 7fb889ffb700 1 -- 172.21.15.57:0/1331558189 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb8900549a0 con 0x7fb8980b0880 2023-11-28T12:04:53.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.113+0000 7fb89f572700 1 -- 172.21.15.57:0/1331558189 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8980743f0 con 0x7fb8980b0880 2023-11-28T12:04:53.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.114+0000 7fb889ffb700 1 -- 172.21.15.57:0/1331558189 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+17876 (secure 0 0 0) 0x7fb890057fc0 con 0x7fb8980b0880 2023-11-28T12:04:53.116 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:53.116 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":62,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:04:52.360071+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:04:50.522537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T12:04:51.333855+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:04:53.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.117+0000 7fb89f572700 1 -- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb87c0682f0 msgr2=0x7fb87c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:53.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.117+0000 7fb89f572700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb87c0682f0 0x7fb87c06a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb894005910 tx=0x7fb894009040 comp rx=0 tx=0).stop 2023-11-28T12:04:53.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.117+0000 7fb89f572700 1 -- 172.21.15.57:0/1331558189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8980b0880 msgr2=0x7fb8980980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:53.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.117+0000 7fb89f572700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8980b0880 0x7fb8980980d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb89000b370 tx=0x7fb890004ad0 comp rx=0 tx=0).stop 2023-11-28T12:04:53.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.117+0000 7fb89f572700 1 -- 172.21.15.57:0/1331558189 shutdown_connections 2023-11-28T12:04:53.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.117+0000 7fb89f572700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb87c0682f0 0x7fb87c06a7a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:53.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.117+0000 7fb89f572700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8980b0880 0x7fb8980980d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:53.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.117+0000 7fb89f572700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb89806b7b0 0x7fb8981f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:53.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.117+0000 7fb89f572700 1 --2- 172.21.15.57:0/1331558189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8980ad1f0 0x7fb898099ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:53.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.117+0000 7fb89f572700 1 -- 172.21.15.57:0/1331558189 >> 172.21.15.57:0/1331558189 conn(0x7fb8980b2fc0 msgr2=0x7fb8980aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:53.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.117+0000 7fb89f572700 1 -- 172.21.15.57:0/1331558189 shutdown_connections 2023-11-28T12:04:53.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.117+0000 7fb89f572700 1 -- 172.21.15.57:0/1331558189 wait complete. 2023-11-28T12:04:53.132 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T12:04:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.270+0000 7f4b8dfd1700 1 Processor -- start 2023-11-28T12:04:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.270+0000 7f4b8dfd1700 1 -- start start 2023-11-28T12:04:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b8dfd1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b8815fe50 0x7f4b8806d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b8dfd1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4b880b27d0 0x7f4b880b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b8dfd1700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b88161370 con 0x7f4b880b3b80 2023-11-28T12:04:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b8dfd1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b880b3190 con 0x7f4b880b27d0 2023-11-28T12:04:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b8dfd1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b88160220 con 0x7f4b8815fe50 2023-11-28T12:04:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b86ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4b880b3b80 0x7f4b88160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37852/0 (socket says 172.21.15.57:37852) 2023-11-28T12:04:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b86ffd700 1 -- 172.21.15.57:0/3347783589 learned_addr learned my addr 172.21.15.57:0/3347783589 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b87fff700 1 --2- 172.21.15.57:0/3347783589 >> v2:172.21.15.181:3300/0 conn(0x7f4b880b27d0 0x7f4b880b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b877fe700 1 --2- 172.21.15.57:0/3347783589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b8815fe50 0x7f4b8806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b867fc700 1 -- 172.21.15.57:0/3347783589 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242297134 0 0) 0x7f4b88161370 con 0x7f4b880b3b80 2023-11-28T12:04:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b867fc700 1 -- 172.21.15.57:0/3347783589 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b740035b0 con 0x7f4b880b3b80 2023-11-28T12:04:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b87fff700 1 -- 172.21.15.57:0/3347783589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b8815fe50 msgr2=0x7f4b8806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b867fc700 1 -- 172.21.15.57:0/3347783589 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3816341021 0 0) 0x7f4b740035b0 con 0x7f4b880b3b80 2023-11-28T12:04:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b87fff700 1 --2- 172.21.15.57:0/3347783589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b8815fe50 0x7f4b8806d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b87fff700 1 -- 172.21.15.57:0/3347783589 >> v1:172.21.15.57:6789/0 conn(0x7f4b880b3b80 legacy=0x7f4b88160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b87fff700 1 -- 172.21.15.57:0/3347783589 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b880cf700 con 0x7f4b880b27d0 2023-11-28T12:04:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.271+0000 7f4b877fe700 1 --2- 172.21.15.57:0/3347783589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b8815fe50 0x7f4b8806d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.272+0000 7f4b87fff700 1 --2- 172.21.15.57:0/3347783589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b880b27d0 0x7f4b880b2c50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009d90 tx=0x7f4b8000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=221d4848d874a4bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.272+0000 7f4b867fc700 1 -- 172.21.15.57:0/3347783589 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000e070 con 0x7f4b880b27d0 2023-11-28T12:04:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.272+0000 7f4b867fc700 1 -- 172.21.15.57:0/3347783589 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b80003a20 con 0x7f4b880b27d0 2023-11-28T12:04:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.272+0000 7f4b867fc700 1 -- 172.21.15.57:0/3347783589 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78001670 con 0x7f4b880b27d0 2023-11-28T12:04:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.272+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/3347783589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b880b27d0 msgr2=0x7f4b880b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.272+0000 7f4b8dfd1700 1 --2- 172.21.15.57:0/3347783589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b880b27d0 0x7f4b880b2c50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009d90 tx=0x7f4b8000b330 comp rx=0 tx=0).stop 2023-11-28T12:04:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.273+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/3347783589 shutdown_connections 2023-11-28T12:04:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.273+0000 7f4b8dfd1700 1 --2- 172.21.15.57:0/3347783589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b880b27d0 0x7f4b880b2c50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.273+0000 7f4b8dfd1700 1 --2- 172.21.15.57:0/3347783589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b8815fe50 0x7f4b8806d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.273+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/3347783589 >> 172.21.15.57:0/3347783589 conn(0x7f4b8806ea80 msgr2=0x7f4b8806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.273+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/3347783589 shutdown_connections 2023-11-28T12:04:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.273+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/3347783589 wait complete. 2023-11-28T12:04:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.274+0000 7f4b8dfd1700 1 Processor -- start 2023-11-28T12:04:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.274+0000 7f4b8dfd1700 1 -- start start 2023-11-28T12:04:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b8dfd1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b880b27d0 0x7f4b880ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b8dfd1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b880b3b80 0x7f4b880c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b8dfd1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b8815fe50 0x7f4b88154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b8dfd1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880c8800 con 0x7f4b880b27d0 2023-11-28T12:04:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b8dfd1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880cacc0 con 0x7f4b880b3b80 2023-11-28T12:04:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b8dfd1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880c99b0 con 0x7f4b8815fe50 2023-11-28T12:04:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b877fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b880b27d0 0x7f4b880ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b877fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b880b27d0 0x7f4b880ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54774/0 (socket says 172.21.15.57:54774) 2023-11-28T12:04:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b877fe700 1 -- 172.21.15.57:0/774698538 learned_addr learned my addr 172.21.15.57:0/774698538 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b87fff700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b8815fe50 0x7f4b88154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b86ffd700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b880b3b80 0x7f4b880c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b877fe700 1 -- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b8815fe50 msgr2=0x7f4b88154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b877fe700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b8815fe50 0x7f4b88154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b877fe700 1 -- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b880b3b80 msgr2=0x7f4b880c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b877fe700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b880b3b80 0x7f4b880c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.275+0000 7f4b877fe700 1 -- 172.21.15.57:0/774698538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b700094a0 con 0x7f4b880b27d0 2023-11-28T12:04:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.276+0000 7f4b877fe700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b880b27d0 0x7f4b880ca780 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000ebe0 tx=0x7f4b70009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.276+0000 7f4b7ffff700 1 -- 172.21.15.57:0/774698538 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7000c8b0 con 0x7f4b880b27d0 2023-11-28T12:04:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.276+0000 7f4b7ffff700 1 -- 172.21.15.57:0/774698538 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7000ca10 con 0x7f4b880b27d0 2023-11-28T12:04:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.276+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/774698538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b80009a70 con 0x7f4b880b27d0 2023-11-28T12:04:53.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.276+0000 7f4b7ffff700 1 -- 172.21.15.57:0/774698538 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7000cda0 con 0x7f4b880b27d0 2023-11-28T12:04:53.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.276+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/774698538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b880cedd0 con 0x7f4b880b27d0 2023-11-28T12:04:53.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.277+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/774698538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b880cf0b0 con 0x7f4b880b27d0 2023-11-28T12:04:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.277+0000 7f4b7ffff700 1 -- 172.21.15.57:0/774698538 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4b70019030 con 0x7f4b880b27d0 2023-11-28T12:04:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.278+0000 7f4b7ffff700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4b680682f0 0x7f4b6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.278+0000 7f4b7ffff700 1 -- 172.21.15.57:0/774698538 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4b70085c90 con 0x7f4b880b27d0 2023-11-28T12:04:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.278+0000 7f4b86ffd700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4b680682f0 0x7f4b6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.278+0000 7f4b86ffd700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4b680682f0 0x7f4b6806a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4b8815fb00 tx=0x7f4b78007a40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.282+0000 7f4b7ffff700 1 -- 172.21.15.57:0/774698538 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4b700524f0 con 0x7f4b880b27d0 2023-11-28T12:04:53.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:53.529+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/774698538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4b880c8940 con 0x7f4b880b27d0 2023-11-28T12:04:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.377+0000 7f4b7ffff700 1 -- 172.21.15.57:0/774698538 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4b880c8940 con 0x7f4b880b27d0 2023-11-28T12:04:54.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.380+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4b680682f0 msgr2=0x7f4b6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:54.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.380+0000 7f4b8dfd1700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4b680682f0 0x7f4b6806a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4b8815fb00 tx=0x7f4b78007a40 comp rx=0 tx=0).stop 2023-11-28T12:04:54.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.380+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/774698538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b880b27d0 msgr2=0x7f4b880ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:54.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.380+0000 7f4b8dfd1700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b880b27d0 0x7f4b880ca780 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000ebe0 tx=0x7f4b70009e20 comp rx=0 tx=0).stop 2023-11-28T12:04:54.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.380+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/774698538 shutdown_connections 2023-11-28T12:04:54.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.380+0000 7f4b8dfd1700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4b680682f0 0x7f4b6806a7a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:54.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.380+0000 7f4b8dfd1700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b8815fe50 0x7f4b88154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:54.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.380+0000 7f4b8dfd1700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b880b27d0 0x7f4b880ca780 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:54.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.380+0000 7f4b8dfd1700 1 --2- 172.21.15.57:0/774698538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b880b3b80 0x7f4b880c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:54.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.380+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/774698538 >> 172.21.15.57:0/774698538 conn(0x7f4b8806ea80 msgr2=0x7f4b8815dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:54.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.380+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/774698538 shutdown_connections 2023-11-28T12:04:54.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.380+0000 7f4b8dfd1700 1 -- 172.21.15.57:0/774698538 wait complete. 2023-11-28T12:04:54.396 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T12:04:54.397 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T12:04:54.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.538+0000 7fb069509700 1 Processor -- start 2023-11-28T12:04:54.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.538+0000 7fb069509700 1 -- start start 2023-11-28T12:04:54.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.539+0000 7fb069509700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb06415fe50 0x7fb06406d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:54.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.539+0000 7fb069509700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb0640b3b80 0x7fb064160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:54.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.539+0000 7fb069509700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb064161370 con 0x7fb0640b27d0 2023-11-28T12:04:54.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.539+0000 7fb069509700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb064160220 con 0x7fb0640b3b80 2023-11-28T12:04:54.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.539+0000 7fb069509700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0640cf700 con 0x7fb06415fe50 2023-11-28T12:04:54.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.539+0000 7fb062ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb06415fe50 0x7fb06406d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:54.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.539+0000 7fb0627fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb0640b3b80 0x7fb064160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:54.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.539+0000 7fb0637fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb0640b27d0 0x7fb0640cfb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:50520/0 (socket says 172.21.15.57:50520) 2023-11-28T12:04:54.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.539+0000 7fb0637fe700 1 -- 172.21.15.57:0/2688214982 learned_addr learned my addr 172.21.15.57:0/2688214982 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:54.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.540+0000 7fb0627fc700 1 -- 172.21.15.57:0/2688214982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb06415fe50 msgr2=0x7fb06406d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:54.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.540+0000 7fb0627fc700 1 --2- 172.21.15.57:0/2688214982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb06415fe50 0x7fb06406d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:54.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.540+0000 7fb0627fc700 1 -- 172.21.15.57:0/2688214982 >> v1:172.21.15.181:6789/0 conn(0x7fb0640b27d0 legacy=0x7fb0640cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:54.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.540+0000 7fb0627fc700 1 -- 172.21.15.57:0/2688214982 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0640b3fd0 con 0x7fb0640b3b80 2023-11-28T12:04:54.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.540+0000 7fb062ffd700 1 --2- 172.21.15.57:0/2688214982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb06415fe50 0x7fb06406d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:04:54.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.540+0000 7fb0627fc700 1 --2- 172.21.15.57:0/2688214982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0640b3b80 0x7fb064160ad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009840 tx=0x7fb04c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baa3ac11345f7e32 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:54.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.540+0000 7fb061ffb700 1 -- 172.21.15.57:0/2688214982 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c011830 con 0x7fb0640b3b80 2023-11-28T12:04:54.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.540+0000 7fb061ffb700 1 -- 172.21.15.57:0/2688214982 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c011990 con 0x7fb0640b3b80 2023-11-28T12:04:54.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.540+0000 7fb061ffb700 1 -- 172.21.15.57:0/2688214982 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c011d20 con 0x7fb0640b3b80 2023-11-28T12:04:54.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.541+0000 7fb069509700 1 -- 172.21.15.57:0/2688214982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0640b3b80 msgr2=0x7fb064160ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:54.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.541+0000 7fb069509700 1 --2- 172.21.15.57:0/2688214982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0640b3b80 0x7fb064160ad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009840 tx=0x7fb04c009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:54.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.541+0000 7fb069509700 1 -- 172.21.15.57:0/2688214982 shutdown_connections 2023-11-28T12:04:54.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.541+0000 7fb069509700 1 --2- 172.21.15.57:0/2688214982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb06415fe50 0x7fb06406d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:54.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.541+0000 7fb069509700 1 --2- 172.21.15.57:0/2688214982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0640b3b80 0x7fb064160ad0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:54.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.541+0000 7fb069509700 1 -- 172.21.15.57:0/2688214982 >> 172.21.15.57:0/2688214982 conn(0x7fb06406ea80 msgr2=0x7fb06406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:54.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.541+0000 7fb069509700 1 -- 172.21.15.57:0/2688214982 shutdown_connections 2023-11-28T12:04:54.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.541+0000 7fb069509700 1 -- 172.21.15.57:0/2688214982 wait complete. 2023-11-28T12:04:54.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.542+0000 7fb069509700 1 Processor -- start 2023-11-28T12:04:54.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.542+0000 7fb069509700 1 -- start start 2023-11-28T12:04:54.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.543+0000 7fb069509700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0640b27d0 0x7fb064154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:54.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.543+0000 7fb069509700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0640b3b80 0x7fb0640c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:54.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.543+0000 7fb069509700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb06415fe50 0x7fb0640c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:54.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.543+0000 7fb069509700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0641e90b0 con 0x7fb0640b3b80 2023-11-28T12:04:54.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.543+0000 7fb069509700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0641e8dd0 con 0x7fb06415fe50 2023-11-28T12:04:54.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.543+0000 7fb069509700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0640aca20 con 0x7fb0640b27d0 2023-11-28T12:04:54.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.543+0000 7fb0627fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0640b3b80 0x7fb0640c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:54.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.543+0000 7fb0627fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0640b3b80 0x7fb0640c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46312/0 (socket says 172.21.15.57:46312) 2023-11-28T12:04:54.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.543+0000 7fb0627fc700 1 -- 172.21.15.57:0/2774477812 learned_addr learned my addr 172.21.15.57:0/2774477812 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:54.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.543+0000 7fb0637fe700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb06415fe50 0x7fb0640c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:54.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.543+0000 7fb062ffd700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0640b27d0 0x7fb064154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:54.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.544+0000 7fb0627fc700 1 -- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0640b27d0 msgr2=0x7fb064154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:54.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.544+0000 7fb0627fc700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0640b27d0 0x7fb064154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:54.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.544+0000 7fb0627fc700 1 -- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb06415fe50 msgr2=0x7fb0640c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:54.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.544+0000 7fb0627fc700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb06415fe50 0x7fb0640c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:54.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.544+0000 7fb0627fc700 1 -- 172.21.15.57:0/2774477812 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540094a0 con 0x7fb0640b3b80 2023-11-28T12:04:54.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.544+0000 7fb0627fc700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0640b3b80 0x7fb0640c7e00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009fd0 tx=0x7fb04c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:54.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.544+0000 7fb05b7fe700 1 -- 172.21.15.57:0/2774477812 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0036a0 con 0x7fb0640b3b80 2023-11-28T12:04:54.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.544+0000 7fb05b7fe700 1 -- 172.21.15.57:0/2774477812 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c01e070 con 0x7fb0640b3b80 2023-11-28T12:04:54.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.544+0000 7fb069509700 1 -- 172.21.15.57:0/2774477812 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c0094a0 con 0x7fb0640b3b80 2023-11-28T12:04:54.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.544+0000 7fb05b7fe700 1 -- 172.21.15.57:0/2774477812 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c003a60 con 0x7fb0640b3b80 2023-11-28T12:04:54.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.544+0000 7fb069509700 1 -- 172.21.15.57:0/2774477812 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0640cf0b0 con 0x7fb0640b3b80 2023-11-28T12:04:54.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.545+0000 7fb069509700 1 -- 172.21.15.57:0/2774477812 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0640cedd0 con 0x7fb0640b3b80 2023-11-28T12:04:54.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.546+0000 7fb05b7fe700 1 -- 172.21.15.57:0/2774477812 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb04c00f050 con 0x7fb0640b3b80 2023-11-28T12:04:54.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.546+0000 7fb05b7fe700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb0440682f0 0x7fb04406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:54.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.546+0000 7fb05b7fe700 1 -- 172.21.15.57:0/2774477812 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb04c086c30 con 0x7fb0640b3b80 2023-11-28T12:04:54.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.546+0000 7fb062ffd700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb0440682f0 0x7fb04406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:54.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.547+0000 7fb062ffd700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb0440682f0 0x7fb04406a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb06415f160 tx=0x7fb054009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:54.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.550+0000 7fb05b7fe700 1 -- 172.21.15.57:0/2774477812 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb04c053490 con 0x7fb0640b3b80 2023-11-28T12:04:54.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:54.800+0000 7fb069509700 1 -- 172.21.15.57:0/2774477812 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb0640cf390 con 0x7fb0640b3b80 2023-11-28T12:04:55.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.380+0000 7fb05b7fe700 1 -- 172.21.15.57:0/2774477812 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb04c020020 con 0x7fb0640b3b80 2023-11-28T12:04:55.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.383+0000 7fb069509700 1 -- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb0440682f0 msgr2=0x7fb04406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:55.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.383+0000 7fb069509700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb0440682f0 0x7fb04406a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb06415f160 tx=0x7fb054009210 comp rx=0 tx=0).stop 2023-11-28T12:04:55.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.383+0000 7fb069509700 1 -- 172.21.15.57:0/2774477812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0640b3b80 msgr2=0x7fb0640c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:55.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.383+0000 7fb069509700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0640b3b80 0x7fb0640c7e00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009fd0 tx=0x7fb04c004750 comp rx=0 tx=0).stop 2023-11-28T12:04:55.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.383+0000 7fb069509700 1 -- 172.21.15.57:0/2774477812 shutdown_connections 2023-11-28T12:04:55.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.383+0000 7fb069509700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb0440682f0 0x7fb04406a7a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.383+0000 7fb069509700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb06415fe50 0x7fb0640c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.383+0000 7fb069509700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0640b27d0 0x7fb064154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.383+0000 7fb069509700 1 --2- 172.21.15.57:0/2774477812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0640b3b80 0x7fb0640c7e00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.383+0000 7fb069509700 1 -- 172.21.15.57:0/2774477812 >> 172.21.15.57:0/2774477812 conn(0x7fb06406ea80 msgr2=0x7fb0640b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:55.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.384+0000 7fb069509700 1 -- 172.21.15.57:0/2774477812 shutdown_connections 2023-11-28T12:04:55.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.384+0000 7fb069509700 1 -- 172.21.15.57:0/2774477812 wait complete. 2023-11-28T12:04:55.402 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T12:04:55.402 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:04:55.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.541+0000 7f7472828700 1 Processor -- start 2023-11-28T12:04:55.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.541+0000 7f7472828700 1 -- start start 2023-11-28T12:04:55.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f7472828700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f746c06a270 0x7f746c071f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f7472828700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f746c0ae520 0x7f746c06a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f7472828700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f746c0b43f0 0x7f746c0b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f7472828700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746c0af0c0 con 0x7f746c0ae520 2023-11-28T12:04:55.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f7472828700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746c0b0590 con 0x7f746c0b43f0 2023-11-28T12:04:55.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f7472828700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c159280 con 0x7f746c06a270 2023-11-28T12:04:55.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f746b7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f746c0ae520 0x7f746c06a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f746b7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f746c0ae520 0x7f746c06a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46320/0 (socket says 172.21.15.57:46320) 2023-11-28T12:04:55.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f746b7fe700 1 -- 172.21.15.57:0/3653283554 learned_addr learned my addr 172.21.15.57:0/3653283554 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:55.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f746bfff700 1 --2- 172.21.15.57:0/3653283554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f746c06a270 0x7f746c071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f7470dc5700 1 --2- 172.21.15.57:0/3653283554 >> v2:172.21.15.181:3300/0 conn(0x7f746c0b43f0 0x7f746c0b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f746b7fe700 1 -- 172.21.15.57:0/3653283554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f746c06a270 msgr2=0x7f746c071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:55.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f746b7fe700 1 --2- 172.21.15.57:0/3653283554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f746c06a270 0x7f746c071f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f746b7fe700 1 -- 172.21.15.57:0/3653283554 >> v2:172.21.15.181:3300/0 conn(0x7f746c0b43f0 msgr2=0x7f746c0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:55.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f746b7fe700 1 --2- 172.21.15.57:0/3653283554 >> v2:172.21.15.181:3300/0 conn(0x7f746c0b43f0 0x7f746c0b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f746b7fe700 1 -- 172.21.15.57:0/3653283554 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c06ab80 con 0x7f746c0ae520 2023-11-28T12:04:55.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.543+0000 7f746b7fe700 1 --2- 172.21.15.57:0/3653283554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f746c0ae520 0x7f746c06a640 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f746c06f2e0 tx=0x7f745400ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d4458618f20094 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:55.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.544+0000 7f746affd700 1 -- 172.21.15.57:0/3653283554 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745400cb70 con 0x7f746c0ae520 2023-11-28T12:04:55.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.544+0000 7f746affd700 1 -- 172.21.15.57:0/3653283554 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745400ccd0 con 0x7f746c0ae520 2023-11-28T12:04:55.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.544+0000 7f746affd700 1 -- 172.21.15.57:0/3653283554 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454018620 con 0x7f746c0ae520 2023-11-28T12:04:55.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.544+0000 7f7472828700 1 -- 172.21.15.57:0/3653283554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f746c0ae520 msgr2=0x7f746c06a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:55.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.544+0000 7f7472828700 1 --2- 172.21.15.57:0/3653283554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f746c0ae520 0x7f746c06a640 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f746c06f2e0 tx=0x7f745400ad70 comp rx=0 tx=0).stop 2023-11-28T12:04:55.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.544+0000 7f7472828700 1 -- 172.21.15.57:0/3653283554 shutdown_connections 2023-11-28T12:04:55.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.544+0000 7f7472828700 1 --2- 172.21.15.57:0/3653283554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f746c06a270 0x7f746c071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.544+0000 7f7472828700 1 --2- 172.21.15.57:0/3653283554 >> v2:172.21.15.181:3300/0 conn(0x7f746c0b43f0 0x7f746c0b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.544+0000 7f7472828700 1 --2- 172.21.15.57:0/3653283554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f746c0ae520 0x7f746c06a640 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.544+0000 7f7472828700 1 -- 172.21.15.57:0/3653283554 >> 172.21.15.57:0/3653283554 conn(0x7f746c0b2fc0 msgr2=0x7f746c071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:55.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.544+0000 7f7472828700 1 -- 172.21.15.57:0/3653283554 shutdown_connections 2023-11-28T12:04:55.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.544+0000 7f7472828700 1 -- 172.21.15.57:0/3653283554 wait complete. 2023-11-28T12:04:55.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.546+0000 7f7472828700 1 Processor -- start 2023-11-28T12:04:55.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.546+0000 7f7472828700 1 -- start start 2023-11-28T12:04:55.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.547+0000 7f7472828700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c06a270 0x7f746c1f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.547+0000 7f7472828700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f746c0ae520 0x7f746c06d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.547+0000 7f7472828700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f746c0b43f0 0x7f746c06db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.547+0000 7f7472828700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c15ad00 con 0x7f746c0b43f0 2023-11-28T12:04:55.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.547+0000 7f7472828700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c1f5610 con 0x7f746c06a270 2023-11-28T12:04:55.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.547+0000 7f7472828700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c15c650 con 0x7f746c0ae520 2023-11-28T12:04:55.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.547+0000 7f7470dc5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f746c0b43f0 0x7f746c06db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.547+0000 7f7470dc5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f746c0b43f0 0x7f746c06db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46336/0 (socket says 172.21.15.57:46336) 2023-11-28T12:04:55.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.547+0000 7f7470dc5700 1 -- 172.21.15.57:0/1814931835 learned_addr learned my addr 172.21.15.57:0/1814931835 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:55.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.547+0000 7f746b7fe700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f746c0ae520 0x7f746c06d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.547+0000 7f746bfff700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c06a270 0x7f746c1f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.548+0000 7f7470dc5700 1 -- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f746c0ae520 msgr2=0x7f746c06d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:55.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.548+0000 7f7470dc5700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f746c0ae520 0x7f746c06d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.548+0000 7f7470dc5700 1 -- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c06a270 msgr2=0x7f746c1f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:55.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.548+0000 7f7470dc5700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c06a270 0x7f746c1f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.548+0000 7f7470dc5700 1 -- 172.21.15.57:0/1814931835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745400a6f0 con 0x7f746c0b43f0 2023-11-28T12:04:55.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.548+0000 7f746b7fe700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f746c0ae520 0x7f746c06d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:04:55.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.548+0000 7f7470dc5700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f746c0b43f0 0x7f746c06db90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7464007e70 tx=0x7f746400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:55.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.548+0000 7f7468ff9700 1 -- 172.21.15.57:0/1814931835 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746400ee50 con 0x7f746c0b43f0 2023-11-28T12:04:55.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.548+0000 7f7468ff9700 1 -- 172.21.15.57:0/1814931835 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7464014420 con 0x7f746c0b43f0 2023-11-28T12:04:55.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.548+0000 7f7468ff9700 1 -- 172.21.15.57:0/1814931835 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74640147d0 con 0x7f746c0b43f0 2023-11-28T12:04:55.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.548+0000 7f7472828700 1 -- 172.21.15.57:0/1814931835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746c072670 con 0x7f746c0b43f0 2023-11-28T12:04:55.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.548+0000 7f7472828700 1 -- 172.21.15.57:0/1814931835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c0b0d10 con 0x7f746c0b43f0 2023-11-28T12:04:55.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.549+0000 7f7472828700 1 -- 172.21.15.57:0/1814931835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746c0afc60 con 0x7f746c0b43f0 2023-11-28T12:04:55.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.550+0000 7f7468ff9700 1 -- 172.21.15.57:0/1814931835 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7464012050 con 0x7f746c0b43f0 2023-11-28T12:04:55.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.550+0000 7f7468ff9700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f744c0682f0 0x7f744c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.550+0000 7f7468ff9700 1 -- 172.21.15.57:0/1814931835 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7464086330 con 0x7f746c0b43f0 2023-11-28T12:04:55.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.550+0000 7f746bfff700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f744c0682f0 0x7f744c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.551+0000 7f746bfff700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f744c0682f0 0x7f744c06a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f745c00b9b0 tx=0x7f745c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:55.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.554+0000 7f7468ff9700 1 -- 172.21.15.57:0/1814931835 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7464052b00 con 0x7f746c0b43f0 2023-11-28T12:04:55.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.802+0000 7f7472828700 1 -- 172.21.15.57:0/1814931835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f746c0cb260 con 0x7f746c0b43f0 2023-11-28T12:04:55.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.803+0000 7f7468ff9700 1 -- 172.21.15.57:0/1814931835 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4652 (secure 0 0 0) 0x7f7464056120 con 0x7f746c0b43f0 2023-11-28T12:04:55.804 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:55.805 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2317025916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2317025916},{"type":"v1","addr":"172.21.15.181:6836","nonce":2317025916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3486085199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3486085199},{"type":"v1","addr":"172.21.15.181:6837","nonce":3486085199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/4272259116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":4272259116},{"type":"v1","addr":"172.21.15.57:6834","nonce":4272259116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:04:52.360416+0000","modified":"2023-11-28T12:04:55.377023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6835/685888843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":685888843},{"type":"v1","addr":"172.21.15.57:6835","nonce":685888843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T12:04:55.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.806+0000 7f7472828700 1 -- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f744c0682f0 msgr2=0x7f744c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:55.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.806+0000 7f7472828700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f744c0682f0 0x7f744c06a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f745c00b9b0 tx=0x7f745c008040 comp rx=0 tx=0).stop 2023-11-28T12:04:55.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.806+0000 7f7472828700 1 -- 172.21.15.57:0/1814931835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f746c0b43f0 msgr2=0x7f746c06db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:55.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.806+0000 7f7472828700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f746c0b43f0 0x7f746c06db90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7464007e70 tx=0x7f746400e3c0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.806+0000 7f7472828700 1 -- 172.21.15.57:0/1814931835 shutdown_connections 2023-11-28T12:04:55.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.806+0000 7f7472828700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f744c0682f0 0x7f744c06a7a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.806+0000 7f7472828700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f746c0b43f0 0x7f746c06db90 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.806+0000 7f7472828700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c06a270 0x7f746c1f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.806+0000 7f7472828700 1 --2- 172.21.15.57:0/1814931835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f746c0ae520 0x7f746c06d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.806+0000 7f7472828700 1 -- 172.21.15.57:0/1814931835 >> 172.21.15.57:0/1814931835 conn(0x7f746c0b2fc0 msgr2=0x7f746c0cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:55.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.806+0000 7f7472828700 1 -- 172.21.15.57:0/1814931835 shutdown_connections 2023-11-28T12:04:55.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.806+0000 7f7472828700 1 -- 172.21.15.57:0/1814931835 wait complete. 2023-11-28T12:04:55.808 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 32 2023-11-28T12:04:55.821 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:04:55.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.956+0000 7efc36d0f700 1 Processor -- start 2023-11-28T12:04:55.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.956+0000 7efc36d0f700 1 -- start start 2023-11-28T12:04:55.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc36d0f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc300ce040 0x7efc3015d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc36d0f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7efc30142db0 0x7efc3013ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc36d0f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc300b1770 0x7efc300b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc36d0f700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc300b2130 con 0x7efc30142db0 2023-11-28T12:04:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc36d0f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3013fc90 con 0x7efc300b1770 2023-11-28T12:04:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc36d0f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3013f6d0 con 0x7efc300ce040 2023-11-28T12:04:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc2ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7efc30142db0 0x7efc3013ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc2ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7efc30142db0 0x7efc3013ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46346/0 (socket says 172.21.15.57:46346) 2023-11-28T12:04:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc2ffff700 1 -- 172.21.15.57:0/3708820595 learned_addr learned my addr 172.21.15.57:0/3708820595 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc352ac700 1 --2- 172.21.15.57:0/3708820595 >> v2:172.21.15.181:3300/0 conn(0x7efc300b1770 0x7efc300b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc34aab700 1 --2- 172.21.15.57:0/3708820595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc300ce040 0x7efc3015d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc2ffff700 1 -- 172.21.15.57:0/3708820595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc300ce040 msgr2=0x7efc3015d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc2ffff700 1 --2- 172.21.15.57:0/3708820595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc300ce040 0x7efc3015d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc2ffff700 1 -- 172.21.15.57:0/3708820595 >> v2:172.21.15.181:3300/0 conn(0x7efc300b1770 msgr2=0x7efc300b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc2ffff700 1 --2- 172.21.15.57:0/3708820595 >> v2:172.21.15.181:3300/0 conn(0x7efc300b1770 0x7efc300b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc2ffff700 1 -- 172.21.15.57:0/3708820595 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3013f9b0 con 0x7efc30142db0 2023-11-28T12:04:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc352ac700 1 --2- 172.21.15.57:0/3708820595 >> v2:172.21.15.181:3300/0 conn(0x7efc300b1770 0x7efc300b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.957+0000 7efc2ffff700 1 --2- 172.21.15.57:0/3708820595 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc30142db0 0x7efc3013ed70 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efc300c37c0 tx=0x7efc1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad98eddd6fefd9c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.958+0000 7efc2f7fe700 1 -- 172.21.15.57:0/3708820595 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00b870 con 0x7efc30142db0 2023-11-28T12:04:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.958+0000 7efc2f7fe700 1 -- 172.21.15.57:0/3708820595 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c00b9d0 con 0x7efc30142db0 2023-11-28T12:04:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.958+0000 7efc2f7fe700 1 -- 172.21.15.57:0/3708820595 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00bd60 con 0x7efc30142db0 2023-11-28T12:04:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.958+0000 7efc36d0f700 1 -- 172.21.15.57:0/3708820595 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc30142db0 msgr2=0x7efc3013ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.958+0000 7efc36d0f700 1 --2- 172.21.15.57:0/3708820595 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc30142db0 0x7efc3013ed70 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efc300c37c0 tx=0x7efc1c009bd0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.958+0000 7efc36d0f700 1 -- 172.21.15.57:0/3708820595 shutdown_connections 2023-11-28T12:04:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.958+0000 7efc36d0f700 1 --2- 172.21.15.57:0/3708820595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc300ce040 0x7efc3015d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.958+0000 7efc36d0f700 1 --2- 172.21.15.57:0/3708820595 >> v2:172.21.15.181:3300/0 conn(0x7efc300b1770 0x7efc300b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.958+0000 7efc36d0f700 1 --2- 172.21.15.57:0/3708820595 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc30142db0 0x7efc3013ed70 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.958+0000 7efc36d0f700 1 -- 172.21.15.57:0/3708820595 >> 172.21.15.57:0/3708820595 conn(0x7efc3006c010 msgr2=0x7efc300cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:55.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.958+0000 7efc36d0f700 1 -- 172.21.15.57:0/3708820595 shutdown_connections 2023-11-28T12:04:55.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.958+0000 7efc36d0f700 1 -- 172.21.15.57:0/3708820595 wait complete. 2023-11-28T12:04:55.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.959+0000 7efc36d0f700 1 Processor -- start 2023-11-28T12:04:55.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.960+0000 7efc36d0f700 1 -- start start 2023-11-28T12:04:55.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.960+0000 7efc36d0f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc300b1770 0x7efc301e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.960+0000 7efc36d0f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc300ce040 0x7efc301433c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.960+0000 7efc36d0f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc30142db0 0x7efc30143900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.960+0000 7efc36d0f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc301e3100 con 0x7efc300ce040 2023-11-28T12:04:55.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.960+0000 7efc36d0f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300c65e0 con 0x7efc300b1770 2023-11-28T12:04:55.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.960+0000 7efc36d0f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc300c59d0 con 0x7efc30142db0 2023-11-28T12:04:55.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc2ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc300ce040 0x7efc301433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc2ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc300ce040 0x7efc301433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46362/0 (socket says 172.21.15.57:46362) 2023-11-28T12:04:55.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc2ffff700 1 -- 172.21.15.57:0/675319210 learned_addr learned my addr 172.21.15.57:0/675319210 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:55.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc352ac700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc30142db0 0x7efc30143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc34aab700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc300b1770 0x7efc301e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc2ffff700 1 -- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc30142db0 msgr2=0x7efc30143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:55.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc2ffff700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc30142db0 0x7efc30143900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc2ffff700 1 -- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc300b1770 msgr2=0x7efc301e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:55.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc2ffff700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc300b1770 0x7efc301e2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:55.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc2ffff700 1 -- 172.21.15.57:0/675319210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c009550 con 0x7efc300ce040 2023-11-28T12:04:55.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc2ffff700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc300ce040 0x7efc301433c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efc30070850 tx=0x7efc1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:55.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc2d7fa700 1 -- 172.21.15.57:0/675319210 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c004310 con 0x7efc300ce040 2023-11-28T12:04:55.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc2d7fa700 1 -- 172.21.15.57:0/675319210 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c01f070 con 0x7efc300ce040 2023-11-28T12:04:55.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc2d7fa700 1 -- 172.21.15.57:0/675319210 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0036a0 con 0x7efc300ce040 2023-11-28T12:04:55.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc36d0f700 1 -- 172.21.15.57:0/675319210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3015d3e0 con 0x7efc300ce040 2023-11-28T12:04:55.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.961+0000 7efc36d0f700 1 -- 172.21.15.57:0/675319210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3015d100 con 0x7efc300ce040 2023-11-28T12:04:55.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.962+0000 7efc36d0f700 1 -- 172.21.15.57:0/675319210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc300ce410 con 0x7efc300ce040 2023-11-28T12:04:55.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.963+0000 7efc2d7fa700 1 -- 172.21.15.57:0/675319210 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efc1c02a020 con 0x7efc300ce040 2023-11-28T12:04:55.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.963+0000 7efc2d7fa700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efc140682f0 0x7efc1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:55.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.963+0000 7efc2d7fa700 1 -- 172.21.15.57:0/675319210 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efc1c087320 con 0x7efc300ce040 2023-11-28T12:04:55.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.964+0000 7efc34aab700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efc140682f0 0x7efc1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:55.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.964+0000 7efc34aab700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efc140682f0 0x7efc1406a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efc24008850 tx=0x7efc24008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:55.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:55.968+0000 7efc2d7fa700 1 -- 172.21.15.57:0/675319210 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efc1c053b80 con 0x7efc300ce040 2023-11-28T12:04:56.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.166+0000 7efc36d0f700 1 -- 172.21.15.57:0/675319210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc300c6770 con 0x7efc300ce040 2023-11-28T12:04:56.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.167+0000 7efc2d7fa700 1 -- 172.21.15.57:0/675319210 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17863 (secure 0 0 0) 0x7efc1c0571a0 con 0x7efc300ce040 2023-11-28T12:04:56.169 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:56.169 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":63,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:04:53.362615+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:04:50.522537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T12:04:51.333855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:04:56.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.170+0000 7efc36d0f700 1 -- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efc140682f0 msgr2=0x7efc1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.170+0000 7efc36d0f700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efc140682f0 0x7efc1406a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efc24008850 tx=0x7efc24008040 comp rx=0 tx=0).stop 2023-11-28T12:04:56.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.170+0000 7efc36d0f700 1 -- 172.21.15.57:0/675319210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc300ce040 msgr2=0x7efc301433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.170+0000 7efc36d0f700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc300ce040 0x7efc301433c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efc30070850 tx=0x7efc1c004750 comp rx=0 tx=0).stop 2023-11-28T12:04:56.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.170+0000 7efc36d0f700 1 -- 172.21.15.57:0/675319210 shutdown_connections 2023-11-28T12:04:56.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.170+0000 7efc36d0f700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efc140682f0 0x7efc1406a7a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.170+0000 7efc36d0f700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc30142db0 0x7efc30143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.170+0000 7efc36d0f700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc300b1770 0x7efc301e2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.170+0000 7efc36d0f700 1 --2- 172.21.15.57:0/675319210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc300ce040 0x7efc301433c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.170+0000 7efc36d0f700 1 -- 172.21.15.57:0/675319210 >> 172.21.15.57:0/675319210 conn(0x7efc3006c010 msgr2=0x7efc300cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:56.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.170+0000 7efc36d0f700 1 -- 172.21.15.57:0/675319210 shutdown_connections 2023-11-28T12:04:56.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.170+0000 7efc36d0f700 1 -- 172.21.15.57:0/675319210 wait complete. 2023-11-28T12:04:56.185 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T12:04:56.185 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:04:56.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.318+0000 7f0232408700 1 Processor -- start 2023-11-28T12:04:56.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.318+0000 7f0232408700 1 -- start start 2023-11-28T12:04:56.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.319+0000 7f0232408700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f022c0ce040 0x7f022c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:56.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.319+0000 7f0232408700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022c13fc90 con 0x7f022c142db0 2023-11-28T12:04:56.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.319+0000 7f0232408700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022c13f480 con 0x7f022c0b1770 2023-11-28T12:04:56.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.319+0000 7f0232408700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022c13f6d0 con 0x7f022c0ce040 2023-11-28T12:04:56.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.320+0000 7f0223fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f022c142db0 0x7f022c13ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55562/0 (socket says 172.21.15.57:55562) 2023-11-28T12:04:56.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.320+0000 7f0223fff700 1 -- 172.21.15.57:0/3872923261 learned_addr learned my addr 172.21.15.57:0/3872923261 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:56.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.320+0000 7f022bfff700 1 --2- 172.21.15.57:0/3872923261 >> v2:172.21.15.181:3300/0 conn(0x7f022c0ce040 0x7f022c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:56.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.320+0000 7f022b7fe700 1 -- 172.21.15.57:0/3872923261 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623505275 0 0) 0x7f022c13fc90 con 0x7f022c142db0 2023-11-28T12:04:56.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.320+0000 7f022b7fe700 1 -- 172.21.15.57:0/3872923261 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02180035b0 con 0x7f022c142db0 2023-11-28T12:04:56.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.320+0000 7f022bfff700 1 -- 172.21.15.57:0/3872923261 >> v1:172.21.15.181:6789/0 conn(0x7f022c0b1770 legacy=0x7f022c0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.320+0000 7f022bfff700 1 -- 172.21.15.57:0/3872923261 >> v1:172.21.15.57:6789/0 conn(0x7f022c142db0 legacy=0x7f022c13ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.320+0000 7f022b7fe700 1 -- 172.21.15.57:0/3872923261 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784718877 0 0) 0x7f022c13f480 con 0x7f022c0b1770 2023-11-28T12:04:56.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.320+0000 7f022bfff700 1 -- 172.21.15.57:0/3872923261 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c13f9b0 con 0x7f022c0ce040 2023-11-28T12:04:56.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.320+0000 7f022bfff700 1 --2- 172.21.15.57:0/3872923261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0ce040 0x7f022c15d900 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f021c009840 tx=0x7f021c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8627eeb136e2e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:56.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.320+0000 7f022b7fe700 1 -- 172.21.15.57:0/3872923261 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c00f8a0 con 0x7f022c0ce040 2023-11-28T12:04:56.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.321+0000 7f022b7fe700 1 -- 172.21.15.57:0/3872923261 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c00fa00 con 0x7f022c0ce040 2023-11-28T12:04:56.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.321+0000 7f022b7fe700 1 -- 172.21.15.57:0/3872923261 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c00fd90 con 0x7f022c0ce040 2023-11-28T12:04:56.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.321+0000 7f0232408700 1 -- 172.21.15.57:0/3872923261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0ce040 msgr2=0x7f022c15d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.321+0000 7f0232408700 1 --2- 172.21.15.57:0/3872923261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0ce040 0x7f022c15d900 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f021c009840 tx=0x7f021c009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:56.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.321+0000 7f0232408700 1 -- 172.21.15.57:0/3872923261 shutdown_connections 2023-11-28T12:04:56.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.321+0000 7f0232408700 1 --2- 172.21.15.57:0/3872923261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0ce040 0x7f022c15d900 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.321+0000 7f0232408700 1 -- 172.21.15.57:0/3872923261 >> 172.21.15.57:0/3872923261 conn(0x7f022c06c010 msgr2=0x7f022c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:56.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.321+0000 7f0232408700 1 -- 172.21.15.57:0/3872923261 shutdown_connections 2023-11-28T12:04:56.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.321+0000 7f0232408700 1 -- 172.21.15.57:0/3872923261 wait complete. 2023-11-28T12:04:56.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.322+0000 7f0232408700 1 Processor -- start 2023-11-28T12:04:56.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.323+0000 7f0232408700 1 -- start start 2023-11-28T12:04:56.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.323+0000 7f0232408700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f022c0b1770 0x7f022c0b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:56.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.323+0000 7f0232408700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0ce040 0x7f022c159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:56.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.323+0000 7f0232408700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f022c142db0 0x7f022c150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:56.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.323+0000 7f0232408700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c1513c0 con 0x7f022c0b1770 2023-11-28T12:04:56.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.323+0000 7f0232408700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0b52d0 con 0x7f022c0ce040 2023-11-28T12:04:56.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.323+0000 7f0232408700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0b55b0 con 0x7f022c142db0 2023-11-28T12:04:56.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.323+0000 7f022bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f022c0b1770 0x7f022c0b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:56.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.323+0000 7f022bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f022c0b1770 0x7f022c0b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46364/0 (socket says 172.21.15.57:46364) 2023-11-28T12:04:56.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.323+0000 7f022bfff700 1 -- 172.21.15.57:0/2427652809 learned_addr learned my addr 172.21.15.57:0/2427652809 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:56.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.324+0000 7f0223fff700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0ce040 0x7f022c159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:56.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.324+0000 7f02309a5700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f022c142db0 0x7f022c150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:56.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.324+0000 7f022bfff700 1 -- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f022c142db0 msgr2=0x7f022c150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.324+0000 7f022bfff700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f022c142db0 0x7f022c150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.324+0000 7f022bfff700 1 -- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0ce040 msgr2=0x7f022c159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.324+0000 7f022bfff700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0ce040 0x7f022c159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.324+0000 7f022bfff700 1 -- 172.21.15.57:0/2427652809 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c0094a0 con 0x7f022c0b1770 2023-11-28T12:04:56.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.324+0000 7f022bfff700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f022c0b1770 0x7f022c0b4c60 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f021c0090b0 tx=0x7f021c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:56.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.324+0000 7f02297fa700 1 -- 172.21.15.57:0/2427652809 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c023400 con 0x7f022c0b1770 2023-11-28T12:04:56.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.324+0000 7f02297fa700 1 -- 172.21.15.57:0/2427652809 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c023560 con 0x7f022c0b1770 2023-11-28T12:04:56.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.324+0000 7f0232408700 1 -- 172.21.15.57:0/2427652809 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022c15d3e0 con 0x7f022c0b1770 2023-11-28T12:04:56.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.324+0000 7f02297fa700 1 -- 172.21.15.57:0/2427652809 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c023910 con 0x7f022c0b1770 2023-11-28T12:04:56.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.324+0000 7f0232408700 1 -- 172.21.15.57:0/2427652809 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022c15d100 con 0x7f022c0b1770 2023-11-28T12:04:56.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.325+0000 7f0232408700 1 -- 172.21.15.57:0/2427652809 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022c0ce410 con 0x7f022c0b1770 2023-11-28T12:04:56.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.326+0000 7f02297fa700 1 -- 172.21.15.57:0/2427652809 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f021c023a70 con 0x7f022c0b1770 2023-11-28T12:04:56.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.326+0000 7f02297fa700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f020c0682f0 0x7f020c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:56.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.326+0000 7f02297fa700 1 -- 172.21.15.57:0/2427652809 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f021c01e070 con 0x7f022c0b1770 2023-11-28T12:04:56.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.326+0000 7f0223fff700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f020c0682f0 0x7f020c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:56.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.327+0000 7f0223fff700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f020c0682f0 0x7f020c06a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f022c142260 tx=0x7f0214009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:56.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.331+0000 7f02297fa700 1 -- 172.21.15.57:0/2427652809 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f021c0562f0 con 0x7f022c0b1770 2023-11-28T12:04:56.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.529+0000 7f0232408700 1 -- 172.21.15.57:0/2427652809 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f022c0b5720 con 0x7f020c0682f0 2023-11-28T12:04:56.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.538+0000 7f02297fa700 1 -- 172.21.15.57:0/2427652809 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+301090 (secure 0 0 0) 0x7f022c0b5720 con 0x7f020c0682f0 2023-11-28T12:04:56.543 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:56.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.541+0000 7f0232408700 1 -- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f020c0682f0 msgr2=0x7f020c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.541+0000 7f0232408700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f020c0682f0 0x7f020c06a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f022c142260 tx=0x7f0214009040 comp rx=0 tx=0).stop 2023-11-28T12:04:56.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.541+0000 7f0232408700 1 -- 172.21.15.57:0/2427652809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f022c0b1770 msgr2=0x7f022c0b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.541+0000 7f0232408700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f022c0b1770 0x7f022c0b4c60 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f021c0090b0 tx=0x7f021c0047c0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.541+0000 7f0232408700 1 -- 172.21.15.57:0/2427652809 shutdown_connections 2023-11-28T12:04:56.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.541+0000 7f0232408700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f020c0682f0 0x7f020c06a7a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.541+0000 7f0232408700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f022c142db0 0x7f022c150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.541+0000 7f0232408700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f022c0b1770 0x7f022c0b4c60 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.542+0000 7f0232408700 1 --2- 172.21.15.57:0/2427652809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0ce040 0x7f022c159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.542+0000 7f0232408700 1 -- 172.21.15.57:0/2427652809 >> 172.21.15.57:0/2427652809 conn(0x7f022c06c010 msgr2=0x7f022c158fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:56.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.542+0000 7f0232408700 1 -- 172.21.15.57:0/2427652809 shutdown_connections 2023-11-28T12:04:56.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.542+0000 7f0232408700 1 -- 172.21.15.57:0/2427652809 wait complete. 2023-11-28T12:04:56.546 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:04:56.556 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2023-11-28T12:04:55.707979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":81352,"kb_used_data":6792,"kb_used_omap":0,"kb_used_meta":74496,"kb_avail":754893368,"statfs":{"total":773094113280,"available":773010808832,"internally_reserved":0,"allocated":6955008,"data_stored":3710024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":76283904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.791532"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921344+0000","last_change":"2023-11-28T12:04:50.921344+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921344+0000","last_undegraded":"2023-11-28T12:04:50.921344+0000","last_fullsized":"2023-11-28T12:04:50.921344+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:15:52.635652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.920764+0000","last_change":"2023-11-28T12:04:50.920764+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.920764+0000","last_undegraded":"2023-11-28T12:04:50.920764+0000","last_fullsized":"2023-11-28T12:04:50.920764+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:00.055068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.922385+0000","last_change":"2023-11-28T12:04:50.922385+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.922385+0000","last_undegraded":"2023-11-28T12:04:50.922385+0000","last_fullsized":"2023-11-28T12:04:50.922385+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:05.119970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921531+0000","last_change":"2023-11-28T12:04:50.921531+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921531+0000","last_undegraded":"2023-11-28T12:04:50.921531+0000","last_fullsized":"2023-11-28T12:04:50.921531+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:48.111122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918811+0000","last_change":"2023-11-28T12:04:50.918811+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918811+0000","last_undegraded":"2023-11-28T12:04:50.918811+0000","last_fullsized":"2023-11-28T12:04:50.918811+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:02.722002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.919427+0000","last_change":"2023-11-28T12:04:50.919427+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.919427+0000","last_undegraded":"2023-11-28T12:04:50.919427+0000","last_fullsized":"2023-11-28T12:04:50.919427+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:51:24.026885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918657+0000","last_change":"2023-11-28T12:04:50.918657+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918657+0000","last_undegraded":"2023-11-28T12:04:50.918657+0000","last_fullsized":"2023-11-28T12:04:50.918657+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:49:50.803373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918668+0000","last_change":"2023-11-28T12:04:50.918668+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918668+0000","last_undegraded":"2023-11-28T12:04:50.918668+0000","last_fullsized":"2023-11-28T12:04:50.918668+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:30.385397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.922083+0000","last_change":"2023-11-28T12:04:50.922083+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.922083+0000","last_undegraded":"2023-11-28T12:04:50.922083+0000","last_fullsized":"2023-11-28T12:04:50.922083+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:20.019844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.920655+0000","last_change":"2023-11-28T12:04:50.920655+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.920655+0000","last_undegraded":"2023-11-28T12:04:50.920655+0000","last_fullsized":"2023-11-28T12:04:50.920655+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:27.335615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.922683+0000","last_change":"2023-11-28T12:04:50.922683+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.922683+0000","last_undegraded":"2023-11-28T12:04:50.922683+0000","last_fullsized":"2023-11-28T12:04:50.922683+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:59.043060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.923929+0000","last_change":"2023-11-28T12:04:50.923929+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.923929+0000","last_undegraded":"2023-11-28T12:04:50.923929+0000","last_fullsized":"2023-11-28T12:04:50.923929+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921783+0000","last_change":"2023-11-28T12:04:50.921783+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921783+0000","last_undegraded":"2023-11-28T12:04:50.921783+0000","last_fullsized":"2023-11-28T12:04:50.921783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:25.076484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.919538+0000","last_change":"2023-11-28T12:04:50.919538+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.919538+0000","last_undegraded":"2023-11-28T12:04:50.919538+0000","last_fullsized":"2023-11-28T12:04:50.919538+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921160+0000","last_change":"2023-11-28T12:04:50.921160+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921160+0000","last_undegraded":"2023-11-28T12:04:50.921160+0000","last_fullsized":"2023-11-28T12:04:50.921160+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.920611+0000","last_change":"2023-11-28T12:04:50.920611+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.920611+0000","last_undegraded":"2023-11-28T12:04:50.920611+0000","last_fullsized":"2023-11-28T12:04:50.920611+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:25.076484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918378+0000","last_change":"2023-11-28T12:04:50.918378+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918378+0000","last_undegraded":"2023-11-28T12:04:50.918378+0000","last_fullsized":"2023-11-28T12:04:50.918378+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918495+0000","last_change":"2023-11-28T12:04:50.918495+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918495+0000","last_undegraded":"2023-11-28T12:04:50.918495+0000","last_fullsized":"2023-11-28T12:04:50.918495+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:01.574232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921872+0000","last_change":"2023-11-28T12:04:50.921872+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921872+0000","last_undegraded":"2023-11-28T12:04:50.921872+0000","last_fullsized":"2023-11-28T12:04:50.921872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:36.797091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921337+0000","last_change":"2023-11-28T12:04:50.921337+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921337+0000","last_undegraded":"2023-11-28T12:04:50.921337+0000","last_fullsized":"2023-11-28T12:04:50.921337+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921338+0000","last_change":"2023-11-28T12:04:50.921338+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921338+0000","last_undegraded":"2023-11-28T12:04:50.921338+0000","last_fullsized":"2023-11-28T12:04:50.921338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918025+0000","last_change":"2023-11-28T12:04:50.918025+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918025+0000","last_undegraded":"2023-11-28T12:04:50.918025+0000","last_fullsized":"2023-11-28T12:04:50.918025+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:15:49.848587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918006+0000","last_change":"2023-11-28T12:04:50.918006+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918006+0000","last_undegraded":"2023-11-28T12:04:50.918006+0000","last_fullsized":"2023-11-28T12:04:50.918006+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:55:26.284594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921434+0000","last_change":"2023-11-28T12:04:50.921434+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921434+0000","last_undegraded":"2023-11-28T12:04:50.921434+0000","last_fullsized":"2023-11-28T12:04:50.921434+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:50.784042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921731+0000","last_change":"2023-11-28T12:04:50.921731+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921731+0000","last_undegraded":"2023-11-28T12:04:50.921731+0000","last_fullsized":"2023-11-28T12:04:50.921731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:27.335615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.167933+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:04:48.167933+0000","last_peered":"2023-11-28T12:04:48.167933+0000","last_clean":"2023-11-28T12:04:48.167933+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:04:48.167933+0000","last_undegraded":"2023-11-28T12:04:48.167933+0000","last_fullsized":"2023-11-28T12:04:48.167933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918131+0000","last_change":"2023-11-28T12:04:50.918131+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918131+0000","last_undegraded":"2023-11-28T12:04:50.918131+0000","last_fullsized":"2023-11-28T12:04:50.918131+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:40.127328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":55,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.170391+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:04:48.170391+0000","last_peered":"2023-11-28T12:04:48.170391+0000","last_clean":"2023-11-28T12:04:48.170391+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:04:48.170391+0000","last_undegraded":"2023-11-28T12:04:48.170391+0000","last_fullsized":"2023-11-28T12:04:48.170391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.920780+0000","last_change":"2023-11-28T12:04:50.920780+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.920780+0000","last_undegraded":"2023-11-28T12:04:50.920780+0000","last_fullsized":"2023-11-28T12:04:50.920780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:05.234114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":49,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:50.454294+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:04:50.454294+0000","last_peered":"2023-11-28T12:04:50.454294+0000","last_clean":"2023-11-28T12:04:50.454294+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:04:50.454294+0000","last_undegraded":"2023-11-28T12:04:50.454294+0000","last_fullsized":"2023-11-28T12:04:50.454294+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":195,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.077002+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:04:48.077002+0000","last_peered":"2023-11-28T12:04:48.077002+0000","last_clean":"2023-11-28T12:04:48.077002+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:04:48.077002+0000","last_undegraded":"2023-11-28T12:04:48.077002+0000","last_fullsized":"2023-11-28T12:04:48.077002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918875+0000","last_change":"2023-11-28T12:04:50.918875+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918875+0000","last_undegraded":"2023-11-28T12:04:50.918875+0000","last_fullsized":"2023-11-28T12:04:50.918875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:50.784042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":46,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.076960+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:04:48.076960+0000","last_peered":"2023-11-28T12:04:48.076960+0000","last_clean":"2023-11-28T12:04:48.076960+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:04:48.076960+0000","last_undegraded":"2023-11-28T12:04:48.076960+0000","last_fullsized":"2023-11-28T12:04:48.076960+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918131+0000","last_change":"2023-11-28T12:04:50.918131+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918131+0000","last_undegraded":"2023-11-28T12:04:50.918131+0000","last_fullsized":"2023-11-28T12:04:50.918131+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:05.119970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":46,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.076960+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:04:48.076960+0000","last_peered":"2023-11-28T12:04:48.076960+0000","last_clean":"2023-11-28T12:04:48.076960+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:04:48.076960+0000","last_undegraded":"2023-11-28T12:04:48.076960+0000","last_fullsized":"2023-11-28T12:04:48.076960+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921035+0000","last_change":"2023-11-28T12:04:50.921035+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921035+0000","last_undegraded":"2023-11-28T12:04:50.921035+0000","last_fullsized":"2023-11-28T12:04:50.921035+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:07.170960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.506371+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:04:48.506371+0000","last_peered":"2023-11-28T12:04:48.506371+0000","last_clean":"2023-11-28T12:04:48.506371+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:04:48.506371+0000","last_undegraded":"2023-11-28T12:04:48.506371+0000","last_fullsized":"2023-11-28T12:04:48.506371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921455+0000","last_change":"2023-11-28T12:04:50.921455+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921455+0000","last_undegraded":"2023-11-28T12:04:50.921455+0000","last_fullsized":"2023-11-28T12:04:50.921455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:01.574232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.167172+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:04:48.167172+0000","last_peered":"2023-11-28T12:04:48.167172+0000","last_clean":"2023-11-28T12:04:48.167172+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:04:48.167172+0000","last_undegraded":"2023-11-28T12:04:48.167172+0000","last_fullsized":"2023-11-28T12:04:48.167172+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918965+0000","last_change":"2023-11-28T12:04:50.918965+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918965+0000","last_undegraded":"2023-11-28T12:04:50.918965+0000","last_fullsized":"2023-11-28T12:04:50.918965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:15:52.635652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.167176+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:04:48.167176+0000","last_peered":"2023-11-28T12:04:48.167176+0000","last_clean":"2023-11-28T12:04:48.167176+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:04:48.167176+0000","last_undegraded":"2023-11-28T12:04:48.167176+0000","last_fullsized":"2023-11-28T12:04:48.167176+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10572,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361268,"statfs":{"total":96636764160,"available":96625938432,"internally_reserved":0,"allocated":1314816,"data_stored":906529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55500000000000005,"15min":0.54800000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69699999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64300000000000002,"5min":0.872,"15min":0.872},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.76500000000000001,"15min":0.84699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73099999999999998,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.63600000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.751,"15min":0.83099999999999996},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70199999999999996,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72399999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72399999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.75600000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64900000000000002,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75700000000000001,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.623}]},{"osd":4,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.505},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.39300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64400000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.499}]},{"osd":5,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.433},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55200000000000005},"min":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73599999999999999,"5min":0.752,"15min":0.752},"last":0.39800000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.70499999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69599999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9160,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362680,"statfs":{"total":96636764160,"available":96627384320,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.44500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":1.446,"15min":1.446},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.72099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.52400000000000002,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73499999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80500000000000005,"5min":0.88,"15min":0.88},"last":0.51000000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.53000000000000003,"15min":0.497},"min":{"1min":0.46899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.76800000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.503},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80400000000000005,"5min":1.254,"15min":1.254},"last":0.52100000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.39500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.46000000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80500000000000005,"5min":1.228,"15min":1.228},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46899999999999997,"15min":0.45200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69499999999999995,"5min":1.204,"15min":1.204},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67000000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.371}]},{"osd":5,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70999999999999996,"5min":0.753,"15min":0.753},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.374,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":1.29,"15min":1.29},"last":0.55300000000000005}]},{"osd":7,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.748,"5min":1.159,"15min":1.159},"last":0.44},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.35499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81799999999999995,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.34100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10380,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361460,"statfs":{"total":96636764160,"available":96626135040,"internally_reserved":0,"allocated":1314816,"data_stored":906529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39500000000000002,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.49199999999999999,"5min":1.2,"15min":1.2},"last":0.311},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.27300000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60799999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.39100000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.37,"15min":0.37},"min":{"1min":0.22900000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.52100000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57499999999999996,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.26400000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47499999999999998,"15min":0.48599999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63900000000000001,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.44500000000000001,"15min":0.45400000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54800000000000004,"5min":1.03,"15min":1.03},"last":0.51000000000000001}]},{"osd":4,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.375,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81000000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81599999999999995,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.48399999999999999}]},{"osd":5,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.627,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97699999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.621},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.503,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.91400000000000003,"5min":0.998,"15min":0.998},"last":0.53100000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65800000000000003,"15min":0.67300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89300000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.871,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.498}]},{"osd":7,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64700000000000002,"15min":0.66000000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94999999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91800000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9416,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362424,"statfs":{"total":96636764160,"available":96627122176,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.50600000000000001,"15min":0.56399999999999995},"min":{"1min":0.26400000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.42999999999999999,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.308},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.312,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51600000000000001,"5min":1.667,"15min":1.667},"last":0.28899999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.53100000000000003,"15min":0.58199999999999996},"min":{"1min":0.31900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.47699999999999998,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42999999999999999,"15min":0.437},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53500000000000003,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.432,"15min":0.44400000000000001},"min":{"1min":0.29999999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.49299999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.59499999999999997,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.52400000000000002,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79800000000000004,"5min":1.536,"15min":1.536},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.72199999999999998,"15min":0.76600000000000001},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76000000000000001,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.63400000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.46500000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79300000000000004,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.624,"15min":0.62},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.78600000000000003,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.52900000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74299999999999999,"15min":0.78100000000000003},"min":{"1min":0.441,"5min":0.44,"15min":0.44},"max":{"1min":0.81899999999999995,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63200000000000001,"15min":0.62},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80800000000000005,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.51700000000000002}]},{"osd":7,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.76000000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.755,"5min":1.605,"15min":1.605},"last":0.50800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9288,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362552,"statfs":{"total":96636764160,"available":96627253248,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.375},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58999999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40100000000000002},"min":{"1min":0.318,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.40500000000000003}]},{"osd":1,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42099999999999999,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.60899999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.54800000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.32400000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41299999999999998,"15min":0.40999999999999998},"min":{"1min":0.31,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.621,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.377},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.59499999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.432,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.627,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90200000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.51300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.92100000000000004}]},{"osd":7,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13768,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":13056,"kb_avail":94358072,"statfs":{"total":96636764160,"available":96622665728,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13369344},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.29599999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42299999999999999,"15min":0.441},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.495,"5min":1.238,"15min":1.238},"last":0.373}]},{"osd":1,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42799999999999999,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56799999999999995,"5min":1.53,"15min":1.53},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.441,"5min":0.48599999999999999,"15min":0.51000000000000001},"min":{"1min":0.27100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.622,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47599999999999998,"15min":0.49199999999999999},"min":{"1min":0.34000000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60299999999999998,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.48499999999999999},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60199999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79000000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.51600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91900000000000004,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67900000000000005,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.99299999999999999,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.497},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.51900000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.879,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48399999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92300000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84699999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.94299999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67400000000000004,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.54400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9416,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362424,"statfs":{"total":96636764160,"available":96627122176,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53700000000000003,"15min":0.52200000000000002},"min":{"1min":0.44800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.72099999999999997}]},{"osd":1,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52900000000000003}]},{"osd":2,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76000000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.69999999999999996}]},{"osd":3,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68999999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.503,"15min":0.497},"min":{"1min":0.35199999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.36399999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.498,"15min":0.5},"min":{"1min":0.39100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46600000000000003}]},{"osd":7,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574891,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9352,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362488,"statfs":{"total":96636764160,"available":96627187712,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71199999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.62},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.443,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.63600000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.80500000000000005,"15min":0.91900000000000004},"min":{"1min":0.45200000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74299999999999999,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.56999999999999995}]},{"osd":2,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.64800000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.432,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77800000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66800000000000004}]},{"osd":3,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.64600000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.81699999999999995}]},{"osd":4,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46700000000000003,"15min":0.47299999999999998},"min":{"1min":0.36099999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66400000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51400000000000001,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66400000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.50600000000000001,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.65300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.375,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63300000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.72099999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53600000000000003,"15min":0.54600000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68899999999999995,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.83899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:04:56.563 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:04:56.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.734+0000 7f7c5314e700 1 Processor -- start 2023-11-28T12:04:56.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.734+0000 7f7c5314e700 1 -- start start 2023-11-28T12:04:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.735+0000 7f7c5314e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7c4c06a270 0x7f7c4c06a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.735+0000 7f7c5314e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c4c15cc80 0x7f7c4c06de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.735+0000 7f7c5314e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7c4c06e3a0 0x7f7c4c06ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.735+0000 7f7c5314e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c4c0b2490 con 0x7f7c4c06a270 2023-11-28T12:04:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.735+0000 7f7c5314e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c4c06ab80 con 0x7f7c4c06e3a0 2023-11-28T12:04:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.735+0000 7f7c5314e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c0b1810 con 0x7f7c4c15cc80 2023-11-28T12:04:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.735+0000 7f7c50eea700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7c4c06a270 0x7f7c4c06a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.735+0000 7f7c50eea700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7c4c06a270 0x7f7c4c06a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46366/0 (socket says 172.21.15.57:46366) 2023-11-28T12:04:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.735+0000 7f7c50eea700 1 -- 172.21.15.57:0/1370134158 learned_addr learned my addr 172.21.15.57:0/1370134158 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:56.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.736+0000 7f7c4bfff700 1 --2- 172.21.15.57:0/1370134158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c4c15cc80 0x7f7c4c06de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.736+0000 7f7c516eb700 1 --2- 172.21.15.57:0/1370134158 >> v2:172.21.15.181:3300/0 conn(0x7f7c4c06e3a0 0x7f7c4c06ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.736+0000 7f7c50eea700 1 -- 172.21.15.57:0/1370134158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c4c15cc80 msgr2=0x7f7c4c06de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.736+0000 7f7c50eea700 1 --2- 172.21.15.57:0/1370134158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c4c15cc80 0x7f7c4c06de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.736+0000 7f7c50eea700 1 -- 172.21.15.57:0/1370134158 >> v2:172.21.15.181:3300/0 conn(0x7f7c4c06e3a0 msgr2=0x7f7c4c06ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.736+0000 7f7c50eea700 1 --2- 172.21.15.57:0/1370134158 >> v2:172.21.15.181:3300/0 conn(0x7f7c4c06e3a0 0x7f7c4c06ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.736+0000 7f7c50eea700 1 -- 172.21.15.57:0/1370134158 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4c0b1af0 con 0x7f7c4c06a270 2023-11-28T12:04:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.736+0000 7f7c4bfff700 1 --2- 172.21.15.57:0/1370134158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c4c15cc80 0x7f7c4c06de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.736+0000 7f7c50eea700 1 --2- 172.21.15.57:0/1370134158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c4c06a270 0x7f7c4c06a640 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0af700 tx=0x7f7c3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50b7b2175572734 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.736+0000 7f7c4b7fe700 1 -- 172.21.15.57:0/1370134158 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c00b870 con 0x7f7c4c06a270 2023-11-28T12:04:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.736+0000 7f7c4b7fe700 1 -- 172.21.15.57:0/1370134158 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c00b9d0 con 0x7f7c4c06a270 2023-11-28T12:04:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.736+0000 7f7c4b7fe700 1 -- 172.21.15.57:0/1370134158 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c00bd60 con 0x7f7c4c06a270 2023-11-28T12:04:56.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.737+0000 7f7c5314e700 1 -- 172.21.15.57:0/1370134158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c4c06a270 msgr2=0x7f7c4c06a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.737+0000 7f7c5314e700 1 --2- 172.21.15.57:0/1370134158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c4c06a270 0x7f7c4c06a640 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0af700 tx=0x7f7c3c009bd0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.737+0000 7f7c5314e700 1 -- 172.21.15.57:0/1370134158 shutdown_connections 2023-11-28T12:04:56.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.737+0000 7f7c5314e700 1 --2- 172.21.15.57:0/1370134158 >> v2:172.21.15.181:3300/0 conn(0x7f7c4c06e3a0 0x7f7c4c06ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.737+0000 7f7c5314e700 1 --2- 172.21.15.57:0/1370134158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c4c06a270 0x7f7c4c06a640 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.737+0000 7f7c5314e700 1 --2- 172.21.15.57:0/1370134158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c4c15cc80 0x7f7c4c06de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.737+0000 7f7c5314e700 1 -- 172.21.15.57:0/1370134158 >> 172.21.15.57:0/1370134158 conn(0x7f7c4c06c010 msgr2=0x7f7c4c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:56.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.737+0000 7f7c5314e700 1 -- 172.21.15.57:0/1370134158 shutdown_connections 2023-11-28T12:04:56.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.737+0000 7f7c5314e700 1 -- 172.21.15.57:0/1370134158 wait complete. 2023-11-28T12:04:56.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.738+0000 7f7c5314e700 1 Processor -- start 2023-11-28T12:04:56.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.738+0000 7f7c5314e700 1 -- start start 2023-11-28T12:04:56.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c5314e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c4c06a270 0x7f7c4c152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:56.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c5314e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c4c06e3a0 0x7f7c4c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:56.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c5314e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c4c15cc80 0x7f7c4c0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:56.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c5314e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c0b5450 con 0x7f7c4c06e3a0 2023-11-28T12:04:56.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c5314e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c0c8030 con 0x7f7c4c15cc80 2023-11-28T12:04:56.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c5314e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c0c8310 con 0x7f7c4c06a270 2023-11-28T12:04:56.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c4bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c4c06e3a0 0x7f7c4c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:56.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c4bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c4c06e3a0 0x7f7c4c0b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46372/0 (socket says 172.21.15.57:46372) 2023-11-28T12:04:56.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c4bfff700 1 -- 172.21.15.57:0/2850861976 learned_addr learned my addr 172.21.15.57:0/2850861976 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:56.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c50eea700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c4c06a270 0x7f7c4c152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:56.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c516eb700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c4c15cc80 0x7f7c4c0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:56.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c4bfff700 1 -- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c4c06a270 msgr2=0x7f7c4c152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c4bfff700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c4c06a270 0x7f7c4c152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c4bfff700 1 -- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c4c15cc80 msgr2=0x7f7c4c0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c4bfff700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c4c15cc80 0x7f7c4c0c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c4bfff700 1 -- 172.21.15.57:0/2850861976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3c009550 con 0x7f7c4c06e3a0 2023-11-28T12:04:56.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.739+0000 7f7c4bfff700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c4c06e3a0 0x7f7c4c0b4b00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000c750 tx=0x7f7c4000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:56.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.740+0000 7f7c497fa700 1 -- 172.21.15.57:0/2850861976 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c40011800 con 0x7f7c4c06e3a0 2023-11-28T12:04:56.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.740+0000 7f7c497fa700 1 -- 172.21.15.57:0/2850861976 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c40011960 con 0x7f7c4c06e3a0 2023-11-28T12:04:56.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.740+0000 7f7c497fa700 1 -- 172.21.15.57:0/2850861976 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c40011cd0 con 0x7f7c4c06e3a0 2023-11-28T12:04:56.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.740+0000 7f7c5314e700 1 -- 172.21.15.57:0/2850861976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4c0b20b0 con 0x7f7c4c06e3a0 2023-11-28T12:04:56.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.740+0000 7f7c5314e700 1 -- 172.21.15.57:0/2850861976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4c0b1dd0 con 0x7f7c4c06e3a0 2023-11-28T12:04:56.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.741+0000 7f7c5314e700 1 -- 172.21.15.57:0/2850861976 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4c06e7e0 con 0x7f7c4c06e3a0 2023-11-28T12:04:56.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.741+0000 7f7c497fa700 1 -- 172.21.15.57:0/2850861976 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7c40020070 con 0x7f7c4c06e3a0 2023-11-28T12:04:56.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.742+0000 7f7c497fa700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7c380682f0 0x7f7c3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:56.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.742+0000 7f7c497fa700 1 -- 172.21.15.57:0/2850861976 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7c40085a70 con 0x7f7c4c06e3a0 2023-11-28T12:04:56.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.742+0000 7f7c50eea700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7c380682f0 0x7f7c3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:56.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.743+0000 7f7c50eea700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7c380682f0 0x7f7c3806a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0ad7b0 tx=0x7f7c3c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:56.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.747+0000 7f7c497fa700 1 -- 172.21.15.57:0/2850861976 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7c400522d0 con 0x7f7c4c06e3a0 2023-11-28T12:04:56.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.941+0000 7f7c5314e700 1 -- 172.21.15.57:0/2850861976 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c4c0743f0 con 0x7f7c380682f0 2023-11-28T12:04:56.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.950+0000 7f7c497fa700 1 -- 172.21.15.57:0/2850861976 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+301090 (secure 0 0 0) 0x7f7c4c0743f0 con 0x7f7c380682f0 2023-11-28T12:04:56.954 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:56.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.953+0000 7f7c5314e700 1 -- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7c380682f0 msgr2=0x7f7c3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.953+0000 7f7c5314e700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7c380682f0 0x7f7c3806a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0ad7b0 tx=0x7f7c3c008030 comp rx=0 tx=0).stop 2023-11-28T12:04:56.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.953+0000 7f7c5314e700 1 -- 172.21.15.57:0/2850861976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c4c06e3a0 msgr2=0x7f7c4c0b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:56.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.953+0000 7f7c5314e700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c4c06e3a0 0x7f7c4c0b4b00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000c750 tx=0x7f7c4000cb10 comp rx=0 tx=0).stop 2023-11-28T12:04:56.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.953+0000 7f7c5314e700 1 -- 172.21.15.57:0/2850861976 shutdown_connections 2023-11-28T12:04:56.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.953+0000 7f7c5314e700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7c380682f0 0x7f7c3806a7a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.953+0000 7f7c5314e700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c4c15cc80 0x7f7c4c0c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.953+0000 7f7c5314e700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c4c06a270 0x7f7c4c152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.953+0000 7f7c5314e700 1 --2- 172.21.15.57:0/2850861976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c4c06e3a0 0x7f7c4c0b4b00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:56.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.953+0000 7f7c5314e700 1 -- 172.21.15.57:0/2850861976 >> 172.21.15.57:0/2850861976 conn(0x7f7c4c06c010 msgr2=0x7f7c4c0ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:56.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.953+0000 7f7c5314e700 1 -- 172.21.15.57:0/2850861976 shutdown_connections 2023-11-28T12:04:56.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:56.953+0000 7f7c5314e700 1 -- 172.21.15.57:0/2850861976 wait complete. 2023-11-28T12:04:56.957 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:04:56.968 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2023-11-28T12:04:55.707979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":81352,"kb_used_data":6792,"kb_used_omap":0,"kb_used_meta":74496,"kb_avail":754893368,"statfs":{"total":773094113280,"available":773010808832,"internally_reserved":0,"allocated":6955008,"data_stored":3710024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":76283904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.791532"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921344+0000","last_change":"2023-11-28T12:04:50.921344+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921344+0000","last_undegraded":"2023-11-28T12:04:50.921344+0000","last_fullsized":"2023-11-28T12:04:50.921344+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:15:52.635652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.920764+0000","last_change":"2023-11-28T12:04:50.920764+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.920764+0000","last_undegraded":"2023-11-28T12:04:50.920764+0000","last_fullsized":"2023-11-28T12:04:50.920764+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:00.055068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.922385+0000","last_change":"2023-11-28T12:04:50.922385+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.922385+0000","last_undegraded":"2023-11-28T12:04:50.922385+0000","last_fullsized":"2023-11-28T12:04:50.922385+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:05.119970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921531+0000","last_change":"2023-11-28T12:04:50.921531+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921531+0000","last_undegraded":"2023-11-28T12:04:50.921531+0000","last_fullsized":"2023-11-28T12:04:50.921531+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:48.111122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918811+0000","last_change":"2023-11-28T12:04:50.918811+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918811+0000","last_undegraded":"2023-11-28T12:04:50.918811+0000","last_fullsized":"2023-11-28T12:04:50.918811+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:02.722002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.919427+0000","last_change":"2023-11-28T12:04:50.919427+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.919427+0000","last_undegraded":"2023-11-28T12:04:50.919427+0000","last_fullsized":"2023-11-28T12:04:50.919427+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:51:24.026885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918657+0000","last_change":"2023-11-28T12:04:50.918657+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918657+0000","last_undegraded":"2023-11-28T12:04:50.918657+0000","last_fullsized":"2023-11-28T12:04:50.918657+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:49:50.803373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918668+0000","last_change":"2023-11-28T12:04:50.918668+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918668+0000","last_undegraded":"2023-11-28T12:04:50.918668+0000","last_fullsized":"2023-11-28T12:04:50.918668+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:30.385397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.922083+0000","last_change":"2023-11-28T12:04:50.922083+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.922083+0000","last_undegraded":"2023-11-28T12:04:50.922083+0000","last_fullsized":"2023-11-28T12:04:50.922083+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:20.019844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.920655+0000","last_change":"2023-11-28T12:04:50.920655+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.920655+0000","last_undegraded":"2023-11-28T12:04:50.920655+0000","last_fullsized":"2023-11-28T12:04:50.920655+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:27.335615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.922683+0000","last_change":"2023-11-28T12:04:50.922683+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.922683+0000","last_undegraded":"2023-11-28T12:04:50.922683+0000","last_fullsized":"2023-11-28T12:04:50.922683+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:59.043060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.923929+0000","last_change":"2023-11-28T12:04:50.923929+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.923929+0000","last_undegraded":"2023-11-28T12:04:50.923929+0000","last_fullsized":"2023-11-28T12:04:50.923929+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921783+0000","last_change":"2023-11-28T12:04:50.921783+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921783+0000","last_undegraded":"2023-11-28T12:04:50.921783+0000","last_fullsized":"2023-11-28T12:04:50.921783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:25.076484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.919538+0000","last_change":"2023-11-28T12:04:50.919538+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.919538+0000","last_undegraded":"2023-11-28T12:04:50.919538+0000","last_fullsized":"2023-11-28T12:04:50.919538+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921160+0000","last_change":"2023-11-28T12:04:50.921160+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921160+0000","last_undegraded":"2023-11-28T12:04:50.921160+0000","last_fullsized":"2023-11-28T12:04:50.921160+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.920611+0000","last_change":"2023-11-28T12:04:50.920611+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.920611+0000","last_undegraded":"2023-11-28T12:04:50.920611+0000","last_fullsized":"2023-11-28T12:04:50.920611+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:25.076484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918378+0000","last_change":"2023-11-28T12:04:50.918378+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918378+0000","last_undegraded":"2023-11-28T12:04:50.918378+0000","last_fullsized":"2023-11-28T12:04:50.918378+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918495+0000","last_change":"2023-11-28T12:04:50.918495+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918495+0000","last_undegraded":"2023-11-28T12:04:50.918495+0000","last_fullsized":"2023-11-28T12:04:50.918495+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:01.574232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921872+0000","last_change":"2023-11-28T12:04:50.921872+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921872+0000","last_undegraded":"2023-11-28T12:04:50.921872+0000","last_fullsized":"2023-11-28T12:04:50.921872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:36.797091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921337+0000","last_change":"2023-11-28T12:04:50.921337+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921337+0000","last_undegraded":"2023-11-28T12:04:50.921337+0000","last_fullsized":"2023-11-28T12:04:50.921337+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921338+0000","last_change":"2023-11-28T12:04:50.921338+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921338+0000","last_undegraded":"2023-11-28T12:04:50.921338+0000","last_fullsized":"2023-11-28T12:04:50.921338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918025+0000","last_change":"2023-11-28T12:04:50.918025+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918025+0000","last_undegraded":"2023-11-28T12:04:50.918025+0000","last_fullsized":"2023-11-28T12:04:50.918025+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:15:49.848587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918006+0000","last_change":"2023-11-28T12:04:50.918006+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918006+0000","last_undegraded":"2023-11-28T12:04:50.918006+0000","last_fullsized":"2023-11-28T12:04:50.918006+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:55:26.284594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921434+0000","last_change":"2023-11-28T12:04:50.921434+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921434+0000","last_undegraded":"2023-11-28T12:04:50.921434+0000","last_fullsized":"2023-11-28T12:04:50.921434+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:50.784042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921731+0000","last_change":"2023-11-28T12:04:50.921731+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921731+0000","last_undegraded":"2023-11-28T12:04:50.921731+0000","last_fullsized":"2023-11-28T12:04:50.921731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:27.335615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.167933+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:04:48.167933+0000","last_peered":"2023-11-28T12:04:48.167933+0000","last_clean":"2023-11-28T12:04:48.167933+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:04:48.167933+0000","last_undegraded":"2023-11-28T12:04:48.167933+0000","last_fullsized":"2023-11-28T12:04:48.167933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918131+0000","last_change":"2023-11-28T12:04:50.918131+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918131+0000","last_undegraded":"2023-11-28T12:04:50.918131+0000","last_fullsized":"2023-11-28T12:04:50.918131+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:40.127328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":55,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.170391+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:04:48.170391+0000","last_peered":"2023-11-28T12:04:48.170391+0000","last_clean":"2023-11-28T12:04:48.170391+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:04:48.170391+0000","last_undegraded":"2023-11-28T12:04:48.170391+0000","last_fullsized":"2023-11-28T12:04:48.170391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.920780+0000","last_change":"2023-11-28T12:04:50.920780+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.920780+0000","last_undegraded":"2023-11-28T12:04:50.920780+0000","last_fullsized":"2023-11-28T12:04:50.920780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:05.234114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":49,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:50.454294+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:04:50.454294+0000","last_peered":"2023-11-28T12:04:50.454294+0000","last_clean":"2023-11-28T12:04:50.454294+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:04:50.454294+0000","last_undegraded":"2023-11-28T12:04:50.454294+0000","last_fullsized":"2023-11-28T12:04:50.454294+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":195,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.077002+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:04:48.077002+0000","last_peered":"2023-11-28T12:04:48.077002+0000","last_clean":"2023-11-28T12:04:48.077002+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:04:48.077002+0000","last_undegraded":"2023-11-28T12:04:48.077002+0000","last_fullsized":"2023-11-28T12:04:48.077002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918875+0000","last_change":"2023-11-28T12:04:50.918875+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918875+0000","last_undegraded":"2023-11-28T12:04:50.918875+0000","last_fullsized":"2023-11-28T12:04:50.918875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:50.784042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":46,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.076960+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:04:48.076960+0000","last_peered":"2023-11-28T12:04:48.076960+0000","last_clean":"2023-11-28T12:04:48.076960+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:04:48.076960+0000","last_undegraded":"2023-11-28T12:04:48.076960+0000","last_fullsized":"2023-11-28T12:04:48.076960+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918131+0000","last_change":"2023-11-28T12:04:50.918131+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918131+0000","last_undegraded":"2023-11-28T12:04:50.918131+0000","last_fullsized":"2023-11-28T12:04:50.918131+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:05.119970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":46,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.076960+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:04:48.076960+0000","last_peered":"2023-11-28T12:04:48.076960+0000","last_clean":"2023-11-28T12:04:48.076960+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:04:48.076960+0000","last_undegraded":"2023-11-28T12:04:48.076960+0000","last_fullsized":"2023-11-28T12:04:48.076960+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921035+0000","last_change":"2023-11-28T12:04:50.921035+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921035+0000","last_undegraded":"2023-11-28T12:04:50.921035+0000","last_fullsized":"2023-11-28T12:04:50.921035+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:07.170960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.506371+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:04:48.506371+0000","last_peered":"2023-11-28T12:04:48.506371+0000","last_clean":"2023-11-28T12:04:48.506371+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:04:48.506371+0000","last_undegraded":"2023-11-28T12:04:48.506371+0000","last_fullsized":"2023-11-28T12:04:48.506371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921455+0000","last_change":"2023-11-28T12:04:50.921455+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921455+0000","last_undegraded":"2023-11-28T12:04:50.921455+0000","last_fullsized":"2023-11-28T12:04:50.921455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:01.574232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.167172+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:04:48.167172+0000","last_peered":"2023-11-28T12:04:48.167172+0000","last_clean":"2023-11-28T12:04:48.167172+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:04:48.167172+0000","last_undegraded":"2023-11-28T12:04:48.167172+0000","last_fullsized":"2023-11-28T12:04:48.167172+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918965+0000","last_change":"2023-11-28T12:04:50.918965+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918965+0000","last_undegraded":"2023-11-28T12:04:50.918965+0000","last_fullsized":"2023-11-28T12:04:50.918965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:15:52.635652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.167176+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:04:48.167176+0000","last_peered":"2023-11-28T12:04:48.167176+0000","last_clean":"2023-11-28T12:04:48.167176+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:04:48.167176+0000","last_undegraded":"2023-11-28T12:04:48.167176+0000","last_fullsized":"2023-11-28T12:04:48.167176+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10572,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361268,"statfs":{"total":96636764160,"available":96625938432,"internally_reserved":0,"allocated":1314816,"data_stored":906529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55500000000000005,"15min":0.54800000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69699999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64300000000000002,"5min":0.872,"15min":0.872},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.76500000000000001,"15min":0.84699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73099999999999998,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.63600000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.751,"15min":0.83099999999999996},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70199999999999996,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72399999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72399999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.75600000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64900000000000002,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75700000000000001,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.623}]},{"osd":4,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.505},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.39300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64400000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.499}]},{"osd":5,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.433},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55200000000000005},"min":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73599999999999999,"5min":0.752,"15min":0.752},"last":0.39800000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.70499999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69599999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9160,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362680,"statfs":{"total":96636764160,"available":96627384320,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.44500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":1.446,"15min":1.446},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.72099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.52400000000000002,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73499999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80500000000000005,"5min":0.88,"15min":0.88},"last":0.51000000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.53000000000000003,"15min":0.497},"min":{"1min":0.46899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.76800000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.503},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80400000000000005,"5min":1.254,"15min":1.254},"last":0.52100000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.39500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.46000000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80500000000000005,"5min":1.228,"15min":1.228},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46899999999999997,"15min":0.45200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69499999999999995,"5min":1.204,"15min":1.204},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67000000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.371}]},{"osd":5,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70999999999999996,"5min":0.753,"15min":0.753},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.374,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":1.29,"15min":1.29},"last":0.55300000000000005}]},{"osd":7,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.748,"5min":1.159,"15min":1.159},"last":0.44},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.35499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81799999999999995,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.34100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10380,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361460,"statfs":{"total":96636764160,"available":96626135040,"internally_reserved":0,"allocated":1314816,"data_stored":906529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39500000000000002,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.49199999999999999,"5min":1.2,"15min":1.2},"last":0.311},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.27300000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60799999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.39100000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.37,"15min":0.37},"min":{"1min":0.22900000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.52100000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57499999999999996,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.26400000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47499999999999998,"15min":0.48599999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63900000000000001,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.44500000000000001,"15min":0.45400000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54800000000000004,"5min":1.03,"15min":1.03},"last":0.51000000000000001}]},{"osd":4,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.375,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81000000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81599999999999995,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.48399999999999999}]},{"osd":5,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.627,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97699999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.621},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.503,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.91400000000000003,"5min":0.998,"15min":0.998},"last":0.53100000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65800000000000003,"15min":0.67300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89300000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.871,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.498}]},{"osd":7,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64700000000000002,"15min":0.66000000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94999999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91800000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9416,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362424,"statfs":{"total":96636764160,"available":96627122176,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.50600000000000001,"15min":0.56399999999999995},"min":{"1min":0.26400000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.42999999999999999,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.308},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.312,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51600000000000001,"5min":1.667,"15min":1.667},"last":0.28899999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.53100000000000003,"15min":0.58199999999999996},"min":{"1min":0.31900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.47699999999999998,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42999999999999999,"15min":0.437},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53500000000000003,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.432,"15min":0.44400000000000001},"min":{"1min":0.29999999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.49299999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.59499999999999997,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.52400000000000002,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79800000000000004,"5min":1.536,"15min":1.536},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.72199999999999998,"15min":0.76600000000000001},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76000000000000001,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.63400000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.46500000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79300000000000004,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.624,"15min":0.62},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.78600000000000003,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.52900000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74299999999999999,"15min":0.78100000000000003},"min":{"1min":0.441,"5min":0.44,"15min":0.44},"max":{"1min":0.81899999999999995,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63200000000000001,"15min":0.62},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80800000000000005,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.51700000000000002}]},{"osd":7,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.76000000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.755,"5min":1.605,"15min":1.605},"last":0.50800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9288,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362552,"statfs":{"total":96636764160,"available":96627253248,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.375},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58999999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40100000000000002},"min":{"1min":0.318,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.40500000000000003}]},{"osd":1,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42099999999999999,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.60899999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.54800000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.32400000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41299999999999998,"15min":0.40999999999999998},"min":{"1min":0.31,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.621,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.377},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.59499999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.432,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.627,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90200000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.51300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.92100000000000004}]},{"osd":7,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13768,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":13056,"kb_avail":94358072,"statfs":{"total":96636764160,"available":96622665728,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13369344},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.29599999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42299999999999999,"15min":0.441},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.495,"5min":1.238,"15min":1.238},"last":0.373}]},{"osd":1,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42799999999999999,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56799999999999995,"5min":1.53,"15min":1.53},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.441,"5min":0.48599999999999999,"15min":0.51000000000000001},"min":{"1min":0.27100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.622,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47599999999999998,"15min":0.49199999999999999},"min":{"1min":0.34000000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60299999999999998,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.48499999999999999},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60199999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79000000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.51600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91900000000000004,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67900000000000005,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.99299999999999999,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.497},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.51900000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.879,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48399999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92300000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84699999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.94299999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67400000000000004,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.54400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9416,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362424,"statfs":{"total":96636764160,"available":96627122176,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53700000000000003,"15min":0.52200000000000002},"min":{"1min":0.44800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.72099999999999997}]},{"osd":1,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52900000000000003}]},{"osd":2,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76000000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.69999999999999996}]},{"osd":3,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68999999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.503,"15min":0.497},"min":{"1min":0.35199999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.36399999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.498,"15min":0.5},"min":{"1min":0.39100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46600000000000003}]},{"osd":7,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574891,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9352,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362488,"statfs":{"total":96636764160,"available":96627187712,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71199999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.62},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.443,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.63600000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.80500000000000005,"15min":0.91900000000000004},"min":{"1min":0.45200000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74299999999999999,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.56999999999999995}]},{"osd":2,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.64800000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.432,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77800000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66800000000000004}]},{"osd":3,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.64600000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.81699999999999995}]},{"osd":4,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46700000000000003,"15min":0.47299999999999998},"min":{"1min":0.36099999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66400000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51400000000000001,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66400000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.50600000000000001,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.65300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.375,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63300000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.72099999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53600000000000003,"15min":0.54600000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68899999999999995,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.83899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:04:56.973 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:04:57.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.146+0000 7f61812e2700 1 Processor -- start 2023-11-28T12:04:57.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.146+0000 7f61812e2700 1 -- start start 2023-11-28T12:04:57.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.147+0000 7f61812e2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f617c06ab10 0x7f617c06aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:57.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.147+0000 7f61812e2700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f617c0d1ee0 con 0x7f617c0cd5a0 2023-11-28T12:04:57.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.147+0000 7f61812e2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f617c161120 con 0x7f617c0d0450 2023-11-28T12:04:57.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.147+0000 7f61812e2700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f617c06b420 con 0x7f617c06ab10 2023-11-28T12:04:57.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.148+0000 7f617affd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f617c06ab10 0x7f617c06aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:57.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.148+0000 7f617a7fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f617c0cd5a0 0x7f617c160300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55570/0 (socket says 172.21.15.57:55570) 2023-11-28T12:04:57.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.148+0000 7f617a7fc700 1 -- 172.21.15.57:0/2829082581 learned_addr learned my addr 172.21.15.57:0/2829082581 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:57.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.148+0000 7f617affd700 1 -- 172.21.15.57:0/2829082581 >> v1:172.21.15.181:6789/0 conn(0x7f617c0d0450 legacy=0x7f617c160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.148+0000 7f617affd700 1 -- 172.21.15.57:0/2829082581 >> v1:172.21.15.57:6789/0 conn(0x7f617c0cd5a0 legacy=0x7f617c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.148+0000 7f617affd700 1 -- 172.21.15.57:0/2829082581 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c0cdb10 con 0x7f617c06ab10 2023-11-28T12:04:57.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.148+0000 7f6179ffb700 1 -- 172.21.15.57:0/2829082581 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997105116 0 0) 0x7f617c0d1ee0 con 0x7f617c0cd5a0 2023-11-28T12:04:57.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.148+0000 7f617affd700 1 --2- 172.21.15.57:0/2829082581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f617c06ab10 0x7f617c06aee0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6164009840 tx=0x7f6164009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02e479c9c09913d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:57.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.148+0000 7f6179ffb700 1 -- 172.21.15.57:0/2829082581 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164011830 con 0x7f617c06ab10 2023-11-28T12:04:57.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.148+0000 7f6179ffb700 1 -- 172.21.15.57:0/2829082581 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6164011990 con 0x7f617c06ab10 2023-11-28T12:04:57.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.148+0000 7f6179ffb700 1 -- 172.21.15.57:0/2829082581 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164011d20 con 0x7f617c06ab10 2023-11-28T12:04:57.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.149+0000 7f61812e2700 1 -- 172.21.15.57:0/2829082581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f617c06ab10 msgr2=0x7f617c06aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.149+0000 7f61812e2700 1 --2- 172.21.15.57:0/2829082581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f617c06ab10 0x7f617c06aee0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6164009840 tx=0x7f6164009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:57.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.149+0000 7f61812e2700 1 -- 172.21.15.57:0/2829082581 shutdown_connections 2023-11-28T12:04:57.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.149+0000 7f61812e2700 1 --2- 172.21.15.57:0/2829082581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f617c06ab10 0x7f617c06aee0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.149+0000 7f61812e2700 1 -- 172.21.15.57:0/2829082581 >> 172.21.15.57:0/2829082581 conn(0x7f617c06c520 msgr2=0x7f617c06c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:57.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.149+0000 7f61812e2700 1 -- 172.21.15.57:0/2829082581 shutdown_connections 2023-11-28T12:04:57.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.149+0000 7f61812e2700 1 -- 172.21.15.57:0/2829082581 wait complete. 2023-11-28T12:04:57.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.150+0000 7f61812e2700 1 Processor -- start 2023-11-28T12:04:57.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.150+0000 7f61812e2700 1 -- start start 2023-11-28T12:04:57.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.151+0000 7f61812e2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f617c06ab10 0x7f617c1513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:57.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.151+0000 7f61812e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f617c0cd5a0 0x7f617c0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:57.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.151+0000 7f61812e2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f617c0d0450 0x7f617c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:57.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.151+0000 7f61812e2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617c1543e0 con 0x7f617c0d0450 2023-11-28T12:04:57.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.151+0000 7f61812e2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617c151910 con 0x7f617c0cd5a0 2023-11-28T12:04:57.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.151+0000 7f61812e2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617c158920 con 0x7f617c06ab10 2023-11-28T12:04:57.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.151+0000 7f617b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f617c0d0450 0x7f617c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:57.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.151+0000 7f617b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f617c0d0450 0x7f617c0d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46382/0 (socket says 172.21.15.57:46382) 2023-11-28T12:04:57.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.151+0000 7f617b7fe700 1 -- 172.21.15.57:0/2641018024 learned_addr learned my addr 172.21.15.57:0/2641018024 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:57.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.152+0000 7f617b7fe700 1 -- 172.21.15.57:0/2641018024 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f617c06ab10 msgr2=0x7f617c1513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.152+0000 7f617b7fe700 1 --2- 172.21.15.57:0/2641018024 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f617c06ab10 0x7f617c1513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.152+0000 7f617b7fe700 1 -- 172.21.15.57:0/2641018024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f617c0cd5a0 msgr2=0x7f617c0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.152+0000 7f617a7fc700 1 --2- 172.21.15.57:0/2641018024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f617c0cd5a0 0x7f617c0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:57.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.152+0000 7f617b7fe700 1 --2- 172.21.15.57:0/2641018024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f617c0cd5a0 0x7f617c0d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.152+0000 7f617b7fe700 1 -- 172.21.15.57:0/2641018024 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640094a0 con 0x7f617c0d0450 2023-11-28T12:04:57.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.152+0000 7f617b7fe700 1 --2- 172.21.15.57:0/2641018024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f617c0d0450 0x7f617c0d1060 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f617400cb50 tx=0x7f617400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:57.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.152+0000 7f61737fe700 1 -- 172.21.15.57:0/2641018024 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174003c00 con 0x7f617c0d0450 2023-11-28T12:04:57.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.152+0000 7f61737fe700 1 -- 172.21.15.57:0/2641018024 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6174003d60 con 0x7f617c0d0450 2023-11-28T12:04:57.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.152+0000 7f61812e2700 1 -- 172.21.15.57:0/2641018024 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617c0d1b70 con 0x7f617c0d0450 2023-11-28T12:04:57.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.152+0000 7f61737fe700 1 -- 172.21.15.57:0/2641018024 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617400ab70 con 0x7f617c0d0450 2023-11-28T12:04:57.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.152+0000 7f61812e2700 1 -- 172.21.15.57:0/2641018024 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617c0d1890 con 0x7f617c0d0450 2023-11-28T12:04:57.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.153+0000 7f61812e2700 1 -- 172.21.15.57:0/2641018024 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617c15cad0 con 0x7f617c0d0450 2023-11-28T12:04:57.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.154+0000 7f61737fe700 1 -- 172.21.15.57:0/2641018024 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6174010050 con 0x7f617c0d0450 2023-11-28T12:04:57.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.154+0000 7f61737fe700 1 --2- 172.21.15.57:0/2641018024 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f615c0682f0 0x7f615c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:57.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.154+0000 7f61737fe700 1 -- 172.21.15.57:0/2641018024 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6174085b50 con 0x7f617c0d0450 2023-11-28T12:04:57.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.154+0000 7f617affd700 1 --2- 172.21.15.57:0/2641018024 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f615c0682f0 0x7f615c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:57.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.155+0000 7f617affd700 1 --2- 172.21.15.57:0/2641018024 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f615c0682f0 0x7f615c06a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6164009b90 tx=0x7f6164010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:57.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.158+0000 7f61737fe700 1 -- 172.21.15.57:0/2641018024 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f61740523b0 con 0x7f617c0d0450 2023-11-28T12:04:57.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.353+0000 7f61812e2700 1 -- 172.21.15.57:0/2641018024 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f617c0743f0 con 0x7f615c0682f0 2023-11-28T12:04:57.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.361+0000 7f61737fe700 1 -- 172.21.15.57:0/2641018024 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+301090 (secure 0 0 0) 0x7f617c0743f0 con 0x7f615c0682f0 2023-11-28T12:04:57.366 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:57.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.365+0000 7f61812e2700 1 -- 172.21.15.57:0/2641018024 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f615c0682f0 msgr2=0x7f615c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.365+0000 7f61812e2700 1 --2- 172.21.15.57:0/2641018024 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f615c0682f0 0x7f615c06a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6164009b90 tx=0x7f6164010000 comp rx=0 tx=0).stop 2023-11-28T12:04:57.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.365+0000 7f61812e2700 1 -- 172.21.15.57:0/2641018024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f617c0d0450 msgr2=0x7f617c0d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.365+0000 7f61812e2700 1 --2- 172.21.15.57:0/2641018024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f617c0d0450 0x7f617c0d1060 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f617400cb50 tx=0x7f617400cf10 comp rx=0 tx=0).stop 2023-11-28T12:04:57.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.365+0000 7f61812e2700 1 -- 172.21.15.57:0/2641018024 shutdown_connections 2023-11-28T12:04:57.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.365+0000 7f61812e2700 1 --2- 172.21.15.57:0/2641018024 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f615c0682f0 0x7f615c06a7a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.365+0000 7f61812e2700 1 --2- 172.21.15.57:0/2641018024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f617c0d0450 0x7f617c0d1060 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.365+0000 7f61812e2700 1 --2- 172.21.15.57:0/2641018024 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f617c06ab10 0x7f617c1513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.365+0000 7f61812e2700 1 --2- 172.21.15.57:0/2641018024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f617c0cd5a0 0x7f617c0d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.365+0000 7f61812e2700 1 -- 172.21.15.57:0/2641018024 >> 172.21.15.57:0/2641018024 conn(0x7f617c06c520 msgr2=0x7f617c0b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:57.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.365+0000 7f61812e2700 1 -- 172.21.15.57:0/2641018024 shutdown_connections 2023-11-28T12:04:57.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.365+0000 7f61812e2700 1 -- 172.21.15.57:0/2641018024 wait complete. 2023-11-28T12:04:57.372 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:04:57.380 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2023-11-28T12:04:55.707979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":81352,"kb_used_data":6792,"kb_used_omap":0,"kb_used_meta":74496,"kb_avail":754893368,"statfs":{"total":773094113280,"available":773010808832,"internally_reserved":0,"allocated":6955008,"data_stored":3710024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":76283904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.791532"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921344+0000","last_change":"2023-11-28T12:04:50.921344+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921344+0000","last_undegraded":"2023-11-28T12:04:50.921344+0000","last_fullsized":"2023-11-28T12:04:50.921344+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:15:52.635652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.920764+0000","last_change":"2023-11-28T12:04:50.920764+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.920764+0000","last_undegraded":"2023-11-28T12:04:50.920764+0000","last_fullsized":"2023-11-28T12:04:50.920764+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:00.055068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.922385+0000","last_change":"2023-11-28T12:04:50.922385+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.922385+0000","last_undegraded":"2023-11-28T12:04:50.922385+0000","last_fullsized":"2023-11-28T12:04:50.922385+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:05.119970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921531+0000","last_change":"2023-11-28T12:04:50.921531+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921531+0000","last_undegraded":"2023-11-28T12:04:50.921531+0000","last_fullsized":"2023-11-28T12:04:50.921531+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:48.111122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918811+0000","last_change":"2023-11-28T12:04:50.918811+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918811+0000","last_undegraded":"2023-11-28T12:04:50.918811+0000","last_fullsized":"2023-11-28T12:04:50.918811+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:02.722002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.919427+0000","last_change":"2023-11-28T12:04:50.919427+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.919427+0000","last_undegraded":"2023-11-28T12:04:50.919427+0000","last_fullsized":"2023-11-28T12:04:50.919427+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:51:24.026885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918657+0000","last_change":"2023-11-28T12:04:50.918657+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918657+0000","last_undegraded":"2023-11-28T12:04:50.918657+0000","last_fullsized":"2023-11-28T12:04:50.918657+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:49:50.803373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918668+0000","last_change":"2023-11-28T12:04:50.918668+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918668+0000","last_undegraded":"2023-11-28T12:04:50.918668+0000","last_fullsized":"2023-11-28T12:04:50.918668+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:30.385397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.922083+0000","last_change":"2023-11-28T12:04:50.922083+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.922083+0000","last_undegraded":"2023-11-28T12:04:50.922083+0000","last_fullsized":"2023-11-28T12:04:50.922083+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:20.019844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.920655+0000","last_change":"2023-11-28T12:04:50.920655+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.920655+0000","last_undegraded":"2023-11-28T12:04:50.920655+0000","last_fullsized":"2023-11-28T12:04:50.920655+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:27.335615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.922683+0000","last_change":"2023-11-28T12:04:50.922683+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.922683+0000","last_undegraded":"2023-11-28T12:04:50.922683+0000","last_fullsized":"2023-11-28T12:04:50.922683+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:59.043060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.923929+0000","last_change":"2023-11-28T12:04:50.923929+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.923929+0000","last_undegraded":"2023-11-28T12:04:50.923929+0000","last_fullsized":"2023-11-28T12:04:50.923929+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921783+0000","last_change":"2023-11-28T12:04:50.921783+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921783+0000","last_undegraded":"2023-11-28T12:04:50.921783+0000","last_fullsized":"2023-11-28T12:04:50.921783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:25.076484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.919538+0000","last_change":"2023-11-28T12:04:50.919538+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.919538+0000","last_undegraded":"2023-11-28T12:04:50.919538+0000","last_fullsized":"2023-11-28T12:04:50.919538+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921160+0000","last_change":"2023-11-28T12:04:50.921160+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921160+0000","last_undegraded":"2023-11-28T12:04:50.921160+0000","last_fullsized":"2023-11-28T12:04:50.921160+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.920611+0000","last_change":"2023-11-28T12:04:50.920611+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.920611+0000","last_undegraded":"2023-11-28T12:04:50.920611+0000","last_fullsized":"2023-11-28T12:04:50.920611+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:25.076484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918378+0000","last_change":"2023-11-28T12:04:50.918378+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918378+0000","last_undegraded":"2023-11-28T12:04:50.918378+0000","last_fullsized":"2023-11-28T12:04:50.918378+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918495+0000","last_change":"2023-11-28T12:04:50.918495+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918495+0000","last_undegraded":"2023-11-28T12:04:50.918495+0000","last_fullsized":"2023-11-28T12:04:50.918495+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:01.574232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921872+0000","last_change":"2023-11-28T12:04:50.921872+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921872+0000","last_undegraded":"2023-11-28T12:04:50.921872+0000","last_fullsized":"2023-11-28T12:04:50.921872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:36.797091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921337+0000","last_change":"2023-11-28T12:04:50.921337+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921337+0000","last_undegraded":"2023-11-28T12:04:50.921337+0000","last_fullsized":"2023-11-28T12:04:50.921337+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921338+0000","last_change":"2023-11-28T12:04:50.921338+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921338+0000","last_undegraded":"2023-11-28T12:04:50.921338+0000","last_fullsized":"2023-11-28T12:04:50.921338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918025+0000","last_change":"2023-11-28T12:04:50.918025+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918025+0000","last_undegraded":"2023-11-28T12:04:50.918025+0000","last_fullsized":"2023-11-28T12:04:50.918025+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:15:49.848587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918006+0000","last_change":"2023-11-28T12:04:50.918006+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918006+0000","last_undegraded":"2023-11-28T12:04:50.918006+0000","last_fullsized":"2023-11-28T12:04:50.918006+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:55:26.284594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921434+0000","last_change":"2023-11-28T12:04:50.921434+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921434+0000","last_undegraded":"2023-11-28T12:04:50.921434+0000","last_fullsized":"2023-11-28T12:04:50.921434+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:50.784042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921731+0000","last_change":"2023-11-28T12:04:50.921731+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921731+0000","last_undegraded":"2023-11-28T12:04:50.921731+0000","last_fullsized":"2023-11-28T12:04:50.921731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:27.335615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.167933+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:04:48.167933+0000","last_peered":"2023-11-28T12:04:48.167933+0000","last_clean":"2023-11-28T12:04:48.167933+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:04:48.167933+0000","last_undegraded":"2023-11-28T12:04:48.167933+0000","last_fullsized":"2023-11-28T12:04:48.167933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918131+0000","last_change":"2023-11-28T12:04:50.918131+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918131+0000","last_undegraded":"2023-11-28T12:04:50.918131+0000","last_fullsized":"2023-11-28T12:04:50.918131+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:40.127328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":55,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.170391+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:04:48.170391+0000","last_peered":"2023-11-28T12:04:48.170391+0000","last_clean":"2023-11-28T12:04:48.170391+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:04:48.170391+0000","last_undegraded":"2023-11-28T12:04:48.170391+0000","last_fullsized":"2023-11-28T12:04:48.170391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.920780+0000","last_change":"2023-11-28T12:04:50.920780+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.920780+0000","last_undegraded":"2023-11-28T12:04:50.920780+0000","last_fullsized":"2023-11-28T12:04:50.920780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:05.234114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":49,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:50.454294+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:04:50.454294+0000","last_peered":"2023-11-28T12:04:50.454294+0000","last_clean":"2023-11-28T12:04:50.454294+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:04:50.454294+0000","last_undegraded":"2023-11-28T12:04:50.454294+0000","last_fullsized":"2023-11-28T12:04:50.454294+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":195,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.077002+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:04:48.077002+0000","last_peered":"2023-11-28T12:04:48.077002+0000","last_clean":"2023-11-28T12:04:48.077002+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:04:48.077002+0000","last_undegraded":"2023-11-28T12:04:48.077002+0000","last_fullsized":"2023-11-28T12:04:48.077002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918875+0000","last_change":"2023-11-28T12:04:50.918875+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918875+0000","last_undegraded":"2023-11-28T12:04:50.918875+0000","last_fullsized":"2023-11-28T12:04:50.918875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:50.784042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":46,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.076960+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:04:48.076960+0000","last_peered":"2023-11-28T12:04:48.076960+0000","last_clean":"2023-11-28T12:04:48.076960+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:04:48.076960+0000","last_undegraded":"2023-11-28T12:04:48.076960+0000","last_fullsized":"2023-11-28T12:04:48.076960+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918131+0000","last_change":"2023-11-28T12:04:50.918131+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918131+0000","last_undegraded":"2023-11-28T12:04:50.918131+0000","last_fullsized":"2023-11-28T12:04:50.918131+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:05.119970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":46,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.076960+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:04:48.076960+0000","last_peered":"2023-11-28T12:04:48.076960+0000","last_clean":"2023-11-28T12:04:48.076960+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:04:48.076960+0000","last_undegraded":"2023-11-28T12:04:48.076960+0000","last_fullsized":"2023-11-28T12:04:48.076960+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921035+0000","last_change":"2023-11-28T12:04:50.921035+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921035+0000","last_undegraded":"2023-11-28T12:04:50.921035+0000","last_fullsized":"2023-11-28T12:04:50.921035+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:07.170960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.506371+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:04:48.506371+0000","last_peered":"2023-11-28T12:04:48.506371+0000","last_clean":"2023-11-28T12:04:48.506371+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:04:48.506371+0000","last_undegraded":"2023-11-28T12:04:48.506371+0000","last_fullsized":"2023-11-28T12:04:48.506371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.921455+0000","last_change":"2023-11-28T12:04:50.921455+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.921455+0000","last_undegraded":"2023-11-28T12:04:50.921455+0000","last_fullsized":"2023-11-28T12:04:50.921455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:01.574232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.167172+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:04:48.167172+0000","last_peered":"2023-11-28T12:04:48.167172+0000","last_clean":"2023-11-28T12:04:48.167172+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:04:48.167172+0000","last_undegraded":"2023-11-28T12:04:48.167172+0000","last_fullsized":"2023-11-28T12:04:48.167172+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T12:04:51.910728+0000","last_change":"2023-11-28T12:04:51.910728+0000","last_active":"2023-11-28T12:04:51.910728+0000","last_peered":"2023-11-28T12:04:51.910728+0000","last_clean":"2023-11-28T12:04:51.910728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:51.910728+0000","last_undegraded":"2023-11-28T12:04:51.910728+0000","last_fullsized":"2023-11-28T12:04:51.910728+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T12:04:50.918965+0000","last_change":"2023-11-28T12:04:50.918965+0000","last_active":"2023-11-28T12:04:50.898186+0000","last_peered":"2023-11-28T12:04:50.898186+0000","last_clean":"2023-11-28T12:04:50.898186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T12:04:50.918965+0000","last_undegraded":"2023-11-28T12:04:50.918965+0000","last_fullsized":"2023-11-28T12:04:50.918965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:15:52.635652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T12:04:48.167176+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:04:48.167176+0000","last_peered":"2023-11-28T12:04:48.167176+0000","last_clean":"2023-11-28T12:04:48.167176+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:04:48.167176+0000","last_undegraded":"2023-11-28T12:04:48.167176+0000","last_fullsized":"2023-11-28T12:04:48.167176+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10572,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361268,"statfs":{"total":96636764160,"available":96625938432,"internally_reserved":0,"allocated":1314816,"data_stored":906529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55500000000000005,"15min":0.54800000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69699999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64300000000000002,"5min":0.872,"15min":0.872},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.76500000000000001,"15min":0.84699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73099999999999998,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.63600000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.751,"15min":0.83099999999999996},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70199999999999996,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72399999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72399999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.75600000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64900000000000002,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75700000000000001,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.623}]},{"osd":4,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.505},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.39300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64400000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.499}]},{"osd":5,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.433},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55200000000000005},"min":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73599999999999999,"5min":0.752,"15min":0.752},"last":0.39800000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.70499999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69599999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9160,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362680,"statfs":{"total":96636764160,"available":96627384320,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.44500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":1.446,"15min":1.446},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.72099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.52400000000000002,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73499999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80500000000000005,"5min":0.88,"15min":0.88},"last":0.51000000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.53000000000000003,"15min":0.497},"min":{"1min":0.46899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.76800000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.503},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80400000000000005,"5min":1.254,"15min":1.254},"last":0.52100000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.39500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.46000000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80500000000000005,"5min":1.228,"15min":1.228},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46899999999999997,"15min":0.45200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69499999999999995,"5min":1.204,"15min":1.204},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67000000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.371}]},{"osd":5,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70999999999999996,"5min":0.753,"15min":0.753},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.374,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":1.29,"15min":1.29},"last":0.55300000000000005}]},{"osd":7,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.748,"5min":1.159,"15min":1.159},"last":0.44},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.35499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81799999999999995,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.34100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10380,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361460,"statfs":{"total":96636764160,"available":96626135040,"internally_reserved":0,"allocated":1314816,"data_stored":906529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39500000000000002,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.49199999999999999,"5min":1.2,"15min":1.2},"last":0.311},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.27300000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60799999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.39100000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.37,"15min":0.37},"min":{"1min":0.22900000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.52100000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57499999999999996,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.26400000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47499999999999998,"15min":0.48599999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63900000000000001,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.44500000000000001,"15min":0.45400000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54800000000000004,"5min":1.03,"15min":1.03},"last":0.51000000000000001}]},{"osd":4,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.375,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81000000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81599999999999995,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.48399999999999999}]},{"osd":5,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.627,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97699999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.621},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.503,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.91400000000000003,"5min":0.998,"15min":0.998},"last":0.53100000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65800000000000003,"15min":0.67300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89300000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.871,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.498}]},{"osd":7,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64700000000000002,"15min":0.66000000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94999999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91800000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9416,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362424,"statfs":{"total":96636764160,"available":96627122176,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.50600000000000001,"15min":0.56399999999999995},"min":{"1min":0.26400000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.42999999999999999,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.308},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.312,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51600000000000001,"5min":1.667,"15min":1.667},"last":0.28899999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.53100000000000003,"15min":0.58199999999999996},"min":{"1min":0.31900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.47699999999999998,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42999999999999999,"15min":0.437},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53500000000000003,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.432,"15min":0.44400000000000001},"min":{"1min":0.29999999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.49299999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.59499999999999997,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.52400000000000002,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79800000000000004,"5min":1.536,"15min":1.536},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.72199999999999998,"15min":0.76600000000000001},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76000000000000001,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.63400000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.46500000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79300000000000004,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.624,"15min":0.62},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.78600000000000003,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.52900000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74299999999999999,"15min":0.78100000000000003},"min":{"1min":0.441,"5min":0.44,"15min":0.44},"max":{"1min":0.81899999999999995,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63200000000000001,"15min":0.62},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80800000000000005,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.51700000000000002}]},{"osd":7,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.76000000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.755,"5min":1.605,"15min":1.605},"last":0.50800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9288,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362552,"statfs":{"total":96636764160,"available":96627253248,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.375},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58999999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40100000000000002},"min":{"1min":0.318,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.40500000000000003}]},{"osd":1,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42099999999999999,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.60899999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.54800000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.32400000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41299999999999998,"15min":0.40999999999999998},"min":{"1min":0.31,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.621,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.377},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.59499999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.432,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.627,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90200000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.51300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.92100000000000004}]},{"osd":7,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13768,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":13056,"kb_avail":94358072,"statfs":{"total":96636764160,"available":96622665728,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13369344},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.29599999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42299999999999999,"15min":0.441},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.495,"5min":1.238,"15min":1.238},"last":0.373}]},{"osd":1,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42799999999999999,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56799999999999995,"5min":1.53,"15min":1.53},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.441,"5min":0.48599999999999999,"15min":0.51000000000000001},"min":{"1min":0.27100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.622,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47599999999999998,"15min":0.49199999999999999},"min":{"1min":0.34000000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60299999999999998,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.48499999999999999},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60199999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79000000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.51600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91900000000000004,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67900000000000005,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.99299999999999999,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.497},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.51900000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.879,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48399999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92300000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84699999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.94299999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67400000000000004,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.54400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9416,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362424,"statfs":{"total":96636764160,"available":96627122176,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53700000000000003,"15min":0.52200000000000002},"min":{"1min":0.44800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.72099999999999997}]},{"osd":1,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52900000000000003}]},{"osd":2,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76000000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.69999999999999996}]},{"osd":3,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68999999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.503,"15min":0.497},"min":{"1min":0.35199999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.36399999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.498,"15min":0.5},"min":{"1min":0.39100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46600000000000003}]},{"osd":7,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574891,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9352,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362488,"statfs":{"total":96636764160,"available":96627187712,"internally_reserved":0,"allocated":720896,"data_stored":316161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71199999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.62},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.443,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.63600000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.80500000000000005,"15min":0.91900000000000004},"min":{"1min":0.45200000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74299999999999999,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.56999999999999995}]},{"osd":2,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.64800000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.432,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77800000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66800000000000004}]},{"osd":3,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.64600000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.81699999999999995}]},{"osd":4,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46700000000000003,"15min":0.47299999999999998},"min":{"1min":0.36099999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66400000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51400000000000001,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66400000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.50600000000000001,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.65300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.375,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63300000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.72099999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53600000000000003,"15min":0.54600000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68899999999999995,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.83899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:04:57.385 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2023-11-28T12:04:57.385 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.385 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2023-11-28T12:04:57.385 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.385 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2023-11-28T12:04:57.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.386 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2023-11-28T12:04:57.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.386 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2023-11-28T12:04:57.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.386 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2023-11-28T12:04:57.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.386 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2023-11-28T12:04:57.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2023-11-28T12:04:57.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2023-11-28T12:04:57.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2023-11-28T12:04:57.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2023-11-28T12:04:57.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2023-11-28T12:04:57.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2023-11-28T12:04:57.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2023-11-28T12:04:57.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2023-11-28T12:04:57.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2023-11-28T12:04:57.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2023-11-28T12:04:57.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2023-11-28T12:04:57.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2023-11-28T12:04:57.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2023-11-28T12:04:57.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2023-11-28T12:04:57.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2023-11-28T12:04:57.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2023-11-28T12:04:57.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2023-11-28T12:04:57.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2023-11-28T12:04:57.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2023-11-28T12:04:57.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2023-11-28T12:04:57.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2023-11-28T12:04:57.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2023-11-28T12:04:57.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2023-11-28T12:04:57.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2023-11-28T12:04:57.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2023-11-28T12:04:57.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2023-11-28T12:04:57.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-11-28T12:04:57.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.921344+0000', 'last_change': '2023-11-28T12:04:50.921344+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.921344+0000', 'last_undegraded': '2023-11-28T12:04:50.921344+0000', 'last_fullsized': '2023-11-28T12:04:50.921344+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:15:52.635652+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T12:04:57.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-11-28T12:04:57.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-11-28T12:04:57.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.920764+0000', 'last_change': '2023-11-28T12:04:50.920764+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.920764+0000', 'last_undegraded': '2023-11-28T12:04:50.920764+0000', 'last_fullsized': '2023-11-28T12:04:50.920764+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T22:34:00.055068+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T12:04:57.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2023-11-28T12:04:57.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-11-28T12:04:57.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.922385+0000', 'last_change': '2023-11-28T12:04:50.922385+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.922385+0000', 'last_undegraded': '2023-11-28T12:04:50.922385+0000', 'last_fullsized': '2023-11-28T12:04:50.922385+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:46:05.119970+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T12:04:57.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-11-28T12:04:57.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-11-28T12:04:57.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.921531+0000', 'last_change': '2023-11-28T12:04:50.921531+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.921531+0000', 'last_undegraded': '2023-11-28T12:04:50.921531+0000', 'last_fullsized': '2023-11-28T12:04:50.921531+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:04:48.111122+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T12:04:57.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2023-11-28T12:04:57.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2023-11-28T12:04:57.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.918811+0000', 'last_change': '2023-11-28T12:04:50.918811+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.918811+0000', 'last_undegraded': '2023-11-28T12:04:50.918811+0000', 'last_fullsized': '2023-11-28T12:04:50.918811+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:48:02.722002+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T12:04:57.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-11-28T12:04:57.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2023-11-28T12:04:57.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.919427+0000', 'last_change': '2023-11-28T12:04:50.919427+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.919427+0000', 'last_undegraded': '2023-11-28T12:04:50.919427+0000', 'last_fullsized': '2023-11-28T12:04:50.919427+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:51:24.026885+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T12:04:57.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2023-11-28T12:04:57.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2023-11-28T12:04:57.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.918657+0000', 'last_change': '2023-11-28T12:04:50.918657+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.918657+0000', 'last_undegraded': '2023-11-28T12:04:50.918657+0000', 'last_fullsized': '2023-11-28T12:04:50.918657+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:49:50.803373+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T12:04:57.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2023-11-28T12:04:57.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-11-28T12:04:57.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.918668+0000', 'last_change': '2023-11-28T12:04:50.918668+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.918668+0000', 'last_undegraded': '2023-11-28T12:04:50.918668+0000', 'last_fullsized': '2023-11-28T12:04:50.918668+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:06:30.385397+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T12:04:57.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-11-28T12:04:57.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-11-28T12:04:57.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.922083+0000', 'last_change': '2023-11-28T12:04:50.922083+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.922083+0000', 'last_undegraded': '2023-11-28T12:04:50.922083+0000', 'last_fullsized': '2023-11-28T12:04:50.922083+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:03:20.019844+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T12:04:57.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2023-11-28T12:04:57.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-11-28T12:04:57.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.920655+0000', 'last_change': '2023-11-28T12:04:50.920655+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.920655+0000', 'last_undegraded': '2023-11-28T12:04:50.920655+0000', 'last_fullsized': '2023-11-28T12:04:50.920655+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T23:57:27.335615+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T12:04:57.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2023-11-28T12:04:57.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-11-28T12:04:57.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.922683+0000', 'last_change': '2023-11-28T12:04:50.922683+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.922683+0000', 'last_undegraded': '2023-11-28T12:04:50.922683+0000', 'last_fullsized': '2023-11-28T12:04:50.922683+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:12:59.043060+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T12:04:57.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-11-28T12:04:57.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-11-28T12:04:57.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.923929+0000', 'last_change': '2023-11-28T12:04:50.923929+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.923929+0000', 'last_undegraded': '2023-11-28T12:04:50.923929+0000', 'last_fullsized': '2023-11-28T12:04:50.923929+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T12:04:57.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-11-28T12:04:57.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-11-28T12:04:57.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.921783+0000', 'last_change': '2023-11-28T12:04:50.921783+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.921783+0000', 'last_undegraded': '2023-11-28T12:04:50.921783+0000', 'last_fullsized': '2023-11-28T12:04:50.921783+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T23:40:25.076484+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T12:04:57.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-11-28T12:04:57.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-11-28T12:04:57.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.919538+0000', 'last_change': '2023-11-28T12:04:50.919538+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.919538+0000', 'last_undegraded': '2023-11-28T12:04:50.919538+0000', 'last_fullsized': '2023-11-28T12:04:50.919538+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T12:04:57.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2023-11-28T12:04:57.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-11-28T12:04:57.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.921160+0000', 'last_change': '2023-11-28T12:04:50.921160+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.921160+0000', 'last_undegraded': '2023-11-28T12:04:50.921160+0000', 'last_fullsized': '2023-11-28T12:04:50.921160+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T12:04:57.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-11-28T12:04:57.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-11-28T12:04:57.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.920611+0000', 'last_change': '2023-11-28T12:04:50.920611+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.920611+0000', 'last_undegraded': '2023-11-28T12:04:50.920611+0000', 'last_fullsized': '2023-11-28T12:04:50.920611+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T23:40:25.076484+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T12:04:57.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2023-11-28T12:04:57.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-11-28T12:04:57.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.918378+0000', 'last_change': '2023-11-28T12:04:50.918378+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.918378+0000', 'last_undegraded': '2023-11-28T12:04:50.918378+0000', 'last_fullsized': '2023-11-28T12:04:50.918378+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T12:04:57.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2023-11-28T12:04:57.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-11-28T12:04:57.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.918495+0000', 'last_change': '2023-11-28T12:04:50.918495+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.918495+0000', 'last_undegraded': '2023-11-28T12:04:50.918495+0000', 'last_fullsized': '2023-11-28T12:04:50.918495+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:39:01.574232+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T12:04:57.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-11-28T12:04:57.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-11-28T12:04:57.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.921872+0000', 'last_change': '2023-11-28T12:04:50.921872+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.921872+0000', 'last_undegraded': '2023-11-28T12:04:50.921872+0000', 'last_fullsized': '2023-11-28T12:04:50.921872+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T22:12:36.797091+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T12:04:57.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2023-11-28T12:04:57.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-11-28T12:04:57.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.921337+0000', 'last_change': '2023-11-28T12:04:50.921337+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.921337+0000', 'last_undegraded': '2023-11-28T12:04:50.921337+0000', 'last_fullsized': '2023-11-28T12:04:50.921337+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T12:04:57.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2023-11-28T12:04:57.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-11-28T12:04:57.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.921338+0000', 'last_change': '2023-11-28T12:04:50.921338+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.921338+0000', 'last_undegraded': '2023-11-28T12:04:50.921338+0000', 'last_fullsized': '2023-11-28T12:04:50.921338+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T12:04:57.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-11-28T12:04:57.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2023-11-28T12:04:57.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.918025+0000', 'last_change': '2023-11-28T12:04:50.918025+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.918025+0000', 'last_undegraded': '2023-11-28T12:04:50.918025+0000', 'last_fullsized': '2023-11-28T12:04:50.918025+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:15:49.848587+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T12:04:57.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-11-28T12:04:57.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2023-11-28T12:04:57.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.918006+0000', 'last_change': '2023-11-28T12:04:50.918006+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.918006+0000', 'last_undegraded': '2023-11-28T12:04:50.918006+0000', 'last_fullsized': '2023-11-28T12:04:50.918006+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:55:26.284594+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T12:04:57.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2023-11-28T12:04:57.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-11-28T12:04:57.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.921434+0000', 'last_change': '2023-11-28T12:04:50.921434+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.921434+0000', 'last_undegraded': '2023-11-28T12:04:50.921434+0000', 'last_fullsized': '2023-11-28T12:04:50.921434+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:03:50.784042+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T12:04:57.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-11-28T12:04:57.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-11-28T12:04:57.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.921731+0000', 'last_change': '2023-11-28T12:04:50.921731+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.921731+0000', 'last_undegraded': '2023-11-28T12:04:50.921731+0000', 'last_fullsized': '2023-11-28T12:04:50.921731+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T23:57:27.335615+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T12:04:57.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2023-11-28T12:04:57.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2023-11-28T12:04:57.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.918131+0000', 'last_change': '2023-11-28T12:04:50.918131+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.918131+0000', 'last_undegraded': '2023-11-28T12:04:50.918131+0000', 'last_fullsized': '2023-11-28T12:04:50.918131+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:15:40.127328+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T12:04:57.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2023-11-28T12:04:57.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-11-28T12:04:57.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.920780+0000', 'last_change': '2023-11-28T12:04:50.920780+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.920780+0000', 'last_undegraded': '2023-11-28T12:04:50.920780+0000', 'last_fullsized': '2023-11-28T12:04:50.920780+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:47:05.234114+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T12:04:57.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-11-28T12:04:57.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-11-28T12:04:57.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.918875+0000', 'last_change': '2023-11-28T12:04:50.918875+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.918875+0000', 'last_undegraded': '2023-11-28T12:04:50.918875+0000', 'last_fullsized': '2023-11-28T12:04:50.918875+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:03:50.784042+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T12:04:57.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2023-11-28T12:04:57.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2023-11-28T12:04:57.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.918131+0000', 'last_change': '2023-11-28T12:04:50.918131+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.918131+0000', 'last_undegraded': '2023-11-28T12:04:50.918131+0000', 'last_fullsized': '2023-11-28T12:04:50.918131+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:46:05.119970+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T12:04:57.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2023-11-28T12:04:57.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-11-28T12:04:57.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.921035+0000', 'last_change': '2023-11-28T12:04:50.921035+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.921035+0000', 'last_undegraded': '2023-11-28T12:04:50.921035+0000', 'last_fullsized': '2023-11-28T12:04:50.921035+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:10:07.170960+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T12:04:57.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2023-11-28T12:04:57.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-11-28T12:04:57.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.921455+0000', 'last_change': '2023-11-28T12:04:50.921455+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.921455+0000', 'last_undegraded': '2023-11-28T12:04:50.921455+0000', 'last_fullsized': '2023-11-28T12:04:50.921455+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:39:01.574232+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T12:04:57.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-11-28T12:04:57.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T12:04:51.910728+0000', 'last_change': '2023-11-28T12:04:51.910728+0000', 'last_active': '2023-11-28T12:04:51.910728+0000', 'last_peered': '2023-11-28T12:04:51.910728+0000', 'last_clean': '2023-11-28T12:04:51.910728+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:51.910728+0000', 'last_undegraded': '2023-11-28T12:04:51.910728+0000', 'last_fullsized': '2023-11-28T12:04:51.910728+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:51.910728+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T12:04:57.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-11-28T12:04:57.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T12:04:50.918965+0000', 'last_change': '2023-11-28T12:04:50.918965+0000', 'last_active': '2023-11-28T12:04:50.898186+0000', 'last_peered': '2023-11-28T12:04:50.898186+0000', 'last_clean': '2023-11-28T12:04:50.898186+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T12:04:50.918965+0000', 'last_undegraded': '2023-11-28T12:04:50.918965+0000', 'last_fullsized': '2023-11-28T12:04:50.918965+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'last_clean_scrub_stamp': '2023-11-28T12:04:50.898186+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:15:52.635652+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T12:04:57.410 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T12:04:57.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.555+0000 7f0ccd34b700 1 Processor -- start 2023-11-28T12:04:57.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.555+0000 7f0ccd34b700 1 -- start start 2023-11-28T12:04:57.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.556+0000 7f0ccd34b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0cc80d0c60 0x7f0cc80b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:57.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.556+0000 7f0ccd34b700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc806e900 con 0x7f0cc806e400 2023-11-28T12:04:57.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.556+0000 7f0ccd34b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc8160390 con 0x7f0cc8160e70 2023-11-28T12:04:57.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.556+0000 7f0ccd34b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc8160670 con 0x7f0cc80d0c60 2023-11-28T12:04:57.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.557+0000 7f0cc67fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f0cc806e400 0x7f0cc80d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55586/0 (socket says 172.21.15.57:55586) 2023-11-28T12:04:57.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.557+0000 7f0cc67fc700 1 -- 172.21.15.57:0/2933527918 learned_addr learned my addr 172.21.15.57:0/2933527918 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:04:57.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.557+0000 7f0cc6ffd700 1 --2- 172.21.15.57:0/2933527918 >> v2:172.21.15.181:3300/0 conn(0x7f0cc80d0c60 0x7f0cc80b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:57.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.557+0000 7f0cc5ffb700 1 -- 172.21.15.57:0/2933527918 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31038053 0 0) 0x7f0cc806e900 con 0x7f0cc806e400 2023-11-28T12:04:57.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.557+0000 7f0cc5ffb700 1 -- 172.21.15.57:0/2933527918 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cb40035b0 con 0x7f0cc806e400 2023-11-28T12:04:57.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.557+0000 7f0cc5ffb700 1 -- 172.21.15.57:0/2933527918 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1252290902 0 0) 0x7f0cb40035b0 con 0x7f0cc806e400 2023-11-28T12:04:57.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.557+0000 7f0cc6ffd700 1 -- 172.21.15.57:0/2933527918 >> v1:172.21.15.181:6789/0 conn(0x7f0cc8160e70 legacy=0x7f0cc815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.557+0000 7f0cc6ffd700 1 -- 172.21.15.57:0/2933527918 >> v1:172.21.15.57:6789/0 conn(0x7f0cc806e400 legacy=0x7f0cc80d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.557+0000 7f0cc6ffd700 1 -- 172.21.15.57:0/2933527918 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc8160950 con 0x7f0cc80d0c60 2023-11-28T12:04:57.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.557+0000 7f0cc6ffd700 1 --2- 172.21.15.57:0/2933527918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cc80d0c60 0x7f0cc80b5be0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009840 tx=0x7f0cb0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63dd5496a04525af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:57.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.558+0000 7f0cc5ffb700 1 -- 172.21.15.57:0/2933527918 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8001670 con 0x7f0cc80d0c60 2023-11-28T12:04:57.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.558+0000 7f0cc5ffb700 1 -- 172.21.15.57:0/2933527918 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb0004030 con 0x7f0cc80d0c60 2023-11-28T12:04:57.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.558+0000 7f0cc5ffb700 1 -- 172.21.15.57:0/2933527918 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb00043a0 con 0x7f0cc80d0c60 2023-11-28T12:04:57.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.558+0000 7f0ccd34b700 1 -- 172.21.15.57:0/2933527918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cc80d0c60 msgr2=0x7f0cc80b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.558+0000 7f0ccd34b700 1 --2- 172.21.15.57:0/2933527918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cc80d0c60 0x7f0cc80b5be0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009840 tx=0x7f0cb0009b50 comp rx=0 tx=0).stop 2023-11-28T12:04:57.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.558+0000 7f0ccd34b700 1 -- 172.21.15.57:0/2933527918 shutdown_connections 2023-11-28T12:04:57.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.558+0000 7f0ccd34b700 1 --2- 172.21.15.57:0/2933527918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cc80d0c60 0x7f0cc80b5be0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.558+0000 7f0ccd34b700 1 -- 172.21.15.57:0/2933527918 >> 172.21.15.57:0/2933527918 conn(0x7f0cc806a270 msgr2=0x7f0cc806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:57.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.558+0000 7f0ccd34b700 1 -- 172.21.15.57:0/2933527918 shutdown_connections 2023-11-28T12:04:57.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.558+0000 7f0ccd34b700 1 -- 172.21.15.57:0/2933527918 wait complete. 2023-11-28T12:04:57.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.560+0000 7f0ccd34b700 1 Processor -- start 2023-11-28T12:04:57.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.560+0000 7f0ccd34b700 1 -- start start 2023-11-28T12:04:57.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0ccd34b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cc806e400 0x7f0cc813d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:57.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0ccd34b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cc80d0c60 0x7f0cc813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:57.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0ccd34b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0cc8160e70 0x7f0cc813d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:57.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0ccd34b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc8152540 con 0x7f0cc8160e70 2023-11-28T12:04:57.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0ccd34b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc8153930 con 0x7f0cc80d0c60 2023-11-28T12:04:57.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0ccd34b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc81409a0 con 0x7f0cc806e400 2023-11-28T12:04:57.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0cc77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0cc8160e70 0x7f0cc813d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:57.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0cc77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0cc8160e70 0x7f0cc813d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46394/0 (socket says 172.21.15.57:46394) 2023-11-28T12:04:57.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0cc77fe700 1 -- 172.21.15.57:0/1898145497 learned_addr learned my addr 172.21.15.57:0/1898145497 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:57.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0cc6ffd700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cc806e400 0x7f0cc813d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:57.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0cc67fc700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cc80d0c60 0x7f0cc813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:57.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0cc77fe700 1 -- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cc806e400 msgr2=0x7f0cc813d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0cc77fe700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cc806e400 0x7f0cc813d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0cc77fe700 1 -- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cc80d0c60 msgr2=0x7f0cc813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0cc77fe700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cc80d0c60 0x7f0cc813fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0cc77fe700 1 -- 172.21.15.57:0/1898145497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb00094a0 con 0x7f0cc8160e70 2023-11-28T12:04:57.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0cc77fe700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0cc8160e70 0x7f0cc813d9e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c7f0 tx=0x7f0cc000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:57.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.561+0000 7f0cbf7fe700 1 -- 172.21.15.57:0/1898145497 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0003e30 con 0x7f0cc8160e70 2023-11-28T12:04:57.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.562+0000 7f0cbf7fe700 1 -- 172.21.15.57:0/1898145497 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc0005070 con 0x7f0cc8160e70 2023-11-28T12:04:57.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.562+0000 7f0ccd34b700 1 -- 172.21.15.57:0/1898145497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc815d080 con 0x7f0cc8160e70 2023-11-28T12:04:57.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.562+0000 7f0cbf7fe700 1 -- 172.21.15.57:0/1898145497 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0005420 con 0x7f0cc8160e70 2023-11-28T12:04:57.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.562+0000 7f0ccd34b700 1 -- 172.21.15.57:0/1898145497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc80d0740 con 0x7f0cc8160e70 2023-11-28T12:04:57.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.563+0000 7f0ccd34b700 1 -- 172.21.15.57:0/1898145497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc815d210 con 0x7f0cc8160e70 2023-11-28T12:04:57.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.563+0000 7f0cbf7fe700 1 -- 172.21.15.57:0/1898145497 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0cc0010070 con 0x7f0cc8160e70 2023-11-28T12:04:57.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.564+0000 7f0cbf7fe700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0ca80682f0 0x7f0ca806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:57.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.564+0000 7f0cbf7fe700 1 -- 172.21.15.57:0/1898145497 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0cc0085700 con 0x7f0cc8160e70 2023-11-28T12:04:57.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.564+0000 7f0cc6ffd700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0ca80682f0 0x7f0ca806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:57.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.564+0000 7f0cc6ffd700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0ca80682f0 0x7f0ca806a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0000c00 tx=0x7f0cb0008000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:57.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.568+0000 7f0cbf7fe700 1 -- 172.21.15.57:0/1898145497 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0cc0051f60 con 0x7f0cc8160e70 2023-11-28T12:04:57.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.854+0000 7f0ccd34b700 1 -- 172.21.15.57:0/1898145497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0cc80743f0 con 0x7f0cc8160e70 2023-11-28T12:04:57.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.854+0000 7f0cbf7fe700 1 -- 172.21.15.57:0/1898145497 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1269 (secure 0 0 0) 0x7f0cc0055580 con 0x7f0cc8160e70 2023-11-28T12:04:57.855 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:57.856 INFO:teuthology.orchestra.run.smithi057.stdout:{"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":212,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":63,"num_osds":8,"num_up_osds":8,"osd_up_since":1701172896,"num_in_osds":8,"osd_in_since":1701172888,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":83304448,"bytes_avail":773010808832,"bytes_total":773094113280,"unknown_pgs_ratio":0.60952383279800415,"inactive_pgs_ratio":0.30476191639900208},"fsmap":{"epoch":32,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":6219}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-11-28T12:04:43.705272+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-28T12:04:57.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.856+0000 7f0ccd34b700 1 -- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0ca80682f0 msgr2=0x7f0ca806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.856+0000 7f0ccd34b700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0ca80682f0 0x7f0ca806a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0000c00 tx=0x7f0cb0008000 comp rx=0 tx=0).stop 2023-11-28T12:04:57.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.856+0000 7f0ccd34b700 1 -- 172.21.15.57:0/1898145497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0cc8160e70 msgr2=0x7f0cc813d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:57.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.856+0000 7f0ccd34b700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0cc8160e70 0x7f0cc813d9e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c7f0 tx=0x7f0cc000cb00 comp rx=0 tx=0).stop 2023-11-28T12:04:57.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.856+0000 7f0ccd34b700 1 -- 172.21.15.57:0/1898145497 shutdown_connections 2023-11-28T12:04:57.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.856+0000 7f0ccd34b700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0ca80682f0 0x7f0ca806a7a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.856+0000 7f0ccd34b700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0cc8160e70 0x7f0cc813d9e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.856+0000 7f0ccd34b700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cc806e400 0x7f0cc813d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.856+0000 7f0ccd34b700 1 --2- 172.21.15.57:0/1898145497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cc80d0c60 0x7f0cc813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:57.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.856+0000 7f0ccd34b700 1 -- 172.21.15.57:0/1898145497 >> 172.21.15.57:0/1898145497 conn(0x7f0cc806a270 msgr2=0x7f0cc815e970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:57.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.856+0000 7f0ccd34b700 1 -- 172.21.15.57:0/1898145497 shutdown_connections 2023-11-28T12:04:57.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:57.857+0000 7f0ccd34b700 1 -- 172.21.15.57:0/1898145497 wait complete. 2023-11-28T12:04:57.871 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T12:04:57.871 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:04:57.892 INFO:tasks.ceph.mds.a.smithi057.stderr:2023-11-28T12:04:57.891+0000 7f92e9fc6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T12:04:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.006+0000 7f007deef700 1 Processor -- start 2023-11-28T12:04:58.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.006+0000 7f007deef700 1 -- start start 2023-11-28T12:04:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f007deef700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f007806b7b0 0x7f0078072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f007deef700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00780ad1f0 0x7f00780ad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f007deef700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00780726c0 con 0x7f00780b0880 2023-11-28T12:04:58.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f007deef700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007806bc50 con 0x7f007806b7b0 2023-11-28T12:04:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f007deef700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00780b43f0 con 0x7f00780ad1f0 2023-11-28T12:04:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f0076ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f007806b7b0 0x7f0078072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f0076ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f007806b7b0 0x7f0078072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46396/0 (socket says 172.21.15.57:46396) 2023-11-28T12:04:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f0076ffd700 1 -- 172.21.15.57:0/442174080 learned_addr learned my addr 172.21.15.57:0/442174080 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f0077fff700 1 --2- 172.21.15.57:0/442174080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00780ad1f0 0x7f00780ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f0076ffd700 1 -- 172.21.15.57:0/442174080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00780ad1f0 msgr2=0x7f00780ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:04:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f0076ffd700 1 --2- 172.21.15.57:0/442174080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00780ad1f0 0x7f00780ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f0076ffd700 1 -- 172.21.15.57:0/442174080 >> v1:172.21.15.181:6789/0 conn(0x7f00780b0880 legacy=0x7f00780aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f0076ffd700 1 -- 172.21.15.57:0/442174080 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00780adb90 con 0x7f007806b7b0 2023-11-28T12:04:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f0076ffd700 1 --2- 172.21.15.57:0/442174080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f007806b7b0 0x7f0078072180 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f006c0098f0 tx=0x7f006c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e505d46631802a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:58.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.007+0000 7f00767fc700 1 -- 172.21.15.57:0/442174080 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c00b8a0 con 0x7f007806b7b0 2023-11-28T12:04:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.008+0000 7f00767fc700 1 -- 172.21.15.57:0/442174080 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c00ba00 con 0x7f007806b7b0 2023-11-28T12:04:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.008+0000 7f00767fc700 1 -- 172.21.15.57:0/442174080 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c00bd90 con 0x7f007806b7b0 2023-11-28T12:04:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.008+0000 7f007deef700 1 -- 172.21.15.57:0/442174080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f007806b7b0 msgr2=0x7f0078072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.008+0000 7f007deef700 1 --2- 172.21.15.57:0/442174080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f007806b7b0 0x7f0078072180 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f006c0098f0 tx=0x7f006c009c00 comp rx=0 tx=0).stop 2023-11-28T12:04:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.008+0000 7f007deef700 1 -- 172.21.15.57:0/442174080 shutdown_connections 2023-11-28T12:04:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.008+0000 7f007deef700 1 --2- 172.21.15.57:0/442174080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00780ad1f0 0x7f00780ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.008+0000 7f007deef700 1 --2- 172.21.15.57:0/442174080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f007806b7b0 0x7f0078072180 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.008+0000 7f007deef700 1 -- 172.21.15.57:0/442174080 >> 172.21.15.57:0/442174080 conn(0x7f00780b2fc0 msgr2=0x7f0078071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.008+0000 7f007deef700 1 -- 172.21.15.57:0/442174080 shutdown_connections 2023-11-28T12:04:58.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.008+0000 7f007deef700 1 -- 172.21.15.57:0/442174080 wait complete. 2023-11-28T12:04:58.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.009+0000 7f007deef700 1 Processor -- start 2023-11-28T12:04:58.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.010+0000 7f007deef700 1 -- start start 2023-11-28T12:04:58.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.010+0000 7f007deef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f007806b7b0 0x7f007815f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:58.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f007deef700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00780ad1f0 0x7f00781f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:58.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f007deef700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00780b0880 0x7f007809aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:58.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f007deef700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007815fe00 con 0x7f00780b0880 2023-11-28T12:04:58.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f007deef700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007809cfe0 con 0x7f007806b7b0 2023-11-28T12:04:58.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f007deef700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007809d2c0 con 0x7f00780ad1f0 2023-11-28T12:04:58.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f0077fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00780b0880 0x7f007809aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:58.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f0077fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00780b0880 0x7f007809aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46402/0 (socket says 172.21.15.57:46402) 2023-11-28T12:04:58.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f0077fff700 1 -- 172.21.15.57:0/3713418284 learned_addr learned my addr 172.21.15.57:0/3713418284 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:04:58.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f0076ffd700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00780ad1f0 0x7f00781f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:58.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f00777fe700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f007806b7b0 0x7f007815f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f0077fff700 1 -- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00780ad1f0 msgr2=0x7f00781f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:58.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f0077fff700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00780ad1f0 0x7f00781f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f0077fff700 1 -- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f007806b7b0 msgr2=0x7f007815f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f0077fff700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f007806b7b0 0x7f007815f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f0077fff700 1 -- 172.21.15.57:0/3713418284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c009550 con 0x7f00780b0880 2023-11-28T12:04:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f00777fe700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f007806b7b0 0x7f007815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:04:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f0077fff700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00780b0880 0x7f007809aa60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0070007e40 tx=0x7f007000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f005ffff700 1 -- 172.21.15.57:0/3713418284 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007000eea0 con 0x7f00780b0880 2023-11-28T12:04:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f005ffff700 1 -- 172.21.15.57:0/3713418284 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0070014420 con 0x7f00780b0880 2023-11-28T12:04:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f007deef700 1 -- 172.21.15.57:0/3713418284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00780b4d00 con 0x7f00780b0880 2023-11-28T12:04:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f005ffff700 1 -- 172.21.15.57:0/3713418284 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00700147b0 con 0x7f00780b0880 2023-11-28T12:04:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.011+0000 7f007deef700 1 -- 172.21.15.57:0/3713418284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00780b46d0 con 0x7f00780b0880 2023-11-28T12:04:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.012+0000 7f007deef700 1 -- 172.21.15.57:0/3713418284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00780b4fe0 con 0x7f00780b0880 2023-11-28T12:04:58.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.013+0000 7f005ffff700 1 -- 172.21.15.57:0/3713418284 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0070012050 con 0x7f00780b0880 2023-11-28T12:04:58.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.013+0000 7f005ffff700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f00640682f0 0x7f006406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:04:58.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.013+0000 7f005ffff700 1 -- 172.21.15.57:0/3713418284 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0070086360 con 0x7f00780b0880 2023-11-28T12:04:58.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.014+0000 7f00777fe700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f00640682f0 0x7f006406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:04:58.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.014+0000 7f00777fe700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f00640682f0 0x7f006406a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0068000f60 tx=0x7f0068009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:04:58.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.018+0000 7f005ffff700 1 -- 172.21.15.57:0/3713418284 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0070052bc0 con 0x7f00780b0880 2023-11-28T12:04:58.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.217+0000 7f007deef700 1 -- 172.21.15.57:0/3713418284 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f007809d150 con 0x7f00640682f0 2023-11-28T12:04:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.227+0000 7f005ffff700 1 -- 172.21.15.57:0/3713418284 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311988 (secure 0 0 0) 0x7f007809d150 con 0x7f00640682f0 2023-11-28T12:04:58.232 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:04:58.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.230+0000 7f007deef700 1 -- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f00640682f0 msgr2=0x7f006406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:58.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.230+0000 7f007deef700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f00640682f0 0x7f006406a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0068000f60 tx=0x7f0068009040 comp rx=0 tx=0).stop 2023-11-28T12:04:58.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.230+0000 7f007deef700 1 -- 172.21.15.57:0/3713418284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00780b0880 msgr2=0x7f007809aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:04:58.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.230+0000 7f007deef700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00780b0880 0x7f007809aa60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0070007e40 tx=0x7f007000e3c0 comp rx=0 tx=0).stop 2023-11-28T12:04:58.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.230+0000 7f007deef700 1 -- 172.21.15.57:0/3713418284 shutdown_connections 2023-11-28T12:04:58.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.230+0000 7f007deef700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f00640682f0 0x7f006406a7a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:58.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.230+0000 7f007deef700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00780b0880 0x7f007809aa60 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:58.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.230+0000 7f007deef700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f007806b7b0 0x7f007815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:58.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.230+0000 7f007deef700 1 --2- 172.21.15.57:0/3713418284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00780ad1f0 0x7f00781f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:04:58.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.230+0000 7f007deef700 1 -- 172.21.15.57:0/3713418284 >> 172.21.15.57:0/3713418284 conn(0x7f00780b2fc0 msgr2=0x7f007806cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:04:58.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.230+0000 7f007deef700 1 -- 172.21.15.57:0/3713418284 shutdown_connections 2023-11-28T12:04:58.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:04:58.230+0000 7f007deef700 1 -- 172.21.15.57:0/3713418284 wait complete. 2023-11-28T12:04:58.238 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:04:58.245 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2023-11-28T12:04:57.708382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":87016,"kb_used_data":7144,"kb_used_omap":0,"kb_used_meta":79808,"kb_avail":754887704,"statfs":{"total":773094113280,"available":773005008896,"internally_reserved":0,"allocated":7315456,"data_stored":3877892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81723392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001231"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.747247+0000","last_change":"2023-11-28T12:04:52.391013+0000","last_active":"2023-11-28T12:04:54.747247+0000","last_peered":"2023-11-28T12:04:54.747247+0000","last_clean":"2023-11-28T12:04:54.747247+0000","last_became_active":"2023-11-28T12:04:52.386457+0000","last_became_peered":"2023-11-28T12:04:52.386457+0000","last_unstale":"2023-11-28T12:04:54.747247+0000","last_undegraded":"2023-11-28T12:04:54.747247+0000","last_fullsized":"2023-11-28T12:04:54.747247+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:14:26.740655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.747121+0000","last_change":"2023-11-28T12:04:52.389120+0000","last_active":"2023-11-28T12:04:54.747121+0000","last_peered":"2023-11-28T12:04:54.747121+0000","last_clean":"2023-11-28T12:04:54.747121+0000","last_became_active":"2023-11-28T12:04:52.388206+0000","last_became_peered":"2023-11-28T12:04:52.388206+0000","last_unstale":"2023-11-28T12:04:54.747121+0000","last_undegraded":"2023-11-28T12:04:54.747121+0000","last_fullsized":"2023-11-28T12:04:54.747121+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:59:12.567356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272642+0000","last_change":"2023-11-28T12:04:52.384787+0000","last_active":"2023-11-28T12:04:55.272642+0000","last_peered":"2023-11-28T12:04:55.272642+0000","last_clean":"2023-11-28T12:04:55.272642+0000","last_became_active":"2023-11-28T12:04:52.384361+0000","last_became_peered":"2023-11-28T12:04:52.384361+0000","last_unstale":"2023-11-28T12:04:55.272642+0000","last_undegraded":"2023-11-28T12:04:55.272642+0000","last_fullsized":"2023-11-28T12:04:55.272642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:02.952081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.378249+0000","last_change":"2023-11-28T12:04:52.387408+0000","last_active":"2023-11-28T12:04:53.378249+0000","last_peered":"2023-11-28T12:04:53.378249+0000","last_clean":"2023-11-28T12:04:53.378249+0000","last_became_active":"2023-11-28T12:04:52.384005+0000","last_became_peered":"2023-11-28T12:04:52.384005+0000","last_unstale":"2023-11-28T12:04:53.378249+0000","last_undegraded":"2023-11-28T12:04:53.378249+0000","last_fullsized":"2023-11-28T12:04:53.378249+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:45.027098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377728+0000","last_change":"2023-11-28T12:04:52.375579+0000","last_active":"2023-11-28T12:04:53.377728+0000","last_peered":"2023-11-28T12:04:53.377728+0000","last_clean":"2023-11-28T12:04:53.377728+0000","last_became_active":"2023-11-28T12:04:52.375281+0000","last_became_peered":"2023-11-28T12:04:52.375281+0000","last_unstale":"2023-11-28T12:04:53.377728+0000","last_undegraded":"2023-11-28T12:04:53.377728+0000","last_fullsized":"2023-11-28T12:04:53.377728+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:16:17.346419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.824337+0000","last_change":"2023-11-28T12:04:52.382894+0000","last_active":"2023-11-28T12:04:54.824337+0000","last_peered":"2023-11-28T12:04:54.824337+0000","last_clean":"2023-11-28T12:04:54.824337+0000","last_became_active":"2023-11-28T12:04:52.382468+0000","last_became_peered":"2023-11-28T12:04:52.382468+0000","last_unstale":"2023-11-28T12:04:54.824337+0000","last_undegraded":"2023-11-28T12:04:54.824337+0000","last_fullsized":"2023-11-28T12:04:54.824337+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:33.123525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.518005+0000","last_change":"2023-11-28T12:04:52.389377+0000","last_active":"2023-11-28T12:04:53.518005+0000","last_peered":"2023-11-28T12:04:53.518005+0000","last_clean":"2023-11-28T12:04:53.518005+0000","last_became_active":"2023-11-28T12:04:52.389094+0000","last_became_peered":"2023-11-28T12:04:52.389094+0000","last_unstale":"2023-11-28T12:04:53.518005+0000","last_undegraded":"2023-11-28T12:04:53.518005+0000","last_fullsized":"2023-11-28T12:04:53.518005+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:02.757783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271031+0000","last_change":"2023-11-28T12:04:52.383250+0000","last_active":"2023-11-28T12:04:55.271031+0000","last_peered":"2023-11-28T12:04:55.271031+0000","last_clean":"2023-11-28T12:04:55.271031+0000","last_became_active":"2023-11-28T12:04:52.382481+0000","last_became_peered":"2023-11-28T12:04:52.382481+0000","last_unstale":"2023-11-28T12:04:55.271031+0000","last_undegraded":"2023-11-28T12:04:55.271031+0000","last_fullsized":"2023-11-28T12:04:55.271031+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:34:16.533129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272870+0000","last_change":"2023-11-28T12:04:52.389880+0000","last_active":"2023-11-28T12:04:55.272870+0000","last_peered":"2023-11-28T12:04:55.272870+0000","last_clean":"2023-11-28T12:04:55.272870+0000","last_became_active":"2023-11-28T12:04:52.389308+0000","last_became_peered":"2023-11-28T12:04:52.389308+0000","last_unstale":"2023-11-28T12:04:55.272870+0000","last_undegraded":"2023-11-28T12:04:55.272870+0000","last_fullsized":"2023-11-28T12:04:55.272870+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:21.970184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272555+0000","last_change":"2023-11-28T12:04:52.391554+0000","last_active":"2023-11-28T12:04:55.272555+0000","last_peered":"2023-11-28T12:04:55.272555+0000","last_clean":"2023-11-28T12:04:55.272555+0000","last_became_active":"2023-11-28T12:04:52.391299+0000","last_became_peered":"2023-11-28T12:04:52.391299+0000","last_unstale":"2023-11-28T12:04:55.272555+0000","last_undegraded":"2023-11-28T12:04:55.272555+0000","last_fullsized":"2023-11-28T12:04:55.272555+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:08.183501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377922+0000","last_change":"2023-11-28T12:04:52.391075+0000","last_active":"2023-11-28T12:04:53.377922+0000","last_peered":"2023-11-28T12:04:53.377922+0000","last_clean":"2023-11-28T12:04:53.377922+0000","last_became_active":"2023-11-28T12:04:52.390792+0000","last_became_peered":"2023-11-28T12:04:52.390792+0000","last_unstale":"2023-11-28T12:04:53.377922+0000","last_undegraded":"2023-11-28T12:04:53.377922+0000","last_fullsized":"2023-11-28T12:04:53.377922+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:02.952081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271248+0000","last_change":"2023-11-28T12:04:52.389229+0000","last_active":"2023-11-28T12:04:55.271248+0000","last_peered":"2023-11-28T12:04:55.271248+0000","last_clean":"2023-11-28T12:04:55.271248+0000","last_became_active":"2023-11-28T12:04:52.388960+0000","last_became_peered":"2023-11-28T12:04:52.388960+0000","last_unstale":"2023-11-28T12:04:55.271248+0000","last_undegraded":"2023-11-28T12:04:55.271248+0000","last_fullsized":"2023-11-28T12:04:55.271248+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:29:42.869610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271194+0000","last_change":"2023-11-28T12:04:52.374641+0000","last_active":"2023-11-28T12:04:55.271194+0000","last_peered":"2023-11-28T12:04:55.271194+0000","last_clean":"2023-11-28T12:04:55.271194+0000","last_became_active":"2023-11-28T12:04:52.374351+0000","last_became_peered":"2023-11-28T12:04:52.374351+0000","last_unstale":"2023-11-28T12:04:55.271194+0000","last_undegraded":"2023-11-28T12:04:55.271194+0000","last_fullsized":"2023-11-28T12:04:55.271194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:21.970184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272487+0000","last_change":"2023-11-28T12:04:52.384898+0000","last_active":"2023-11-28T12:04:55.272487+0000","last_peered":"2023-11-28T12:04:55.272487+0000","last_clean":"2023-11-28T12:04:55.272487+0000","last_became_active":"2023-11-28T12:04:52.384398+0000","last_became_peered":"2023-11-28T12:04:52.384398+0000","last_unstale":"2023-11-28T12:04:55.272487+0000","last_undegraded":"2023-11-28T12:04:55.272487+0000","last_fullsized":"2023-11-28T12:04:55.272487+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:45.027098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272622+0000","last_change":"2023-11-28T12:04:52.393915+0000","last_active":"2023-11-28T12:04:55.272622+0000","last_peered":"2023-11-28T12:04:55.272622+0000","last_clean":"2023-11-28T12:04:55.272622+0000","last_became_active":"2023-11-28T12:04:52.393655+0000","last_became_peered":"2023-11-28T12:04:52.393655+0000","last_unstale":"2023-11-28T12:04:55.272622+0000","last_undegraded":"2023-11-28T12:04:55.272622+0000","last_fullsized":"2023-11-28T12:04:55.272622+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:25.673535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.517952+0000","last_change":"2023-11-28T12:04:52.379165+0000","last_active":"2023-11-28T12:04:53.517952+0000","last_peered":"2023-11-28T12:04:53.517952+0000","last_clean":"2023-11-28T12:04:53.517952+0000","last_became_active":"2023-11-28T12:04:52.375881+0000","last_became_peered":"2023-11-28T12:04:52.375881+0000","last_unstale":"2023-11-28T12:04:53.517952+0000","last_undegraded":"2023-11-28T12:04:53.517952+0000","last_fullsized":"2023-11-28T12:04:53.517952+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:12:27.795181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.747352+0000","last_change":"2023-11-28T12:04:52.385438+0000","last_active":"2023-11-28T12:04:54.747352+0000","last_peered":"2023-11-28T12:04:54.747352+0000","last_clean":"2023-11-28T12:04:54.747352+0000","last_became_active":"2023-11-28T12:04:52.385167+0000","last_became_peered":"2023-11-28T12:04:52.385167+0000","last_unstale":"2023-11-28T12:04:54.747352+0000","last_undegraded":"2023-11-28T12:04:54.747352+0000","last_fullsized":"2023-11-28T12:04:54.747352+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:21.970184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377997+0000","last_change":"2023-11-28T12:04:52.384637+0000","last_active":"2023-11-28T12:04:53.377997+0000","last_peered":"2023-11-28T12:04:53.377997+0000","last_clean":"2023-11-28T12:04:53.377997+0000","last_became_active":"2023-11-28T12:04:52.383699+0000","last_became_peered":"2023-11-28T12:04:52.383699+0000","last_unstale":"2023-11-28T12:04:53.377997+0000","last_undegraded":"2023-11-28T12:04:53.377997+0000","last_fullsized":"2023-11-28T12:04:53.377997+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:25.673535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377627+0000","last_change":"2023-11-28T12:04:52.382384+0000","last_active":"2023-11-28T12:04:53.377627+0000","last_peered":"2023-11-28T12:04:53.377627+0000","last_clean":"2023-11-28T12:04:53.377627+0000","last_became_active":"2023-11-28T12:04:52.382095+0000","last_became_peered":"2023-11-28T12:04:52.382095+0000","last_unstale":"2023-11-28T12:04:53.377627+0000","last_undegraded":"2023-11-28T12:04:53.377627+0000","last_fullsized":"2023-11-28T12:04:53.377627+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:28.348156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.371843+0000","last_change":"2023-11-28T12:04:52.372907+0000","last_active":"2023-11-28T12:04:53.371843+0000","last_peered":"2023-11-28T12:04:53.371843+0000","last_clean":"2023-11-28T12:04:53.371843+0000","last_became_active":"2023-11-28T12:04:52.372662+0000","last_became_peered":"2023-11-28T12:04:52.372662+0000","last_unstale":"2023-11-28T12:04:53.371843+0000","last_undegraded":"2023-11-28T12:04:53.371843+0000","last_fullsized":"2023-11-28T12:04:53.371843+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:35:28.129566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271322+0000","last_change":"2023-11-28T12:04:52.382559+0000","last_active":"2023-11-28T12:04:55.271322+0000","last_peered":"2023-11-28T12:04:55.271322+0000","last_clean":"2023-11-28T12:04:55.271322+0000","last_became_active":"2023-11-28T12:04:52.382278+0000","last_became_peered":"2023-11-28T12:04:52.382278+0000","last_unstale":"2023-11-28T12:04:55.271322+0000","last_undegraded":"2023-11-28T12:04:55.271322+0000","last_fullsized":"2023-11-28T12:04:55.271322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:53.408359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.371918+0000","last_change":"2023-11-28T12:04:52.384805+0000","last_active":"2023-11-28T12:04:53.371918+0000","last_peered":"2023-11-28T12:04:53.371918+0000","last_clean":"2023-11-28T12:04:53.371918+0000","last_became_active":"2023-11-28T12:04:52.384525+0000","last_became_peered":"2023-11-28T12:04:52.384525+0000","last_unstale":"2023-11-28T12:04:53.371918+0000","last_undegraded":"2023-11-28T12:04:53.371918+0000","last_fullsized":"2023-11-28T12:04:53.371918+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:02:34.621157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.371754+0000","last_change":"2023-11-28T12:04:52.389833+0000","last_active":"2023-11-28T12:04:53.371754+0000","last_peered":"2023-11-28T12:04:53.371754+0000","last_clean":"2023-11-28T12:04:53.371754+0000","last_became_active":"2023-11-28T12:04:52.389538+0000","last_became_peered":"2023-11-28T12:04:52.389538+0000","last_unstale":"2023-11-28T12:04:53.371754+0000","last_undegraded":"2023-11-28T12:04:53.371754+0000","last_fullsized":"2023-11-28T12:04:53.371754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:47:06.821673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272517+0000","last_change":"2023-11-28T12:04:52.385183+0000","last_active":"2023-11-28T12:04:55.272517+0000","last_peered":"2023-11-28T12:04:55.272517+0000","last_clean":"2023-11-28T12:04:55.272517+0000","last_became_active":"2023-11-28T12:04:52.384407+0000","last_became_peered":"2023-11-28T12:04:52.384407+0000","last_unstale":"2023-11-28T12:04:55.272517+0000","last_undegraded":"2023-11-28T12:04:55.272517+0000","last_fullsized":"2023-11-28T12:04:55.272517+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:37.694660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271468+0000","last_change":"2023-11-28T12:04:52.380426+0000","last_active":"2023-11-28T12:04:55.271468+0000","last_peered":"2023-11-28T12:04:55.271468+0000","last_clean":"2023-11-28T12:04:55.271468+0000","last_became_active":"2023-11-28T12:04:52.380112+0000","last_became_peered":"2023-11-28T12:04:52.380112+0000","last_unstale":"2023-11-28T12:04:55.271468+0000","last_undegraded":"2023-11-28T12:04:55.271468+0000","last_fullsized":"2023-11-28T12:04:55.271468+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:02.952081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271292+0000","last_change":"2023-11-28T12:04:52.371983+0000","last_active":"2023-11-28T12:04:55.271292+0000","last_peered":"2023-11-28T12:04:55.271292+0000","last_clean":"2023-11-28T12:04:55.271292+0000","last_became_active":"2023-11-28T12:04:52.371719+0000","last_became_peered":"2023-11-28T12:04:52.371719+0000","last_unstale":"2023-11-28T12:04:55.271292+0000","last_undegraded":"2023-11-28T12:04:55.271292+0000","last_fullsized":"2023-11-28T12:04:55.271292+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:37.694660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377988+0000","last_change":"2023-11-28T12:04:52.386382+0000","last_active":"2023-11-28T12:04:53.377988+0000","last_peered":"2023-11-28T12:04:53.377988+0000","last_clean":"2023-11-28T12:04:53.377988+0000","last_became_active":"2023-11-28T12:04:52.386100+0000","last_became_peered":"2023-11-28T12:04:52.386100+0000","last_unstale":"2023-11-28T12:04:53.377988+0000","last_undegraded":"2023-11-28T12:04:53.377988+0000","last_fullsized":"2023-11-28T12:04:53.377988+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:48.028736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.747184+0000","last_change":"2023-11-28T12:04:52.388852+0000","last_active":"2023-11-28T12:04:54.747184+0000","last_peered":"2023-11-28T12:04:54.747184+0000","last_clean":"2023-11-28T12:04:54.747184+0000","last_became_active":"2023-11-28T12:04:52.388577+0000","last_became_peered":"2023-11-28T12:04:52.388577+0000","last_unstale":"2023-11-28T12:04:54.747184+0000","last_undegraded":"2023-11-28T12:04:54.747184+0000","last_fullsized":"2023-11-28T12:04:54.747184+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:53.408359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272556+0000","last_change":"2023-11-28T12:04:52.387833+0000","last_active":"2023-11-28T12:04:55.272556+0000","last_peered":"2023-11-28T12:04:55.272556+0000","last_clean":"2023-11-28T12:04:55.272556+0000","last_became_active":"2023-11-28T12:04:52.387501+0000","last_became_peered":"2023-11-28T12:04:52.387501+0000","last_unstale":"2023-11-28T12:04:55.272556+0000","last_undegraded":"2023-11-28T12:04:55.272556+0000","last_fullsized":"2023-11-28T12:04:55.272556+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:34:16.533129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271952+0000","last_change":"2023-11-28T12:04:52.383408+0000","last_active":"2023-11-28T12:04:55.271952+0000","last_peered":"2023-11-28T12:04:55.271952+0000","last_clean":"2023-11-28T12:04:55.271952+0000","last_became_active":"2023-11-28T12:04:52.382571+0000","last_became_peered":"2023-11-28T12:04:52.382571+0000","last_unstale":"2023-11-28T12:04:55.271952+0000","last_undegraded":"2023-11-28T12:04:55.271952+0000","last_fullsized":"2023-11-28T12:04:55.271952+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:43:03.206592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377016+0000","last_change":"2023-11-28T12:04:52.387026+0000","last_active":"2023-11-28T12:04:53.377016+0000","last_peered":"2023-11-28T12:04:53.377016+0000","last_clean":"2023-11-28T12:04:53.377016+0000","last_became_active":"2023-11-28T12:04:52.385351+0000","last_became_peered":"2023-11-28T12:04:52.385351+0000","last_unstale":"2023-11-28T12:04:53.377016+0000","last_undegraded":"2023-11-28T12:04:53.377016+0000","last_fullsized":"2023-11-28T12:04:53.377016+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:34:16.533129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.824149+0000","last_change":"2023-11-28T12:04:52.384434+0000","last_active":"2023-11-28T12:04:54.824149+0000","last_peered":"2023-11-28T12:04:54.824149+0000","last_clean":"2023-11-28T12:04:54.824149+0000","last_became_active":"2023-11-28T12:04:52.384128+0000","last_became_peered":"2023-11-28T12:04:52.384128+0000","last_unstale":"2023-11-28T12:04:54.824149+0000","last_undegraded":"2023-11-28T12:04:54.824149+0000","last_fullsized":"2023-11-28T12:04:54.824149+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:53:53.840908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377495+0000","last_change":"2023-11-28T12:04:52.391481+0000","last_active":"2023-11-28T12:04:53.377495+0000","last_peered":"2023-11-28T12:04:53.377495+0000","last_clean":"2023-11-28T12:04:53.377495+0000","last_became_active":"2023-11-28T12:04:52.391179+0000","last_became_peered":"2023-11-28T12:04:52.391179+0000","last_unstale":"2023-11-28T12:04:53.377495+0000","last_undegraded":"2023-11-28T12:04:53.377495+0000","last_fullsized":"2023-11-28T12:04:53.377495+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:59:12.567356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"62'4","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.392550+0000","last_change":"2023-11-28T12:04:51.944589+0000","last_active":"2023-11-28T12:04:52.392550+0000","last_peered":"2023-11-28T12:04:52.392550+0000","last_clean":"2023-11-28T12:04:52.392550+0000","last_became_active":"2023-11-28T12:04:51.944202+0000","last_became_peered":"2023-11-28T12:04:51.944202+0000","last_unstale":"2023-11-28T12:04:52.392550+0000","last_undegraded":"2023-11-28T12:04:52.392550+0000","last_fullsized":"2023-11-28T12:04:52.392550+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:15:52.635652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.378041+0000","last_change":"2023-11-28T12:04:52.378216+0000","last_active":"2023-11-28T12:04:53.378041+0000","last_peered":"2023-11-28T12:04:53.378041+0000","last_clean":"2023-11-28T12:04:53.378041+0000","last_became_active":"2023-11-28T12:04:52.377807+0000","last_became_peered":"2023-11-28T12:04:52.377807+0000","last_unstale":"2023-11-28T12:04:53.378041+0000","last_undegraded":"2023-11-28T12:04:53.378041+0000","last_fullsized":"2023-11-28T12:04:53.378041+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:06.246656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.393297+0000","last_change":"2023-11-28T12:04:51.939589+0000","last_active":"2023-11-28T12:04:52.393297+0000","last_peered":"2023-11-28T12:04:52.393297+0000","last_clean":"2023-11-28T12:04:52.393297+0000","last_became_active":"2023-11-28T12:04:51.939256+0000","last_became_peered":"2023-11-28T12:04:51.939256+0000","last_unstale":"2023-11-28T12:04:52.393297+0000","last_undegraded":"2023-11-28T12:04:52.393297+0000","last_fullsized":"2023-11-28T12:04:52.393297+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:00.055068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.824644+0000","last_change":"2023-11-28T12:04:52.386301+0000","last_active":"2023-11-28T12:04:54.824644+0000","last_peered":"2023-11-28T12:04:54.824644+0000","last_clean":"2023-11-28T12:04:54.824644+0000","last_became_active":"2023-11-28T12:04:52.384226+0000","last_became_peered":"2023-11-28T12:04:52.384226+0000","last_unstale":"2023-11-28T12:04:54.824644+0000","last_undegraded":"2023-11-28T12:04:54.824644+0000","last_fullsized":"2023-11-28T12:04:54.824644+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:55:46.190010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.383119+0000","last_change":"2023-11-28T12:04:51.935882+0000","last_active":"2023-11-28T12:04:52.383119+0000","last_peered":"2023-11-28T12:04:52.383119+0000","last_clean":"2023-11-28T12:04:52.383119+0000","last_became_active":"2023-11-28T12:04:51.934809+0000","last_became_peered":"2023-11-28T12:04:51.934809+0000","last_unstale":"2023-11-28T12:04:52.383119+0000","last_undegraded":"2023-11-28T12:04:52.383119+0000","last_fullsized":"2023-11-28T12:04:52.383119+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:05.119970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272939+0000","last_change":"2023-11-28T12:04:52.391923+0000","last_active":"2023-11-28T12:04:55.272939+0000","last_peered":"2023-11-28T12:04:55.272939+0000","last_clean":"2023-11-28T12:04:55.272939+0000","last_became_active":"2023-11-28T12:04:52.391688+0000","last_became_peered":"2023-11-28T12:04:52.391688+0000","last_unstale":"2023-11-28T12:04:55.272939+0000","last_undegraded":"2023-11-28T12:04:55.272939+0000","last_fullsized":"2023-11-28T12:04:55.272939+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:01.067609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.387644+0000","last_change":"2023-11-28T12:04:51.941486+0000","last_active":"2023-11-28T12:04:52.387644+0000","last_peered":"2023-11-28T12:04:52.387644+0000","last_clean":"2023-11-28T12:04:52.387644+0000","last_became_active":"2023-11-28T12:04:51.941144+0000","last_became_peered":"2023-11-28T12:04:51.941144+0000","last_unstale":"2023-11-28T12:04:52.387644+0000","last_undegraded":"2023-11-28T12:04:52.387644+0000","last_fullsized":"2023-11-28T12:04:52.387644+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:48.111122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.518069+0000","last_change":"2023-11-28T12:04:52.383946+0000","last_active":"2023-11-28T12:04:53.518069+0000","last_peered":"2023-11-28T12:04:53.518069+0000","last_clean":"2023-11-28T12:04:53.518069+0000","last_became_active":"2023-11-28T12:04:52.383666+0000","last_became_peered":"2023-11-28T12:04:52.383666+0000","last_unstale":"2023-11-28T12:04:53.518069+0000","last_undegraded":"2023-11-28T12:04:53.518069+0000","last_fullsized":"2023-11-28T12:04:53.518069+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:28.371921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.378932+0000","last_change":"2023-11-28T12:04:51.938573+0000","last_active":"2023-11-28T12:04:52.378932+0000","last_peered":"2023-11-28T12:04:52.378932+0000","last_clean":"2023-11-28T12:04:52.378932+0000","last_became_active":"2023-11-28T12:04:51.937184+0000","last_became_peered":"2023-11-28T12:04:51.937184+0000","last_unstale":"2023-11-28T12:04:52.378932+0000","last_undegraded":"2023-11-28T12:04:52.378932+0000","last_fullsized":"2023-11-28T12:04:52.378932+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:02.722002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.746780+0000","last_change":"2023-11-28T12:04:52.385183+0000","last_active":"2023-11-28T12:04:54.746780+0000","last_peered":"2023-11-28T12:04:54.746780+0000","last_clean":"2023-11-28T12:04:54.746780+0000","last_became_active":"2023-11-28T12:04:52.383229+0000","last_became_peered":"2023-11-28T12:04:52.383229+0000","last_unstale":"2023-11-28T12:04:54.746780+0000","last_undegraded":"2023-11-28T12:04:54.746780+0000","last_fullsized":"2023-11-28T12:04:54.746780+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:16:17.346419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.381680+0000","last_change":"2023-11-28T12:04:51.935707+0000","last_active":"2023-11-28T12:04:52.381680+0000","last_peered":"2023-11-28T12:04:52.381680+0000","last_clean":"2023-11-28T12:04:52.381680+0000","last_became_active":"2023-11-28T12:04:51.935328+0000","last_became_peered":"2023-11-28T12:04:51.935328+0000","last_unstale":"2023-11-28T12:04:52.381680+0000","last_undegraded":"2023-11-28T12:04:52.381680+0000","last_fullsized":"2023-11-28T12:04:52.381680+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:51:24.026885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377312+0000","last_change":"2023-11-28T12:04:52.376721+0000","last_active":"2023-11-28T12:04:53.377312+0000","last_peered":"2023-11-28T12:04:53.377312+0000","last_clean":"2023-11-28T12:04:53.377312+0000","last_became_active":"2023-11-28T12:04:52.376451+0000","last_became_peered":"2023-11-28T12:04:52.376451+0000","last_unstale":"2023-11-28T12:04:53.377312+0000","last_undegraded":"2023-11-28T12:04:53.377312+0000","last_fullsized":"2023-11-28T12:04:53.377312+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:55.240985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.364546+0000","last_change":"2023-11-28T12:04:51.942735+0000","last_active":"2023-11-28T12:04:52.364546+0000","last_peered":"2023-11-28T12:04:52.364546+0000","last_clean":"2023-11-28T12:04:52.364546+0000","last_became_active":"2023-11-28T12:04:51.942397+0000","last_became_peered":"2023-11-28T12:04:51.942397+0000","last_unstale":"2023-11-28T12:04:52.364546+0000","last_undegraded":"2023-11-28T12:04:52.364546+0000","last_fullsized":"2023-11-28T12:04:52.364546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:49:50.803373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.372067+0000","last_change":"2023-11-28T12:04:52.375404+0000","last_active":"2023-11-28T12:04:53.372067+0000","last_peered":"2023-11-28T12:04:53.372067+0000","last_clean":"2023-11-28T12:04:53.372067+0000","last_became_active":"2023-11-28T12:04:52.375170+0000","last_became_peered":"2023-11-28T12:04:52.375170+0000","last_unstale":"2023-11-28T12:04:53.372067+0000","last_undegraded":"2023-11-28T12:04:53.372067+0000","last_fullsized":"2023-11-28T12:04:53.372067+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:25.580039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.367699+0000","last_change":"2023-11-28T12:04:51.941216+0000","last_active":"2023-11-28T12:04:52.367699+0000","last_peered":"2023-11-28T12:04:52.367699+0000","last_clean":"2023-11-28T12:04:52.367699+0000","last_became_active":"2023-11-28T12:04:51.940857+0000","last_became_peered":"2023-11-28T12:04:51.940857+0000","last_unstale":"2023-11-28T12:04:52.367699+0000","last_undegraded":"2023-11-28T12:04:52.367699+0000","last_fullsized":"2023-11-28T12:04:52.367699+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:30.385397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.746932+0000","last_change":"2023-11-28T12:04:52.388818+0000","last_active":"2023-11-28T12:04:54.746932+0000","last_peered":"2023-11-28T12:04:54.746932+0000","last_clean":"2023-11-28T12:04:54.746932+0000","last_became_active":"2023-11-28T12:04:52.388236+0000","last_became_peered":"2023-11-28T12:04:52.388236+0000","last_unstale":"2023-11-28T12:04:54.746932+0000","last_undegraded":"2023-11-28T12:04:54.746932+0000","last_fullsized":"2023-11-28T12:04:54.746932+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:48.028736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.379423+0000","last_change":"2023-11-28T12:04:51.945702+0000","last_active":"2023-11-28T12:04:52.379423+0000","last_peered":"2023-11-28T12:04:52.379423+0000","last_clean":"2023-11-28T12:04:52.379423+0000","last_became_active":"2023-11-28T12:04:51.945398+0000","last_became_peered":"2023-11-28T12:04:51.945398+0000","last_unstale":"2023-11-28T12:04:52.379423+0000","last_undegraded":"2023-11-28T12:04:52.379423+0000","last_fullsized":"2023-11-28T12:04:52.379423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:20.019844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.371918+0000","last_change":"2023-11-28T12:04:52.371074+0000","last_active":"2023-11-28T12:04:53.371918+0000","last_peered":"2023-11-28T12:04:53.371918+0000","last_clean":"2023-11-28T12:04:53.371918+0000","last_became_active":"2023-11-28T12:04:52.370751+0000","last_became_peered":"2023-11-28T12:04:52.370751+0000","last_unstale":"2023-11-28T12:04:53.371918+0000","last_undegraded":"2023-11-28T12:04:53.371918+0000","last_fullsized":"2023-11-28T12:04:53.371918+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:44:32.011577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.375010+0000","last_change":"2023-11-28T12:04:51.935817+0000","last_active":"2023-11-28T12:04:52.375010+0000","last_peered":"2023-11-28T12:04:52.375010+0000","last_clean":"2023-11-28T12:04:52.375010+0000","last_became_active":"2023-11-28T12:04:51.935546+0000","last_became_peered":"2023-11-28T12:04:51.935546+0000","last_unstale":"2023-11-28T12:04:52.375010+0000","last_undegraded":"2023-11-28T12:04:52.375010+0000","last_fullsized":"2023-11-28T12:04:52.375010+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:27.335615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.378247+0000","last_change":"2023-11-28T12:04:52.372158+0000","last_active":"2023-11-28T12:04:53.378247+0000","last_peered":"2023-11-28T12:04:53.378247+0000","last_clean":"2023-11-28T12:04:53.378247+0000","last_became_active":"2023-11-28T12:04:52.371714+0000","last_became_peered":"2023-11-28T12:04:52.371714+0000","last_unstale":"2023-11-28T12:04:53.378247+0000","last_undegraded":"2023-11-28T12:04:53.378247+0000","last_fullsized":"2023-11-28T12:04:53.378247+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:26.089026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"62'3","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.392142+0000","last_change":"2023-11-28T12:04:51.944007+0000","last_active":"2023-11-28T12:04:52.392142+0000","last_peered":"2023-11-28T12:04:52.392142+0000","last_clean":"2023-11-28T12:04:52.392142+0000","last_became_active":"2023-11-28T12:04:51.943700+0000","last_became_peered":"2023-11-28T12:04:51.943700+0000","last_unstale":"2023-11-28T12:04:52.392142+0000","last_undegraded":"2023-11-28T12:04:52.392142+0000","last_fullsized":"2023-11-28T12:04:52.392142+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:59.043060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.378307+0000","last_change":"2023-11-28T12:04:52.385932+0000","last_active":"2023-11-28T12:04:53.378307+0000","last_peered":"2023-11-28T12:04:53.378307+0000","last_clean":"2023-11-28T12:04:53.378307+0000","last_became_active":"2023-11-28T12:04:52.382785+0000","last_became_peered":"2023-11-28T12:04:52.382785+0000","last_unstale":"2023-11-28T12:04:53.378307+0000","last_undegraded":"2023-11-28T12:04:53.378307+0000","last_fullsized":"2023-11-28T12:04:53.378307+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:01.067609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"62'6","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.396071+0000","last_change":"2023-11-28T12:04:51.940572+0000","last_active":"2023-11-28T12:04:52.396071+0000","last_peered":"2023-11-28T12:04:52.396071+0000","last_clean":"2023-11-28T12:04:52.396071+0000","last_became_active":"2023-11-28T12:04:51.939863+0000","last_became_peered":"2023-11-28T12:04:51.939863+0000","last_unstale":"2023-11-28T12:04:52.396071+0000","last_undegraded":"2023-11-28T12:04:52.396071+0000","last_fullsized":"2023-11-28T12:04:52.396071+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377208+0000","last_change":"2023-11-28T12:04:52.385917+0000","last_active":"2023-11-28T12:04:53.377208+0000","last_peered":"2023-11-28T12:04:53.377208+0000","last_clean":"2023-11-28T12:04:53.377208+0000","last_became_active":"2023-11-28T12:04:52.385614+0000","last_became_peered":"2023-11-28T12:04:52.385614+0000","last_unstale":"2023-11-28T12:04:53.377208+0000","last_undegraded":"2023-11-28T12:04:53.377208+0000","last_fullsized":"2023-11-28T12:04:53.377208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:37.694660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.388967+0000","last_change":"2023-11-28T12:04:51.939230+0000","last_active":"2023-11-28T12:04:52.388967+0000","last_peered":"2023-11-28T12:04:52.388967+0000","last_clean":"2023-11-28T12:04:52.388967+0000","last_became_active":"2023-11-28T12:04:51.938945+0000","last_became_peered":"2023-11-28T12:04:51.938945+0000","last_unstale":"2023-11-28T12:04:52.388967+0000","last_undegraded":"2023-11-28T12:04:52.388967+0000","last_fullsized":"2023-11-28T12:04:52.388967+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:25.076484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271127+0000","last_change":"2023-11-28T12:04:52.376236+0000","last_active":"2023-11-28T12:04:55.271127+0000","last_peered":"2023-11-28T12:04:55.271127+0000","last_clean":"2023-11-28T12:04:55.271127+0000","last_became_active":"2023-11-28T12:04:52.375938+0000","last_became_peered":"2023-11-28T12:04:52.375938+0000","last_unstale":"2023-11-28T12:04:55.271127+0000","last_undegraded":"2023-11-28T12:04:55.271127+0000","last_fullsized":"2023-11-28T12:04:55.271127+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:45.027098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.366472+0000","last_change":"2023-11-28T12:04:51.943415+0000","last_active":"2023-11-28T12:04:52.366472+0000","last_peered":"2023-11-28T12:04:52.366472+0000","last_clean":"2023-11-28T12:04:52.366472+0000","last_became_active":"2023-11-28T12:04:51.943175+0000","last_became_peered":"2023-11-28T12:04:51.943175+0000","last_unstale":"2023-11-28T12:04:52.366472+0000","last_undegraded":"2023-11-28T12:04:52.366472+0000","last_fullsized":"2023-11-28T12:04:52.366472+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377409+0000","last_change":"2023-11-28T12:04:52.383381+0000","last_active":"2023-11-28T12:04:53.377409+0000","last_peered":"2023-11-28T12:04:53.377409+0000","last_clean":"2023-11-28T12:04:53.377409+0000","last_became_active":"2023-11-28T12:04:52.379867+0000","last_became_peered":"2023-11-28T12:04:52.379867+0000","last_unstale":"2023-11-28T12:04:53.377409+0000","last_undegraded":"2023-11-28T12:04:53.377409+0000","last_fullsized":"2023-11-28T12:04:53.377409+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:00:20.462743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.380545+0000","last_change":"2023-11-28T12:04:51.940514+0000","last_active":"2023-11-28T12:04:52.380545+0000","last_peered":"2023-11-28T12:04:52.380545+0000","last_clean":"2023-11-28T12:04:52.380545+0000","last_became_active":"2023-11-28T12:04:51.939694+0000","last_became_peered":"2023-11-28T12:04:51.939694+0000","last_unstale":"2023-11-28T12:04:52.380545+0000","last_undegraded":"2023-11-28T12:04:52.380545+0000","last_fullsized":"2023-11-28T12:04:52.380545+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.747005+0000","last_change":"2023-11-28T12:04:52.386666+0000","last_active":"2023-11-28T12:04:54.747005+0000","last_peered":"2023-11-28T12:04:54.747005+0000","last_clean":"2023-11-28T12:04:54.747005+0000","last_became_active":"2023-11-28T12:04:52.386329+0000","last_became_peered":"2023-11-28T12:04:52.386329+0000","last_unstale":"2023-11-28T12:04:54.747005+0000","last_undegraded":"2023-11-28T12:04:54.747005+0000","last_fullsized":"2023-11-28T12:04:54.747005+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:43:03.206592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"62'8","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.390122+0000","last_change":"2023-11-28T12:04:51.940379+0000","last_active":"2023-11-28T12:04:52.390122+0000","last_peered":"2023-11-28T12:04:52.390122+0000","last_clean":"2023-11-28T12:04:52.390122+0000","last_became_active":"2023-11-28T12:04:51.940118+0000","last_became_peered":"2023-11-28T12:04:51.940118+0000","last_unstale":"2023-11-28T12:04:52.390122+0000","last_undegraded":"2023-11-28T12:04:52.390122+0000","last_fullsized":"2023-11-28T12:04:52.390122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:25.076484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.824326+0000","last_change":"2023-11-28T12:04:52.393348+0000","last_active":"2023-11-28T12:04:54.824326+0000","last_peered":"2023-11-28T12:04:54.824326+0000","last_clean":"2023-11-28T12:04:54.824326+0000","last_became_active":"2023-11-28T12:04:52.393002+0000","last_became_peered":"2023-11-28T12:04:52.393002+0000","last_unstale":"2023-11-28T12:04:54.824326+0000","last_undegraded":"2023-11-28T12:04:54.824326+0000","last_fullsized":"2023-11-28T12:04:54.824326+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:33.184300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.383268+0000","last_change":"2023-11-28T12:04:51.939307+0000","last_active":"2023-11-28T12:04:52.383268+0000","last_peered":"2023-11-28T12:04:52.383268+0000","last_clean":"2023-11-28T12:04:52.383268+0000","last_became_active":"2023-11-28T12:04:51.939004+0000","last_became_peered":"2023-11-28T12:04:51.939004+0000","last_unstale":"2023-11-28T12:04:52.383268+0000","last_undegraded":"2023-11-28T12:04:52.383268+0000","last_fullsized":"2023-11-28T12:04:52.383268+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.518114+0000","last_change":"2023-11-28T12:04:52.383898+0000","last_active":"2023-11-28T12:04:53.518114+0000","last_peered":"2023-11-28T12:04:53.518114+0000","last_clean":"2023-11-28T12:04:53.518114+0000","last_became_active":"2023-11-28T12:04:52.383743+0000","last_became_peered":"2023-11-28T12:04:52.383743+0000","last_unstale":"2023-11-28T12:04:53.518114+0000","last_undegraded":"2023-11-28T12:04:53.518114+0000","last_fullsized":"2023-11-28T12:04:53.518114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:59:10.723237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.366188+0000","last_change":"2023-11-28T12:04:51.937807+0000","last_active":"2023-11-28T12:04:52.366188+0000","last_peered":"2023-11-28T12:04:52.366188+0000","last_clean":"2023-11-28T12:04:52.366188+0000","last_became_active":"2023-11-28T12:04:51.937514+0000","last_became_peered":"2023-11-28T12:04:51.937514+0000","last_unstale":"2023-11-28T12:04:52.366188+0000","last_undegraded":"2023-11-28T12:04:52.366188+0000","last_fullsized":"2023-11-28T12:04:52.366188+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:01.574232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271360+0000","last_change":"2023-11-28T12:04:52.384074+0000","last_active":"2023-11-28T12:04:55.271360+0000","last_peered":"2023-11-28T12:04:55.271360+0000","last_clean":"2023-11-28T12:04:55.271360+0000","last_became_active":"2023-11-28T12:04:52.383778+0000","last_became_peered":"2023-11-28T12:04:52.383778+0000","last_unstale":"2023-11-28T12:04:55.271360+0000","last_undegraded":"2023-11-28T12:04:55.271360+0000","last_fullsized":"2023-11-28T12:04:55.271360+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:14:26.740655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.375818+0000","last_change":"2023-11-28T12:04:51.934819+0000","last_active":"2023-11-28T12:04:52.375818+0000","last_peered":"2023-11-28T12:04:52.375818+0000","last_clean":"2023-11-28T12:04:52.375818+0000","last_became_active":"2023-11-28T12:04:51.934462+0000","last_became_peered":"2023-11-28T12:04:51.934462+0000","last_unstale":"2023-11-28T12:04:52.375818+0000","last_undegraded":"2023-11-28T12:04:52.375818+0000","last_fullsized":"2023-11-28T12:04:52.375818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:36.797091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.378300+0000","last_change":"2023-11-28T12:04:52.380427+0000","last_active":"2023-11-28T12:04:53.378300+0000","last_peered":"2023-11-28T12:04:53.378300+0000","last_clean":"2023-11-28T12:04:53.378300+0000","last_became_active":"2023-11-28T12:04:52.379655+0000","last_became_peered":"2023-11-28T12:04:52.379655+0000","last_unstale":"2023-11-28T12:04:53.378300+0000","last_undegraded":"2023-11-28T12:04:53.378300+0000","last_fullsized":"2023-11-28T12:04:53.378300+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:13.075372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.376654+0000","last_change":"2023-11-28T12:04:51.940150+0000","last_active":"2023-11-28T12:04:52.376654+0000","last_peered":"2023-11-28T12:04:52.376654+0000","last_clean":"2023-11-28T12:04:52.376654+0000","last_became_active":"2023-11-28T12:04:51.939467+0000","last_became_peered":"2023-11-28T12:04:51.939467+0000","last_unstale":"2023-11-28T12:04:52.376654+0000","last_undegraded":"2023-11-28T12:04:52.376654+0000","last_fullsized":"2023-11-28T12:04:52.376654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377076+0000","last_change":"2023-11-28T12:04:52.380806+0000","last_active":"2023-11-28T12:04:53.377076+0000","last_peered":"2023-11-28T12:04:53.377076+0000","last_clean":"2023-11-28T12:04:53.377076+0000","last_became_active":"2023-11-28T12:04:52.380550+0000","last_became_peered":"2023-11-28T12:04:52.380550+0000","last_unstale":"2023-11-28T12:04:53.377076+0000","last_undegraded":"2023-11-28T12:04:53.377076+0000","last_fullsized":"2023-11-28T12:04:53.377076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:02.586773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.373642+0000","last_change":"2023-11-28T12:04:51.942228+0000","last_active":"2023-11-28T12:04:52.373642+0000","last_peered":"2023-11-28T12:04:52.373642+0000","last_clean":"2023-11-28T12:04:52.373642+0000","last_became_active":"2023-11-28T12:04:51.941878+0000","last_became_peered":"2023-11-28T12:04:51.941878+0000","last_unstale":"2023-11-28T12:04:52.373642+0000","last_undegraded":"2023-11-28T12:04:52.373642+0000","last_fullsized":"2023-11-28T12:04:52.373642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377075+0000","last_change":"2023-11-28T12:04:52.389871+0000","last_active":"2023-11-28T12:04:53.377075+0000","last_peered":"2023-11-28T12:04:53.377075+0000","last_clean":"2023-11-28T12:04:53.377075+0000","last_became_active":"2023-11-28T12:04:52.389588+0000","last_became_peered":"2023-11-28T12:04:52.389588+0000","last_unstale":"2023-11-28T12:04:53.377075+0000","last_undegraded":"2023-11-28T12:04:53.377075+0000","last_fullsized":"2023-11-28T12:04:53.377075+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:14:26.740655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.365217+0000","last_change":"2023-11-28T12:04:51.940161+0000","last_active":"2023-11-28T12:04:52.365217+0000","last_peered":"2023-11-28T12:04:52.365217+0000","last_clean":"2023-11-28T12:04:52.365217+0000","last_became_active":"2023-11-28T12:04:51.939789+0000","last_became_peered":"2023-11-28T12:04:51.939789+0000","last_unstale":"2023-11-28T12:04:52.365217+0000","last_undegraded":"2023-11-28T12:04:52.365217+0000","last_fullsized":"2023-11-28T12:04:52.365217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:15:49.848587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.746914+0000","last_change":"2023-11-28T12:04:52.385238+0000","last_active":"2023-11-28T12:04:54.746914+0000","last_peered":"2023-11-28T12:04:54.746914+0000","last_clean":"2023-11-28T12:04:54.746914+0000","last_became_active":"2023-11-28T12:04:52.383242+0000","last_became_peered":"2023-11-28T12:04:52.383242+0000","last_unstale":"2023-11-28T12:04:54.746914+0000","last_undegraded":"2023-11-28T12:04:54.746914+0000","last_fullsized":"2023-11-28T12:04:54.746914+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:29:42.869610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.364639+0000","last_change":"2023-11-28T12:04:51.942255+0000","last_active":"2023-11-28T12:04:52.364639+0000","last_peered":"2023-11-28T12:04:52.364639+0000","last_clean":"2023-11-28T12:04:52.364639+0000","last_became_active":"2023-11-28T12:04:51.941749+0000","last_became_peered":"2023-11-28T12:04:51.941749+0000","last_unstale":"2023-11-28T12:04:52.364639+0000","last_undegraded":"2023-11-28T12:04:52.364639+0000","last_fullsized":"2023-11-28T12:04:52.364639+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:55:26.284594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.517975+0000","last_change":"2023-11-28T12:04:52.374068+0000","last_active":"2023-11-28T12:04:53.517975+0000","last_peered":"2023-11-28T12:04:53.517975+0000","last_clean":"2023-11-28T12:04:53.517975+0000","last_became_active":"2023-11-28T12:04:52.373799+0000","last_became_peered":"2023-11-28T12:04:52.373799+0000","last_unstale":"2023-11-28T12:04:53.517975+0000","last_undegraded":"2023-11-28T12:04:53.517975+0000","last_fullsized":"2023-11-28T12:04:53.517975+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:53.258677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.375980+0000","last_change":"2023-11-28T12:04:51.939116+0000","last_active":"2023-11-28T12:04:52.375980+0000","last_peered":"2023-11-28T12:04:52.375980+0000","last_clean":"2023-11-28T12:04:52.375980+0000","last_became_active":"2023-11-28T12:04:51.938814+0000","last_became_peered":"2023-11-28T12:04:51.938814+0000","last_unstale":"2023-11-28T12:04:52.375980+0000","last_undegraded":"2023-11-28T12:04:52.375980+0000","last_fullsized":"2023-11-28T12:04:52.375980+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:50.784042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.378113+0000","last_change":"2023-11-28T12:04:52.380401+0000","last_active":"2023-11-28T12:04:53.378113+0000","last_peered":"2023-11-28T12:04:53.378113+0000","last_clean":"2023-11-28T12:04:53.378113+0000","last_became_active":"2023-11-28T12:04:52.378912+0000","last_became_peered":"2023-11-28T12:04:52.378912+0000","last_unstale":"2023-11-28T12:04:53.378113+0000","last_undegraded":"2023-11-28T12:04:53.378113+0000","last_fullsized":"2023-11-28T12:04:53.378113+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:08.183501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.392063+0000","last_change":"2023-11-28T12:04:51.944525+0000","last_active":"2023-11-28T12:04:52.392063+0000","last_peered":"2023-11-28T12:04:52.392063+0000","last_clean":"2023-11-28T12:04:52.392063+0000","last_became_active":"2023-11-28T12:04:51.944298+0000","last_became_peered":"2023-11-28T12:04:51.944298+0000","last_unstale":"2023-11-28T12:04:52.392063+0000","last_undegraded":"2023-11-28T12:04:52.392063+0000","last_fullsized":"2023-11-28T12:04:52.392063+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:27.335615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.375194+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:04:52.375194+0000","last_peered":"2023-11-28T12:04:52.375194+0000","last_clean":"2023-11-28T12:04:52.375194+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:04:52.375194+0000","last_undegraded":"2023-11-28T12:04:52.375194+0000","last_fullsized":"2023-11-28T12:04:52.375194+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377762+0000","last_change":"2023-11-28T12:04:52.385572+0000","last_active":"2023-11-28T12:04:53.377762+0000","last_peered":"2023-11-28T12:04:53.377762+0000","last_clean":"2023-11-28T12:04:53.377762+0000","last_became_active":"2023-11-28T12:04:52.385271+0000","last_became_peered":"2023-11-28T12:04:52.385271+0000","last_unstale":"2023-11-28T12:04:53.377762+0000","last_undegraded":"2023-11-28T12:04:53.377762+0000","last_fullsized":"2023-11-28T12:04:53.377762+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:00.055601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.365759+0000","last_change":"2023-11-28T12:04:51.939162+0000","last_active":"2023-11-28T12:04:52.365759+0000","last_peered":"2023-11-28T12:04:52.365759+0000","last_clean":"2023-11-28T12:04:52.365759+0000","last_became_active":"2023-11-28T12:04:51.938867+0000","last_became_peered":"2023-11-28T12:04:51.938867+0000","last_unstale":"2023-11-28T12:04:52.365759+0000","last_undegraded":"2023-11-28T12:04:52.365759+0000","last_fullsized":"2023-11-28T12:04:52.365759+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:40.127328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.365825+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:04:52.365825+0000","last_peered":"2023-11-28T12:04:52.365825+0000","last_clean":"2023-11-28T12:04:52.365825+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:04:52.365825+0000","last_undegraded":"2023-11-28T12:04:52.365825+0000","last_fullsized":"2023-11-28T12:04:52.365825+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.824239+0000","last_change":"2023-11-28T12:04:52.383789+0000","last_active":"2023-11-28T12:04:54.824239+0000","last_peered":"2023-11-28T12:04:54.824239+0000","last_clean":"2023-11-28T12:04:54.824239+0000","last_became_active":"2023-11-28T12:04:52.383605+0000","last_became_peered":"2023-11-28T12:04:52.383605+0000","last_unstale":"2023-11-28T12:04:54.824239+0000","last_undegraded":"2023-11-28T12:04:54.824239+0000","last_fullsized":"2023-11-28T12:04:54.824239+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:59:29.170255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.387281+0000","last_change":"2023-11-28T12:04:51.938688+0000","last_active":"2023-11-28T12:04:52.387281+0000","last_peered":"2023-11-28T12:04:52.387281+0000","last_clean":"2023-11-28T12:04:52.387281+0000","last_became_active":"2023-11-28T12:04:51.938320+0000","last_became_peered":"2023-11-28T12:04:51.938320+0000","last_unstale":"2023-11-28T12:04:52.387281+0000","last_undegraded":"2023-11-28T12:04:52.387281+0000","last_fullsized":"2023-11-28T12:04:52.387281+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:05.234114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.377234+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:04:52.377234+0000","last_peered":"2023-11-28T12:04:52.377234+0000","last_clean":"2023-11-28T12:04:52.377234+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:04:52.377234+0000","last_undegraded":"2023-11-28T12:04:52.377234+0000","last_fullsized":"2023-11-28T12:04:52.377234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":198,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.375347+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:04:52.375347+0000","last_peered":"2023-11-28T12:04:52.375347+0000","last_clean":"2023-11-28T12:04:52.375347+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:04:52.375347+0000","last_undegraded":"2023-11-28T12:04:52.375347+0000","last_fullsized":"2023-11-28T12:04:52.375347+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271035+0000","last_change":"2023-11-28T12:04:52.373760+0000","last_active":"2023-11-28T12:04:55.271035+0000","last_peered":"2023-11-28T12:04:55.271035+0000","last_clean":"2023-11-28T12:04:55.271035+0000","last_became_active":"2023-11-28T12:04:52.373477+0000","last_became_peered":"2023-11-28T12:04:52.373477+0000","last_unstale":"2023-11-28T12:04:55.271035+0000","last_undegraded":"2023-11-28T12:04:55.271035+0000","last_fullsized":"2023-11-28T12:04:55.271035+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:25.673535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.366504+0000","last_change":"2023-11-28T12:04:51.943124+0000","last_active":"2023-11-28T12:04:52.366504+0000","last_peered":"2023-11-28T12:04:52.366504+0000","last_clean":"2023-11-28T12:04:52.366504+0000","last_became_active":"2023-11-28T12:04:51.942813+0000","last_became_peered":"2023-11-28T12:04:51.942813+0000","last_unstale":"2023-11-28T12:04:52.366504+0000","last_undegraded":"2023-11-28T12:04:52.366504+0000","last_fullsized":"2023-11-28T12:04:52.366504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:50.784042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.373075+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:04:52.373075+0000","last_peered":"2023-11-28T12:04:52.373075+0000","last_clean":"2023-11-28T12:04:52.373075+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:04:52.373075+0000","last_undegraded":"2023-11-28T12:04:52.373075+0000","last_fullsized":"2023-11-28T12:04:52.373075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377620+0000","last_change":"2023-11-28T12:04:52.382618+0000","last_active":"2023-11-28T12:04:53.377620+0000","last_peered":"2023-11-28T12:04:53.377620+0000","last_clean":"2023-11-28T12:04:53.377620+0000","last_became_active":"2023-11-28T12:04:52.382345+0000","last_became_peered":"2023-11-28T12:04:52.382345+0000","last_unstale":"2023-11-28T12:04:53.377620+0000","last_undegraded":"2023-11-28T12:04:53.377620+0000","last_fullsized":"2023-11-28T12:04:53.377620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:21.032385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.364471+0000","last_change":"2023-11-28T12:04:51.936566+0000","last_active":"2023-11-28T12:04:52.364471+0000","last_peered":"2023-11-28T12:04:52.364471+0000","last_clean":"2023-11-28T12:04:52.364471+0000","last_became_active":"2023-11-28T12:04:51.936221+0000","last_became_peered":"2023-11-28T12:04:51.936221+0000","last_unstale":"2023-11-28T12:04:52.364471+0000","last_undegraded":"2023-11-28T12:04:52.364471+0000","last_fullsized":"2023-11-28T12:04:52.364471+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:05.119970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.375434+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:04:52.375434+0000","last_peered":"2023-11-28T12:04:52.375434+0000","last_clean":"2023-11-28T12:04:52.375434+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:04:52.375434+0000","last_undegraded":"2023-11-28T12:04:52.375434+0000","last_fullsized":"2023-11-28T12:04:52.375434+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377900+0000","last_change":"2023-11-28T12:04:52.376429+0000","last_active":"2023-11-28T12:04:53.377900+0000","last_peered":"2023-11-28T12:04:53.377900+0000","last_clean":"2023-11-28T12:04:53.377900+0000","last_became_active":"2023-11-28T12:04:52.376077+0000","last_became_peered":"2023-11-28T12:04:52.376077+0000","last_unstale":"2023-11-28T12:04:53.377900+0000","last_undegraded":"2023-11-28T12:04:53.377900+0000","last_fullsized":"2023-11-28T12:04:53.377900+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:51.796583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.373175+0000","last_change":"2023-11-28T12:04:51.939473+0000","last_active":"2023-11-28T12:04:52.373175+0000","last_peered":"2023-11-28T12:04:52.373175+0000","last_clean":"2023-11-28T12:04:52.373175+0000","last_became_active":"2023-11-28T12:04:51.939141+0000","last_became_peered":"2023-11-28T12:04:51.939141+0000","last_unstale":"2023-11-28T12:04:52.373175+0000","last_undegraded":"2023-11-28T12:04:52.373175+0000","last_fullsized":"2023-11-28T12:04:52.373175+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:07.170960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.366113+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:04:52.366113+0000","last_peered":"2023-11-28T12:04:52.366113+0000","last_clean":"2023-11-28T12:04:52.366113+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:04:52.366113+0000","last_undegraded":"2023-11-28T12:04:52.366113+0000","last_fullsized":"2023-11-28T12:04:52.366113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377687+0000","last_change":"2023-11-28T12:04:52.375680+0000","last_active":"2023-11-28T12:04:53.377687+0000","last_peered":"2023-11-28T12:04:53.377687+0000","last_clean":"2023-11-28T12:04:53.377687+0000","last_became_active":"2023-11-28T12:04:52.375252+0000","last_became_peered":"2023-11-28T12:04:52.375252+0000","last_unstale":"2023-11-28T12:04:53.377687+0000","last_undegraded":"2023-11-28T12:04:53.377687+0000","last_fullsized":"2023-11-28T12:04:53.377687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:17.163166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.389550+0000","last_change":"2023-11-28T12:04:51.939352+0000","last_active":"2023-11-28T12:04:52.389550+0000","last_peered":"2023-11-28T12:04:52.389550+0000","last_clean":"2023-11-28T12:04:52.389550+0000","last_became_active":"2023-11-28T12:04:51.939082+0000","last_became_peered":"2023-11-28T12:04:51.939082+0000","last_unstale":"2023-11-28T12:04:52.389550+0000","last_undegraded":"2023-11-28T12:04:52.389550+0000","last_fullsized":"2023-11-28T12:04:52.389550+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:01.574232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.367469+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:04:52.367469+0000","last_peered":"2023-11-28T12:04:52.367469+0000","last_clean":"2023-11-28T12:04:52.367469+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:04:52.367469+0000","last_undegraded":"2023-11-28T12:04:52.367469+0000","last_fullsized":"2023-11-28T12:04:52.367469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272584+0000","last_change":"2023-11-28T12:04:52.390267+0000","last_active":"2023-11-28T12:04:55.272584+0000","last_peered":"2023-11-28T12:04:55.272584+0000","last_clean":"2023-11-28T12:04:55.272584+0000","last_became_active":"2023-11-28T12:04:52.389948+0000","last_became_peered":"2023-11-28T12:04:52.389948+0000","last_unstale":"2023-11-28T12:04:55.272584+0000","last_undegraded":"2023-11-28T12:04:55.272584+0000","last_fullsized":"2023-11-28T12:04:55.272584+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:48.028736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.386252+0000","last_change":"2023-11-28T12:04:51.944547+0000","last_active":"2023-11-28T12:04:52.386252+0000","last_peered":"2023-11-28T12:04:52.386252+0000","last_clean":"2023-11-28T12:04:52.386252+0000","last_became_active":"2023-11-28T12:04:51.944273+0000","last_became_peered":"2023-11-28T12:04:51.944273+0000","last_unstale":"2023-11-28T12:04:52.386252+0000","last_undegraded":"2023-11-28T12:04:52.386252+0000","last_fullsized":"2023-11-28T12:04:52.386252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:15:52.635652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.368579+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:04:52.368579+0000","last_peered":"2023-11-28T12:04:52.368579+0000","last_clean":"2023-11-28T12:04:52.368579+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:04:52.368579+0000","last_undegraded":"2023-11-28T12:04:52.368579+0000","last_fullsized":"2023-11-28T12:04:52.368579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10816,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94361024,"statfs":{"total":96636764160,"available":96625688576,"internally_reserved":0,"allocated":1368064,"data_stored":927690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55500000000000005,"15min":0.54800000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69699999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64300000000000002,"5min":0.872,"15min":0.872},"last":0.44},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.76500000000000001,"15min":0.84699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73099999999999998,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.751,"15min":0.83099999999999996},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70199999999999996,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72399999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.39000000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.75600000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64900000000000002,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75700000000000001,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.52200000000000002}]},{"osd":4,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.505},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.501},{"interface":"front","average":{"1min":0.503,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.39300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64400000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.49299999999999999}]},{"osd":5,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55200000000000005},"min":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73599999999999999,"5min":0.752,"15min":0.752},"last":0.57999999999999996}]},{"osd":6,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.70499999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69599999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9328,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362512,"statfs":{"total":96636764160,"available":96627212288,"internally_reserved":0,"allocated":761856,"data_stored":336596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.44500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":1.446,"15min":1.446},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.72099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.39800000000000002}]},{"osd":1,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.52400000000000002,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73499999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80500000000000005,"5min":0.88,"15min":0.88},"last":0.51900000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.53000000000000003,"15min":0.497},"min":{"1min":0.46899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.76800000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80400000000000005,"5min":1.254,"15min":1.254},"last":0.48799999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.39500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.46000000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80500000000000005,"5min":1.228,"15min":1.228},"last":0.55800000000000005}]},{"osd":4,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46899999999999997,"15min":0.45200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69499999999999995,"5min":1.204,"15min":1.204},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67000000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.46100000000000002}]},{"osd":5,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70999999999999996,"5min":0.753,"15min":0.753},"last":0.439},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.374,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":1.29,"15min":1.29},"last":0.60699999999999998}]},{"osd":7,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.748,"5min":1.159,"15min":1.159},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.35499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81799999999999995,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10636,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361204,"statfs":{"total":96636764160,"available":96625872896,"internally_reserved":0,"allocated":1380352,"data_stored":929226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39500000000000002,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.49199999999999999,"5min":1.2,"15min":1.2},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.27300000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60799999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.30599999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.37,"15min":0.37},"min":{"1min":0.22900000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.52100000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57499999999999996,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.38100000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47499999999999998,"15min":0.48599999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63900000000000001,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.44500000000000001,"15min":0.45400000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54800000000000004,"5min":1.03,"15min":1.03},"last":0.37}]},{"osd":4,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.375,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81000000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81599999999999995,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.58099999999999996}]},{"osd":5,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.627,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97699999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.503,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.91400000000000003,"5min":0.998,"15min":0.998},"last":0.49399999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65800000000000003,"15min":0.67300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89300000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.871,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.59499999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64700000000000002,"15min":0.66000000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94999999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91800000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.54000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9580,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362260,"statfs":{"total":96636764160,"available":96626954240,"internally_reserved":0,"allocated":757760,"data_stored":336562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.50600000000000001,"15min":0.56399999999999995},"min":{"1min":0.26400000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.42999999999999999,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.312,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51600000000000001,"5min":1.667,"15min":1.667},"last":0.35899999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.53100000000000003,"15min":0.58199999999999996},"min":{"1min":0.31900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.47699999999999998,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42999999999999999,"15min":0.437},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53500000000000003,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.308}]},{"osd":3,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.432,"15min":0.44400000000000001},"min":{"1min":0.29999999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.49299999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.251},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.59499999999999997,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.52400000000000002,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.33400000000000002}]},{"osd":4,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79800000000000004,"5min":1.536,"15min":1.536},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.72199999999999998,"15min":0.76600000000000001},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76000000000000001,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.46500000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79300000000000004,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.624,"15min":0.62},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.78600000000000003,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.53500000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74299999999999999,"15min":0.78100000000000003},"min":{"1min":0.441,"5min":0.44,"15min":0.44},"max":{"1min":0.81899999999999995,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63200000000000001,"15min":0.62},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80800000000000005,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.51400000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.76000000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.755,"5min":1.605,"15min":1.605},"last":0.49299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13548,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":12800,"kb_avail":94358292,"statfs":{"total":96636764160,"available":96622891008,"internally_reserved":0,"allocated":757760,"data_stored":336562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13107200},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.375},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58999999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.309},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40100000000000002},"min":{"1min":0.318,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.25800000000000001}]},{"osd":1,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42099999999999999,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.60899999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.54800000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.28699999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41299999999999998,"15min":0.40999999999999998},"min":{"1min":0.31,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.621,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.432,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.627,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.59799999999999998}]},{"osd":5,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.503},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.625},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.51300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68799999999999994}]},{"osd":7,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.63700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13932,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":13184,"kb_avail":94357908,"statfs":{"total":96636764160,"available":96622497792,"internally_reserved":0,"allocated":757760,"data_stored":336562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13500416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.29599999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42299999999999999,"15min":0.441},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.495,"5min":1.238,"15min":1.238},"last":0.36499999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42799999999999999,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56799999999999995,"5min":1.53,"15min":1.53},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.48599999999999999,"15min":0.51000000000000001},"min":{"1min":0.27100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.622,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.44700000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47599999999999998,"15min":0.49199999999999999},"min":{"1min":0.34000000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60299999999999998,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.48499999999999999},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60199999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79000000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.51600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91900000000000004,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.70799999999999996}]},{"osd":5,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67900000000000005,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.99299999999999999,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.51900000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.879,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.503}]},{"osd":6,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92300000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84699999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.755}]},{"osd":7,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.94299999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67400000000000004,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9656,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362184,"statfs":{"total":96636764160,"available":96626876416,"internally_reserved":0,"allocated":770048,"data_stored":338110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53700000000000003,"15min":0.52200000000000002},"min":{"1min":0.44800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61499999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.54100000000000004}]},{"osd":2,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.628},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76000000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.57399999999999995}]},{"osd":3,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.70299999999999996}]},{"osd":5,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68999999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.503,"15min":0.497},"min":{"1min":0.35199999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48399999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.498,"15min":0.5},"min":{"1min":0.39100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.503}]}]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9520,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362320,"statfs":{"total":96636764160,"available":96627015680,"internally_reserved":0,"allocated":761856,"data_stored":336584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71199999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.504},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.443,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.56100000000000005}]},{"osd":1,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.63600000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.80500000000000005,"15min":0.91900000000000004},"min":{"1min":0.45200000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74299999999999999,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.53600000000000003}]},{"osd":2,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.64800000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.432,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77800000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51400000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.64600000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46700000000000003,"15min":0.47299999999999998},"min":{"1min":0.36099999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66400000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51400000000000001,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66400000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.376}]},{"osd":6,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.50600000000000001,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.65300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.439},{"interface":"front","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.375,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63300000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.46700000000000003}]},{"osd":7,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53600000000000003,"15min":0.54600000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68899999999999995,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.57499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:05:01.251 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:05:01.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.394+0000 7f4c54cd0700 1 Processor -- start 2023-11-28T12:05:01.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.395+0000 7f4c54cd0700 1 -- start start 2023-11-28T12:05:01.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.395+0000 7f4c54cd0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c500ce040 0x7f4c5015d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:01.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c54cd0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4c500b1770 0x7f4c500b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:01.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c54cd0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5013fc90 con 0x7f4c50142db0 2023-11-28T12:05:01.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c54cd0700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c500b2130 con 0x7f4c500b1770 2023-11-28T12:05:01.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c54cd0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5013f480 con 0x7f4c500ce040 2023-11-28T12:05:01.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c4ed9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4c500b1770 0x7f4c500b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:01.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c4ed9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4c500b1770 0x7f4c500b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46416/0 (socket says 172.21.15.57:46416) 2023-11-28T12:05:01.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c4ed9d700 1 -- 172.21.15.57:0/2091130138 learned_addr learned my addr 172.21.15.57:0/2091130138 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:01.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c4e59c700 1 --2- 172.21.15.57:0/2091130138 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c500ce040 0x7f4c5015d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:01.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c4ed9d700 1 -- 172.21.15.57:0/2091130138 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c500ce040 msgr2=0x7f4c5015d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:01.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c4ed9d700 1 --2- 172.21.15.57:0/2091130138 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c500ce040 0x7f4c5015d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c4ed9d700 1 -- 172.21.15.57:0/2091130138 >> v1:172.21.15.181:6789/0 conn(0x7f4c50142db0 legacy=0x7f4c5013ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:01.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c4ed9d700 1 -- 172.21.15.57:0/2091130138 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5013f6d0 con 0x7f4c500b1770 2023-11-28T12:05:01.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c4ed9d700 1 --2- 172.21.15.57:0/2091130138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c500b1770 0x7f4c500b1bf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800d150 tx=0x7f4c4800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84a6a3399de7bbc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:01.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c4d59a700 1 -- 172.21.15.57:0/2091130138 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48010070 con 0x7f4c500b1770 2023-11-28T12:05:01.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c4d59a700 1 -- 172.21.15.57:0/2091130138 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4800ddc0 con 0x7f4c500b1770 2023-11-28T12:05:01.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.396+0000 7f4c4d59a700 1 -- 172.21.15.57:0/2091130138 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4800b310 con 0x7f4c500b1770 2023-11-28T12:05:01.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.397+0000 7f4c54cd0700 1 -- 172.21.15.57:0/2091130138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c500b1770 msgr2=0x7f4c500b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:01.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.397+0000 7f4c54cd0700 1 --2- 172.21.15.57:0/2091130138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c500b1770 0x7f4c500b1bf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800d150 tx=0x7f4c4800d460 comp rx=0 tx=0).stop 2023-11-28T12:05:01.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.397+0000 7f4c54cd0700 1 -- 172.21.15.57:0/2091130138 shutdown_connections 2023-11-28T12:05:01.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.397+0000 7f4c54cd0700 1 --2- 172.21.15.57:0/2091130138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c500b1770 0x7f4c500b1bf0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.397+0000 7f4c54cd0700 1 --2- 172.21.15.57:0/2091130138 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c500ce040 0x7f4c5015d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.397+0000 7f4c54cd0700 1 -- 172.21.15.57:0/2091130138 >> 172.21.15.57:0/2091130138 conn(0x7f4c5006c010 msgr2=0x7f4c500cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:01.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.397+0000 7f4c54cd0700 1 -- 172.21.15.57:0/2091130138 shutdown_connections 2023-11-28T12:05:01.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.397+0000 7f4c54cd0700 1 -- 172.21.15.57:0/2091130138 wait complete. 2023-11-28T12:05:01.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.399+0000 7f4c54cd0700 1 Processor -- start 2023-11-28T12:05:01.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.399+0000 7f4c54cd0700 1 -- start start 2023-11-28T12:05:01.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.399+0000 7f4c54cd0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c500b1770 0x7f4c500c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:01.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.399+0000 7f4c54cd0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c500ce040 0x7f4c501f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:01.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.399+0000 7f4c54cd0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c50142db0 0x7f4c500c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:01.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.399+0000 7f4c54cd0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c500c60f0 con 0x7f4c50142db0 2023-11-28T12:05:01.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.399+0000 7f4c54cd0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c500c54e0 con 0x7f4c500ce040 2023-11-28T12:05:01.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.399+0000 7f4c54cd0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c500c57c0 con 0x7f4c500b1770 2023-11-28T12:05:01.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c4ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c50142db0 0x7f4c500c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:01.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c4ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c50142db0 0x7f4c500c3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46420/0 (socket says 172.21.15.57:46420) 2023-11-28T12:05:01.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c4ed9d700 1 -- 172.21.15.57:0/732623635 learned_addr learned my addr 172.21.15.57:0/732623635 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:01.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c4dd9b700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c500ce040 0x7f4c501f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:01.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c4e59c700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c500b1770 0x7f4c500c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:01.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c4ed9d700 1 -- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c500b1770 msgr2=0x7f4c500c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:01.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c4ed9d700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c500b1770 0x7f4c500c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c4ed9d700 1 -- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c500ce040 msgr2=0x7f4c501f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:01.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c4ed9d700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c500ce040 0x7f4c501f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c4ed9d700 1 -- 172.21.15.57:0/732623635 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4800bd40 con 0x7f4c50142db0 2023-11-28T12:05:01.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c4dd9b700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c500ce040 0x7f4c501f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:01.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c4e59c700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c500b1770 0x7f4c500c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:01.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c4ed9d700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c50142db0 0x7f4c500c3400 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4c50070850 tx=0x7f4c48004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:01.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c46ffd700 1 -- 172.21.15.57:0/732623635 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4800b4c0 con 0x7f4c50142db0 2023-11-28T12:05:01.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c46ffd700 1 -- 172.21.15.57:0/732623635 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c48009810 con 0x7f4c50142db0 2023-11-28T12:05:01.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c54cd0700 1 -- 172.21.15.57:0/732623635 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5015d100 con 0x7f4c50142db0 2023-11-28T12:05:01.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c46ffd700 1 -- 172.21.15.57:0/732623635 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48009bc0 con 0x7f4c50142db0 2023-11-28T12:05:01.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.400+0000 7f4c54cd0700 1 -- 172.21.15.57:0/732623635 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5013f9b0 con 0x7f4c50142db0 2023-11-28T12:05:01.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.401+0000 7f4c54cd0700 1 -- 172.21.15.57:0/732623635 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5015d3e0 con 0x7f4c50142db0 2023-11-28T12:05:01.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.402+0000 7f4c46ffd700 1 -- 172.21.15.57:0/732623635 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4c48009d20 con 0x7f4c50142db0 2023-11-28T12:05:01.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.402+0000 7f4c46ffd700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4c300682f0 0x7f4c3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:01.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.402+0000 7f4c46ffd700 1 -- 172.21.15.57:0/732623635 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c48087350 con 0x7f4c50142db0 2023-11-28T12:05:01.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.403+0000 7f4c4e59c700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4c300682f0 0x7f4c3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:01.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.403+0000 7f4c4e59c700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4c300682f0 0x7f4c3006a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4c400099f0 tx=0x7f4c40008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:01.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.407+0000 7f4c46ffd700 1 -- 172.21.15.57:0/732623635 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4c48053bb0 con 0x7f4c50142db0 2023-11-28T12:05:01.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.600+0000 7f4c54cd0700 1 -- 172.21.15.57:0/732623635 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c500743f0 con 0x7f4c300682f0 2023-11-28T12:05:01.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.609+0000 7f4c46ffd700 1 -- 172.21.15.57:0/732623635 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311988 (secure 0 0 0) 0x7f4c500743f0 con 0x7f4c300682f0 2023-11-28T12:05:01.614 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:05:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.612+0000 7f4c54cd0700 1 -- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4c300682f0 msgr2=0x7f4c3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.612+0000 7f4c54cd0700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4c300682f0 0x7f4c3006a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4c400099f0 tx=0x7f4c40008040 comp rx=0 tx=0).stop 2023-11-28T12:05:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.612+0000 7f4c54cd0700 1 -- 172.21.15.57:0/732623635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c50142db0 msgr2=0x7f4c500c3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.612+0000 7f4c54cd0700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c50142db0 0x7f4c500c3400 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4c50070850 tx=0x7f4c48004ad0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.612+0000 7f4c54cd0700 1 -- 172.21.15.57:0/732623635 shutdown_connections 2023-11-28T12:05:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.612+0000 7f4c54cd0700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4c300682f0 0x7f4c3006a7a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.612+0000 7f4c54cd0700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c50142db0 0x7f4c500c3400 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.612+0000 7f4c54cd0700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c500b1770 0x7f4c500c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.612+0000 7f4c54cd0700 1 --2- 172.21.15.57:0/732623635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c500ce040 0x7f4c501f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.612+0000 7f4c54cd0700 1 -- 172.21.15.57:0/732623635 >> 172.21.15.57:0/732623635 conn(0x7f4c5006c010 msgr2=0x7f4c500cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.612+0000 7f4c54cd0700 1 -- 172.21.15.57:0/732623635 shutdown_connections 2023-11-28T12:05:01.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.612+0000 7f4c54cd0700 1 -- 172.21.15.57:0/732623635 wait complete. 2023-11-28T12:05:01.618 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:05:01.627 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2023-11-28T12:04:59.708956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":87016,"kb_used_data":7144,"kb_used_omap":0,"kb_used_meta":79808,"kb_avail":754887704,"statfs":{"total":773094113280,"available":773005008896,"internally_reserved":0,"allocated":7315456,"data_stored":3877892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81723392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.787265"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.747247+0000","last_change":"2023-11-28T12:04:52.391013+0000","last_active":"2023-11-28T12:04:54.747247+0000","last_peered":"2023-11-28T12:04:54.747247+0000","last_clean":"2023-11-28T12:04:54.747247+0000","last_became_active":"2023-11-28T12:04:52.386457+0000","last_became_peered":"2023-11-28T12:04:52.386457+0000","last_unstale":"2023-11-28T12:04:54.747247+0000","last_undegraded":"2023-11-28T12:04:54.747247+0000","last_fullsized":"2023-11-28T12:04:54.747247+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:14:26.740655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.747121+0000","last_change":"2023-11-28T12:04:52.389120+0000","last_active":"2023-11-28T12:04:54.747121+0000","last_peered":"2023-11-28T12:04:54.747121+0000","last_clean":"2023-11-28T12:04:54.747121+0000","last_became_active":"2023-11-28T12:04:52.388206+0000","last_became_peered":"2023-11-28T12:04:52.388206+0000","last_unstale":"2023-11-28T12:04:54.747121+0000","last_undegraded":"2023-11-28T12:04:54.747121+0000","last_fullsized":"2023-11-28T12:04:54.747121+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:59:12.567356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272642+0000","last_change":"2023-11-28T12:04:52.384787+0000","last_active":"2023-11-28T12:04:55.272642+0000","last_peered":"2023-11-28T12:04:55.272642+0000","last_clean":"2023-11-28T12:04:55.272642+0000","last_became_active":"2023-11-28T12:04:52.384361+0000","last_became_peered":"2023-11-28T12:04:52.384361+0000","last_unstale":"2023-11-28T12:04:55.272642+0000","last_undegraded":"2023-11-28T12:04:55.272642+0000","last_fullsized":"2023-11-28T12:04:55.272642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:02.952081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.378249+0000","last_change":"2023-11-28T12:04:52.387408+0000","last_active":"2023-11-28T12:04:53.378249+0000","last_peered":"2023-11-28T12:04:53.378249+0000","last_clean":"2023-11-28T12:04:53.378249+0000","last_became_active":"2023-11-28T12:04:52.384005+0000","last_became_peered":"2023-11-28T12:04:52.384005+0000","last_unstale":"2023-11-28T12:04:53.378249+0000","last_undegraded":"2023-11-28T12:04:53.378249+0000","last_fullsized":"2023-11-28T12:04:53.378249+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:45.027098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377728+0000","last_change":"2023-11-28T12:04:52.375579+0000","last_active":"2023-11-28T12:04:53.377728+0000","last_peered":"2023-11-28T12:04:53.377728+0000","last_clean":"2023-11-28T12:04:53.377728+0000","last_became_active":"2023-11-28T12:04:52.375281+0000","last_became_peered":"2023-11-28T12:04:52.375281+0000","last_unstale":"2023-11-28T12:04:53.377728+0000","last_undegraded":"2023-11-28T12:04:53.377728+0000","last_fullsized":"2023-11-28T12:04:53.377728+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:16:17.346419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.824337+0000","last_change":"2023-11-28T12:04:52.382894+0000","last_active":"2023-11-28T12:04:54.824337+0000","last_peered":"2023-11-28T12:04:54.824337+0000","last_clean":"2023-11-28T12:04:54.824337+0000","last_became_active":"2023-11-28T12:04:52.382468+0000","last_became_peered":"2023-11-28T12:04:52.382468+0000","last_unstale":"2023-11-28T12:04:54.824337+0000","last_undegraded":"2023-11-28T12:04:54.824337+0000","last_fullsized":"2023-11-28T12:04:54.824337+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:33.123525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.518005+0000","last_change":"2023-11-28T12:04:52.389377+0000","last_active":"2023-11-28T12:04:53.518005+0000","last_peered":"2023-11-28T12:04:53.518005+0000","last_clean":"2023-11-28T12:04:53.518005+0000","last_became_active":"2023-11-28T12:04:52.389094+0000","last_became_peered":"2023-11-28T12:04:52.389094+0000","last_unstale":"2023-11-28T12:04:53.518005+0000","last_undegraded":"2023-11-28T12:04:53.518005+0000","last_fullsized":"2023-11-28T12:04:53.518005+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:02.757783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271031+0000","last_change":"2023-11-28T12:04:52.383250+0000","last_active":"2023-11-28T12:04:55.271031+0000","last_peered":"2023-11-28T12:04:55.271031+0000","last_clean":"2023-11-28T12:04:55.271031+0000","last_became_active":"2023-11-28T12:04:52.382481+0000","last_became_peered":"2023-11-28T12:04:52.382481+0000","last_unstale":"2023-11-28T12:04:55.271031+0000","last_undegraded":"2023-11-28T12:04:55.271031+0000","last_fullsized":"2023-11-28T12:04:55.271031+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:34:16.533129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272870+0000","last_change":"2023-11-28T12:04:52.389880+0000","last_active":"2023-11-28T12:04:55.272870+0000","last_peered":"2023-11-28T12:04:55.272870+0000","last_clean":"2023-11-28T12:04:55.272870+0000","last_became_active":"2023-11-28T12:04:52.389308+0000","last_became_peered":"2023-11-28T12:04:52.389308+0000","last_unstale":"2023-11-28T12:04:55.272870+0000","last_undegraded":"2023-11-28T12:04:55.272870+0000","last_fullsized":"2023-11-28T12:04:55.272870+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:21.970184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272555+0000","last_change":"2023-11-28T12:04:52.391554+0000","last_active":"2023-11-28T12:04:55.272555+0000","last_peered":"2023-11-28T12:04:55.272555+0000","last_clean":"2023-11-28T12:04:55.272555+0000","last_became_active":"2023-11-28T12:04:52.391299+0000","last_became_peered":"2023-11-28T12:04:52.391299+0000","last_unstale":"2023-11-28T12:04:55.272555+0000","last_undegraded":"2023-11-28T12:04:55.272555+0000","last_fullsized":"2023-11-28T12:04:55.272555+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:08.183501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377922+0000","last_change":"2023-11-28T12:04:52.391075+0000","last_active":"2023-11-28T12:04:53.377922+0000","last_peered":"2023-11-28T12:04:53.377922+0000","last_clean":"2023-11-28T12:04:53.377922+0000","last_became_active":"2023-11-28T12:04:52.390792+0000","last_became_peered":"2023-11-28T12:04:52.390792+0000","last_unstale":"2023-11-28T12:04:53.377922+0000","last_undegraded":"2023-11-28T12:04:53.377922+0000","last_fullsized":"2023-11-28T12:04:53.377922+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:02.952081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271248+0000","last_change":"2023-11-28T12:04:52.389229+0000","last_active":"2023-11-28T12:04:55.271248+0000","last_peered":"2023-11-28T12:04:55.271248+0000","last_clean":"2023-11-28T12:04:55.271248+0000","last_became_active":"2023-11-28T12:04:52.388960+0000","last_became_peered":"2023-11-28T12:04:52.388960+0000","last_unstale":"2023-11-28T12:04:55.271248+0000","last_undegraded":"2023-11-28T12:04:55.271248+0000","last_fullsized":"2023-11-28T12:04:55.271248+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:29:42.869610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271194+0000","last_change":"2023-11-28T12:04:52.374641+0000","last_active":"2023-11-28T12:04:55.271194+0000","last_peered":"2023-11-28T12:04:55.271194+0000","last_clean":"2023-11-28T12:04:55.271194+0000","last_became_active":"2023-11-28T12:04:52.374351+0000","last_became_peered":"2023-11-28T12:04:52.374351+0000","last_unstale":"2023-11-28T12:04:55.271194+0000","last_undegraded":"2023-11-28T12:04:55.271194+0000","last_fullsized":"2023-11-28T12:04:55.271194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:21.970184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272487+0000","last_change":"2023-11-28T12:04:52.384898+0000","last_active":"2023-11-28T12:04:55.272487+0000","last_peered":"2023-11-28T12:04:55.272487+0000","last_clean":"2023-11-28T12:04:55.272487+0000","last_became_active":"2023-11-28T12:04:52.384398+0000","last_became_peered":"2023-11-28T12:04:52.384398+0000","last_unstale":"2023-11-28T12:04:55.272487+0000","last_undegraded":"2023-11-28T12:04:55.272487+0000","last_fullsized":"2023-11-28T12:04:55.272487+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:45.027098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272622+0000","last_change":"2023-11-28T12:04:52.393915+0000","last_active":"2023-11-28T12:04:55.272622+0000","last_peered":"2023-11-28T12:04:55.272622+0000","last_clean":"2023-11-28T12:04:55.272622+0000","last_became_active":"2023-11-28T12:04:52.393655+0000","last_became_peered":"2023-11-28T12:04:52.393655+0000","last_unstale":"2023-11-28T12:04:55.272622+0000","last_undegraded":"2023-11-28T12:04:55.272622+0000","last_fullsized":"2023-11-28T12:04:55.272622+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:25.673535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.517952+0000","last_change":"2023-11-28T12:04:52.379165+0000","last_active":"2023-11-28T12:04:53.517952+0000","last_peered":"2023-11-28T12:04:53.517952+0000","last_clean":"2023-11-28T12:04:53.517952+0000","last_became_active":"2023-11-28T12:04:52.375881+0000","last_became_peered":"2023-11-28T12:04:52.375881+0000","last_unstale":"2023-11-28T12:04:53.517952+0000","last_undegraded":"2023-11-28T12:04:53.517952+0000","last_fullsized":"2023-11-28T12:04:53.517952+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:12:27.795181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.747352+0000","last_change":"2023-11-28T12:04:52.385438+0000","last_active":"2023-11-28T12:04:54.747352+0000","last_peered":"2023-11-28T12:04:54.747352+0000","last_clean":"2023-11-28T12:04:54.747352+0000","last_became_active":"2023-11-28T12:04:52.385167+0000","last_became_peered":"2023-11-28T12:04:52.385167+0000","last_unstale":"2023-11-28T12:04:54.747352+0000","last_undegraded":"2023-11-28T12:04:54.747352+0000","last_fullsized":"2023-11-28T12:04:54.747352+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:21.970184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377997+0000","last_change":"2023-11-28T12:04:52.384637+0000","last_active":"2023-11-28T12:04:53.377997+0000","last_peered":"2023-11-28T12:04:53.377997+0000","last_clean":"2023-11-28T12:04:53.377997+0000","last_became_active":"2023-11-28T12:04:52.383699+0000","last_became_peered":"2023-11-28T12:04:52.383699+0000","last_unstale":"2023-11-28T12:04:53.377997+0000","last_undegraded":"2023-11-28T12:04:53.377997+0000","last_fullsized":"2023-11-28T12:04:53.377997+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:25.673535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377627+0000","last_change":"2023-11-28T12:04:52.382384+0000","last_active":"2023-11-28T12:04:53.377627+0000","last_peered":"2023-11-28T12:04:53.377627+0000","last_clean":"2023-11-28T12:04:53.377627+0000","last_became_active":"2023-11-28T12:04:52.382095+0000","last_became_peered":"2023-11-28T12:04:52.382095+0000","last_unstale":"2023-11-28T12:04:53.377627+0000","last_undegraded":"2023-11-28T12:04:53.377627+0000","last_fullsized":"2023-11-28T12:04:53.377627+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:28.348156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.371843+0000","last_change":"2023-11-28T12:04:52.372907+0000","last_active":"2023-11-28T12:04:53.371843+0000","last_peered":"2023-11-28T12:04:53.371843+0000","last_clean":"2023-11-28T12:04:53.371843+0000","last_became_active":"2023-11-28T12:04:52.372662+0000","last_became_peered":"2023-11-28T12:04:52.372662+0000","last_unstale":"2023-11-28T12:04:53.371843+0000","last_undegraded":"2023-11-28T12:04:53.371843+0000","last_fullsized":"2023-11-28T12:04:53.371843+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:35:28.129566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271322+0000","last_change":"2023-11-28T12:04:52.382559+0000","last_active":"2023-11-28T12:04:55.271322+0000","last_peered":"2023-11-28T12:04:55.271322+0000","last_clean":"2023-11-28T12:04:55.271322+0000","last_became_active":"2023-11-28T12:04:52.382278+0000","last_became_peered":"2023-11-28T12:04:52.382278+0000","last_unstale":"2023-11-28T12:04:55.271322+0000","last_undegraded":"2023-11-28T12:04:55.271322+0000","last_fullsized":"2023-11-28T12:04:55.271322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:53.408359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.371918+0000","last_change":"2023-11-28T12:04:52.384805+0000","last_active":"2023-11-28T12:04:53.371918+0000","last_peered":"2023-11-28T12:04:53.371918+0000","last_clean":"2023-11-28T12:04:53.371918+0000","last_became_active":"2023-11-28T12:04:52.384525+0000","last_became_peered":"2023-11-28T12:04:52.384525+0000","last_unstale":"2023-11-28T12:04:53.371918+0000","last_undegraded":"2023-11-28T12:04:53.371918+0000","last_fullsized":"2023-11-28T12:04:53.371918+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:02:34.621157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.371754+0000","last_change":"2023-11-28T12:04:52.389833+0000","last_active":"2023-11-28T12:04:53.371754+0000","last_peered":"2023-11-28T12:04:53.371754+0000","last_clean":"2023-11-28T12:04:53.371754+0000","last_became_active":"2023-11-28T12:04:52.389538+0000","last_became_peered":"2023-11-28T12:04:52.389538+0000","last_unstale":"2023-11-28T12:04:53.371754+0000","last_undegraded":"2023-11-28T12:04:53.371754+0000","last_fullsized":"2023-11-28T12:04:53.371754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:47:06.821673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272517+0000","last_change":"2023-11-28T12:04:52.385183+0000","last_active":"2023-11-28T12:04:55.272517+0000","last_peered":"2023-11-28T12:04:55.272517+0000","last_clean":"2023-11-28T12:04:55.272517+0000","last_became_active":"2023-11-28T12:04:52.384407+0000","last_became_peered":"2023-11-28T12:04:52.384407+0000","last_unstale":"2023-11-28T12:04:55.272517+0000","last_undegraded":"2023-11-28T12:04:55.272517+0000","last_fullsized":"2023-11-28T12:04:55.272517+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:37.694660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271468+0000","last_change":"2023-11-28T12:04:52.380426+0000","last_active":"2023-11-28T12:04:55.271468+0000","last_peered":"2023-11-28T12:04:55.271468+0000","last_clean":"2023-11-28T12:04:55.271468+0000","last_became_active":"2023-11-28T12:04:52.380112+0000","last_became_peered":"2023-11-28T12:04:52.380112+0000","last_unstale":"2023-11-28T12:04:55.271468+0000","last_undegraded":"2023-11-28T12:04:55.271468+0000","last_fullsized":"2023-11-28T12:04:55.271468+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:02.952081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271292+0000","last_change":"2023-11-28T12:04:52.371983+0000","last_active":"2023-11-28T12:04:55.271292+0000","last_peered":"2023-11-28T12:04:55.271292+0000","last_clean":"2023-11-28T12:04:55.271292+0000","last_became_active":"2023-11-28T12:04:52.371719+0000","last_became_peered":"2023-11-28T12:04:52.371719+0000","last_unstale":"2023-11-28T12:04:55.271292+0000","last_undegraded":"2023-11-28T12:04:55.271292+0000","last_fullsized":"2023-11-28T12:04:55.271292+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:37.694660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377988+0000","last_change":"2023-11-28T12:04:52.386382+0000","last_active":"2023-11-28T12:04:53.377988+0000","last_peered":"2023-11-28T12:04:53.377988+0000","last_clean":"2023-11-28T12:04:53.377988+0000","last_became_active":"2023-11-28T12:04:52.386100+0000","last_became_peered":"2023-11-28T12:04:52.386100+0000","last_unstale":"2023-11-28T12:04:53.377988+0000","last_undegraded":"2023-11-28T12:04:53.377988+0000","last_fullsized":"2023-11-28T12:04:53.377988+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:48.028736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.747184+0000","last_change":"2023-11-28T12:04:52.388852+0000","last_active":"2023-11-28T12:04:54.747184+0000","last_peered":"2023-11-28T12:04:54.747184+0000","last_clean":"2023-11-28T12:04:54.747184+0000","last_became_active":"2023-11-28T12:04:52.388577+0000","last_became_peered":"2023-11-28T12:04:52.388577+0000","last_unstale":"2023-11-28T12:04:54.747184+0000","last_undegraded":"2023-11-28T12:04:54.747184+0000","last_fullsized":"2023-11-28T12:04:54.747184+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:53.408359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272556+0000","last_change":"2023-11-28T12:04:52.387833+0000","last_active":"2023-11-28T12:04:55.272556+0000","last_peered":"2023-11-28T12:04:55.272556+0000","last_clean":"2023-11-28T12:04:55.272556+0000","last_became_active":"2023-11-28T12:04:52.387501+0000","last_became_peered":"2023-11-28T12:04:52.387501+0000","last_unstale":"2023-11-28T12:04:55.272556+0000","last_undegraded":"2023-11-28T12:04:55.272556+0000","last_fullsized":"2023-11-28T12:04:55.272556+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:34:16.533129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271952+0000","last_change":"2023-11-28T12:04:52.383408+0000","last_active":"2023-11-28T12:04:55.271952+0000","last_peered":"2023-11-28T12:04:55.271952+0000","last_clean":"2023-11-28T12:04:55.271952+0000","last_became_active":"2023-11-28T12:04:52.382571+0000","last_became_peered":"2023-11-28T12:04:52.382571+0000","last_unstale":"2023-11-28T12:04:55.271952+0000","last_undegraded":"2023-11-28T12:04:55.271952+0000","last_fullsized":"2023-11-28T12:04:55.271952+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:43:03.206592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377016+0000","last_change":"2023-11-28T12:04:52.387026+0000","last_active":"2023-11-28T12:04:53.377016+0000","last_peered":"2023-11-28T12:04:53.377016+0000","last_clean":"2023-11-28T12:04:53.377016+0000","last_became_active":"2023-11-28T12:04:52.385351+0000","last_became_peered":"2023-11-28T12:04:52.385351+0000","last_unstale":"2023-11-28T12:04:53.377016+0000","last_undegraded":"2023-11-28T12:04:53.377016+0000","last_fullsized":"2023-11-28T12:04:53.377016+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:34:16.533129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.824149+0000","last_change":"2023-11-28T12:04:52.384434+0000","last_active":"2023-11-28T12:04:54.824149+0000","last_peered":"2023-11-28T12:04:54.824149+0000","last_clean":"2023-11-28T12:04:54.824149+0000","last_became_active":"2023-11-28T12:04:52.384128+0000","last_became_peered":"2023-11-28T12:04:52.384128+0000","last_unstale":"2023-11-28T12:04:54.824149+0000","last_undegraded":"2023-11-28T12:04:54.824149+0000","last_fullsized":"2023-11-28T12:04:54.824149+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:53:53.840908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377495+0000","last_change":"2023-11-28T12:04:52.391481+0000","last_active":"2023-11-28T12:04:53.377495+0000","last_peered":"2023-11-28T12:04:53.377495+0000","last_clean":"2023-11-28T12:04:53.377495+0000","last_became_active":"2023-11-28T12:04:52.391179+0000","last_became_peered":"2023-11-28T12:04:52.391179+0000","last_unstale":"2023-11-28T12:04:53.377495+0000","last_undegraded":"2023-11-28T12:04:53.377495+0000","last_fullsized":"2023-11-28T12:04:53.377495+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:59:12.567356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"62'4","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.392550+0000","last_change":"2023-11-28T12:04:51.944589+0000","last_active":"2023-11-28T12:04:52.392550+0000","last_peered":"2023-11-28T12:04:52.392550+0000","last_clean":"2023-11-28T12:04:52.392550+0000","last_became_active":"2023-11-28T12:04:51.944202+0000","last_became_peered":"2023-11-28T12:04:51.944202+0000","last_unstale":"2023-11-28T12:04:52.392550+0000","last_undegraded":"2023-11-28T12:04:52.392550+0000","last_fullsized":"2023-11-28T12:04:52.392550+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:15:52.635652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.378041+0000","last_change":"2023-11-28T12:04:52.378216+0000","last_active":"2023-11-28T12:04:53.378041+0000","last_peered":"2023-11-28T12:04:53.378041+0000","last_clean":"2023-11-28T12:04:53.378041+0000","last_became_active":"2023-11-28T12:04:52.377807+0000","last_became_peered":"2023-11-28T12:04:52.377807+0000","last_unstale":"2023-11-28T12:04:53.378041+0000","last_undegraded":"2023-11-28T12:04:53.378041+0000","last_fullsized":"2023-11-28T12:04:53.378041+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:06.246656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.393297+0000","last_change":"2023-11-28T12:04:51.939589+0000","last_active":"2023-11-28T12:04:52.393297+0000","last_peered":"2023-11-28T12:04:52.393297+0000","last_clean":"2023-11-28T12:04:52.393297+0000","last_became_active":"2023-11-28T12:04:51.939256+0000","last_became_peered":"2023-11-28T12:04:51.939256+0000","last_unstale":"2023-11-28T12:04:52.393297+0000","last_undegraded":"2023-11-28T12:04:52.393297+0000","last_fullsized":"2023-11-28T12:04:52.393297+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:00.055068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.824644+0000","last_change":"2023-11-28T12:04:52.386301+0000","last_active":"2023-11-28T12:04:54.824644+0000","last_peered":"2023-11-28T12:04:54.824644+0000","last_clean":"2023-11-28T12:04:54.824644+0000","last_became_active":"2023-11-28T12:04:52.384226+0000","last_became_peered":"2023-11-28T12:04:52.384226+0000","last_unstale":"2023-11-28T12:04:54.824644+0000","last_undegraded":"2023-11-28T12:04:54.824644+0000","last_fullsized":"2023-11-28T12:04:54.824644+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:55:46.190010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.383119+0000","last_change":"2023-11-28T12:04:51.935882+0000","last_active":"2023-11-28T12:04:52.383119+0000","last_peered":"2023-11-28T12:04:52.383119+0000","last_clean":"2023-11-28T12:04:52.383119+0000","last_became_active":"2023-11-28T12:04:51.934809+0000","last_became_peered":"2023-11-28T12:04:51.934809+0000","last_unstale":"2023-11-28T12:04:52.383119+0000","last_undegraded":"2023-11-28T12:04:52.383119+0000","last_fullsized":"2023-11-28T12:04:52.383119+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:05.119970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272939+0000","last_change":"2023-11-28T12:04:52.391923+0000","last_active":"2023-11-28T12:04:55.272939+0000","last_peered":"2023-11-28T12:04:55.272939+0000","last_clean":"2023-11-28T12:04:55.272939+0000","last_became_active":"2023-11-28T12:04:52.391688+0000","last_became_peered":"2023-11-28T12:04:52.391688+0000","last_unstale":"2023-11-28T12:04:55.272939+0000","last_undegraded":"2023-11-28T12:04:55.272939+0000","last_fullsized":"2023-11-28T12:04:55.272939+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:01.067609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.387644+0000","last_change":"2023-11-28T12:04:51.941486+0000","last_active":"2023-11-28T12:04:52.387644+0000","last_peered":"2023-11-28T12:04:52.387644+0000","last_clean":"2023-11-28T12:04:52.387644+0000","last_became_active":"2023-11-28T12:04:51.941144+0000","last_became_peered":"2023-11-28T12:04:51.941144+0000","last_unstale":"2023-11-28T12:04:52.387644+0000","last_undegraded":"2023-11-28T12:04:52.387644+0000","last_fullsized":"2023-11-28T12:04:52.387644+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:48.111122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.518069+0000","last_change":"2023-11-28T12:04:52.383946+0000","last_active":"2023-11-28T12:04:53.518069+0000","last_peered":"2023-11-28T12:04:53.518069+0000","last_clean":"2023-11-28T12:04:53.518069+0000","last_became_active":"2023-11-28T12:04:52.383666+0000","last_became_peered":"2023-11-28T12:04:52.383666+0000","last_unstale":"2023-11-28T12:04:53.518069+0000","last_undegraded":"2023-11-28T12:04:53.518069+0000","last_fullsized":"2023-11-28T12:04:53.518069+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:28.371921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.378932+0000","last_change":"2023-11-28T12:04:51.938573+0000","last_active":"2023-11-28T12:04:52.378932+0000","last_peered":"2023-11-28T12:04:52.378932+0000","last_clean":"2023-11-28T12:04:52.378932+0000","last_became_active":"2023-11-28T12:04:51.937184+0000","last_became_peered":"2023-11-28T12:04:51.937184+0000","last_unstale":"2023-11-28T12:04:52.378932+0000","last_undegraded":"2023-11-28T12:04:52.378932+0000","last_fullsized":"2023-11-28T12:04:52.378932+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:02.722002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.746780+0000","last_change":"2023-11-28T12:04:52.385183+0000","last_active":"2023-11-28T12:04:54.746780+0000","last_peered":"2023-11-28T12:04:54.746780+0000","last_clean":"2023-11-28T12:04:54.746780+0000","last_became_active":"2023-11-28T12:04:52.383229+0000","last_became_peered":"2023-11-28T12:04:52.383229+0000","last_unstale":"2023-11-28T12:04:54.746780+0000","last_undegraded":"2023-11-28T12:04:54.746780+0000","last_fullsized":"2023-11-28T12:04:54.746780+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:16:17.346419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.381680+0000","last_change":"2023-11-28T12:04:51.935707+0000","last_active":"2023-11-28T12:04:52.381680+0000","last_peered":"2023-11-28T12:04:52.381680+0000","last_clean":"2023-11-28T12:04:52.381680+0000","last_became_active":"2023-11-28T12:04:51.935328+0000","last_became_peered":"2023-11-28T12:04:51.935328+0000","last_unstale":"2023-11-28T12:04:52.381680+0000","last_undegraded":"2023-11-28T12:04:52.381680+0000","last_fullsized":"2023-11-28T12:04:52.381680+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:51:24.026885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377312+0000","last_change":"2023-11-28T12:04:52.376721+0000","last_active":"2023-11-28T12:04:53.377312+0000","last_peered":"2023-11-28T12:04:53.377312+0000","last_clean":"2023-11-28T12:04:53.377312+0000","last_became_active":"2023-11-28T12:04:52.376451+0000","last_became_peered":"2023-11-28T12:04:52.376451+0000","last_unstale":"2023-11-28T12:04:53.377312+0000","last_undegraded":"2023-11-28T12:04:53.377312+0000","last_fullsized":"2023-11-28T12:04:53.377312+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:55.240985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.364546+0000","last_change":"2023-11-28T12:04:51.942735+0000","last_active":"2023-11-28T12:04:52.364546+0000","last_peered":"2023-11-28T12:04:52.364546+0000","last_clean":"2023-11-28T12:04:52.364546+0000","last_became_active":"2023-11-28T12:04:51.942397+0000","last_became_peered":"2023-11-28T12:04:51.942397+0000","last_unstale":"2023-11-28T12:04:52.364546+0000","last_undegraded":"2023-11-28T12:04:52.364546+0000","last_fullsized":"2023-11-28T12:04:52.364546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:49:50.803373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.372067+0000","last_change":"2023-11-28T12:04:52.375404+0000","last_active":"2023-11-28T12:04:53.372067+0000","last_peered":"2023-11-28T12:04:53.372067+0000","last_clean":"2023-11-28T12:04:53.372067+0000","last_became_active":"2023-11-28T12:04:52.375170+0000","last_became_peered":"2023-11-28T12:04:52.375170+0000","last_unstale":"2023-11-28T12:04:53.372067+0000","last_undegraded":"2023-11-28T12:04:53.372067+0000","last_fullsized":"2023-11-28T12:04:53.372067+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:25.580039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.367699+0000","last_change":"2023-11-28T12:04:51.941216+0000","last_active":"2023-11-28T12:04:52.367699+0000","last_peered":"2023-11-28T12:04:52.367699+0000","last_clean":"2023-11-28T12:04:52.367699+0000","last_became_active":"2023-11-28T12:04:51.940857+0000","last_became_peered":"2023-11-28T12:04:51.940857+0000","last_unstale":"2023-11-28T12:04:52.367699+0000","last_undegraded":"2023-11-28T12:04:52.367699+0000","last_fullsized":"2023-11-28T12:04:52.367699+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:30.385397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.746932+0000","last_change":"2023-11-28T12:04:52.388818+0000","last_active":"2023-11-28T12:04:54.746932+0000","last_peered":"2023-11-28T12:04:54.746932+0000","last_clean":"2023-11-28T12:04:54.746932+0000","last_became_active":"2023-11-28T12:04:52.388236+0000","last_became_peered":"2023-11-28T12:04:52.388236+0000","last_unstale":"2023-11-28T12:04:54.746932+0000","last_undegraded":"2023-11-28T12:04:54.746932+0000","last_fullsized":"2023-11-28T12:04:54.746932+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:48.028736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.379423+0000","last_change":"2023-11-28T12:04:51.945702+0000","last_active":"2023-11-28T12:04:52.379423+0000","last_peered":"2023-11-28T12:04:52.379423+0000","last_clean":"2023-11-28T12:04:52.379423+0000","last_became_active":"2023-11-28T12:04:51.945398+0000","last_became_peered":"2023-11-28T12:04:51.945398+0000","last_unstale":"2023-11-28T12:04:52.379423+0000","last_undegraded":"2023-11-28T12:04:52.379423+0000","last_fullsized":"2023-11-28T12:04:52.379423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:20.019844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.371918+0000","last_change":"2023-11-28T12:04:52.371074+0000","last_active":"2023-11-28T12:04:53.371918+0000","last_peered":"2023-11-28T12:04:53.371918+0000","last_clean":"2023-11-28T12:04:53.371918+0000","last_became_active":"2023-11-28T12:04:52.370751+0000","last_became_peered":"2023-11-28T12:04:52.370751+0000","last_unstale":"2023-11-28T12:04:53.371918+0000","last_undegraded":"2023-11-28T12:04:53.371918+0000","last_fullsized":"2023-11-28T12:04:53.371918+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:44:32.011577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.375010+0000","last_change":"2023-11-28T12:04:51.935817+0000","last_active":"2023-11-28T12:04:52.375010+0000","last_peered":"2023-11-28T12:04:52.375010+0000","last_clean":"2023-11-28T12:04:52.375010+0000","last_became_active":"2023-11-28T12:04:51.935546+0000","last_became_peered":"2023-11-28T12:04:51.935546+0000","last_unstale":"2023-11-28T12:04:52.375010+0000","last_undegraded":"2023-11-28T12:04:52.375010+0000","last_fullsized":"2023-11-28T12:04:52.375010+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:27.335615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.378247+0000","last_change":"2023-11-28T12:04:52.372158+0000","last_active":"2023-11-28T12:04:53.378247+0000","last_peered":"2023-11-28T12:04:53.378247+0000","last_clean":"2023-11-28T12:04:53.378247+0000","last_became_active":"2023-11-28T12:04:52.371714+0000","last_became_peered":"2023-11-28T12:04:52.371714+0000","last_unstale":"2023-11-28T12:04:53.378247+0000","last_undegraded":"2023-11-28T12:04:53.378247+0000","last_fullsized":"2023-11-28T12:04:53.378247+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:26.089026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"62'3","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.392142+0000","last_change":"2023-11-28T12:04:51.944007+0000","last_active":"2023-11-28T12:04:52.392142+0000","last_peered":"2023-11-28T12:04:52.392142+0000","last_clean":"2023-11-28T12:04:52.392142+0000","last_became_active":"2023-11-28T12:04:51.943700+0000","last_became_peered":"2023-11-28T12:04:51.943700+0000","last_unstale":"2023-11-28T12:04:52.392142+0000","last_undegraded":"2023-11-28T12:04:52.392142+0000","last_fullsized":"2023-11-28T12:04:52.392142+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:59.043060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.378307+0000","last_change":"2023-11-28T12:04:52.385932+0000","last_active":"2023-11-28T12:04:53.378307+0000","last_peered":"2023-11-28T12:04:53.378307+0000","last_clean":"2023-11-28T12:04:53.378307+0000","last_became_active":"2023-11-28T12:04:52.382785+0000","last_became_peered":"2023-11-28T12:04:52.382785+0000","last_unstale":"2023-11-28T12:04:53.378307+0000","last_undegraded":"2023-11-28T12:04:53.378307+0000","last_fullsized":"2023-11-28T12:04:53.378307+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:01.067609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"62'6","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.396071+0000","last_change":"2023-11-28T12:04:51.940572+0000","last_active":"2023-11-28T12:04:52.396071+0000","last_peered":"2023-11-28T12:04:52.396071+0000","last_clean":"2023-11-28T12:04:52.396071+0000","last_became_active":"2023-11-28T12:04:51.939863+0000","last_became_peered":"2023-11-28T12:04:51.939863+0000","last_unstale":"2023-11-28T12:04:52.396071+0000","last_undegraded":"2023-11-28T12:04:52.396071+0000","last_fullsized":"2023-11-28T12:04:52.396071+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377208+0000","last_change":"2023-11-28T12:04:52.385917+0000","last_active":"2023-11-28T12:04:53.377208+0000","last_peered":"2023-11-28T12:04:53.377208+0000","last_clean":"2023-11-28T12:04:53.377208+0000","last_became_active":"2023-11-28T12:04:52.385614+0000","last_became_peered":"2023-11-28T12:04:52.385614+0000","last_unstale":"2023-11-28T12:04:53.377208+0000","last_undegraded":"2023-11-28T12:04:53.377208+0000","last_fullsized":"2023-11-28T12:04:53.377208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:37.694660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.388967+0000","last_change":"2023-11-28T12:04:51.939230+0000","last_active":"2023-11-28T12:04:52.388967+0000","last_peered":"2023-11-28T12:04:52.388967+0000","last_clean":"2023-11-28T12:04:52.388967+0000","last_became_active":"2023-11-28T12:04:51.938945+0000","last_became_peered":"2023-11-28T12:04:51.938945+0000","last_unstale":"2023-11-28T12:04:52.388967+0000","last_undegraded":"2023-11-28T12:04:52.388967+0000","last_fullsized":"2023-11-28T12:04:52.388967+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:25.076484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271127+0000","last_change":"2023-11-28T12:04:52.376236+0000","last_active":"2023-11-28T12:04:55.271127+0000","last_peered":"2023-11-28T12:04:55.271127+0000","last_clean":"2023-11-28T12:04:55.271127+0000","last_became_active":"2023-11-28T12:04:52.375938+0000","last_became_peered":"2023-11-28T12:04:52.375938+0000","last_unstale":"2023-11-28T12:04:55.271127+0000","last_undegraded":"2023-11-28T12:04:55.271127+0000","last_fullsized":"2023-11-28T12:04:55.271127+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:45.027098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.366472+0000","last_change":"2023-11-28T12:04:51.943415+0000","last_active":"2023-11-28T12:04:52.366472+0000","last_peered":"2023-11-28T12:04:52.366472+0000","last_clean":"2023-11-28T12:04:52.366472+0000","last_became_active":"2023-11-28T12:04:51.943175+0000","last_became_peered":"2023-11-28T12:04:51.943175+0000","last_unstale":"2023-11-28T12:04:52.366472+0000","last_undegraded":"2023-11-28T12:04:52.366472+0000","last_fullsized":"2023-11-28T12:04:52.366472+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377409+0000","last_change":"2023-11-28T12:04:52.383381+0000","last_active":"2023-11-28T12:04:53.377409+0000","last_peered":"2023-11-28T12:04:53.377409+0000","last_clean":"2023-11-28T12:04:53.377409+0000","last_became_active":"2023-11-28T12:04:52.379867+0000","last_became_peered":"2023-11-28T12:04:52.379867+0000","last_unstale":"2023-11-28T12:04:53.377409+0000","last_undegraded":"2023-11-28T12:04:53.377409+0000","last_fullsized":"2023-11-28T12:04:53.377409+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:00:20.462743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.380545+0000","last_change":"2023-11-28T12:04:51.940514+0000","last_active":"2023-11-28T12:04:52.380545+0000","last_peered":"2023-11-28T12:04:52.380545+0000","last_clean":"2023-11-28T12:04:52.380545+0000","last_became_active":"2023-11-28T12:04:51.939694+0000","last_became_peered":"2023-11-28T12:04:51.939694+0000","last_unstale":"2023-11-28T12:04:52.380545+0000","last_undegraded":"2023-11-28T12:04:52.380545+0000","last_fullsized":"2023-11-28T12:04:52.380545+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.747005+0000","last_change":"2023-11-28T12:04:52.386666+0000","last_active":"2023-11-28T12:04:54.747005+0000","last_peered":"2023-11-28T12:04:54.747005+0000","last_clean":"2023-11-28T12:04:54.747005+0000","last_became_active":"2023-11-28T12:04:52.386329+0000","last_became_peered":"2023-11-28T12:04:52.386329+0000","last_unstale":"2023-11-28T12:04:54.747005+0000","last_undegraded":"2023-11-28T12:04:54.747005+0000","last_fullsized":"2023-11-28T12:04:54.747005+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:43:03.206592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"62'8","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.390122+0000","last_change":"2023-11-28T12:04:51.940379+0000","last_active":"2023-11-28T12:04:52.390122+0000","last_peered":"2023-11-28T12:04:52.390122+0000","last_clean":"2023-11-28T12:04:52.390122+0000","last_became_active":"2023-11-28T12:04:51.940118+0000","last_became_peered":"2023-11-28T12:04:51.940118+0000","last_unstale":"2023-11-28T12:04:52.390122+0000","last_undegraded":"2023-11-28T12:04:52.390122+0000","last_fullsized":"2023-11-28T12:04:52.390122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:25.076484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.824326+0000","last_change":"2023-11-28T12:04:52.393348+0000","last_active":"2023-11-28T12:04:54.824326+0000","last_peered":"2023-11-28T12:04:54.824326+0000","last_clean":"2023-11-28T12:04:54.824326+0000","last_became_active":"2023-11-28T12:04:52.393002+0000","last_became_peered":"2023-11-28T12:04:52.393002+0000","last_unstale":"2023-11-28T12:04:54.824326+0000","last_undegraded":"2023-11-28T12:04:54.824326+0000","last_fullsized":"2023-11-28T12:04:54.824326+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:33.184300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.383268+0000","last_change":"2023-11-28T12:04:51.939307+0000","last_active":"2023-11-28T12:04:52.383268+0000","last_peered":"2023-11-28T12:04:52.383268+0000","last_clean":"2023-11-28T12:04:52.383268+0000","last_became_active":"2023-11-28T12:04:51.939004+0000","last_became_peered":"2023-11-28T12:04:51.939004+0000","last_unstale":"2023-11-28T12:04:52.383268+0000","last_undegraded":"2023-11-28T12:04:52.383268+0000","last_fullsized":"2023-11-28T12:04:52.383268+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.518114+0000","last_change":"2023-11-28T12:04:52.383898+0000","last_active":"2023-11-28T12:04:53.518114+0000","last_peered":"2023-11-28T12:04:53.518114+0000","last_clean":"2023-11-28T12:04:53.518114+0000","last_became_active":"2023-11-28T12:04:52.383743+0000","last_became_peered":"2023-11-28T12:04:52.383743+0000","last_unstale":"2023-11-28T12:04:53.518114+0000","last_undegraded":"2023-11-28T12:04:53.518114+0000","last_fullsized":"2023-11-28T12:04:53.518114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:59:10.723237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.366188+0000","last_change":"2023-11-28T12:04:51.937807+0000","last_active":"2023-11-28T12:04:52.366188+0000","last_peered":"2023-11-28T12:04:52.366188+0000","last_clean":"2023-11-28T12:04:52.366188+0000","last_became_active":"2023-11-28T12:04:51.937514+0000","last_became_peered":"2023-11-28T12:04:51.937514+0000","last_unstale":"2023-11-28T12:04:52.366188+0000","last_undegraded":"2023-11-28T12:04:52.366188+0000","last_fullsized":"2023-11-28T12:04:52.366188+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:01.574232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271360+0000","last_change":"2023-11-28T12:04:52.384074+0000","last_active":"2023-11-28T12:04:55.271360+0000","last_peered":"2023-11-28T12:04:55.271360+0000","last_clean":"2023-11-28T12:04:55.271360+0000","last_became_active":"2023-11-28T12:04:52.383778+0000","last_became_peered":"2023-11-28T12:04:52.383778+0000","last_unstale":"2023-11-28T12:04:55.271360+0000","last_undegraded":"2023-11-28T12:04:55.271360+0000","last_fullsized":"2023-11-28T12:04:55.271360+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:14:26.740655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.375818+0000","last_change":"2023-11-28T12:04:51.934819+0000","last_active":"2023-11-28T12:04:52.375818+0000","last_peered":"2023-11-28T12:04:52.375818+0000","last_clean":"2023-11-28T12:04:52.375818+0000","last_became_active":"2023-11-28T12:04:51.934462+0000","last_became_peered":"2023-11-28T12:04:51.934462+0000","last_unstale":"2023-11-28T12:04:52.375818+0000","last_undegraded":"2023-11-28T12:04:52.375818+0000","last_fullsized":"2023-11-28T12:04:52.375818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:36.797091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.378300+0000","last_change":"2023-11-28T12:04:52.380427+0000","last_active":"2023-11-28T12:04:53.378300+0000","last_peered":"2023-11-28T12:04:53.378300+0000","last_clean":"2023-11-28T12:04:53.378300+0000","last_became_active":"2023-11-28T12:04:52.379655+0000","last_became_peered":"2023-11-28T12:04:52.379655+0000","last_unstale":"2023-11-28T12:04:53.378300+0000","last_undegraded":"2023-11-28T12:04:53.378300+0000","last_fullsized":"2023-11-28T12:04:53.378300+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:13.075372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.376654+0000","last_change":"2023-11-28T12:04:51.940150+0000","last_active":"2023-11-28T12:04:52.376654+0000","last_peered":"2023-11-28T12:04:52.376654+0000","last_clean":"2023-11-28T12:04:52.376654+0000","last_became_active":"2023-11-28T12:04:51.939467+0000","last_became_peered":"2023-11-28T12:04:51.939467+0000","last_unstale":"2023-11-28T12:04:52.376654+0000","last_undegraded":"2023-11-28T12:04:52.376654+0000","last_fullsized":"2023-11-28T12:04:52.376654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:54.228443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377076+0000","last_change":"2023-11-28T12:04:52.380806+0000","last_active":"2023-11-28T12:04:53.377076+0000","last_peered":"2023-11-28T12:04:53.377076+0000","last_clean":"2023-11-28T12:04:53.377076+0000","last_became_active":"2023-11-28T12:04:52.380550+0000","last_became_peered":"2023-11-28T12:04:52.380550+0000","last_unstale":"2023-11-28T12:04:53.377076+0000","last_undegraded":"2023-11-28T12:04:53.377076+0000","last_fullsized":"2023-11-28T12:04:53.377076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:02.586773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.373642+0000","last_change":"2023-11-28T12:04:51.942228+0000","last_active":"2023-11-28T12:04:52.373642+0000","last_peered":"2023-11-28T12:04:52.373642+0000","last_clean":"2023-11-28T12:04:52.373642+0000","last_became_active":"2023-11-28T12:04:51.941878+0000","last_became_peered":"2023-11-28T12:04:51.941878+0000","last_unstale":"2023-11-28T12:04:52.373642+0000","last_undegraded":"2023-11-28T12:04:52.373642+0000","last_fullsized":"2023-11-28T12:04:52.373642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:12.062831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377075+0000","last_change":"2023-11-28T12:04:52.389871+0000","last_active":"2023-11-28T12:04:53.377075+0000","last_peered":"2023-11-28T12:04:53.377075+0000","last_clean":"2023-11-28T12:04:53.377075+0000","last_became_active":"2023-11-28T12:04:52.389588+0000","last_became_peered":"2023-11-28T12:04:52.389588+0000","last_unstale":"2023-11-28T12:04:53.377075+0000","last_undegraded":"2023-11-28T12:04:53.377075+0000","last_fullsized":"2023-11-28T12:04:53.377075+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:14:26.740655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.365217+0000","last_change":"2023-11-28T12:04:51.940161+0000","last_active":"2023-11-28T12:04:52.365217+0000","last_peered":"2023-11-28T12:04:52.365217+0000","last_clean":"2023-11-28T12:04:52.365217+0000","last_became_active":"2023-11-28T12:04:51.939789+0000","last_became_peered":"2023-11-28T12:04:51.939789+0000","last_unstale":"2023-11-28T12:04:52.365217+0000","last_undegraded":"2023-11-28T12:04:52.365217+0000","last_fullsized":"2023-11-28T12:04:52.365217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:15:49.848587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.746914+0000","last_change":"2023-11-28T12:04:52.385238+0000","last_active":"2023-11-28T12:04:54.746914+0000","last_peered":"2023-11-28T12:04:54.746914+0000","last_clean":"2023-11-28T12:04:54.746914+0000","last_became_active":"2023-11-28T12:04:52.383242+0000","last_became_peered":"2023-11-28T12:04:52.383242+0000","last_unstale":"2023-11-28T12:04:54.746914+0000","last_undegraded":"2023-11-28T12:04:54.746914+0000","last_fullsized":"2023-11-28T12:04:54.746914+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:29:42.869610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.364639+0000","last_change":"2023-11-28T12:04:51.942255+0000","last_active":"2023-11-28T12:04:52.364639+0000","last_peered":"2023-11-28T12:04:52.364639+0000","last_clean":"2023-11-28T12:04:52.364639+0000","last_became_active":"2023-11-28T12:04:51.941749+0000","last_became_peered":"2023-11-28T12:04:51.941749+0000","last_unstale":"2023-11-28T12:04:52.364639+0000","last_undegraded":"2023-11-28T12:04:52.364639+0000","last_fullsized":"2023-11-28T12:04:52.364639+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:55:26.284594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.517975+0000","last_change":"2023-11-28T12:04:52.374068+0000","last_active":"2023-11-28T12:04:53.517975+0000","last_peered":"2023-11-28T12:04:53.517975+0000","last_clean":"2023-11-28T12:04:53.517975+0000","last_became_active":"2023-11-28T12:04:52.373799+0000","last_became_peered":"2023-11-28T12:04:52.373799+0000","last_unstale":"2023-11-28T12:04:53.517975+0000","last_undegraded":"2023-11-28T12:04:53.517975+0000","last_fullsized":"2023-11-28T12:04:53.517975+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:53.258677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.375980+0000","last_change":"2023-11-28T12:04:51.939116+0000","last_active":"2023-11-28T12:04:52.375980+0000","last_peered":"2023-11-28T12:04:52.375980+0000","last_clean":"2023-11-28T12:04:52.375980+0000","last_became_active":"2023-11-28T12:04:51.938814+0000","last_became_peered":"2023-11-28T12:04:51.938814+0000","last_unstale":"2023-11-28T12:04:52.375980+0000","last_undegraded":"2023-11-28T12:04:52.375980+0000","last_fullsized":"2023-11-28T12:04:52.375980+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:50.784042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.378113+0000","last_change":"2023-11-28T12:04:52.380401+0000","last_active":"2023-11-28T12:04:53.378113+0000","last_peered":"2023-11-28T12:04:53.378113+0000","last_clean":"2023-11-28T12:04:53.378113+0000","last_became_active":"2023-11-28T12:04:52.378912+0000","last_became_peered":"2023-11-28T12:04:52.378912+0000","last_unstale":"2023-11-28T12:04:53.378113+0000","last_undegraded":"2023-11-28T12:04:53.378113+0000","last_fullsized":"2023-11-28T12:04:53.378113+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:08.183501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.392063+0000","last_change":"2023-11-28T12:04:51.944525+0000","last_active":"2023-11-28T12:04:52.392063+0000","last_peered":"2023-11-28T12:04:52.392063+0000","last_clean":"2023-11-28T12:04:52.392063+0000","last_became_active":"2023-11-28T12:04:51.944298+0000","last_became_peered":"2023-11-28T12:04:51.944298+0000","last_unstale":"2023-11-28T12:04:52.392063+0000","last_undegraded":"2023-11-28T12:04:52.392063+0000","last_fullsized":"2023-11-28T12:04:52.392063+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:27.335615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.375194+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:04:52.375194+0000","last_peered":"2023-11-28T12:04:52.375194+0000","last_clean":"2023-11-28T12:04:52.375194+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:04:52.375194+0000","last_undegraded":"2023-11-28T12:04:52.375194+0000","last_fullsized":"2023-11-28T12:04:52.375194+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377762+0000","last_change":"2023-11-28T12:04:52.385572+0000","last_active":"2023-11-28T12:04:53.377762+0000","last_peered":"2023-11-28T12:04:53.377762+0000","last_clean":"2023-11-28T12:04:53.377762+0000","last_became_active":"2023-11-28T12:04:52.385271+0000","last_became_peered":"2023-11-28T12:04:52.385271+0000","last_unstale":"2023-11-28T12:04:53.377762+0000","last_undegraded":"2023-11-28T12:04:53.377762+0000","last_fullsized":"2023-11-28T12:04:53.377762+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:00.055601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.365759+0000","last_change":"2023-11-28T12:04:51.939162+0000","last_active":"2023-11-28T12:04:52.365759+0000","last_peered":"2023-11-28T12:04:52.365759+0000","last_clean":"2023-11-28T12:04:52.365759+0000","last_became_active":"2023-11-28T12:04:51.938867+0000","last_became_peered":"2023-11-28T12:04:51.938867+0000","last_unstale":"2023-11-28T12:04:52.365759+0000","last_undegraded":"2023-11-28T12:04:52.365759+0000","last_fullsized":"2023-11-28T12:04:52.365759+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:40.127328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.365825+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:04:52.365825+0000","last_peered":"2023-11-28T12:04:52.365825+0000","last_clean":"2023-11-28T12:04:52.365825+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:04:52.365825+0000","last_undegraded":"2023-11-28T12:04:52.365825+0000","last_fullsized":"2023-11-28T12:04:52.365825+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:54.824239+0000","last_change":"2023-11-28T12:04:52.383789+0000","last_active":"2023-11-28T12:04:54.824239+0000","last_peered":"2023-11-28T12:04:54.824239+0000","last_clean":"2023-11-28T12:04:54.824239+0000","last_became_active":"2023-11-28T12:04:52.383605+0000","last_became_peered":"2023-11-28T12:04:52.383605+0000","last_unstale":"2023-11-28T12:04:54.824239+0000","last_undegraded":"2023-11-28T12:04:54.824239+0000","last_fullsized":"2023-11-28T12:04:54.824239+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:59:29.170255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.387281+0000","last_change":"2023-11-28T12:04:51.938688+0000","last_active":"2023-11-28T12:04:52.387281+0000","last_peered":"2023-11-28T12:04:52.387281+0000","last_clean":"2023-11-28T12:04:52.387281+0000","last_became_active":"2023-11-28T12:04:51.938320+0000","last_became_peered":"2023-11-28T12:04:51.938320+0000","last_unstale":"2023-11-28T12:04:52.387281+0000","last_undegraded":"2023-11-28T12:04:52.387281+0000","last_fullsized":"2023-11-28T12:04:52.387281+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:05.234114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.377234+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:04:52.377234+0000","last_peered":"2023-11-28T12:04:52.377234+0000","last_clean":"2023-11-28T12:04:52.377234+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:04:52.377234+0000","last_undegraded":"2023-11-28T12:04:52.377234+0000","last_fullsized":"2023-11-28T12:04:52.377234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":198,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.375347+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:04:52.375347+0000","last_peered":"2023-11-28T12:04:52.375347+0000","last_clean":"2023-11-28T12:04:52.375347+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:04:52.375347+0000","last_undegraded":"2023-11-28T12:04:52.375347+0000","last_fullsized":"2023-11-28T12:04:52.375347+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.271035+0000","last_change":"2023-11-28T12:04:52.373760+0000","last_active":"2023-11-28T12:04:55.271035+0000","last_peered":"2023-11-28T12:04:55.271035+0000","last_clean":"2023-11-28T12:04:55.271035+0000","last_became_active":"2023-11-28T12:04:52.373477+0000","last_became_peered":"2023-11-28T12:04:52.373477+0000","last_unstale":"2023-11-28T12:04:55.271035+0000","last_undegraded":"2023-11-28T12:04:55.271035+0000","last_fullsized":"2023-11-28T12:04:55.271035+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:25.673535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.366504+0000","last_change":"2023-11-28T12:04:51.943124+0000","last_active":"2023-11-28T12:04:52.366504+0000","last_peered":"2023-11-28T12:04:52.366504+0000","last_clean":"2023-11-28T12:04:52.366504+0000","last_became_active":"2023-11-28T12:04:51.942813+0000","last_became_peered":"2023-11-28T12:04:51.942813+0000","last_unstale":"2023-11-28T12:04:52.366504+0000","last_undegraded":"2023-11-28T12:04:52.366504+0000","last_fullsized":"2023-11-28T12:04:52.366504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:50.784042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.373075+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:04:52.373075+0000","last_peered":"2023-11-28T12:04:52.373075+0000","last_clean":"2023-11-28T12:04:52.373075+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:04:52.373075+0000","last_undegraded":"2023-11-28T12:04:52.373075+0000","last_fullsized":"2023-11-28T12:04:52.373075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377620+0000","last_change":"2023-11-28T12:04:52.382618+0000","last_active":"2023-11-28T12:04:53.377620+0000","last_peered":"2023-11-28T12:04:53.377620+0000","last_clean":"2023-11-28T12:04:53.377620+0000","last_became_active":"2023-11-28T12:04:52.382345+0000","last_became_peered":"2023-11-28T12:04:52.382345+0000","last_unstale":"2023-11-28T12:04:53.377620+0000","last_undegraded":"2023-11-28T12:04:53.377620+0000","last_fullsized":"2023-11-28T12:04:53.377620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:21.032385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.364471+0000","last_change":"2023-11-28T12:04:51.936566+0000","last_active":"2023-11-28T12:04:52.364471+0000","last_peered":"2023-11-28T12:04:52.364471+0000","last_clean":"2023-11-28T12:04:52.364471+0000","last_became_active":"2023-11-28T12:04:51.936221+0000","last_became_peered":"2023-11-28T12:04:51.936221+0000","last_unstale":"2023-11-28T12:04:52.364471+0000","last_undegraded":"2023-11-28T12:04:52.364471+0000","last_fullsized":"2023-11-28T12:04:52.364471+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:05.119970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.375434+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:04:52.375434+0000","last_peered":"2023-11-28T12:04:52.375434+0000","last_clean":"2023-11-28T12:04:52.375434+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:04:52.375434+0000","last_undegraded":"2023-11-28T12:04:52.375434+0000","last_fullsized":"2023-11-28T12:04:52.375434+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377900+0000","last_change":"2023-11-28T12:04:52.376429+0000","last_active":"2023-11-28T12:04:53.377900+0000","last_peered":"2023-11-28T12:04:53.377900+0000","last_clean":"2023-11-28T12:04:53.377900+0000","last_became_active":"2023-11-28T12:04:52.376077+0000","last_became_peered":"2023-11-28T12:04:52.376077+0000","last_unstale":"2023-11-28T12:04:53.377900+0000","last_undegraded":"2023-11-28T12:04:53.377900+0000","last_fullsized":"2023-11-28T12:04:53.377900+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:51.796583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.373175+0000","last_change":"2023-11-28T12:04:51.939473+0000","last_active":"2023-11-28T12:04:52.373175+0000","last_peered":"2023-11-28T12:04:52.373175+0000","last_clean":"2023-11-28T12:04:52.373175+0000","last_became_active":"2023-11-28T12:04:51.939141+0000","last_became_peered":"2023-11-28T12:04:51.939141+0000","last_unstale":"2023-11-28T12:04:52.373175+0000","last_undegraded":"2023-11-28T12:04:52.373175+0000","last_fullsized":"2023-11-28T12:04:52.373175+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:07.170960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.366113+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:04:52.366113+0000","last_peered":"2023-11-28T12:04:52.366113+0000","last_clean":"2023-11-28T12:04:52.366113+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:04:52.366113+0000","last_undegraded":"2023-11-28T12:04:52.366113+0000","last_fullsized":"2023-11-28T12:04:52.366113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:53.377687+0000","last_change":"2023-11-28T12:04:52.375680+0000","last_active":"2023-11-28T12:04:53.377687+0000","last_peered":"2023-11-28T12:04:53.377687+0000","last_clean":"2023-11-28T12:04:53.377687+0000","last_became_active":"2023-11-28T12:04:52.375252+0000","last_became_peered":"2023-11-28T12:04:52.375252+0000","last_unstale":"2023-11-28T12:04:53.377687+0000","last_undegraded":"2023-11-28T12:04:53.377687+0000","last_fullsized":"2023-11-28T12:04:53.377687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:17.163166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.389550+0000","last_change":"2023-11-28T12:04:51.939352+0000","last_active":"2023-11-28T12:04:52.389550+0000","last_peered":"2023-11-28T12:04:52.389550+0000","last_clean":"2023-11-28T12:04:52.389550+0000","last_became_active":"2023-11-28T12:04:51.939082+0000","last_became_peered":"2023-11-28T12:04:51.939082+0000","last_unstale":"2023-11-28T12:04:52.389550+0000","last_undegraded":"2023-11-28T12:04:52.389550+0000","last_fullsized":"2023-11-28T12:04:52.389550+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:01.574232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.367469+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:04:52.367469+0000","last_peered":"2023-11-28T12:04:52.367469+0000","last_clean":"2023-11-28T12:04:52.367469+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:04:52.367469+0000","last_undegraded":"2023-11-28T12:04:52.367469+0000","last_fullsized":"2023-11-28T12:04:52.367469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T12:04:55.272584+0000","last_change":"2023-11-28T12:04:52.390267+0000","last_active":"2023-11-28T12:04:55.272584+0000","last_peered":"2023-11-28T12:04:55.272584+0000","last_clean":"2023-11-28T12:04:55.272584+0000","last_became_active":"2023-11-28T12:04:52.389948+0000","last_became_peered":"2023-11-28T12:04:52.389948+0000","last_unstale":"2023-11-28T12:04:55.272584+0000","last_undegraded":"2023-11-28T12:04:55.272584+0000","last_fullsized":"2023-11-28T12:04:55.272584+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:51.910728+0000","last_clean_scrub_stamp":"2023-11-28T12:04:51.910728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:48.028736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.386252+0000","last_change":"2023-11-28T12:04:51.944547+0000","last_active":"2023-11-28T12:04:52.386252+0000","last_peered":"2023-11-28T12:04:52.386252+0000","last_clean":"2023-11-28T12:04:52.386252+0000","last_became_active":"2023-11-28T12:04:51.944273+0000","last_became_peered":"2023-11-28T12:04:51.944273+0000","last_unstale":"2023-11-28T12:04:52.386252+0000","last_undegraded":"2023-11-28T12:04:52.386252+0000","last_fullsized":"2023-11-28T12:04:52.386252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:04:50.898186+0000","last_clean_scrub_stamp":"2023-11-28T12:04:50.898186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:15:52.635652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T12:04:52.368579+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:04:52.368579+0000","last_peered":"2023-11-28T12:04:52.368579+0000","last_clean":"2023-11-28T12:04:52.368579+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:04:52.368579+0000","last_undegraded":"2023-11-28T12:04:52.368579+0000","last_fullsized":"2023-11-28T12:04:52.368579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10816,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94361024,"statfs":{"total":96636764160,"available":96625688576,"internally_reserved":0,"allocated":1368064,"data_stored":927690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55500000000000005,"15min":0.54800000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69699999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64300000000000002,"5min":0.872,"15min":0.872},"last":0.44},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.76500000000000001,"15min":0.84699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73099999999999998,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.751,"15min":0.83099999999999996},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70199999999999996,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72399999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.39000000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.75600000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64900000000000002,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75700000000000001,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.52200000000000002}]},{"osd":4,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.505},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.501},{"interface":"front","average":{"1min":0.503,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.39300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64400000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.49299999999999999}]},{"osd":5,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55200000000000005},"min":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73599999999999999,"5min":0.752,"15min":0.752},"last":0.57999999999999996}]},{"osd":6,"last update":"Tue Nov 28 12:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.70499999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69599999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9328,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362512,"statfs":{"total":96636764160,"available":96627212288,"internally_reserved":0,"allocated":761856,"data_stored":336596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.44500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":1.446,"15min":1.446},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.72099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.39800000000000002}]},{"osd":1,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.52400000000000002,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73499999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80500000000000005,"5min":0.88,"15min":0.88},"last":0.51900000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.53000000000000003,"15min":0.497},"min":{"1min":0.46899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.76800000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80400000000000005,"5min":1.254,"15min":1.254},"last":0.48799999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.39500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.46000000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80500000000000005,"5min":1.228,"15min":1.228},"last":0.55800000000000005}]},{"osd":4,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46899999999999997,"15min":0.45200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69499999999999995,"5min":1.204,"15min":1.204},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67000000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.46100000000000002}]},{"osd":5,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70999999999999996,"5min":0.753,"15min":0.753},"last":0.439},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.374,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":1.29,"15min":1.29},"last":0.60699999999999998}]},{"osd":7,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.748,"5min":1.159,"15min":1.159},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.35499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81799999999999995,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10636,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361204,"statfs":{"total":96636764160,"available":96625872896,"internally_reserved":0,"allocated":1380352,"data_stored":929226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39500000000000002,"15min":0.40699999999999997},"min":{"1min":0.26500000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.49199999999999999,"5min":1.2,"15min":1.2},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40000000000000002,"15min":0.40999999999999998},"min":{"1min":0.27300000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60799999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.30599999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.37,"15min":0.37},"min":{"1min":0.22900000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.52100000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57499999999999996,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.38100000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47499999999999998,"15min":0.48599999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63900000000000001,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.44500000000000001,"15min":0.45400000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54800000000000004,"5min":1.03,"15min":1.03},"last":0.37}]},{"osd":4,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.375,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81000000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81599999999999995,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.58099999999999996}]},{"osd":5,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.627,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97699999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.503,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.91400000000000003,"5min":0.998,"15min":0.998},"last":0.49399999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65800000000000003,"15min":0.67300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89300000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.871,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.59499999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64700000000000002,"15min":0.66000000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94999999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91800000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.54000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9580,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362260,"statfs":{"total":96636764160,"available":96626954240,"internally_reserved":0,"allocated":757760,"data_stored":336562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.50600000000000001,"15min":0.56399999999999995},"min":{"1min":0.26400000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.42999999999999999,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.312,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51600000000000001,"5min":1.667,"15min":1.667},"last":0.35899999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.53100000000000003,"15min":0.58199999999999996},"min":{"1min":0.31900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.47699999999999998,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42999999999999999,"15min":0.437},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53500000000000003,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.308}]},{"osd":3,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.432,"15min":0.44400000000000001},"min":{"1min":0.29999999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.49299999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.251},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.59499999999999997,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.52400000000000002,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.33400000000000002}]},{"osd":4,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79800000000000004,"5min":1.536,"15min":1.536},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.72199999999999998,"15min":0.76600000000000001},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76000000000000001,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.46500000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79300000000000004,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.624,"15min":0.62},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.78600000000000003,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.53500000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74299999999999999,"15min":0.78100000000000003},"min":{"1min":0.441,"5min":0.44,"15min":0.44},"max":{"1min":0.81899999999999995,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63200000000000001,"15min":0.62},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80800000000000005,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.51400000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.76000000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.755,"5min":1.605,"15min":1.605},"last":0.49299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13548,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":12800,"kb_avail":94358292,"statfs":{"total":96636764160,"available":96622891008,"internally_reserved":0,"allocated":757760,"data_stored":336562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13107200},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.375},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58999999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.309},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.40999999999999998,"15min":0.40100000000000002},"min":{"1min":0.318,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.25800000000000001}]},{"osd":1,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42099999999999999,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.60899999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.54800000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.28699999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41299999999999998,"15min":0.40999999999999998},"min":{"1min":0.31,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.621,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.432,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.627,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.59799999999999998}]},{"osd":5,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.503},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.625},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.51300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68799999999999994}]},{"osd":7,"last update":"Tue Nov 28 12:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.63700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13932,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":13184,"kb_avail":94357908,"statfs":{"total":96636764160,"available":96622497792,"internally_reserved":0,"allocated":757760,"data_stored":336562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13500416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.29599999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42299999999999999,"15min":0.441},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.495,"5min":1.238,"15min":1.238},"last":0.36499999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42799999999999999,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56799999999999995,"5min":1.53,"15min":1.53},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.48599999999999999,"15min":0.51000000000000001},"min":{"1min":0.27100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.622,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.44700000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47599999999999998,"15min":0.49199999999999999},"min":{"1min":0.34000000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60299999999999998,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.48499999999999999},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60199999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79000000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.51600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91900000000000004,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.70799999999999996}]},{"osd":5,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67900000000000005,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.99299999999999999,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.51900000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.879,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.503}]},{"osd":6,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92300000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84699999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.755}]},{"osd":7,"last update":"Tue Nov 28 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.94299999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67400000000000004,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9656,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362184,"statfs":{"total":96636764160,"available":96626876416,"internally_reserved":0,"allocated":770048,"data_stored":338110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53700000000000003,"15min":0.52200000000000002},"min":{"1min":0.44800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61499999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.54100000000000004}]},{"osd":2,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.628},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76000000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.57399999999999995}]},{"osd":3,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.70299999999999996}]},{"osd":5,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68999999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.503,"15min":0.497},"min":{"1min":0.35199999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48399999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.498,"15min":0.5},"min":{"1min":0.39100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.503}]}]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9520,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362320,"statfs":{"total":96636764160,"available":96627015680,"internally_reserved":0,"allocated":761856,"data_stored":336584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71199999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.504},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.443,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.56100000000000005}]},{"osd":1,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.63600000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.80500000000000005,"15min":0.91900000000000004},"min":{"1min":0.45200000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74299999999999999,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.53600000000000003}]},{"osd":2,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.64800000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.432,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77800000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51400000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.64600000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46700000000000003,"15min":0.47299999999999998},"min":{"1min":0.36099999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66400000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51400000000000001,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66400000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.376}]},{"osd":6,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.50600000000000001,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.65300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.439},{"interface":"front","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.375,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63300000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.46700000000000003}]},{"osd":7,"last update":"Tue Nov 28 12:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53600000000000003,"15min":0.54600000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68899999999999995,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.57499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:05:01.633 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T12:05:01.633 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T12:05:01.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.813+0000 7f0299efa700 1 Processor -- start 2023-11-28T12:05:01.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.813+0000 7f0299efa700 1 -- start start 2023-11-28T12:05:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.814+0000 7f0299efa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02940b1650 0x7f02940b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.814+0000 7f0299efa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f02941599b0 0x7f029415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.814+0000 7f0299efa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0294159ec0 con 0x7f02940af0a0 2023-11-28T12:05:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.814+0000 7f0299efa700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029415aa60 con 0x7f02941599b0 2023-11-28T12:05:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.814+0000 7f0299efa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02940b1f60 con 0x7f02940b1650 2023-11-28T12:05:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.814+0000 7f0292ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f02941599b0 0x7f029415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.814+0000 7f02937fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02940b1650 0x7f02940b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.814+0000 7f0293fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f02940af0a0 0x7f029415a350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:50562/0 (socket says 172.21.15.57:50562) 2023-11-28T12:05:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.814+0000 7f0292ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f02941599b0 0x7f029415d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:50616/0 (socket says 172.21.15.57:50616) 2023-11-28T12:05:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.814+0000 7f0292ffd700 1 -- 172.21.15.57:0/1126476411 learned_addr learned my addr 172.21.15.57:0/1126476411 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.815+0000 7f02937fe700 1 -- 172.21.15.57:0/1126476411 >> v2:172.21.15.181:3300/0 conn(0x7f02941599b0 msgr2=0x7f029415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.815+0000 7f02937fe700 1 --2- 172.21.15.57:0/1126476411 >> v2:172.21.15.181:3300/0 conn(0x7f02941599b0 0x7f029415d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.815+0000 7f02937fe700 1 -- 172.21.15.57:0/1126476411 >> v1:172.21.15.181:6789/0 conn(0x7f02940af0a0 legacy=0x7f029415a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.815+0000 7f02937fe700 1 -- 172.21.15.57:0/1126476411 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029415d8a0 con 0x7f02940b1650 2023-11-28T12:05:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.815+0000 7f0292ffd700 1 --2- 172.21.15.57:0/1126476411 >> v2:172.21.15.181:3300/0 conn(0x7f02941599b0 0x7f029415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:05:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.815+0000 7f02937fe700 1 --2- 172.21.15.57:0/1126476411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02940b1650 0x7f02940b1a20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0284009840 tx=0x7f0284009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daa8750668979828 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.815+0000 7f02927fc700 1 -- 172.21.15.57:0/1126476411 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284011830 con 0x7f02940b1650 2023-11-28T12:05:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.815+0000 7f02927fc700 1 -- 172.21.15.57:0/1126476411 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0284011990 con 0x7f02940b1650 2023-11-28T12:05:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.815+0000 7f02927fc700 1 -- 172.21.15.57:0/1126476411 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284011d20 con 0x7f02940b1650 2023-11-28T12:05:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.816+0000 7f0299efa700 1 -- 172.21.15.57:0/1126476411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02940b1650 msgr2=0x7f02940b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.816+0000 7f0299efa700 1 --2- 172.21.15.57:0/1126476411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02940b1650 0x7f02940b1a20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0284009840 tx=0x7f0284009b50 comp rx=0 tx=0).stop 2023-11-28T12:05:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.816+0000 7f0299efa700 1 -- 172.21.15.57:0/1126476411 shutdown_connections 2023-11-28T12:05:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.816+0000 7f0299efa700 1 --2- 172.21.15.57:0/1126476411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02940b1650 0x7f02940b1a20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.816+0000 7f0299efa700 1 --2- 172.21.15.57:0/1126476411 >> v2:172.21.15.181:3300/0 conn(0x7f02941599b0 0x7f029415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.816+0000 7f0299efa700 1 -- 172.21.15.57:0/1126476411 >> 172.21.15.57:0/1126476411 conn(0x7f029406c010 msgr2=0x7f02940cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.816+0000 7f0299efa700 1 -- 172.21.15.57:0/1126476411 shutdown_connections 2023-11-28T12:05:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.816+0000 7f0299efa700 1 -- 172.21.15.57:0/1126476411 wait complete. 2023-11-28T12:05:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.817+0000 7f0299efa700 1 Processor -- start 2023-11-28T12:05:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.817+0000 7f0299efa700 1 -- start start 2023-11-28T12:05:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.818+0000 7f0299efa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02940af0a0 0x7f0294158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.818+0000 7f0299efa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02940b1650 0x7f02940b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.818+0000 7f0299efa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0294159140 0x7f02940b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.818+0000 7f0299efa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02941e6fc0 con 0x7f02940b1650 2023-11-28T12:05:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.818+0000 7f0299efa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02940c37a0 con 0x7f0294159140 2023-11-28T12:05:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.818+0000 7f0299efa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02940c3a80 con 0x7f02940af0a0 2023-11-28T12:05:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.818+0000 7f0292ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02940b1650 0x7f02940b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.818+0000 7f0292ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02940b1650 0x7f02940b0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46436/0 (socket says 172.21.15.57:46436) 2023-11-28T12:05:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.818+0000 7f0292ffd700 1 -- 172.21.15.57:0/2029646366 learned_addr learned my addr 172.21.15.57:0/2029646366 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.818+0000 7f0293fff700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0294159140 0x7f02940b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.818+0000 7f02937fe700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02940af0a0 0x7f0294158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.819+0000 7f0292ffd700 1 -- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02940af0a0 msgr2=0x7f0294158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.819+0000 7f0292ffd700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02940af0a0 0x7f0294158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.819+0000 7f0292ffd700 1 -- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0294159140 msgr2=0x7f02940b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.819+0000 7f0292ffd700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0294159140 0x7f02940b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.819+0000 7f0292ffd700 1 -- 172.21.15.57:0/2029646366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c009550 con 0x7f02940b1650 2023-11-28T12:05:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.819+0000 7f0292ffd700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02940b1650 0x7f02940b0a20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e990 tx=0x7f027c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.819+0000 7f028bfff700 1 -- 172.21.15.57:0/2029646366 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c00c690 con 0x7f02940b1650 2023-11-28T12:05:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.819+0000 7f028bfff700 1 -- 172.21.15.57:0/2029646366 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c00c7f0 con 0x7f02940b1650 2023-11-28T12:05:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.819+0000 7f0299efa700 1 -- 172.21.15.57:0/2029646366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02840094a0 con 0x7f02940b1650 2023-11-28T12:05:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.819+0000 7f028bfff700 1 -- 172.21.15.57:0/2029646366 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c00cba0 con 0x7f02940b1650 2023-11-28T12:05:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.819+0000 7f0299efa700 1 -- 172.21.15.57:0/2029646366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029415de60 con 0x7f02940b1650 2023-11-28T12:05:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.820+0000 7f0299efa700 1 -- 172.21.15.57:0/2029646366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029415db80 con 0x7f02940b1650 2023-11-28T12:05:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.821+0000 7f028bfff700 1 -- 172.21.15.57:0/2029646366 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f027c019030 con 0x7f02940b1650 2023-11-28T12:05:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.821+0000 7f028bfff700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f02740682f0 0x7f027406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.822+0000 7f028bfff700 1 -- 172.21.15.57:0/2029646366 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f027c0861f0 con 0x7f02940b1650 2023-11-28T12:05:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.822+0000 7f02937fe700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f02740682f0 0x7f027406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.822+0000 7f02937fe700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f02740682f0 0x7f027406a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0284000c00 tx=0x7f028401c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:01.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:01.826+0000 7f028bfff700 1 -- 172.21.15.57:0/2029646366 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f027c052af0 con 0x7f02940b1650 2023-11-28T12:05:02.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.106+0000 7f0299efa700 1 -- 172.21.15.57:0/2029646366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f029415d590 con 0x7f02940b1650 2023-11-28T12:05:02.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.111+0000 7f028bfff700 1 -- 172.21.15.57:0/2029646366 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7f027c056110 con 0x7f02940b1650 2023-11-28T12:05:02.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.114+0000 7f0299efa700 1 -- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f02740682f0 msgr2=0x7f027406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:02.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.114+0000 7f0299efa700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f02740682f0 0x7f027406a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0284000c00 tx=0x7f028401c040 comp rx=0 tx=0).stop 2023-11-28T12:05:02.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.114+0000 7f0299efa700 1 -- 172.21.15.57:0/2029646366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02940b1650 msgr2=0x7f02940b0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:02.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.114+0000 7f0299efa700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02940b1650 0x7f02940b0a20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e990 tx=0x7f027c00eca0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.114+0000 7f0299efa700 1 -- 172.21.15.57:0/2029646366 shutdown_connections 2023-11-28T12:05:02.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.114+0000 7f0299efa700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f02740682f0 0x7f027406a7a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.114+0000 7f0299efa700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0294159140 0x7f02940b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.114+0000 7f0299efa700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02940af0a0 0x7f0294158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.114+0000 7f0299efa700 1 --2- 172.21.15.57:0/2029646366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02940b1650 0x7f02940b0a20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.114+0000 7f0299efa700 1 -- 172.21.15.57:0/2029646366 >> 172.21.15.57:0/2029646366 conn(0x7f029406c010 msgr2=0x7f02940cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:02.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.114+0000 7f0299efa700 1 -- 172.21.15.57:0/2029646366 shutdown_connections 2023-11-28T12:05:02.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.114+0000 7f0299efa700 1 -- 172.21.15.57:0/2029646366 wait complete. 2023-11-28T12:05:02.116 INFO:teuthology.orchestra.run.smithi057.stderr:updated caps for client.0 2023-11-28T12:05:02.128 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:05:02.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.262+0000 7f8c5b44e700 1 Processor -- start 2023-11-28T12:05:02.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.263+0000 7f8c5b44e700 1 -- start start 2023-11-28T12:05:02.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.263+0000 7f8c5b44e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c540ce040 0x7f8c5415d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:02.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.263+0000 7f8c5b44e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8c54142db0 0x7f8c5413ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:02.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.263+0000 7f8c5b44e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5413fc90 con 0x7f8c540b1770 2023-11-28T12:05:02.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.263+0000 7f8c5b44e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c5413f6d0 con 0x7f8c54142db0 2023-11-28T12:05:02.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.263+0000 7f8c5b44e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5413f9b0 con 0x7f8c540ce040 2023-11-28T12:05:02.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c589e9700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8c54142db0 0x7f8c5413ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:02.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c599eb700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8c540b1770 0x7f8c5413cd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55600/0 (socket says 172.21.15.57:55600) 2023-11-28T12:05:02.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c599eb700 1 -- 172.21.15.57:0/2771914659 learned_addr learned my addr 172.21.15.57:0/2771914659 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:05:02.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c591ea700 1 --2- 172.21.15.57:0/2771914659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c540ce040 0x7f8c5415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:02.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c4bfff700 1 -- 172.21.15.57:0/2771914659 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258977042 0 0) 0x7f8c5413fc90 con 0x7f8c540b1770 2023-11-28T12:05:02.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c4bfff700 1 -- 172.21.15.57:0/2771914659 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c400035b0 con 0x7f8c540b1770 2023-11-28T12:05:02.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c589e9700 1 -- 172.21.15.57:0/2771914659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c540ce040 msgr2=0x7f8c5415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:02.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c589e9700 1 --2- 172.21.15.57:0/2771914659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c540ce040 0x7f8c5415d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c589e9700 1 -- 172.21.15.57:0/2771914659 >> v1:172.21.15.57:6789/0 conn(0x7f8c540b1770 legacy=0x7f8c5413cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:02.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c589e9700 1 -- 172.21.15.57:0/2771914659 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5415d100 con 0x7f8c54142db0 2023-11-28T12:05:02.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c591ea700 1 --2- 172.21.15.57:0/2771914659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c540ce040 0x7f8c5415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:05:02.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c589e9700 1 --2- 172.21.15.57:0/2771914659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c54142db0 0x7f8c5413ed70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009840 tx=0x7f8c44009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e60e12b9b7139 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:02.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c4bfff700 1 -- 172.21.15.57:0/2771914659 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44011830 con 0x7f8c54142db0 2023-11-28T12:05:02.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c4bfff700 1 -- 172.21.15.57:0/2771914659 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c44011990 con 0x7f8c54142db0 2023-11-28T12:05:02.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.264+0000 7f8c4bfff700 1 -- 172.21.15.57:0/2771914659 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44011d20 con 0x7f8c54142db0 2023-11-28T12:05:02.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.265+0000 7f8c5b44e700 1 -- 172.21.15.57:0/2771914659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c54142db0 msgr2=0x7f8c5413ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:02.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.265+0000 7f8c5b44e700 1 --2- 172.21.15.57:0/2771914659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c54142db0 0x7f8c5413ed70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009840 tx=0x7f8c44009b50 comp rx=0 tx=0).stop 2023-11-28T12:05:02.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.265+0000 7f8c5b44e700 1 -- 172.21.15.57:0/2771914659 shutdown_connections 2023-11-28T12:05:02.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.265+0000 7f8c5b44e700 1 --2- 172.21.15.57:0/2771914659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c540ce040 0x7f8c5415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.265+0000 7f8c5b44e700 1 --2- 172.21.15.57:0/2771914659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c54142db0 0x7f8c5413ed70 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.265+0000 7f8c5b44e700 1 -- 172.21.15.57:0/2771914659 >> 172.21.15.57:0/2771914659 conn(0x7f8c5406c010 msgr2=0x7f8c540cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:02.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.265+0000 7f8c5b44e700 1 -- 172.21.15.57:0/2771914659 shutdown_connections 2023-11-28T12:05:02.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.265+0000 7f8c5b44e700 1 -- 172.21.15.57:0/2771914659 wait complete. 2023-11-28T12:05:02.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.266+0000 7f8c5b44e700 1 Processor -- start 2023-11-28T12:05:02.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.266+0000 7f8c5b44e700 1 -- start start 2023-11-28T12:05:02.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.267+0000 7f8c5b44e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c540b1770 0x7f8c541efb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:02.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.267+0000 7f8c5b44e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c540ce040 0x7f8c540b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:02.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.267+0000 7f8c5b44e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c54142db0 0x7f8c54151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:02.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.267+0000 7f8c5b44e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540c50b0 con 0x7f8c540b1770 2023-11-28T12:05:02.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.267+0000 7f8c5b44e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540c4200 con 0x7f8c54142db0 2023-11-28T12:05:02.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.267+0000 7f8c5b44e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540c44a0 con 0x7f8c540ce040 2023-11-28T12:05:02.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.267+0000 7f8c591ea700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c540b1770 0x7f8c541efb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:02.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.267+0000 7f8c591ea700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c540b1770 0x7f8c541efb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46458/0 (socket says 172.21.15.57:46458) 2023-11-28T12:05:02.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.267+0000 7f8c591ea700 1 -- 172.21.15.57:0/1597759400 learned_addr learned my addr 172.21.15.57:0/1597759400 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:02.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c599eb700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c54142db0 0x7f8c54151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:02.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c589e9700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c540ce040 0x7f8c540b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:02.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c591ea700 1 -- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c540ce040 msgr2=0x7f8c540b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:02.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c591ea700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c540ce040 0x7f8c540b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c591ea700 1 -- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c54142db0 msgr2=0x7f8c54151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:02.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c591ea700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c54142db0 0x7f8c54151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c591ea700 1 -- 172.21.15.57:0/1597759400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c440094a0 con 0x7f8c540b1770 2023-11-28T12:05:02.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c599eb700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c54142db0 0x7f8c54151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:05:02.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c591ea700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c540b1770 0x7f8c541efb50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000c750 tx=0x7f8c5000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:02.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c49ffb700 1 -- 172.21.15.57:0/1597759400 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c50011800 con 0x7f8c540b1770 2023-11-28T12:05:02.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c49ffb700 1 -- 172.21.15.57:0/1597759400 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c50011960 con 0x7f8c540b1770 2023-11-28T12:05:02.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c49ffb700 1 -- 172.21.15.57:0/1597759400 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c50011cd0 con 0x7f8c540b1770 2023-11-28T12:05:02.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c5b44e700 1 -- 172.21.15.57:0/1597759400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c540ce410 con 0x7f8c540b1770 2023-11-28T12:05:02.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.268+0000 7f8c5b44e700 1 -- 172.21.15.57:0/1597759400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5415d3e0 con 0x7f8c540b1770 2023-11-28T12:05:02.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.269+0000 7f8c5b44e700 1 -- 172.21.15.57:0/1597759400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c540ce5a0 con 0x7f8c540b1770 2023-11-28T12:05:02.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.269+0000 7f8c49ffb700 1 -- 172.21.15.57:0/1597759400 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8c50020070 con 0x7f8c540b1770 2023-11-28T12:05:02.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.270+0000 7f8c49ffb700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c380682f0 0x7f8c3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:02.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.270+0000 7f8c49ffb700 1 -- 172.21.15.57:0/1597759400 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8c50085b10 con 0x7f8c540b1770 2023-11-28T12:05:02.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.270+0000 7f8c589e9700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c380682f0 0x7f8c3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:02.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.271+0000 7f8c589e9700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c380682f0 0x7f8c3806a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8c541f37f0 tx=0x7f8c44008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:02.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.274+0000 7f8c49ffb700 1 -- 172.21.15.57:0/1597759400 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8c50052370 con 0x7f8c540b1770 2023-11-28T12:05:02.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.519+0000 7f8c5b44e700 1 -- 172.21.15.57:0/1597759400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c540743f0 con 0x7f8c540b1770 2023-11-28T12:05:02.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.520+0000 7f8c49ffb700 1 -- 172.21.15.57:0/1597759400 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4652 (secure 0 0 0) 0x7f8c50055990 con 0x7f8c540b1770 2023-11-28T12:05:02.521 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:05:02.521 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2317025916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2317025916},{"type":"v1","addr":"172.21.15.181:6836","nonce":2317025916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3486085199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3486085199},{"type":"v1","addr":"172.21.15.181:6837","nonce":3486085199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/4272259116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":4272259116},{"type":"v1","addr":"172.21.15.57:6834","nonce":4272259116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:04:52.360416+0000","modified":"2023-11-28T12:04:55.377023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6835/685888843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":685888843},{"type":"v1","addr":"172.21.15.57:6835","nonce":685888843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T12:05:02.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.522+0000 7f8c5b44e700 1 -- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c380682f0 msgr2=0x7f8c3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:02.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.522+0000 7f8c5b44e700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c380682f0 0x7f8c3806a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8c541f37f0 tx=0x7f8c44008030 comp rx=0 tx=0).stop 2023-11-28T12:05:02.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.522+0000 7f8c5b44e700 1 -- 172.21.15.57:0/1597759400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c540b1770 msgr2=0x7f8c541efb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:02.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.522+0000 7f8c5b44e700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c540b1770 0x7f8c541efb50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000c750 tx=0x7f8c5000cb10 comp rx=0 tx=0).stop 2023-11-28T12:05:02.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.522+0000 7f8c5b44e700 1 -- 172.21.15.57:0/1597759400 shutdown_connections 2023-11-28T12:05:02.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.522+0000 7f8c5b44e700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8c380682f0 0x7f8c3806a7a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.522+0000 7f8c5b44e700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c54142db0 0x7f8c54151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.522+0000 7f8c5b44e700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c540b1770 0x7f8c541efb50 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.522+0000 7f8c5b44e700 1 --2- 172.21.15.57:0/1597759400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c540ce040 0x7f8c540b52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:02.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.522+0000 7f8c5b44e700 1 -- 172.21.15.57:0/1597759400 >> 172.21.15.57:0/1597759400 conn(0x7f8c5406c010 msgr2=0x7f8c540cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:02.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.522+0000 7f8c5b44e700 1 -- 172.21.15.57:0/1597759400 shutdown_connections 2023-11-28T12:05:02.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:02.522+0000 7f8c5b44e700 1 -- 172.21.15.57:0/1597759400 wait complete. 2023-11-28T12:05:02.525 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 32 2023-11-28T12:05:02.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T12:04:52.360416+0000', 'modified': '2023-11-28T12:04:55.377023+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6219}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6219': {'gid': 6219, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.57:6835/685888843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6833', 'nonce': 685888843}, {'type': 'v1', 'addr': '172.21.15.57:6835', 'nonce': 685888843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T12:05:02.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T12:05:02.939 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T12:05:03.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.084+0000 7fc2a2c20700 1 Processor -- start 2023-11-28T12:05:03.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.084+0000 7fc2a2c20700 1 -- start start 2023-11-28T12:05:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc2a2c20700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc29c0cf710 0x7fc29c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc2a2c20700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc29c15cc10 con 0x7fc29c071890 2023-11-28T12:05:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc2a2c20700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc29c06c9b0 con 0x7fc29c06a980 2023-11-28T12:05:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc2a2c20700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc29c06cc90 con 0x7fc29c0cf710 2023-11-28T12:05:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29bfff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc29c071890 0x7fc29c06a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55606/0 (socket says 172.21.15.57:55606) 2023-11-28T12:05:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29bfff700 1 -- 172.21.15.57:0/4165715352 learned_addr learned my addr 172.21.15.57:0/4165715352 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:05:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/4165715352 >> v2:172.21.15.181:3300/0 conn(0x7fc29c0cf710 0x7fc29c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090085878 0 0) 0x7fc29c15cc10 con 0x7fc29c071890 2023-11-28T12:05:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2880035b0 con 0x7fc29c071890 2023-11-28T12:05:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2594933067 0 0) 0x7fc2880035b0 con 0x7fc29c071890 2023-11-28T12:05:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 >> v2:172.21.15.181:3300/0 conn(0x7fc29c0cf710 msgr2=0x7fc29c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29b7fe700 1 --2- 172.21.15.57:0/4165715352 >> v2:172.21.15.181:3300/0 conn(0x7fc29c0cf710 0x7fc29c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 >> v1:172.21.15.181:6789/0 conn(0x7fc29c06a980 legacy=0x7fc29c160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c06cf70 con 0x7fc29c071890 2023-11-28T12:05:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/4165715352 >> v2:172.21.15.181:3300/0 conn(0x7fc29c0cf710 0x7fc29c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:05:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fc284001670 con 0x7fc29c071890 2023-11-28T12:05:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 >> v1:172.21.15.57:6789/0 conn(0x7fc29c071890 legacy=0x7fc29c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29b7fe700 1 --2- 172.21.15.57:0/4165715352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc288003a00 0x7fc288003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29b7fe700 1 --2- 172.21.15.57:0/4165715352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc288004850 0x7fc288004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.085+0000 7fc29b7fe700 1 --2- 172.21.15.57:0/4165715352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc288005200 0x7fc28800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c06cc90 con 0x7fc288005200 2023-11-28T12:05:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28800c110 con 0x7fc288004850 2023-11-28T12:05:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28800c2d0 con 0x7fc288003a00 2023-11-28T12:05:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc2a11bd700 1 --2- 172.21.15.57:0/4165715352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc288005200 0x7fc28800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/4165715352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc288003a00 0x7fc288003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc29bfff700 1 --2- 172.21.15.57:0/4165715352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc288004850 0x7fc288004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc2a11bd700 1 -- 172.21.15.57:0/4165715352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc288003a00 msgr2=0x7fc288003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:03.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc2a11bd700 1 --2- 172.21.15.57:0/4165715352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc288003a00 0x7fc288003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc2a11bd700 1 -- 172.21.15.57:0/4165715352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc288004850 msgr2=0x7fc288004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:03.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc2a11bd700 1 --2- 172.21.15.57:0/4165715352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc288004850 0x7fc288004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc2a11bd700 1 -- 172.21.15.57:0/4165715352 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880035b0 con 0x7fc288005200 2023-11-28T12:05:03.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc2a11bd700 1 -- 172.21.15.57:0/4165715352 reap_dead start 2023-11-28T12:05:03.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc2a11bd700 1 --2- 172.21.15.57:0/4165715352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc288005200 0x7fc28800ba10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc29400c160 tx=0x7fc288005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97d7936f34f8e11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:03.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294001db0 con 0x7fc288005200 2023-11-28T12:05:03.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29400ccc0 con 0x7fc288005200 2023-11-28T12:05:03.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.086+0000 7fc29b7fe700 1 -- 172.21.15.57:0/4165715352 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294003c40 con 0x7fc288005200 2023-11-28T12:05:03.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.087+0000 7fc2a2c20700 1 -- 172.21.15.57:0/4165715352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc288005200 msgr2=0x7fc28800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.087+0000 7fc2a2c20700 1 --2- 172.21.15.57:0/4165715352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc288005200 0x7fc28800ba10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc29400c160 tx=0x7fc288005880 comp rx=0 tx=0).stop 2023-11-28T12:05:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.087+0000 7fc2a2c20700 1 -- 172.21.15.57:0/4165715352 shutdown_connections 2023-11-28T12:05:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.087+0000 7fc2a2c20700 1 --2- 172.21.15.57:0/4165715352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc288005200 0x7fc28800ba10 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.087+0000 7fc2a2c20700 1 -- 172.21.15.57:0/4165715352 >> 172.21.15.57:0/4165715352 conn(0x7fc29c06c010 msgr2=0x7fc29c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.087+0000 7fc2a2c20700 1 -- 172.21.15.57:0/4165715352 shutdown_connections 2023-11-28T12:05:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.087+0000 7fc2a2c20700 1 -- 172.21.15.57:0/4165715352 wait complete. 2023-11-28T12:05:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.088+0000 7fc2a2c20700 1 Processor -- start 2023-11-28T12:05:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.088+0000 7fc2a2c20700 1 -- start start 2023-11-28T12:05:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.089+0000 7fc2a2c20700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc288005200 0x7fc29c1537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.089+0000 7fc2a2c20700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc29c152220 0x7fc29c0b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.089+0000 7fc2a2c20700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc29c155070 0x7fc29c1554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.089+0000 7fc2a2c20700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c0b42a0 con 0x7fc288005200 2023-11-28T12:05:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.089+0000 7fc2a2c20700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c0b4580 con 0x7fc29c152220 2023-11-28T12:05:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.089+0000 7fc2a2c20700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c15dec0 con 0x7fc29c155070 2023-11-28T12:05:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.089+0000 7fc2a09bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc288005200 0x7fc29c1537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.089+0000 7fc2a09bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc288005200 0x7fc29c1537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46484/0 (socket says 172.21.15.57:46484) 2023-11-28T12:05:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.089+0000 7fc2a09bc700 1 -- 172.21.15.57:0/830381303 learned_addr learned my addr 172.21.15.57:0/830381303 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.089+0000 7fc2a11bd700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc29c155070 0x7fc29c1554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.089+0000 7fc29bfff700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc29c152220 0x7fc29c0b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.090+0000 7fc2a09bc700 1 -- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc29c155070 msgr2=0x7fc29c1554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.090+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc29c155070 0x7fc29c1554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.090+0000 7fc2a09bc700 1 -- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc29c152220 msgr2=0x7fc29c0b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.090+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc29c152220 0x7fc29c0b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.090+0000 7fc2a09bc700 1 -- 172.21.15.57:0/830381303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c06d530 con 0x7fc288005200 2023-11-28T12:05:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.090+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc288005200 0x7fc29c1537b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc28c011ca0 tx=0x7fc28c009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.090+0000 7fc2997fa700 1 -- 172.21.15.57:0/830381303 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c004020 con 0x7fc288005200 2023-11-28T12:05:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.090+0000 7fc2997fa700 1 -- 172.21.15.57:0/830381303 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c004180 con 0x7fc288005200 2023-11-28T12:05:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.090+0000 7fc2997fa700 1 -- 172.21.15.57:0/830381303 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c004530 con 0x7fc288005200 2023-11-28T12:05:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.090+0000 7fc2a2c20700 1 -- 172.21.15.57:0/830381303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29c06d250 con 0x7fc288005200 2023-11-28T12:05:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.090+0000 7fc2a2c20700 1 -- 172.21.15.57:0/830381303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29c0ce740 con 0x7fc288005200 2023-11-28T12:05:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.092+0000 7fc2997fa700 1 -- 172.21.15.57:0/830381303 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc28c024070 con 0x7fc288005200 2023-11-28T12:05:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.092+0000 7fc2997fa700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc27c0682f0 0x7fc27c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.092+0000 7fc2997fa700 1 -- 172.21.15.57:0/830381303 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc28c085f60 con 0x7fc288005200 2023-11-28T12:05:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.092+0000 7fc29bfff700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc27c0682f0 0x7fc27c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.093+0000 7fc29bfff700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc27c0682f0 0x7fc27c06a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc284001c90 tx=0x7fc284001b20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.094+0000 7fc2aedbfb80 1 Processor -- start 2023-11-28T12:05:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.094+0000 7fc2aedbfb80 1 -- start start 2023-11-28T12:05:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2aedbfb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 0x5621e1fdb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2aedbfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e2022320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2aedbfb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e1fdcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2aedbfb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e1f59de0 con 0x5621e2004660 2023-11-28T12:05:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2aedbfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e1ec6140 con 0x5621e2018ac0 2023-11-28T12:05:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2aedbfb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621e1f34db0 con 0x5621e2021f50 2023-11-28T12:05:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2a11bd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e1fdcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2a11bd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e1fdcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46496/0 (socket says 172.21.15.57:46496) 2023-11-28T12:05:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2a11bd700 1 -- 172.21.15.57:0/467061563 learned_addr learned my addr 172.21.15.57:0/467061563 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/467061563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 0x5621e1fdb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc29bfff700 1 --2- 172.21.15.57:0/467061563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e2022320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2a11bd700 1 -- 172.21.15.57:0/467061563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 msgr2=0x5621e1fdb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2a11bd700 1 --2- 172.21.15.57:0/467061563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 0x5621e1fdb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2a11bd700 1 -- 172.21.15.57:0/467061563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 msgr2=0x5621e2022320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2a11bd700 1 --2- 172.21.15.57:0/467061563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e2022320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc2a11bd700 1 -- 172.21.15.57:0/467061563 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621e1d22790 con 0x5621e2004660 2023-11-28T12:05:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.095+0000 7fc29bfff700 1 --2- 172.21.15.57:0/467061563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e2022320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.096+0000 7fc2a11bd700 1 --2- 172.21.15.57:0/467061563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e1fdcb60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0672e0 tx=0x7fc294011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e935c9f5f250bb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.096+0000 7fc2927fc700 1 -- 172.21.15.57:0/467061563 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29400ceb0 con 0x5621e2004660 2023-11-28T12:05:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.096+0000 7fc2927fc700 1 -- 172.21.15.57:0/467061563 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc294001cd0 con 0x5621e2004660 2023-11-28T12:05:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.096+0000 7fc2927fc700 1 -- 172.21.15.57:0/467061563 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294003c20 con 0x5621e2004660 2023-11-28T12:05:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.096+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/467061563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 msgr2=0x5621e1fdcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.096+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/467061563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e1fdcb60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0672e0 tx=0x7fc294011000 comp rx=0 tx=0).stop 2023-11-28T12:05:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.097+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/467061563 shutdown_connections 2023-11-28T12:05:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.097+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/467061563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e1fdcb60 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.097+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/467061563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 0x5621e1fdb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.097+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/467061563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e2022320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.097+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/467061563 >> 172.21.15.57:0/467061563 conn(0x5621e1fd1d20 msgr2=0x5621e1fdaf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.097+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/467061563 shutdown_connections 2023-11-28T12:05:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.097+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/467061563 wait complete. 2023-11-28T12:05:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.097+0000 7fc2aedbfb80 1 -- start start 2023-11-28T12:05:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.098+0000 7fc2aedbfb80 1 Processor -- start 2023-11-28T12:05:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.098+0000 7fc2aedbfb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e200ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.098+0000 7fc2aedbfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e1fffab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.098+0000 7fc2aedbfb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 0x5621e2034990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.098+0000 7fc2aedbfb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e1f2e120 con 0x5621e2004660 2023-11-28T12:05:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.098+0000 7fc2aedbfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e1d23390 con 0x5621e2018ac0 2023-11-28T12:05:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.098+0000 7fc2aedbfb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621e1ec6340 con 0x5621e2021f50 2023-11-28T12:05:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.098+0000 7fc2aedbfb80 10 client.0 _ll_register_callbacks cb 0x5621e1ecc400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:05:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.098+0000 7fc2a09bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e200ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.098+0000 7fc2a09bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e200ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46502/0 (socket says 172.21.15.57:46502) 2023-11-28T12:05:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.098+0000 7fc2a09bc700 1 -- 172.21.15.57:0/4202805125 learned_addr learned my addr 172.21.15.57:0/4202805125 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc2a11bd700 1 --2- 172.21.15.57:0/4202805125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 0x5621e2034990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc29bfff700 1 --2- 172.21.15.57:0/4202805125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e1fffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc2a09bc700 1 -- 172.21.15.57:0/4202805125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 msgr2=0x5621e2034990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/4202805125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 0x5621e2034990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc2a09bc700 1 -- 172.21.15.57:0/4202805125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 msgr2=0x5621e1fffab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/4202805125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e1fffab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc2a09bc700 1 -- 172.21.15.57:0/4202805125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2880035b0 con 0x5621e2004660 2023-11-28T12:05:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/4202805125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e200ac10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0563b0 tx=0x7fc28c055a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c848edc7b6c53019 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc27b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:05:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc27b7fe700 1 -- 172.21.15.57:0/4202805125 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c069dc0 con 0x5621e2004660 2023-11-28T12:05:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc27b7fe700 1 -- 172.21.15.57:0/4202805125 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc28c052270 con 0x5621e2004660 2023-11-28T12:05:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/4202805125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621e1f32030 con 0x5621e2004660 2023-11-28T12:05:03.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc27b7fe700 1 -- 172.21.15.57:0/4202805125 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc28c052430 con 0x5621e2004660 2023-11-28T12:05:03.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc2aedbfb80 10 client.6375 fetch_fsmap learned FSMap version 32 2023-11-28T12:05:03.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/4202805125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5621e1e0f0a0 con 0x5621e2004660 2023-11-28T12:05:03.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc27b7fe700 1 -- 172.21.15.57:0/4202805125 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5621e1f32030 con 0x5621e2004660 2023-11-28T12:05:03.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.099+0000 7fc27b7fe700 1 -- 172.21.15.57:0/4202805125 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc28c0553e0 con 0x5621e2004660 2023-11-28T12:05:03.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.100+0000 7fc2aedbfb80 10 client.6375 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T12:05:03.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.100+0000 7fc2aedbfb80 10 client.6375 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T12:05:03.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.100+0000 7fc2aedbfb80 20 client.6375 populate_metadata read hostname 'smithi057' 2023-11-28T12:05:03.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.100+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/4202805125 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5621e20190e0 0x5621e20194b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.100+0000 7fc2aedbfb80 4 client.6375 mds_command: new command op to 6219 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T12:05:03.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.100+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/4202805125 --> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621e1d7def0 con 0x5621e20190e0 2023-11-28T12:05:03.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.100+0000 7fc29bfff700 1 --2- 172.21.15.57:0/4202805125 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5621e20190e0 0x5621e20194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.100+0000 7fc27b7fe700 10 client.6375 ms_handle_connect on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.100+0000 7fc29bfff700 1 --2- 172.21.15.57:0/4202805125 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5621e20190e0 0x5621e20194b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c93981a497fa05f2 server_cookie=4006ba2df33861ec in_seq=0 out_seq=0 2023-11-28T12:05:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc27b7fe700 1 -- 172.21.15.57:0/4202805125 <== mds.0 v2:172.21.15.57:6833/685888843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5621e1d7def0 con 0x5621e20190e0 2023-11-28T12:05:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc27b7fe700 10 client.6375 handle_command_reply: tid=0 2023-11-28T12:05:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc2aedbfb80 1 client.6375 shutdown 2023-11-28T12:05:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/4202805125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 msgr2=0x5621e200ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/4202805125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e200ac10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0563b0 tx=0x7fc28c055a20 comp rx=0 tx=0).stop 2023-11-28T12:05:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/4202805125 shutdown_connections 2023-11-28T12:05:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/4202805125 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5621e20190e0 0x5621e20194b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/4202805125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 0x5621e2034990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/4202805125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e200ac10 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc27b7fe700 0 client.6375 ms_handle_reset on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:03.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/4202805125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e1fffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/4202805125 >> 172.21.15.57:0/4202805125 conn(0x5621e1fd1d20 msgr2=0x5621e1fd6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:03.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/4202805125 shutdown_connections 2023-11-28T12:05:03.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/4202805125 wait complete. 2023-11-28T12:05:03.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.101+0000 7fc2aedbfb80 20 client.6375 trim_cache size 0 max 16384 2023-11-28T12:05:03.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.119+0000 7fc2aedbfb80 1 Processor -- start 2023-11-28T12:05:03.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.119+0000 7fc2aedbfb80 1 -- start start 2023-11-28T12:05:03.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc2aedbfb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e1fd7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc2aedbfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e2004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc2aedbfb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 0x5621e2005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc2aedbfb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e20073e0 con 0x5621e2004660 2023-11-28T12:05:03.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc2aedbfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e203b700 con 0x5621e2018ac0 2023-11-28T12:05:03.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc2aedbfb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621e2007b90 con 0x5621e2021f50 2023-11-28T12:05:03.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc2a09bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e1fd7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc2a09bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e1fd7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46506/0 (socket says 172.21.15.57:46506) 2023-11-28T12:05:03.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc2a09bc700 1 -- 172.21.15.57:0/712418970 learned_addr learned my addr 172.21.15.57:0/712418970 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:03.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc2a11bd700 1 --2- 172.21.15.57:0/712418970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 0x5621e2005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc29bfff700 1 --2- 172.21.15.57:0/712418970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e2004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc2a09bc700 1 -- 172.21.15.57:0/712418970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 msgr2=0x5621e2005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:03.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/712418970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 0x5621e2005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.120+0000 7fc2a09bc700 1 -- 172.21.15.57:0/712418970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 msgr2=0x5621e2004bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:03.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/712418970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e2004bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc2a09bc700 1 -- 172.21.15.57:0/712418970 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621e1ef52e0 con 0x5621e2004660 2023-11-28T12:05:03.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc2a11bd700 1 --2- 172.21.15.57:0/712418970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 0x5621e2005130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:03.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc29bfff700 1 --2- 172.21.15.57:0/712418970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e2004bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:03.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/712418970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e1fd7250 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc28c052980 tx=0x7fc28c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9e0341975b8b50 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:03.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc292ffd700 1 -- 172.21.15.57:0/712418970 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0553e0 con 0x5621e2004660 2023-11-28T12:05:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc292ffd700 1 -- 172.21.15.57:0/712418970 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c055540 con 0x5621e2004660 2023-11-28T12:05:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc292ffd700 1 -- 172.21.15.57:0/712418970 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c069dc0 con 0x5621e2004660 2023-11-28T12:05:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/712418970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 msgr2=0x5621e1fd7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/712418970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e1fd7250 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc28c052980 tx=0x7fc28c017000 comp rx=0 tx=0).stop 2023-11-28T12:05:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/712418970 shutdown_connections 2023-11-28T12:05:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/712418970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2021f50 0x5621e2005130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/712418970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e1fd7250 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/712418970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2018ac0 0x5621e2004bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.121+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/712418970 >> 172.21.15.57:0/712418970 conn(0x5621e1fd1d20 msgr2=0x5621e2022320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.122+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/712418970 shutdown_connections 2023-11-28T12:05:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.122+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/712418970 wait complete. 2023-11-28T12:05:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.122+0000 7fc2aedbfb80 1 -- start start 2023-11-28T12:05:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.122+0000 7fc2aedbfb80 1 Processor -- start 2023-11-28T12:05:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.123+0000 7fc2aedbfb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e2002b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.123+0000 7fc2aedbfb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2018ac0 0x5621e20833f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.123+0000 7fc2aedbfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2021f50 0x5621e2004b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.123+0000 7fc2aedbfb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e2024af0 con 0x5621e2004660 2023-11-28T12:05:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.123+0000 7fc2aedbfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e2083930 con 0x5621e2021f50 2023-11-28T12:05:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.123+0000 7fc2aedbfb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621e2024f50 con 0x5621e2018ac0 2023-11-28T12:05:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.123+0000 7fc2aedbfb80 10 client.0 _ll_register_callbacks cb 0x5621e1fd80b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:05:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.123+0000 7fc2a09bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e2002b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.123+0000 7fc2a09bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e2002b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46520/0 (socket says 172.21.15.57:46520) 2023-11-28T12:05:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.123+0000 7fc2a09bc700 1 -- 172.21.15.57:0/427887606 learned_addr learned my addr 172.21.15.57:0/427887606 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.123+0000 7fc2a11bd700 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2021f50 0x5621e2004b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.123+0000 7fc29bfff700 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2018ac0 0x5621e20833f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc2a09bc700 1 -- 172.21.15.57:0/427887606 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2018ac0 msgr2=0x5621e20833f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:03.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2018ac0 0x5621e20833f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc2a09bc700 1 -- 172.21.15.57:0/427887606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2021f50 msgr2=0x5621e2004b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:03.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2021f50 0x5621e2004b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc2a09bc700 1 -- 172.21.15.57:0/427887606 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621e1e0f0a0 con 0x5621e2004660 2023-11-28T12:05:03.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc29bfff700 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2018ac0 0x5621e20833f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:03.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc2a11bd700 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2021f50 0x5621e2004b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:03.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc2a09bc700 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e2002b90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc28c052980 tx=0x7fc28c0528f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf027d08fcb690b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:03.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc27b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:05:03.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc27b7fe700 1 -- 172.21.15.57:0/427887606 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c069dc0 con 0x5621e2004660 2023-11-28T12:05:03.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc27b7fe700 1 -- 172.21.15.57:0/427887606 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc28c052270 con 0x5621e2004660 2023-11-28T12:05:03.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/427887606 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621e1db3040 con 0x5621e2004660 2023-11-28T12:05:03.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc27b7fe700 1 -- 172.21.15.57:0/427887606 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc28c052430 con 0x5621e2004660 2023-11-28T12:05:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc2aedbfb80 10 client.6381 fetch_fsmap learned FSMap version 32 2023-11-28T12:05:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/427887606 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5621e1d831d0 con 0x5621e2004660 2023-11-28T12:05:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc27b7fe700 1 -- 172.21.15.57:0/427887606 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c075020 con 0x5621e2004660 2023-11-28T12:05:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc27b7fe700 1 -- 172.21.15.57:0/427887606 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc28c005a20 con 0x5621e2004660 2023-11-28T12:05:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc2aedbfb80 10 client.6381 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T12:05:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc2aedbfb80 10 client.6381 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T12:05:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.124+0000 7fc2aedbfb80 20 client.6381 populate_metadata read hostname 'smithi057' 2023-11-28T12:05:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5621e2086170 0x5621e2086540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 4 client.6381 mds_command: new command op to 6219 tid=0[{"prefix": "status"}] 2023-11-28T12:05:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/427887606 --> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5621e1e703c0 con 0x5621e2086170 2023-11-28T12:05:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc29bfff700 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5621e2086170 0x5621e2086540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc27b7fe700 10 client.6381 ms_handle_connect on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc29bfff700 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5621e2086170 0x5621e2086540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d132bdf16fe1a86 server_cookie=e32127f41f0b23a5 in_seq=0 out_seq=0 2023-11-28T12:05:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc27b7fe700 1 -- 172.21.15.57:0/427887606 <== mds.0 v2:172.21.15.57:6833/685888843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5621e1e703c0 con 0x5621e2086170 2023-11-28T12:05:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc27b7fe700 10 client.6381 handle_command_reply: tid=0 2023-11-28T12:05:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 1 client.6381 shutdown 2023-11-28T12:05:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/427887606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 msgr2=0x5621e2002b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e2002b90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc28c052980 tx=0x7fc28c0528f0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/427887606 shutdown_connections 2023-11-28T12:05:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5621e2086170 0x5621e2086540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621e2021f50 0x5621e2004b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5621e2004660 0x5621e2002b90 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 1 --2- 172.21.15.57:0/427887606 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5621e2018ac0 0x5621e20833f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc27b7fe700 0 client.6381 ms_handle_reset on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:03.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/427887606 >> 172.21.15.57:0/427887606 conn(0x5621e1fd1d20 msgr2=0x5621e1ffef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:03.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/427887606 shutdown_connections 2023-11-28T12:05:03.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.125+0000 7fc2aedbfb80 1 -- 172.21.15.57:0/427887606 wait complete. 2023-11-28T12:05:03.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.126+0000 7fc2aedbfb80 20 client.6381 trim_cache size 0 max 16384 2023-11-28T12:05:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:05:03.134 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "0309b83f-6570-4eee-96fa-279822cd7766", 2023-11-28T12:05:03.134 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2023-11-28T12:05:03.134 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 6219, 2023-11-28T12:05:03.135 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-11-28T12:05:03.135 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-11-28T12:05:03.135 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-11-28T12:05:03.135 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 10.763895684, 2023-11-28T12:05:03.135 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 32, 2023-11-28T12:05:03.135 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 63, 2023-11-28T12:05:03.135 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 63, 2023-11-28T12:05:03.135 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 16.003851022999999 2023-11-28T12:05:03.135 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:05:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.126+0000 7fc2a2c20700 1 -- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc27c0682f0 msgr2=0x7fc27c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.126+0000 7fc2a2c20700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc27c0682f0 0x7fc27c06a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc284001c90 tx=0x7fc284001b20 comp rx=0 tx=0).stop 2023-11-28T12:05:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.126+0000 7fc2a2c20700 1 -- 172.21.15.57:0/830381303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc288005200 msgr2=0x7fc29c1537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.126+0000 7fc2a2c20700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc288005200 0x7fc29c1537b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc28c011ca0 tx=0x7fc28c009e80 comp rx=0 tx=0).stop 2023-11-28T12:05:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.126+0000 7fc2a2c20700 1 -- 172.21.15.57:0/830381303 shutdown_connections 2023-11-28T12:05:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.126+0000 7fc2a2c20700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc27c0682f0 0x7fc27c06a7a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.126+0000 7fc2a2c20700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc29c155070 0x7fc29c1554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.126+0000 7fc2a2c20700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc288005200 0x7fc29c1537b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.126+0000 7fc2a2c20700 1 --2- 172.21.15.57:0/830381303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc29c152220 0x7fc29c0b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.126+0000 7fc2a2c20700 1 -- 172.21.15.57:0/830381303 >> 172.21.15.57:0/830381303 conn(0x7fc29c06c010 msgr2=0x7fc29c15bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:03.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.126+0000 7fc2a2c20700 1 -- 172.21.15.57:0/830381303 shutdown_connections 2023-11-28T12:05:03.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.126+0000 7fc2a2c20700 1 -- 172.21.15.57:0/830381303 wait complete. 2023-11-28T12:05:03.141 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T12:05:03.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.274+0000 7fb315690700 1 Processor -- start 2023-11-28T12:05:03.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.274+0000 7fb315690700 1 -- start start 2023-11-28T12:05:03.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.275+0000 7fb315690700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb31006a270 0x7fb31006a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.275+0000 7fb315690700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb31015e0b0 0x7fb31015e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.275+0000 7fb315690700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb310151a50 con 0x7fb310070d90 2023-11-28T12:05:03.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.275+0000 7fb315690700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31015b440 con 0x7fb31015e0b0 2023-11-28T12:05:03.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.275+0000 7fb315690700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31006ab80 con 0x7fb31006a270 2023-11-28T12:05:03.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.275+0000 7fb30f7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb31015e0b0 0x7fb31015e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.275+0000 7fb30f7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb31015e0b0 0x7fb31015e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46526/0 (socket says 172.21.15.57:46526) 2023-11-28T12:05:03.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.275+0000 7fb30f7fe700 1 -- 172.21.15.57:0/1421653018 learned_addr learned my addr 172.21.15.57:0/1421653018 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:03.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.275+0000 7fb30effd700 1 --2- 172.21.15.57:0/1421653018 >> v2:172.21.15.181:3300/0 conn(0x7fb31006a270 0x7fb31006a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.275+0000 7fb30f7fe700 1 -- 172.21.15.57:0/1421653018 >> v2:172.21.15.181:3300/0 conn(0x7fb31006a270 msgr2=0x7fb31006a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:03.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.275+0000 7fb30f7fe700 1 --2- 172.21.15.57:0/1421653018 >> v2:172.21.15.181:3300/0 conn(0x7fb31006a270 0x7fb31006a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.275+0000 7fb30f7fe700 1 -- 172.21.15.57:0/1421653018 >> v1:172.21.15.181:6789/0 conn(0x7fb310070d90 legacy=0x7fb3100711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.275+0000 7fb30f7fe700 1 -- 172.21.15.57:0/1421653018 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb310071ac0 con 0x7fb31015e0b0 2023-11-28T12:05:03.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.276+0000 7fb30f7fe700 1 --2- 172.21.15.57:0/1421653018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb31015e0b0 0x7fb31015e4f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb3100cd250 tx=0x7fb300009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe1b3565605dc1b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:03.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.276+0000 7fb30dffb700 1 -- 172.21.15.57:0/1421653018 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300011830 con 0x7fb31015e0b0 2023-11-28T12:05:03.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.276+0000 7fb30dffb700 1 -- 172.21.15.57:0/1421653018 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb300011990 con 0x7fb31015e0b0 2023-11-28T12:05:03.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.276+0000 7fb30dffb700 1 -- 172.21.15.57:0/1421653018 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300011d20 con 0x7fb31015e0b0 2023-11-28T12:05:03.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.276+0000 7fb315690700 1 -- 172.21.15.57:0/1421653018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb31015e0b0 msgr2=0x7fb31015e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.276+0000 7fb315690700 1 --2- 172.21.15.57:0/1421653018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb31015e0b0 0x7fb31015e4f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb3100cd250 tx=0x7fb300009bd0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.276+0000 7fb315690700 1 -- 172.21.15.57:0/1421653018 shutdown_connections 2023-11-28T12:05:03.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.276+0000 7fb315690700 1 --2- 172.21.15.57:0/1421653018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb31015e0b0 0x7fb31015e4f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.276+0000 7fb315690700 1 --2- 172.21.15.57:0/1421653018 >> v2:172.21.15.181:3300/0 conn(0x7fb31006a270 0x7fb31006a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.276+0000 7fb315690700 1 -- 172.21.15.57:0/1421653018 >> 172.21.15.57:0/1421653018 conn(0x7fb31006c010 msgr2=0x7fb3100cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:03.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.276+0000 7fb315690700 1 -- 172.21.15.57:0/1421653018 shutdown_connections 2023-11-28T12:05:03.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.276+0000 7fb315690700 1 -- 172.21.15.57:0/1421653018 wait complete. 2023-11-28T12:05:03.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.278+0000 7fb315690700 1 Processor -- start 2023-11-28T12:05:03.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.278+0000 7fb315690700 1 -- start start 2023-11-28T12:05:03.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb315690700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb31006a270 0x7fb3100c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb315690700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb310070d90 0x7fb3100c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb315690700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb31015e0b0 0x7fb310152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb315690700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100c4290 con 0x7fb31015e0b0 2023-11-28T12:05:03.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb315690700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb310150e50 con 0x7fb31006a270 2023-11-28T12:05:03.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb315690700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100c6800 con 0x7fb310070d90 2023-11-28T12:05:03.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb30f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb31015e0b0 0x7fb310152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb30f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb31015e0b0 0x7fb310152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46532/0 (socket says 172.21.15.57:46532) 2023-11-28T12:05:03.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb30f7fe700 1 -- 172.21.15.57:0/2877001546 learned_addr learned my addr 172.21.15.57:0/2877001546 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:03.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb30e7fc700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb310070d90 0x7fb3100c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb30effd700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb31006a270 0x7fb3100c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb30f7fe700 1 -- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb310070d90 msgr2=0x7fb3100c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb30f7fe700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb310070d90 0x7fb3100c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb30f7fe700 1 -- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb31006a270 msgr2=0x7fb3100c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb30f7fe700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb31006a270 0x7fb3100c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb30f7fe700 1 -- 172.21.15.57:0/2877001546 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb300009550 con 0x7fb31015e0b0 2023-11-28T12:05:03.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb30f7fe700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb31015e0b0 0x7fb310152090 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb300009c10 tx=0x7fb300004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:03.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.279+0000 7fb3077fe700 1 -- 172.21.15.57:0/2877001546 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300004570 con 0x7fb31015e0b0 2023-11-28T12:05:03.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.280+0000 7fb3077fe700 1 -- 172.21.15.57:0/2877001546 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30001f070 con 0x7fb31015e0b0 2023-11-28T12:05:03.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.280+0000 7fb315690700 1 -- 172.21.15.57:0/2877001546 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31015b0d0 con 0x7fb31015e0b0 2023-11-28T12:05:03.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.280+0000 7fb3077fe700 1 -- 172.21.15.57:0/2877001546 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3000038e0 con 0x7fb31015e0b0 2023-11-28T12:05:03.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.280+0000 7fb315690700 1 -- 172.21.15.57:0/2877001546 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31015adf0 con 0x7fb31015e0b0 2023-11-28T12:05:03.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.281+0000 7fb315690700 1 -- 172.21.15.57:0/2877001546 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3100c94f0 con 0x7fb31015e0b0 2023-11-28T12:05:03.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.281+0000 7fb3077fe700 1 -- 172.21.15.57:0/2877001546 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb30002a020 con 0x7fb31015e0b0 2023-11-28T12:05:03.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.282+0000 7fb3077fe700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb2f0068320 0x7fb2f006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:03.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.282+0000 7fb3077fe700 1 -- 172.21.15.57:0/2877001546 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb3000872e0 con 0x7fb31015e0b0 2023-11-28T12:05:03.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.282+0000 7fb30effd700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb2f0068320 0x7fb2f006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:03.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.282+0000 7fb30effd700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb2f0068320 0x7fb2f006a7d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb3080073f0 tx=0x7fb30800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:03.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.286+0000 7fb3077fe700 1 -- 172.21.15.57:0/2877001546 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb3100c94f0 con 0x7fb31015e0b0 2023-11-28T12:05:03.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.536+0000 7fb315690700 1 -- 172.21.15.57:0/2877001546 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb3100c94f0 con 0x7fb31015e0b0 2023-11-28T12:05:03.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.536+0000 7fb3077fe700 1 -- 172.21.15.57:0/2877001546 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb3100c94f0 con 0x7fb31015e0b0 2023-11-28T12:05:03.538 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T12:05:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.539+0000 7fb315690700 1 -- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb2f0068320 msgr2=0x7fb2f006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.539+0000 7fb315690700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb2f0068320 0x7fb2f006a7d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb3080073f0 tx=0x7fb30800b040 comp rx=0 tx=0).stop 2023-11-28T12:05:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.539+0000 7fb315690700 1 -- 172.21.15.57:0/2877001546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb31015e0b0 msgr2=0x7fb310152090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.539+0000 7fb315690700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb31015e0b0 0x7fb310152090 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb300009c10 tx=0x7fb300004750 comp rx=0 tx=0).stop 2023-11-28T12:05:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.539+0000 7fb315690700 1 -- 172.21.15.57:0/2877001546 shutdown_connections 2023-11-28T12:05:03.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.539+0000 7fb315690700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb2f0068320 0x7fb2f006a7d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.539+0000 7fb315690700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb31015e0b0 0x7fb310152090 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.539+0000 7fb315690700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb31006a270 0x7fb3100c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.539+0000 7fb315690700 1 --2- 172.21.15.57:0/2877001546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb310070d90 0x7fb3100c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:03.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.539+0000 7fb315690700 1 -- 172.21.15.57:0/2877001546 >> 172.21.15.57:0/2877001546 conn(0x7fb31006c010 msgr2=0x7fb3100cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:03.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.539+0000 7fb315690700 1 -- 172.21.15.57:0/2877001546 shutdown_connections 2023-11-28T12:05:03.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:03.539+0000 7fb315690700 1 -- 172.21.15.57:0/2877001546 wait complete. 2023-11-28T12:05:03.553 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T12:05:03.553 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T12:05:03.553 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2023-11-28T12:05:03.554 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T12:05:03.554 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T12:05:03.554 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T12:05:03.554 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T12:05:03.554 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T12:05:03.554 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T12:05:03.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:05:03.554 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2023-11-28T12:05:03.577 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T12:05:03.577 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T12:05:03.577 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T12:05:03.577 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:05:03.577 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-11-28T12:05:03.578 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:05:03.578 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:05:03.578 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:05:03.578 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f0 2023-11-28T12:05:03.578 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:05:03.578 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:05:03.578 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f1 2023-11-28T12:05:03.578 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T12:05:03.578 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:05:03.578 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f0 2023-11-28T12:05:03.579 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T12:05:03.579 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 41756sec preferred_lft 41756sec 2023-11-28T12:05:03.579 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-11-28T12:05:03.579 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:05:03.579 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T12:05:03.579 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2023-11-28T12:05:03.579 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f1 2023-11-28T12:05:03.579 INFO:teuthology.orchestra.run.smithi057.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T12:05:03.579 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 8e:b7:77:39:69:ff brd ff:ff:ff:ff:ff:ff 2023-11-28T12:05:03.579 INFO:teuthology.orchestra.run.smithi057.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T12:05:03.579 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:05:03.579 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::3f:caff:fe00:648e/64 scope link 2023-11-28T12:05:03.580 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:05:03.580 INFO:teuthology.orchestra.run.smithi057.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T12:05:03.580 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether f2:ef:c1:11:c5:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T12:05:03.580 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::f0ef:c1ff:fe11:c5fb/64 scope link 2023-11-28T12:05:03.580 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:05:03.580 INFO:teuthology.orchestra.run.smithi057.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T12:05:03.580 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 8e:b7:77:39:69:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T12:05:03.580 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::8cb7:77ff:fe39:69ff/64 scope link 2023-11-28T12:05:03.580 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-11-28T12:05:03.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:05:03.581 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T12:05:03.649 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-11-28T12:05:03.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:05:03.651 DEBUG:teuthology.orchestra.run.smithi057:> route 2023-11-28T12:05:03.716 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2023-11-28T12:05:03.716 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T12:05:03.716 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T12:05:03.717 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T12:05:03.717 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T12:05:03.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:05:03.717 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-11-28T12:05:03.717 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-11-28T12:05:03.717 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T12:05:03.717 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T12:05:03.717 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T12:05:03.717 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-11-28T12:05:03.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:05:03.839 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-11-28T12:05:03.861 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T12:05:03.861 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T12:05:03.861 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T12:05:03.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:05:03.862 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2023-11-28T12:05:03.937 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T12:05:03.938 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:05:03.938 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T12:05:04.003 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T12:05:04.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T12:05:04.003 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T12:05:04.066 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-11-28T12:05:04.141 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:05:04.141 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:05:04.217 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:05:04.217 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:05:04.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:05:04.218 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-11-28T12:05:04.281 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T12:05:04.281 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/) 2023-11-28T12:05:04.324 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-11-28T12:05:04.359 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T12:05:04.359 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T12:05:04.429 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:2023-11-28T12:05:04.428+0000 7f5e77b4e5c0 -1 init, newargv = 0x558382901cc0 newargc=15 2023-11-28T12:05:04.429 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[145733]: starting ceph client 2023-11-28T12:05:04.430 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T12:05:04.432 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T12:05:04.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:05:04.432 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-11-28T12:05:04.434 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[145733]: starting fuse 2023-11-28T12:05:04.454 INFO:teuthology.orchestra.run.smithi057.stdout:47 2023-11-28T12:05:04.454 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T12:05:04.454 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:05:04.454 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2023-11-28T12:05:04.454 DEBUG:teuthology.orchestra.run.smithi057:> import re 2023-11-28T12:05:04.454 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-11-28T12:05:04.455 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2023-11-28T12:05:04.455 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:05:04.455 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2023-11-28T12:05:04.455 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T12:05:04.455 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2023-11-28T12:05:04.455 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T12:05:04.455 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:05:04.455 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2023-11-28T12:05:04.455 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2023-11-28T12:05:04.456 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2023-11-28T12:05:04.456 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2023-11-28T12:05:04.456 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:05:04.456 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2023-11-28T12:05:04.456 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T12:05:04.456 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T12:05:04.456 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T12:05:04.456 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2023-11-28T12:05:04.456 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2023-11-28T12:05:04.456 DEBUG:teuthology.orchestra.run.smithi057:> return f 2023-11-28T12:05:04.456 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T12:05:04.456 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:05:04.457 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.0")) 2023-11-28T12:05:04.457 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-11-28T12:05:04.590 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.0.145733.asok 2023-11-28T12:05:04.770 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.145733.asok 2023-11-28T12:05:04.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:05:04.771 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.145733.asok status 2023-11-28T12:05:04.949 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:05:04.949 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2023-11-28T12:05:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T12:05:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T12:05:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "0", 2023-11-28T12:05:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2023-11-28T12:05:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T12:05:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "145733", 2023-11-28T12:05:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2023-11-28T12:05:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:05:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2023-11-28T12:05:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2023-11-28T12:05:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 6396, 2023-11-28T12:05:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2023-11-28T12:05:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2023-11-28T12:05:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2023-11-28T12:05:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 6396 2023-11-28T12:05:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:05:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-11-28T12:05:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-11-28T12:05:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-11-28T12:05:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 4279255996 2023-11-28T12:05:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:05:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:05:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-11-28T12:05:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-11-28T12:05:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-11-28T12:05:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 4279255996 2023-11-28T12:05:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:05:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.6396 192.168.0.1:0/4279255996", 2023-11-28T12:05:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.1:0/4279255996", 2023-11-28T12:05:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2023-11-28T12:05:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 32, 2023-11-28T12:05:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 63, 2023-11-28T12:05:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2023-11-28T12:05:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2023-11-28T12:05:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2023-11-28T12:05:04.952 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:05:04.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:05:04.961 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2023-11-28T12:05:04.961 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T12:05:04.981 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2023-11-28T12:05:04.981 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T12:05:04.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:05:04.982 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T12:05:05.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:05:05.055 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2023-11-28T12:05:05.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:05:05.125 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2023-11-28T12:05:05.191 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T12:05:05.191 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-11-28T12:05:05.191 DEBUG:teuthology.orchestra.run.smithi057:> import stat 2023-11-28T12:05:05.192 DEBUG:teuthology.orchestra.run.smithi057:> 2023-11-28T12:05:05.192 DEBUG:teuthology.orchestra.run.smithi057:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2023-11-28T12:05:05.192 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-11-28T12:05:05.315 INFO:teuthology.orchestra.run.smithi057.stdout:1099511627777 2023-11-28T12:05:05.502 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:05:05.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.689+0000 7f15dadf2700 1 Processor -- start 2023-11-28T12:05:05.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.689+0000 7f15dadf2700 1 -- start start 2023-11-28T12:05:05.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.690+0000 7f15dadf2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f15d406b7b0 0x7f15d4072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:05.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.690+0000 7f15dadf2700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d40726c0 con 0x7f15d40ad1f0 2023-11-28T12:05:05.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.690+0000 7f15dadf2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d40b43f0 con 0x7f15d40b0880 2023-11-28T12:05:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.690+0000 7f15dadf2700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d406bc50 con 0x7f15d406b7b0 2023-11-28T12:05:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.690+0000 7f15d3fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f15d406b7b0 0x7f15d4072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.690+0000 7f15d938f700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f15d40ad1f0 0x7f15d40ad610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54310/0 (socket says 172.21.15.57:54310) 2023-11-28T12:05:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.690+0000 7f15d3fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f15d406b7b0 0x7f15d4072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47026/0 (socket says 172.21.15.57:47026) 2023-11-28T12:05:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.690+0000 7f15d938f700 1 -- 172.21.15.57:0/969395950 learned_addr learned my addr 172.21.15.57:0/969395950 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:05:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.691+0000 7f15d3fff700 1 -- 172.21.15.57:0/969395950 >> v1:172.21.15.181:6789/0 conn(0x7f15d40b0880 legacy=0x7f15d40aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.691+0000 7f15d3fff700 1 -- 172.21.15.57:0/969395950 >> v1:172.21.15.57:6789/0 conn(0x7f15d40ad1f0 legacy=0x7f15d40ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.691+0000 7f15d3fff700 1 -- 172.21.15.57:0/969395950 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40b46d0 con 0x7f15d406b7b0 2023-11-28T12:05:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.691+0000 7f15d37fe700 1 -- 172.21.15.57:0/969395950 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616384957 0 0) 0x7f15d40726c0 con 0x7f15d40ad1f0 2023-11-28T12:05:05.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.691+0000 7f15d3fff700 1 --2- 172.21.15.57:0/969395950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15d406b7b0 0x7f15d4072180 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009840 tx=0x7f15bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351146ec2e4a3f99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:05.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.691+0000 7f15d37fe700 1 -- 172.21.15.57:0/969395950 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc011830 con 0x7f15d406b7b0 2023-11-28T12:05:05.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.691+0000 7f15d37fe700 1 -- 172.21.15.57:0/969395950 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc011990 con 0x7f15d406b7b0 2023-11-28T12:05:05.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.691+0000 7f15d37fe700 1 -- 172.21.15.57:0/969395950 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc011d20 con 0x7f15d406b7b0 2023-11-28T12:05:05.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.692+0000 7f15dadf2700 1 -- 172.21.15.57:0/969395950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15d406b7b0 msgr2=0x7f15d4072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:05.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.692+0000 7f15dadf2700 1 --2- 172.21.15.57:0/969395950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15d406b7b0 0x7f15d4072180 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009840 tx=0x7f15bc009b50 comp rx=0 tx=0).stop 2023-11-28T12:05:05.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.692+0000 7f15dadf2700 1 -- 172.21.15.57:0/969395950 shutdown_connections 2023-11-28T12:05:05.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.692+0000 7f15dadf2700 1 --2- 172.21.15.57:0/969395950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15d406b7b0 0x7f15d4072180 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:05.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.692+0000 7f15dadf2700 1 -- 172.21.15.57:0/969395950 >> 172.21.15.57:0/969395950 conn(0x7f15d40b2fc0 msgr2=0x7f15d4071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:05.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.692+0000 7f15dadf2700 1 -- 172.21.15.57:0/969395950 shutdown_connections 2023-11-28T12:05:05.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.692+0000 7f15dadf2700 1 -- 172.21.15.57:0/969395950 wait complete. 2023-11-28T12:05:05.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.693+0000 7f15dadf2700 1 Processor -- start 2023-11-28T12:05:05.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.693+0000 7f15dadf2700 1 -- start start 2023-11-28T12:05:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.694+0000 7f15dadf2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15d406b7b0 0x7f15d41f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.694+0000 7f15dadf2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15d40ad1f0 0x7f15d40c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.694+0000 7f15dadf2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15d40b0880 0x7f15d406d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.694+0000 7f15dadf2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d4152a50 con 0x7f15d40b0880 2023-11-28T12:05:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.694+0000 7f15dadf2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d41535d0 con 0x7f15d40ad1f0 2023-11-28T12:05:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.694+0000 7f15dadf2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d4152ca0 con 0x7f15d406b7b0 2023-11-28T12:05:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.694+0000 7f15d938f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15d40b0880 0x7f15d406d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.694+0000 7f15d938f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15d40b0880 0x7f15d406d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47032/0 (socket says 172.21.15.57:47032) 2023-11-28T12:05:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.694+0000 7f15d938f700 1 -- 172.21.15.57:0/1193648322 learned_addr learned my addr 172.21.15.57:0/1193648322 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.694+0000 7f15d3fff700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15d40ad1f0 0x7f15d40c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.694+0000 7f15d8b8e700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15d406b7b0 0x7f15d41f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.695+0000 7f15d938f700 1 -- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15d406b7b0 msgr2=0x7f15d41f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.695+0000 7f15d938f700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15d406b7b0 0x7f15d41f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.695+0000 7f15d938f700 1 -- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15d40ad1f0 msgr2=0x7f15d40c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:05.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.695+0000 7f15d938f700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15d40ad1f0 0x7f15d40c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:05.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.695+0000 7f15d938f700 1 -- 172.21.15.57:0/1193648322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc0094a0 con 0x7f15d40b0880 2023-11-28T12:05:05.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.695+0000 7f15d938f700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15d40b0880 0x7f15d406d020 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00ccd0 tx=0x7f15cc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:05.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.695+0000 7f15d17fa700 1 -- 172.21.15.57:0/1193648322 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00ada0 con 0x7f15d40b0880 2023-11-28T12:05:05.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.695+0000 7f15d17fa700 1 -- 172.21.15.57:0/1193648322 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc0070d0 con 0x7f15d40b0880 2023-11-28T12:05:05.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.695+0000 7f15dadf2700 1 -- 172.21.15.57:0/1193648322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d40b4fe0 con 0x7f15d40b0880 2023-11-28T12:05:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.695+0000 7f15d17fa700 1 -- 172.21.15.57:0/1193648322 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc007480 con 0x7f15d40b0880 2023-11-28T12:05:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.695+0000 7f15dadf2700 1 -- 172.21.15.57:0/1193648322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d40b4d00 con 0x7f15d40b0880 2023-11-28T12:05:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.696+0000 7f15dadf2700 1 -- 172.21.15.57:0/1193648322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d40b49b0 con 0x7f15d40b0880 2023-11-28T12:05:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.697+0000 7f15d17fa700 1 -- 172.21.15.57:0/1193648322 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f15cc01a030 con 0x7f15d40b0880 2023-11-28T12:05:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.697+0000 7f15d17fa700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15b40682f0 0x7f15b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.697+0000 7f15d17fa700 1 -- 172.21.15.57:0/1193648322 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f15cc085a90 con 0x7f15d40b0880 2023-11-28T12:05:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.697+0000 7f15d8b8e700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15b40682f0 0x7f15b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.698+0000 7f15d8b8e700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15b40682f0 0x7f15b406a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f15c4000f60 tx=0x7f15c4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.701+0000 7f15d17fa700 1 -- 172.21.15.57:0/1193648322 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f15cc0522f0 con 0x7f15d40b0880 2023-11-28T12:05:05.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.949+0000 7f15dadf2700 1 -- 172.21.15.57:0/1193648322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15d40743f0 con 0x7f15d40b0880 2023-11-28T12:05:05.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.950+0000 7f15d17fa700 1 -- 172.21.15.57:0/1193648322 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4652 (secure 0 0 0) 0x7f15cc018090 con 0x7f15d40b0880 2023-11-28T12:05:05.951 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:05:05.951 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2317025916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2317025916},{"type":"v1","addr":"172.21.15.181:6836","nonce":2317025916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3486085199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3486085199},{"type":"v1","addr":"172.21.15.181:6837","nonce":3486085199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/4272259116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":4272259116},{"type":"v1","addr":"172.21.15.57:6834","nonce":4272259116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:04:52.360416+0000","modified":"2023-11-28T12:04:55.377023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6835/685888843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":685888843},{"type":"v1","addr":"172.21.15.57:6835","nonce":685888843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T12:05:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.952+0000 7f15dadf2700 1 -- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15b40682f0 msgr2=0x7f15b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.952+0000 7f15dadf2700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15b40682f0 0x7f15b406a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f15c4000f60 tx=0x7f15c4009040 comp rx=0 tx=0).stop 2023-11-28T12:05:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.952+0000 7f15dadf2700 1 -- 172.21.15.57:0/1193648322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15d40b0880 msgr2=0x7f15d406d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.952+0000 7f15dadf2700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15d40b0880 0x7f15d406d020 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00ccd0 tx=0x7f15cc00a340 comp rx=0 tx=0).stop 2023-11-28T12:05:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.952+0000 7f15dadf2700 1 -- 172.21.15.57:0/1193648322 shutdown_connections 2023-11-28T12:05:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.952+0000 7f15dadf2700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15b40682f0 0x7f15b406a7a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.952+0000 7f15dadf2700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15d40b0880 0x7f15d406d020 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.952+0000 7f15dadf2700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15d406b7b0 0x7f15d41f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.952+0000 7f15dadf2700 1 --2- 172.21.15.57:0/1193648322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15d40ad1f0 0x7f15d40c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.952+0000 7f15dadf2700 1 -- 172.21.15.57:0/1193648322 >> 172.21.15.57:0/1193648322 conn(0x7f15d40b2fc0 msgr2=0x7f15d415b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.952+0000 7f15dadf2700 1 -- 172.21.15.57:0/1193648322 shutdown_connections 2023-11-28T12:05:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:05.952+0000 7f15dadf2700 1 -- 172.21.15.57:0/1193648322 wait complete. 2023-11-28T12:05:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 32 2023-11-28T12:05:05.967 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-28T12:05:06.130 INFO:teuthology.orchestra.run.smithi057.stdout:{"message":"","return_code":0} 2023-11-28T12:05:06.131 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T12:05:06.131 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T12:05:06.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.179+0000 7f9ba7ed30c0 1 Processor -- start 2023-11-28T12:05:06.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.179+0000 7f9ba7ed30c0 1 -- start start 2023-11-28T12:05:06.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9ba7ed30c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a196201f0 0x558a195d75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9ba7ed30c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558a196a2c70 0x558a196a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9ba7ed30c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a196a3040 0x558a19606430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9ba7ed30c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a195d71d0 con 0x558a196a2c70 2023-11-28T12:05:06.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9ba7ed30c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a195d6550 con 0x558a196201f0 2023-11-28T12:05:06.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9ba7ed30c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a195d6830 con 0x558a196a3040 2023-11-28T12:05:06.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9b9d7be700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558a196a2c70 0x558a196a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9b9d7be700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558a196a2c70 0x558a196a22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47050/0 (socket says 172.21.15.57:47050) 2023-11-28T12:05:06.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9b9d7be700 1 -- 172.21.15.57:0/560834005 learned_addr learned my addr 172.21.15.57:0/560834005 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9b9dfbf700 1 --2- 172.21.15.57:0/560834005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a196201f0 0x558a195d75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9b9e7c0700 1 --2- 172.21.15.57:0/560834005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a196a3040 0x558a19606430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9b9d7be700 1 -- 172.21.15.57:0/560834005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a196a3040 msgr2=0x558a19606430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9b9d7be700 1 --2- 172.21.15.57:0/560834005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a196a3040 0x558a19606430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9b9d7be700 1 -- 172.21.15.57:0/560834005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a196201f0 msgr2=0x558a195d75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9b9d7be700 1 --2- 172.21.15.57:0/560834005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a196201f0 0x558a195d75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9b9d7be700 1 -- 172.21.15.57:0/560834005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9800a1c0 con 0x558a196a2c70 2023-11-28T12:05:06.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9b9dfbf700 1 --2- 172.21.15.57:0/560834005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a196201f0 0x558a195d75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:05:06.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.180+0000 7f9b9e7c0700 1 --2- 172.21.15.57:0/560834005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a196a3040 0x558a19606430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:05:06.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.181+0000 7f9b9d7be700 1 --2- 172.21.15.57:0/560834005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558a196a2c70 0x558a196a22f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9b900095a0 tx=0x7f9b900098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7ae0dc72968523 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.181+0000 7f9b97fff700 1 -- 172.21.15.57:0/560834005 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9000e070 con 0x558a196a2c70 2023-11-28T12:05:06.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.181+0000 7f9b97fff700 1 -- 172.21.15.57:0/560834005 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b90006440 con 0x558a196a2c70 2023-11-28T12:05:06.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.181+0000 7f9b97fff700 1 -- 172.21.15.57:0/560834005 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b900067d0 con 0x558a196a2c70 2023-11-28T12:05:06.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.181+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/560834005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558a196a2c70 msgr2=0x558a196a22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.181+0000 7f9ba7ed30c0 1 --2- 172.21.15.57:0/560834005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558a196a2c70 0x558a196a22f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9b900095a0 tx=0x7f9b900098b0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.181+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/560834005 shutdown_connections 2023-11-28T12:05:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.181+0000 7f9ba7ed30c0 1 --2- 172.21.15.57:0/560834005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a196a3040 0x558a19606430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.181+0000 7f9ba7ed30c0 1 --2- 172.21.15.57:0/560834005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a196201f0 0x558a195d75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.181+0000 7f9ba7ed30c0 1 --2- 172.21.15.57:0/560834005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558a196a2c70 0x558a196a22f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.181+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/560834005 >> 172.21.15.57:0/560834005 conn(0x558a19622620 msgr2=0x558a19605610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.181+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/560834005 shutdown_connections 2023-11-28T12:05:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.181+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/560834005 wait complete. 2023-11-28T12:05:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.182+0000 7f9ba7ed30c0 1 Processor -- start 2023-11-28T12:05:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.182+0000 7f9ba7ed30c0 1 -- start start 2023-11-28T12:05:06.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9ba7ed30c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558a196201f0 0x558a196999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9ba7ed30c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a196a2c70 0x558a196a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9ba7ed30c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a196a3040 0x558a1969f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9ba7ed30c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1969d890 con 0x558a196201f0 2023-11-28T12:05:06.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9ba7ed30c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1969db50 con 0x558a196a3040 2023-11-28T12:05:06.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9ba7ed30c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a1969de30 con 0x558a196a2c70 2023-11-28T12:05:06.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9b9dfbf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558a196201f0 0x558a196999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9b9dfbf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558a196201f0 0x558a196999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47060/0 (socket says 172.21.15.57:47060) 2023-11-28T12:05:06.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9b9dfbf700 1 -- 172.21.15.57:0/1576891522 learned_addr learned my addr 172.21.15.57:0/1576891522 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9b9d7be700 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a196a2c70 0x558a196a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9b9e7c0700 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a196a3040 0x558a1969f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9b9dfbf700 1 -- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a196a2c70 msgr2=0x558a196a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9b9dfbf700 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a196a2c70 0x558a196a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9b9dfbf700 1 -- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a196a3040 msgr2=0x558a1969f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9b9dfbf700 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a196a3040 0x558a1969f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9b9dfbf700 1 -- 172.21.15.57:0/1576891522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b90005e40 con 0x558a196201f0 2023-11-28T12:05:06.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9b9e7c0700 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a196a3040 0x558a1969f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:06.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.183+0000 7f9b9dfbf700 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558a196201f0 0x558a196999e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9b88007e20 tx=0x7f9b88006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.184+0000 7f9b96ffd700 1 -- 172.21.15.57:0/1576891522 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88012640 con 0x558a196201f0 2023-11-28T12:05:06.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.184+0000 7f9b96ffd700 1 -- 172.21.15.57:0/1576891522 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b880127a0 con 0x558a196201f0 2023-11-28T12:05:06.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.184+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/1576891522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558a195444f0 con 0x558a196201f0 2023-11-28T12:05:06.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.184+0000 7f9b96ffd700 1 -- 172.21.15.57:0/1576891522 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88012b30 con 0x558a196201f0 2023-11-28T12:05:06.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.184+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/1576891522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a1952a120 con 0x558a196201f0 2023-11-28T12:05:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.185+0000 7f9b96ffd700 1 -- 172.21.15.57:0/1576891522 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9b88019030 con 0x558a196201f0 2023-11-28T12:05:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.185+0000 7f9b96ffd700 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9b80066c10 0x7f9b800690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.186+0000 7f9b96ffd700 1 -- 172.21.15.57:0/1576891522 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9b88083dd0 con 0x558a196201f0 2023-11-28T12:05:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.186+0000 7f9ba7ed30c0 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x558a19699f20 0x558a197337e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.186+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/1576891522 --> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63) v8 -- 0x558a19733d20 con 0x558a19699f20 2023-11-28T12:05:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.186+0000 7f9b9d7be700 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9b80066c10 0x7f9b800690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.186+0000 7f9b9e7c0700 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x558a19699f20 0x558a197337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.186+0000 7f9b9d7be700 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9b80066c10 0x7f9b800690c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x558a19621a00 tx=0x7f9b9000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.187+0000 7f9b9e7c0700 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x558a19699f20 0x558a197337e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.187+0000 7f9b9e7c0700 1 -- 172.21.15.57:0/1576891522 <== osd.6 v2:172.21.15.181:6816/120984 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f9b9800e9e0 con 0x558a19699f20 2023-11-28T12:05:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.187+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x558a19699f20 msgr2=0x558a197337e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.187+0000 7f9ba7ed30c0 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x558a19699f20 0x558a197337e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.187+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9b80066c10 msgr2=0x7f9b800690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.187+0000 7f9ba7ed30c0 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9b80066c10 0x7f9b800690c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x558a19621a00 tx=0x7f9b9000f000 comp rx=0 tx=0).stop 2023-11-28T12:05:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.187+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/1576891522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558a196201f0 msgr2=0x558a196999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.187+0000 7f9ba7ed30c0 1 --2- 172.21.15.57:0/1576891522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558a196201f0 0x558a196999e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9b88007e20 tx=0x7f9b88006660 comp rx=0 tx=0).stop 2023-11-28T12:05:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.187+0000 7f9b9e7c0700 1 -- 172.21.15.57:0/1576891522 reap_dead start 2023-11-28T12:05:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.187+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/1576891522 shutdown_connections 2023-11-28T12:05:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.187+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/1576891522 >> 172.21.15.57:0/1576891522 conn(0x558a19622620 msgr2=0x558a1961b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.187+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/1576891522 shutdown_connections 2023-11-28T12:05:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.187+0000 7f9ba7ed30c0 1 -- 172.21.15.57:0/1576891522 wait complete. 2023-11-28T12:05:06.194 INFO:teuthology.orchestra.run.smithi057.stdout:Xfiledir_test 2023-11-28T12:05:06.194 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:05:06.194 DEBUG:teuthology.orchestra.run.smithi057:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T12:05:06.259 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:05:06.259 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511627777, 2023-11-28T12:05:06.259 INFO:teuthology.orchestra.run.smithi057.stdout: "ancestors": [ 2023-11-28T12:05:06.259 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:05:06.259 INFO:teuthology.orchestra.run.smithi057.stdout: "dirino": 1099511627776, 2023-11-28T12:05:06.259 INFO:teuthology.orchestra.run.smithi057.stdout: "dname": "file", 2023-11-28T12:05:06.259 INFO:teuthology.orchestra.run.smithi057.stdout: "version": 2 2023-11-28T12:05:06.259 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-11-28T12:05:06.259 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:05:06.260 INFO:teuthology.orchestra.run.smithi057.stdout: "dirino": 1, 2023-11-28T12:05:06.260 INFO:teuthology.orchestra.run.smithi057.stdout: "dname": "dir_test", 2023-11-28T12:05:06.260 INFO:teuthology.orchestra.run.smithi057.stdout: "version": 4 2023-11-28T12:05:06.260 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:05:06.260 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-11-28T12:05:06.260 INFO:teuthology.orchestra.run.smithi057.stdout: "pool": 10, 2023-11-28T12:05:06.260 INFO:teuthology.orchestra.run.smithi057.stdout: "old_pools": [] 2023-11-28T12:05:06.260 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:05:06.260 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:05:06.261 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2023-11-28T12:05:06.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.351+0000 7fb6e8cc40c0 1 Processor -- start 2023-11-28T12:05:06.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.351+0000 7fb6e8cc40c0 1 -- start start 2023-11-28T12:05:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.352+0000 7fb6e8cc40c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe8c18c7e0 0x55fe8c238ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.352+0000 7fb6e8cc40c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fe8c1ab7e0 0x55fe8c2366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.352+0000 7fb6e8cc40c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe8c23a1a0 0x55fe8c236c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.352+0000 7fb6e8cc40c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8c09cfe0 con 0x55fe8c23a1a0 2023-11-28T12:05:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.352+0000 7fb6e8cc40c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8c23a570 con 0x55fe8c18c7e0 2023-11-28T12:05:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.352+0000 7fb6e8cc40c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe8c23aaa0 con 0x55fe8c1ab7e0 2023-11-28T12:05:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.352+0000 7fb6df5b1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe8c23a1a0 0x55fe8c236c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.352+0000 7fb6df5b1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe8c23a1a0 0x55fe8c236c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47076/0 (socket says 172.21.15.57:47076) 2023-11-28T12:05:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.352+0000 7fb6df5b1700 1 -- 172.21.15.57:0/1242901415 learned_addr learned my addr 172.21.15.57:0/1242901415 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.352+0000 7fb6dedb0700 1 --2- 172.21.15.57:0/1242901415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe8c18c7e0 0x55fe8c238ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.352+0000 7fb6de5af700 1 --2- 172.21.15.57:0/1242901415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fe8c1ab7e0 0x55fe8c2366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.353+0000 7fb6df5b1700 1 -- 172.21.15.57:0/1242901415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fe8c1ab7e0 msgr2=0x55fe8c2366d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.353+0000 7fb6df5b1700 1 --2- 172.21.15.57:0/1242901415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fe8c1ab7e0 0x55fe8c2366d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.353+0000 7fb6df5b1700 1 -- 172.21.15.57:0/1242901415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe8c18c7e0 msgr2=0x55fe8c238ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.353+0000 7fb6df5b1700 1 --2- 172.21.15.57:0/1242901415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe8c18c7e0 0x55fe8c238ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.353+0000 7fb6df5b1700 1 -- 172.21.15.57:0/1242901415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8005de0 con 0x55fe8c23a1a0 2023-11-28T12:05:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.353+0000 7fb6df5b1700 1 --2- 172.21.15.57:0/1242901415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe8c23a1a0 0x55fe8c236c10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800d2f0 tx=0x7fb6d800d600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df655f90f338488d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.353+0000 7fb6dd5ad700 1 -- 172.21.15.57:0/1242901415 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8006230 con 0x55fe8c23a1a0 2023-11-28T12:05:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.353+0000 7fb6dd5ad700 1 -- 172.21.15.57:0/1242901415 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d80040d0 con 0x55fe8c23a1a0 2023-11-28T12:05:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.353+0000 7fb6dd5ad700 1 -- 172.21.15.57:0/1242901415 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8001760 con 0x55fe8c23a1a0 2023-11-28T12:05:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.353+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/1242901415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe8c23a1a0 msgr2=0x55fe8c236c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.353+0000 7fb6e8cc40c0 1 --2- 172.21.15.57:0/1242901415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe8c23a1a0 0x55fe8c236c10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800d2f0 tx=0x7fb6d800d600 comp rx=0 tx=0).stop 2023-11-28T12:05:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.354+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/1242901415 shutdown_connections 2023-11-28T12:05:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.354+0000 7fb6e8cc40c0 1 --2- 172.21.15.57:0/1242901415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe8c23a1a0 0x55fe8c236c10 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.354+0000 7fb6e8cc40c0 1 --2- 172.21.15.57:0/1242901415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe8c18c7e0 0x55fe8c238ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.354+0000 7fb6e8cc40c0 1 --2- 172.21.15.57:0/1242901415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fe8c1ab7e0 0x55fe8c2366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.354+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/1242901415 >> 172.21.15.57:0/1242901415 conn(0x55fe8c1aa150 msgr2=0x55fe8c1a7870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.354+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/1242901415 shutdown_connections 2023-11-28T12:05:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.354+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/1242901415 wait complete. 2023-11-28T12:05:06.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.354+0000 7fb6e8cc40c0 1 Processor -- start 2023-11-28T12:05:06.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.354+0000 7fb6e8cc40c0 1 -- start start 2023-11-28T12:05:06.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.355+0000 7fb6e8cc40c0 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fe8c18c7e0 0x55fe8c2262a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.355+0000 7fb6e8cc40c0 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe8c1ab7e0 0x55fe8c1a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.355+0000 7fb6e8cc40c0 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe8c23a1a0 0x55fe8c237770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.355+0000 7fb6e8cc40c0 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8c2372e0 con 0x55fe8c23a1a0 2023-11-28T12:05:06.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.355+0000 7fb6e8cc40c0 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8c2267e0 con 0x55fe8c1ab7e0 2023-11-28T12:05:06.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.355+0000 7fb6e8cc40c0 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe8c2366d0 con 0x55fe8c18c7e0 2023-11-28T12:05:06.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.355+0000 7fb6df5b1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe8c23a1a0 0x55fe8c237770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.355+0000 7fb6df5b1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe8c23a1a0 0x55fe8c237770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47092/0 (socket says 172.21.15.57:47092) 2023-11-28T12:05:06.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.355+0000 7fb6df5b1700 1 -- 172.21.15.57:0/3969624493 learned_addr learned my addr 172.21.15.57:0/3969624493 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.355+0000 7fb6de5af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe8c1ab7e0 0x55fe8c1a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.355+0000 7fb6dedb0700 1 --2- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fe8c18c7e0 0x55fe8c2262a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.356+0000 7fb6df5b1700 1 -- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fe8c18c7e0 msgr2=0x55fe8c2262a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.356+0000 7fb6df5b1700 1 --2- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fe8c18c7e0 0x55fe8c2262a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.356+0000 7fb6df5b1700 1 -- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe8c1ab7e0 msgr2=0x55fe8c1a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.356+0000 7fb6df5b1700 1 --2- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe8c1ab7e0 0x55fe8c1a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.356+0000 7fb6df5b1700 1 -- 172.21.15.57:0/3969624493 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe8c0a84a0 con 0x55fe8c23a1a0 2023-11-28T12:05:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.356+0000 7fb6df5b1700 1 --2- 172.21.15.57:0/3969624493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe8c23a1a0 0x55fe8c237770 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800d950 tx=0x7fb6d8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.356+0000 7fb6d7fff700 1 -- 172.21.15.57:0/3969624493 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8001950 con 0x55fe8c23a1a0 2023-11-28T12:05:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.356+0000 7fb6d7fff700 1 -- 172.21.15.57:0/3969624493 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d801e070 con 0x55fe8c23a1a0 2023-11-28T12:05:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.356+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/3969624493 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fe8c0b5120 con 0x55fe8c23a1a0 2023-11-28T12:05:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.356+0000 7fb6d7fff700 1 -- 172.21.15.57:0/3969624493 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8001d10 con 0x55fe8c23a1a0 2023-11-28T12:05:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.356+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/3969624493 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe8c0a8830 con 0x55fe8c23a1a0 2023-11-28T12:05:06.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.358+0000 7fb6d7fff700 1 -- 172.21.15.57:0/3969624493 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb6d802b070 con 0x55fe8c23a1a0 2023-11-28T12:05:06.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.358+0000 7fb6d7fff700 1 --2- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb6c0066c10 0x7fb6c00690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.358+0000 7fb6d7fff700 1 -- 172.21.15.57:0/3969624493 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb6d8085b90 con 0x55fe8c23a1a0 2023-11-28T12:05:06.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.358+0000 7fb6dedb0700 1 --2- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb6c0066c10 0x7fb6c00690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.358+0000 7fb6e8cc40c0 1 --2- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x55fe8c1a7190 0x55fe8c2d0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.358+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/3969624493 --> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63) v8 -- 0x55fe8c18e000 con 0x55fe8c1a7190 2023-11-28T12:05:06.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.358+0000 7fb6de5af700 1 --2- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x55fe8c1a7190 0x55fe8c2d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.359+0000 7fb6dedb0700 1 --2- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb6c0066c10 0x7fb6c00690c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800a8c0 tx=0x7fb6c800f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.359+0000 7fb6de5af700 1 --2- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x55fe8c1a7190 0x55fe8c2d0160 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.360+0000 7fb6de5af700 1 -- 172.21.15.57:0/3969624493 <== osd.6 v2:172.21.15.181:6816/120984 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v63'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb6d0012040 con 0x55fe8c1a7190 2023-11-28T12:05:06.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.360+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x55fe8c1a7190 msgr2=0x55fe8c2d0160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.360+0000 7fb6e8cc40c0 1 --2- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x55fe8c1a7190 0x55fe8c2d0160 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.360+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb6c0066c10 msgr2=0x7fb6c00690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.360+0000 7fb6e8cc40c0 1 --2- 172.21.15.57:0/3969624493 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb6c0066c10 0x7fb6c00690c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800a8c0 tx=0x7fb6c800f000 comp rx=0 tx=0).stop 2023-11-28T12:05:06.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.360+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/3969624493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe8c23a1a0 msgr2=0x55fe8c237770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.360+0000 7fb6e8cc40c0 1 --2- 172.21.15.57:0/3969624493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe8c23a1a0 0x55fe8c237770 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800d950 tx=0x7fb6d8001250 comp rx=0 tx=0).stop 2023-11-28T12:05:06.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.360+0000 7fb6df5b1700 1 -- 172.21.15.57:0/3969624493 reap_dead start 2023-11-28T12:05:06.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.361+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/3969624493 shutdown_connections 2023-11-28T12:05:06.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.361+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/3969624493 >> 172.21.15.57:0/3969624493 conn(0x55fe8c1aa150 msgr2=0x55fe8c1aa830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.361+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/3969624493 shutdown_connections 2023-11-28T12:05:06.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.361+0000 7fb6e8cc40c0 1 -- 172.21.15.57:0/3969624493 wait complete. 2023-11-28T12:05:06.366 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2023-11-28T12:05:06.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.506+0000 7fdc9865b700 1 Processor -- start 2023-11-28T12:05:06.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.506+0000 7fdc9865b700 1 -- start start 2023-11-28T12:05:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.507+0000 7fdc9865b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9015cce0 0x7fdc900b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.507+0000 7fdc9865b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdc900b22e0 0x7fdc900d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.507+0000 7fdc9865b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc900b32b0 con 0x7fdc900b1c90 2023-11-28T12:05:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.507+0000 7fdc9865b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc900b26e0 con 0x7fdc900b22e0 2023-11-28T12:05:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.507+0000 7fdc9865b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc900b2980 con 0x7fdc9015cce0 2023-11-28T12:05:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc95bf6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdc900b22e0 0x7fdc900d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc963f7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9015cce0 0x7fdc900b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc95bf6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdc900b22e0 0x7fdc900d0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:55502/0 (socket says 172.21.15.57:55502) 2023-11-28T12:05:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc95bf6700 1 -- 172.21.15.57:0/335877144 learned_addr learned my addr 172.21.15.57:0/335877144 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc96bf8700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdc900b1c90 0x7fdc900afd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:54260/0 (socket says 172.21.15.57:54260) 2023-11-28T12:05:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc963f7700 1 --2- 172.21.15.57:0/335877144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9015cce0 0x7fdc900b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:33856/0 (socket says 172.21.15.57:33856) 2023-11-28T12:05:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc963f7700 1 -- 172.21.15.57:0/335877144 >> v2:172.21.15.181:3300/0 conn(0x7fdc900b22e0 msgr2=0x7fdc900d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc963f7700 1 --2- 172.21.15.57:0/335877144 >> v2:172.21.15.181:3300/0 conn(0x7fdc900b22e0 0x7fdc900d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc963f7700 1 -- 172.21.15.57:0/335877144 >> v1:172.21.15.181:6789/0 conn(0x7fdc900b1c90 legacy=0x7fdc900afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc963f7700 1 -- 172.21.15.57:0/335877144 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc900b2c60 con 0x7fdc9015cce0 2023-11-28T12:05:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc963f7700 1 --2- 172.21.15.57:0/335877144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9015cce0 0x7fdc900b1750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009840 tx=0x7fdc8c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=981a2278573407e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc953f5700 1 -- 172.21.15.57:0/335877144 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c011830 con 0x7fdc9015cce0 2023-11-28T12:05:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc953f5700 1 -- 172.21.15.57:0/335877144 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c011990 con 0x7fdc9015cce0 2023-11-28T12:05:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.508+0000 7fdc953f5700 1 -- 172.21.15.57:0/335877144 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c011d20 con 0x7fdc9015cce0 2023-11-28T12:05:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.509+0000 7fdc9865b700 1 -- 172.21.15.57:0/335877144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9015cce0 msgr2=0x7fdc900b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.509+0000 7fdc9865b700 1 --2- 172.21.15.57:0/335877144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9015cce0 0x7fdc900b1750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009840 tx=0x7fdc8c009b50 comp rx=0 tx=0).stop 2023-11-28T12:05:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.509+0000 7fdc9865b700 1 -- 172.21.15.57:0/335877144 shutdown_connections 2023-11-28T12:05:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.509+0000 7fdc9865b700 1 --2- 172.21.15.57:0/335877144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9015cce0 0x7fdc900b1750 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.509+0000 7fdc9865b700 1 --2- 172.21.15.57:0/335877144 >> v2:172.21.15.181:3300/0 conn(0x7fdc900b22e0 0x7fdc900d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.509+0000 7fdc9865b700 1 -- 172.21.15.57:0/335877144 >> 172.21.15.57:0/335877144 conn(0x7fdc9009c4d0 msgr2=0x7fdc900d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.509+0000 7fdc9865b700 1 -- 172.21.15.57:0/335877144 shutdown_connections 2023-11-28T12:05:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.509+0000 7fdc9865b700 1 -- 172.21.15.57:0/335877144 wait complete. 2023-11-28T12:05:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.510+0000 7fdc9865b700 1 Processor -- start 2023-11-28T12:05:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.510+0000 7fdc9865b700 1 -- start start 2023-11-28T12:05:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.511+0000 7fdc9865b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc900b1c90 0x7fdc900cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.511+0000 7fdc9865b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc9015cce0 0x7fdc9015bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.511+0000 7fdc9865b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9006bd90 0x7fdc9006c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.511+0000 7fdc9865b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc900b1660 con 0x7fdc900b1c90 2023-11-28T12:05:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.511+0000 7fdc9865b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9006c750 con 0x7fdc9015cce0 2023-11-28T12:05:06.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.511+0000 7fdc9865b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc900caeb0 con 0x7fdc9006bd90 2023-11-28T12:05:06.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.511+0000 7fdc963f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc900b1c90 0x7fdc900cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.511+0000 7fdc963f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc900b1c90 0x7fdc900cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47098/0 (socket says 172.21.15.57:47098) 2023-11-28T12:05:06.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.511+0000 7fdc963f7700 1 -- 172.21.15.57:0/164144296 learned_addr learned my addr 172.21.15.57:0/164144296 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.511+0000 7fdc96bf8700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9006bd90 0x7fdc9006c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.511+0000 7fdc95bf6700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc9015cce0 0x7fdc9015bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.512+0000 7fdc963f7700 1 -- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9006bd90 msgr2=0x7fdc9006c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.512+0000 7fdc963f7700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9006bd90 0x7fdc9006c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.512+0000 7fdc963f7700 1 -- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc9015cce0 msgr2=0x7fdc9015bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.512+0000 7fdc963f7700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc9015cce0 0x7fdc9015bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.512+0000 7fdc963f7700 1 -- 172.21.15.57:0/164144296 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c0094a0 con 0x7fdc900b1c90 2023-11-28T12:05:06.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.512+0000 7fdc96bf8700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9006bd90 0x7fdc9006c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:06.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.512+0000 7fdc95bf6700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc9015cce0 0x7fdc9015bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:06.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.512+0000 7fdc963f7700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc900b1c90 0x7fdc900cbe60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0090b0 tx=0x7fdc8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.512+0000 7fdc82ffd700 1 -- 172.21.15.57:0/164144296 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c0038a0 con 0x7fdc900b1c90 2023-11-28T12:05:06.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.512+0000 7fdc82ffd700 1 -- 172.21.15.57:0/164144296 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c01f070 con 0x7fdc900b1c90 2023-11-28T12:05:06.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.512+0000 7fdc9865b700 1 -- 172.21.15.57:0/164144296 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc900b4750 con 0x7fdc900b1c90 2023-11-28T12:05:06.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.512+0000 7fdc82ffd700 1 -- 172.21.15.57:0/164144296 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c003c00 con 0x7fdc900b1c90 2023-11-28T12:05:06.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.512+0000 7fdc9865b700 1 -- 172.21.15.57:0/164144296 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc900b2f40 con 0x7fdc900b1c90 2023-11-28T12:05:06.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.514+0000 7fdc82ffd700 1 -- 172.21.15.57:0/164144296 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdc8c00f050 con 0x7fdc900b1c90 2023-11-28T12:05:06.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.514+0000 7fdc82ffd700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdc740682f0 0x7fdc7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.514+0000 7fdc82ffd700 1 -- 172.21.15.57:0/164144296 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc8c021070 con 0x7fdc900b1c90 2023-11-28T12:05:06.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.514+0000 7fdc95bf6700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdc740682f0 0x7fdc7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.515+0000 7fdc95bf6700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdc740682f0 0x7fdc7406a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdc900a1050 tx=0x7fdc84008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.516+0000 7fdca47fab80 1 Processor -- start 2023-11-28T12:05:06.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.516+0000 7fdca47fab80 1 -- start start 2023-11-28T12:05:06.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.517+0000 7fdca47fab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc138000 0x55cfdc0f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.517+0000 7fdca47fab80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc12eb70 0x55cfdc1383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.517+0000 7fdca47fab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc11a710 0x55cfdc0f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.517+0000 7fdca47fab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfdc06fe90 con 0x55cfdc138000 2023-11-28T12:05:06.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.517+0000 7fdca47fab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfdbfdc1f0 con 0x55cfdc11a710 2023-11-28T12:05:06.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.517+0000 7fdca47fab80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfdc04ae60 con 0x55cfdc12eb70 2023-11-28T12:05:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc963f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc138000 0x55cfdc0f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc963f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc138000 0x55cfdc0f1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47110/0 (socket says 172.21.15.57:47110) 2023-11-28T12:05:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc963f7700 1 -- 172.21.15.57:0/1697692883 learned_addr learned my addr 172.21.15.57:0/1697692883 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc95bf6700 1 --2- 172.21.15.57:0/1697692883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc12eb70 0x55cfdc1383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc96bf8700 1 --2- 172.21.15.57:0/1697692883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc11a710 0x55cfdc0f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc963f7700 1 -- 172.21.15.57:0/1697692883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc12eb70 msgr2=0x55cfdc1383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc963f7700 1 --2- 172.21.15.57:0/1697692883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc12eb70 0x55cfdc1383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc963f7700 1 -- 172.21.15.57:0/1697692883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc11a710 msgr2=0x55cfdc0f2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc963f7700 1 --2- 172.21.15.57:0/1697692883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc11a710 0x55cfdc0f2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc963f7700 1 -- 172.21.15.57:0/1697692883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfdbe38840 con 0x55cfdc138000 2023-11-28T12:05:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc963f7700 1 --2- 172.21.15.57:0/1697692883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc138000 0x55cfdc0f1aa0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009f20 tx=0x7fdc8c025fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0faf49f33bb6966 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc73fff700 1 -- 172.21.15.57:0/1697692883 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c003d60 con 0x55cfdc138000 2023-11-28T12:05:06.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc73fff700 1 -- 172.21.15.57:0/1697692883 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c083070 con 0x55cfdc138000 2023-11-28T12:05:06.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.518+0000 7fdc73fff700 1 -- 172.21.15.57:0/1697692883 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c057280 con 0x55cfdc138000 2023-11-28T12:05:06.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.519+0000 7fdca47fab80 1 -- 172.21.15.57:0/1697692883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc138000 msgr2=0x55cfdc0f1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.519+0000 7fdca47fab80 1 --2- 172.21.15.57:0/1697692883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc138000 0x55cfdc0f1aa0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009f20 tx=0x7fdc8c025fa0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.519+0000 7fdca47fab80 1 -- 172.21.15.57:0/1697692883 shutdown_connections 2023-11-28T12:05:06.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.519+0000 7fdca47fab80 1 --2- 172.21.15.57:0/1697692883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc11a710 0x55cfdc0f2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.519+0000 7fdca47fab80 1 --2- 172.21.15.57:0/1697692883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc138000 0x55cfdc0f1aa0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.519+0000 7fdca47fab80 1 --2- 172.21.15.57:0/1697692883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc12eb70 0x55cfdc1383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.519+0000 7fdca47fab80 1 -- 172.21.15.57:0/1697692883 >> 172.21.15.57:0/1697692883 conn(0x55cfdc0e7dd0 msgr2=0x55cfdc0f0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.519+0000 7fdca47fab80 1 -- 172.21.15.57:0/1697692883 shutdown_connections 2023-11-28T12:05:06.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.519+0000 7fdca47fab80 1 -- 172.21.15.57:0/1697692883 wait complete. 2023-11-28T12:05:06.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.520+0000 7fdca47fab80 1 -- start start 2023-11-28T12:05:06.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.520+0000 7fdca47fab80 1 Processor -- start 2023-11-28T12:05:06.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdca47fab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc12efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdca47fab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc12eb70 0x55cfdc147a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdca47fab80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc138000 0x55cfdc11b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdca47fab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfdc0441d0 con 0x55cfdc11a710 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdca47fab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfdbe39440 con 0x55cfdc12eb70 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdca47fab80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfdbfdc3f0 con 0x55cfdc138000 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdca47fab80 10 client.0 _ll_register_callbacks cb 0x55cfdbfe24b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdc963f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc12efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdc963f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc12efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47112/0 (socket says 172.21.15.57:47112) 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdc963f7700 1 -- 172.21.15.57:0/577577197 learned_addr learned my addr 172.21.15.57:0/577577197 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdc95bf6700 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc12eb70 0x55cfdc147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdc96bf8700 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc138000 0x55cfdc11b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdc963f7700 1 -- 172.21.15.57:0/577577197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc138000 msgr2=0x55cfdc11b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdc963f7700 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc138000 0x55cfdc11b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdc963f7700 1 -- 172.21.15.57:0/577577197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc12eb70 msgr2=0x55cfdc147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdc963f7700 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc12eb70 0x55cfdc147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdc963f7700 1 -- 172.21.15.57:0/577577197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc900b2f40 con 0x55cfdc11a710 2023-11-28T12:05:06.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdc96bf8700 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc138000 0x55cfdc11b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:06.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdc963f7700 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc12efa0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009f20 tx=0x7fdc8c087050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7c033a2583010e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.521+0000 7fdc717fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:05:06.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdc717fa700 1 -- 172.21.15.57:0/577577197 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c06b020 con 0x55cfdc11a710 2023-11-28T12:05:06.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdc717fa700 1 -- 172.21.15.57:0/577577197 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc8c06ac00 con 0x55cfdc11a710 2023-11-28T12:05:06.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdca47fab80 1 -- 172.21.15.57:0/577577197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfdc0480e0 con 0x55cfdc11a710 2023-11-28T12:05:06.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdc717fa700 1 -- 172.21.15.57:0/577577197 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc8c06adc0 con 0x55cfdc11a710 2023-11-28T12:05:06.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdca47fab80 10 client.6432 fetch_fsmap learned FSMap version 32 2023-11-28T12:05:06.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdca47fab80 1 -- 172.21.15.57:0/577577197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55cfdbf25150 con 0x55cfdc11a710 2023-11-28T12:05:06.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdc717fa700 1 -- 172.21.15.57:0/577577197 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cfdc0480e0 con 0x55cfdc11a710 2023-11-28T12:05:06.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdc717fa700 1 -- 172.21.15.57:0/577577197 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fdc8c057040 con 0x55cfdc11a710 2023-11-28T12:05:06.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdca47fab80 10 client.6432 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T12:05:06.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdca47fab80 10 client.6432 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T12:05:06.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdca47fab80 20 client.6432 populate_metadata read hostname 'smithi057' 2023-11-28T12:05:06.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdca47fab80 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55cfdc11bc90 0x55cfdc11c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdca47fab80 4 client.6432 mds_command: new command op to 6219 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T12:05:06.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdca47fab80 1 -- 172.21.15.57:0/577577197 --> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfdbe93fa0 con 0x55cfdc11bc90 2023-11-28T12:05:06.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.522+0000 7fdc95bf6700 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55cfdc11bc90 0x55cfdc11c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.523+0000 7fdc717fa700 10 client.6432 ms_handle_connect on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:06.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.523+0000 7fdc95bf6700 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55cfdc11bc90 0x55cfdc11c8e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1577b622f6cee6ca server_cookie=fa6dcfaf15086899 in_seq=0 out_seq=0 2023-11-28T12:05:06.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.523+0000 7fdc717fa700 1 -- 172.21.15.57:0/577577197 <== mds.0 v2:172.21.15.57:6833/685888843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55cfdbe93fa0 con 0x55cfdc11bc90 2023-11-28T12:05:06.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.523+0000 7fdc717fa700 10 client.6432 handle_command_reply: tid=0 2023-11-28T12:05:06.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.523+0000 7fdca47fab80 1 client.6432 shutdown 2023-11-28T12:05:06.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.523+0000 7fdca47fab80 1 -- 172.21.15.57:0/577577197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 msgr2=0x55cfdc12efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.523+0000 7fdca47fab80 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc12efa0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009f20 tx=0x7fdc8c087050 comp rx=0 tx=0).stop 2023-11-28T12:05:06.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.524+0000 7fdca47fab80 1 -- 172.21.15.57:0/577577197 shutdown_connections 2023-11-28T12:05:06.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.524+0000 7fdca47fab80 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55cfdc11bc90 0x55cfdc11c8e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.524+0000 7fdca47fab80 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc138000 0x55cfdc11b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.524+0000 7fdca47fab80 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc12efa0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.524+0000 7fdc717fa700 0 client.6432 ms_handle_reset on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:06.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.524+0000 7fdca47fab80 1 --2- 172.21.15.57:0/577577197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc12eb70 0x55cfdc147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.524+0000 7fdca47fab80 1 -- 172.21.15.57:0/577577197 >> 172.21.15.57:0/577577197 conn(0x55cfdc0e7dd0 msgr2=0x55cfdc119c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.524+0000 7fdca47fab80 1 -- 172.21.15.57:0/577577197 shutdown_connections 2023-11-28T12:05:06.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.524+0000 7fdca47fab80 1 -- 172.21.15.57:0/577577197 wait complete. 2023-11-28T12:05:06.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.524+0000 7fdca47fab80 20 client.6432 trim_cache size 0 max 16384 2023-11-28T12:05:06.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.541+0000 7fdca47fab80 1 Processor -- start 2023-11-28T12:05:06.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.541+0000 7fdca47fab80 1 -- start start 2023-11-28T12:05:06.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.542+0000 7fdca47fab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc117eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.542+0000 7fdca47fab80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc12eb70 0x55cfdc130c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.542+0000 7fdca47fab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc138000 0x55cfdc116e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.542+0000 7fdca47fab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfdc118490 con 0x55cfdc11a710 2023-11-28T12:05:06.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.542+0000 7fdca47fab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfdc131160 con 0x55cfdc138000 2023-11-28T12:05:06.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.542+0000 7fdca47fab80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfdc11e340 con 0x55cfdc12eb70 2023-11-28T12:05:06.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc963f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc117eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc963f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc117eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47116/0 (socket says 172.21.15.57:47116) 2023-11-28T12:05:06.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc963f7700 1 -- 172.21.15.57:0/1810434152 learned_addr learned my addr 172.21.15.57:0/1810434152 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc95bf6700 1 --2- 172.21.15.57:0/1810434152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc12eb70 0x55cfdc130c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc96bf8700 1 --2- 172.21.15.57:0/1810434152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc138000 0x55cfdc116e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc963f7700 1 -- 172.21.15.57:0/1810434152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc12eb70 msgr2=0x55cfdc130c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc963f7700 1 --2- 172.21.15.57:0/1810434152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc12eb70 0x55cfdc130c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc963f7700 1 -- 172.21.15.57:0/1810434152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc138000 msgr2=0x55cfdc116e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc963f7700 1 --2- 172.21.15.57:0/1810434152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc138000 0x55cfdc116e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc963f7700 1 -- 172.21.15.57:0/1810434152 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfdc1386a0 con 0x55cfdc11a710 2023-11-28T12:05:06.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc96bf8700 1 --2- 172.21.15.57:0/1810434152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc138000 0x55cfdc116e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:05:06.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc95bf6700 1 --2- 172.21.15.57:0/1810434152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc12eb70 0x55cfdc130c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:05:06.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc963f7700 1 --2- 172.21.15.57:0/1810434152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc117eb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009b90 tx=0x7fdc8c064000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b03baad17b2200 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc80ff9700 1 -- 172.21.15.57:0/1810434152 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c057690 con 0x55cfdc11a710 2023-11-28T12:05:06.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc80ff9700 1 -- 172.21.15.57:0/1810434152 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c003d60 con 0x55cfdc11a710 2023-11-28T12:05:06.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.543+0000 7fdc80ff9700 1 -- 172.21.15.57:0/1810434152 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c05f750 con 0x55cfdc11a710 2023-11-28T12:05:06.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.544+0000 7fdca47fab80 1 -- 172.21.15.57:0/1810434152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 msgr2=0x55cfdc117eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.544+0000 7fdca47fab80 1 --2- 172.21.15.57:0/1810434152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc117eb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009b90 tx=0x7fdc8c064000 comp rx=0 tx=0).stop 2023-11-28T12:05:06.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.544+0000 7fdca47fab80 1 -- 172.21.15.57:0/1810434152 shutdown_connections 2023-11-28T12:05:06.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.544+0000 7fdca47fab80 1 --2- 172.21.15.57:0/1810434152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc138000 0x55cfdc116e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.544+0000 7fdca47fab80 1 --2- 172.21.15.57:0/1810434152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc117eb0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.544+0000 7fdca47fab80 1 --2- 172.21.15.57:0/1810434152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc12eb70 0x55cfdc130c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.544+0000 7fdca47fab80 1 -- 172.21.15.57:0/1810434152 >> 172.21.15.57:0/1810434152 conn(0x55cfdc0e7dd0 msgr2=0x55cfdc14a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.544+0000 7fdca47fab80 1 -- 172.21.15.57:0/1810434152 shutdown_connections 2023-11-28T12:05:06.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.544+0000 7fdca47fab80 1 -- 172.21.15.57:0/1810434152 wait complete. 2023-11-28T12:05:06.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.544+0000 7fdca47fab80 1 -- start start 2023-11-28T12:05:06.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.545+0000 7fdca47fab80 1 Processor -- start 2023-11-28T12:05:06.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.546+0000 7fdca47fab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc19d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.546+0000 7fdca47fab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc12eb70 0x55cfdc0ec6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.546+0000 7fdca47fab80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc138000 0x55cfdc0ecbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.546+0000 7fdca47fab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfdc0ebf60 con 0x55cfdc11a710 2023-11-28T12:05:06.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.546+0000 7fdca47fab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfdc19ddc0 con 0x55cfdc12eb70 2023-11-28T12:05:06.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.546+0000 7fdca47fab80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfdc130860 con 0x55cfdc138000 2023-11-28T12:05:06.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.546+0000 7fdca47fab80 10 client.0 _ll_register_callbacks cb 0x55cfdbfe24b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:05:06.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.546+0000 7fdc963f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc19d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.546+0000 7fdc963f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc19d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47120/0 (socket says 172.21.15.57:47120) 2023-11-28T12:05:06.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.546+0000 7fdc963f7700 1 -- 172.21.15.57:0/2579599940 learned_addr learned my addr 172.21.15.57:0/2579599940 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.546+0000 7fdc95bf6700 1 --2- 172.21.15.57:0/2579599940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc12eb70 0x55cfdc0ec6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.546+0000 7fdc96bf8700 1 --2- 172.21.15.57:0/2579599940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc138000 0x55cfdc0ecbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdc963f7700 1 -- 172.21.15.57:0/2579599940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc138000 msgr2=0x55cfdc0ecbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdc963f7700 1 --2- 172.21.15.57:0/2579599940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc138000 0x55cfdc0ecbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdc963f7700 1 -- 172.21.15.57:0/2579599940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc12eb70 msgr2=0x55cfdc0ec6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdc963f7700 1 --2- 172.21.15.57:0/2579599940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc12eb70 0x55cfdc0ec6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdc963f7700 1 -- 172.21.15.57:0/2579599940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfdbf25150 con 0x55cfdc11a710 2023-11-28T12:05:06.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdc963f7700 1 --2- 172.21.15.57:0/2579599940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc19d370 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c057850 tx=0x7fdc8c053fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f93e0d1ccde107 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdc717fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:05:06.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdc717fa700 1 -- 172.21.15.57:0/2579599940 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c057690 con 0x55cfdc11a710 2023-11-28T12:05:06.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdc717fa700 1 -- 172.21.15.57:0/2579599940 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc8c07ac00 con 0x55cfdc11a710 2023-11-28T12:05:06.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdca47fab80 1 -- 172.21.15.57:0/2579599940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc8c083070 con 0x55cfdc11a710 2023-11-28T12:05:06.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdc717fa700 1 -- 172.21.15.57:0/2579599940 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc8c07adc0 con 0x55cfdc11a710 2023-11-28T12:05:06.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdca47fab80 10 client.6438 fetch_fsmap learned FSMap version 32 2023-11-28T12:05:06.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdca47fab80 1 -- 172.21.15.57:0/2579599940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55cfdbff1bc0 con 0x55cfdc11a710 2023-11-28T12:05:06.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdc717fa700 1 -- 172.21.15.57:0/2579599940 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c083070 con 0x55cfdc11a710 2023-11-28T12:05:06.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.547+0000 7fdc717fa700 1 -- 172.21.15.57:0/2579599940 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fdc8c053940 con 0x55cfdc11a710 2023-11-28T12:05:06.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.548+0000 7fdca47fab80 10 client.6438 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T12:05:06.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.548+0000 7fdca47fab80 10 client.6438 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T12:05:06.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.548+0000 7fdca47fab80 20 client.6438 populate_metadata read hostname 'smithi057' 2023-11-28T12:05:06.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.548+0000 7fdca47fab80 1 --2- 172.21.15.57:0/2579599940 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55cfdc117e60 0x55cfdc13b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.548+0000 7fdca47fab80 4 client.6438 mds_command: new command op to 6219 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2023-11-28T12:05:06.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.548+0000 7fdca47fab80 1 -- 172.21.15.57:0/2579599940 --> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x55cfdbec90f0 con 0x55cfdc117e60 2023-11-28T12:05:06.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.548+0000 7fdc95bf6700 1 --2- 172.21.15.57:0/2579599940 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55cfdc117e60 0x55cfdc13b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.548+0000 7fdc717fa700 10 client.6438 ms_handle_connect on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:06.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.548+0000 7fdc95bf6700 1 --2- 172.21.15.57:0/2579599940 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55cfdc117e60 0x55cfdc13b7b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6228dcdec8d760bf server_cookie=f5e38ced04e7d1dd in_seq=0 out_seq=0 2023-11-28T12:05:06.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdc717fa700 1 -- 172.21.15.57:0/2579599940 <== mds.0 v2:172.21.15.57:6833/685888843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cfdbec90f0 con 0x55cfdc117e60 2023-11-28T12:05:06.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdc717fa700 10 client.6438 handle_command_reply: tid=0 2023-11-28T12:05:06.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdca47fab80 1 client.6438 shutdown 2023-11-28T12:05:06.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdca47fab80 1 -- 172.21.15.57:0/2579599940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 msgr2=0x55cfdc19d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdca47fab80 1 --2- 172.21.15.57:0/2579599940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc19d370 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c057850 tx=0x7fdc8c053fc0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.555 INFO:tasks.ceph.mds.a.smithi057.stderr:2023-11-28T12:05:06.549+0000 7f92e67bf700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6396=0-4194304@1} caps={6396=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6396 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5596a631fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T12:05:06.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdca47fab80 1 -- 172.21.15.57:0/2579599940 shutdown_connections 2023-11-28T12:05:06.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdca47fab80 1 --2- 172.21.15.57:0/2579599940 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55cfdc117e60 0x55cfdc13b7b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdca47fab80 1 --2- 172.21.15.57:0/2579599940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cfdc138000 0x55cfdc0ecbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdca47fab80 1 --2- 172.21.15.57:0/2579599940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfdc11a710 0x55cfdc19d370 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdc717fa700 0 client.6438 ms_handle_reset on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:06.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdca47fab80 1 --2- 172.21.15.57:0/2579599940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cfdc12eb70 0x55cfdc0ec6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdca47fab80 1 -- 172.21.15.57:0/2579599940 >> 172.21.15.57:0/2579599940 conn(0x55cfdc0e7dd0 msgr2=0x55cfdc147300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdca47fab80 1 -- 172.21.15.57:0/2579599940 shutdown_connections 2023-11-28T12:05:06.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdca47fab80 1 -- 172.21.15.57:0/2579599940 wait complete. 2023-11-28T12:05:06.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.549+0000 7fdca47fab80 20 client.6438 trim_cache size 0 max 16384 2023-11-28T12:05:06.557 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:05:06.557 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2023-11-28T12:05:06.557 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "b4d60452-6a11-4833-acda-f42833416fec", 2023-11-28T12:05:06.557 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2023-11-28T12:05:06.557 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:05:06.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.550+0000 7fdc9865b700 1 -- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdc740682f0 msgr2=0x7fdc7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.550+0000 7fdc9865b700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdc740682f0 0x7fdc7406a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdc900a1050 tx=0x7fdc84008040 comp rx=0 tx=0).stop 2023-11-28T12:05:06.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.550+0000 7fdc9865b700 1 -- 172.21.15.57:0/164144296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc900b1c90 msgr2=0x7fdc900cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.550+0000 7fdc9865b700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc900b1c90 0x7fdc900cbe60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0090b0 tx=0x7fdc8c004750 comp rx=0 tx=0).stop 2023-11-28T12:05:06.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.550+0000 7fdc9865b700 1 -- 172.21.15.57:0/164144296 shutdown_connections 2023-11-28T12:05:06.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.550+0000 7fdc9865b700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fdc740682f0 0x7fdc7406a7a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.550+0000 7fdc9865b700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9006bd90 0x7fdc9006c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.550+0000 7fdc9865b700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc900b1c90 0x7fdc900cbe60 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.550+0000 7fdc9865b700 1 --2- 172.21.15.57:0/164144296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc9015cce0 0x7fdc9015bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.550+0000 7fdc9865b700 1 -- 172.21.15.57:0/164144296 >> 172.21.15.57:0/164144296 conn(0x7fdc9009c4d0 msgr2=0x7fdc900b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.550+0000 7fdc9865b700 1 -- 172.21.15.57:0/164144296 shutdown_connections 2023-11-28T12:05:06.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.550+0000 7fdc9865b700 1 -- 172.21.15.57:0/164144296 wait complete. 2023-11-28T12:05:06.564 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T12:05:06.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.700+0000 7f70eaeb6700 1 Processor -- start 2023-11-28T12:05:06.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.700+0000 7f70eaeb6700 1 -- start start 2023-11-28T12:05:06.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.701+0000 7f70eaeb6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e40b2ee0 0x7f70e40d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.701+0000 7f70eaeb6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f70e40cc200 0x7f70e40ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.701+0000 7f70eaeb6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e415a9c0 con 0x7f70e415a5f0 2023-11-28T12:05:06.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.701+0000 7f70eaeb6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e40c7b80 con 0x7f70e40cc200 2023-11-28T12:05:06.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.701+0000 7f70eaeb6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40cc990 con 0x7f70e40b2ee0 2023-11-28T12:05:06.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.701+0000 7f70e9453700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f70e40cc200 0x7f70e40ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.701+0000 7f70e3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e40b2ee0 0x7f70e40d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.702+0000 7f70e8c52700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f70e415a5f0 0x7f70e40b27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:54264/0 (socket says 172.21.15.57:54264) 2023-11-28T12:05:06.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.702+0000 7f70e8c52700 1 -- 172.21.15.57:0/3963368424 learned_addr learned my addr 172.21.15.57:0/3963368424 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:05:06.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.702+0000 7f70e37fe700 1 -- 172.21.15.57:0/3963368424 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996712235 0 0) 0x7f70e415a9c0 con 0x7f70e415a5f0 2023-11-28T12:05:06.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.702+0000 7f70e37fe700 1 -- 172.21.15.57:0/3963368424 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70cc0035b0 con 0x7f70e415a5f0 2023-11-28T12:05:06.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.702+0000 7f70e3fff700 1 -- 172.21.15.57:0/3963368424 >> v2:172.21.15.181:3300/0 conn(0x7f70e40cc200 msgr2=0x7f70e40ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.702+0000 7f70e3fff700 1 --2- 172.21.15.57:0/3963368424 >> v2:172.21.15.181:3300/0 conn(0x7f70e40cc200 0x7f70e40ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.702+0000 7f70e3fff700 1 -- 172.21.15.57:0/3963368424 >> v1:172.21.15.181:6789/0 conn(0x7f70e415a5f0 legacy=0x7f70e40b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.702+0000 7f70e3fff700 1 -- 172.21.15.57:0/3963368424 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e40ccc70 con 0x7f70e40b2ee0 2023-11-28T12:05:06.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.702+0000 7f70e3fff700 1 --2- 172.21.15.57:0/3963368424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e40b2ee0 0x7f70e40d0ad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009840 tx=0x7f70d8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb1ca96ee10027eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.702+0000 7f70e37fe700 1 -- 172.21.15.57:0/3963368424 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8011830 con 0x7f70e40b2ee0 2023-11-28T12:05:06.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.702+0000 7f70e37fe700 1 -- 172.21.15.57:0/3963368424 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d8011990 con 0x7f70e40b2ee0 2023-11-28T12:05:06.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.702+0000 7f70e37fe700 1 -- 172.21.15.57:0/3963368424 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8011d20 con 0x7f70e40b2ee0 2023-11-28T12:05:06.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.703+0000 7f70eaeb6700 1 -- 172.21.15.57:0/3963368424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e40b2ee0 msgr2=0x7f70e40d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.703+0000 7f70eaeb6700 1 --2- 172.21.15.57:0/3963368424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e40b2ee0 0x7f70e40d0ad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009840 tx=0x7f70d8009b50 comp rx=0 tx=0).stop 2023-11-28T12:05:06.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.703+0000 7f70eaeb6700 1 -- 172.21.15.57:0/3963368424 shutdown_connections 2023-11-28T12:05:06.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.703+0000 7f70eaeb6700 1 --2- 172.21.15.57:0/3963368424 >> v2:172.21.15.181:3300/0 conn(0x7f70e40cc200 0x7f70e40ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.703+0000 7f70eaeb6700 1 --2- 172.21.15.57:0/3963368424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e40b2ee0 0x7f70e40d0ad0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.703+0000 7f70eaeb6700 1 -- 172.21.15.57:0/3963368424 >> 172.21.15.57:0/3963368424 conn(0x7f70e406ea80 msgr2=0x7f70e406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.703+0000 7f70eaeb6700 1 -- 172.21.15.57:0/3963368424 shutdown_connections 2023-11-28T12:05:06.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.703+0000 7f70eaeb6700 1 -- 172.21.15.57:0/3963368424 wait complete. 2023-11-28T12:05:06.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.704+0000 7f70eaeb6700 1 Processor -- start 2023-11-28T12:05:06.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.704+0000 7f70eaeb6700 1 -- start start 2023-11-28T12:05:06.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.705+0000 7f70eaeb6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e40b2ee0 0x7f70e40ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.705+0000 7f70eaeb6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e40cc200 0x7f70e406c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.705+0000 7f70eaeb6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e415a5f0 0x7f70e40c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.705+0000 7f70eaeb6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e41f2480 con 0x7f70e415a5f0 2023-11-28T12:05:06.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.705+0000 7f70eaeb6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40cadd0 con 0x7f70e40b2ee0 2023-11-28T12:05:06.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.705+0000 7f70eaeb6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e406b940 con 0x7f70e40cc200 2023-11-28T12:05:06.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.705+0000 7f70e9453700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e415a5f0 0x7f70e40c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.705+0000 7f70e9453700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e415a5f0 0x7f70e40c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47134/0 (socket says 172.21.15.57:47134) 2023-11-28T12:05:06.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.705+0000 7f70e9453700 1 -- 172.21.15.57:0/1942304272 learned_addr learned my addr 172.21.15.57:0/1942304272 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70e3fff700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e40cc200 0x7f70e406c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70e8c52700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e40b2ee0 0x7f70e40ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70e9453700 1 -- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e40cc200 msgr2=0x7f70e406c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70e9453700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e40cc200 0x7f70e406c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70e9453700 1 -- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e40b2ee0 msgr2=0x7f70e40ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70e9453700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e40b2ee0 0x7f70e40ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70e9453700 1 -- 172.21.15.57:0/1942304272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d80094a0 con 0x7f70e415a5f0 2023-11-28T12:05:06.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70e8c52700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e40b2ee0 0x7f70e40ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:05:06.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70e9453700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e415a5f0 0x7f70e40c7e00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00efc0 tx=0x7f70dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70e17fa700 1 -- 172.21.15.57:0/1942304272 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc00cd40 con 0x7f70e415a5f0 2023-11-28T12:05:06.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70e17fa700 1 -- 172.21.15.57:0/1942304272 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc00f040 con 0x7f70e415a5f0 2023-11-28T12:05:06.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70eaeb6700 1 -- 172.21.15.57:0/1942304272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e40cd230 con 0x7f70e415a5f0 2023-11-28T12:05:06.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70e17fa700 1 -- 172.21.15.57:0/1942304272 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0146c0 con 0x7f70e415a5f0 2023-11-28T12:05:06.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.706+0000 7f70eaeb6700 1 -- 172.21.15.57:0/1942304272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e40ccf50 con 0x7f70e415a5f0 2023-11-28T12:05:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.708+0000 7f70e17fa700 1 -- 172.21.15.57:0/1942304272 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f70dc014820 con 0x7f70e415a5f0 2023-11-28T12:05:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.708+0000 7f70e17fa700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f70d00682f0 0x7f70d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.708+0000 7f70e17fa700 1 -- 172.21.15.57:0/1942304272 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70dc086700 con 0x7f70e415a5f0 2023-11-28T12:05:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.708+0000 7f70e8c52700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f70d00682f0 0x7f70d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.709+0000 7f70e8c52700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f70d00682f0 0x7f70d006a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70d40011e0 tx=0x7f70d4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.710+0000 7f70f7055b80 1 Processor -- start 2023-11-28T12:05:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.710+0000 7f70f7055b80 1 -- start start 2023-11-28T12:05:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70f7055b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89bc8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70f7055b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 0x55bf89c0f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70f7055b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89bf16b0 0x55bf89bc9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70f7055b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf89b46e30 con 0x55bf89c0efa0 2023-11-28T12:05:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70f7055b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf89ab3190 con 0x55bf89bf16b0 2023-11-28T12:05:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70f7055b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf89b21e00 con 0x55bf89c05b10 2023-11-28T12:05:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70e8c52700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89bc8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70e8c52700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89bc8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47138/0 (socket says 172.21.15.57:47138) 2023-11-28T12:05:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70e8c52700 1 -- 172.21.15.57:0/1363524511 learned_addr learned my addr 172.21.15.57:0/1363524511 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70e3fff700 1 --2- 172.21.15.57:0/1363524511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 0x55bf89c0f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70e9453700 1 --2- 172.21.15.57:0/1363524511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89bf16b0 0x55bf89bc9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70e8c52700 1 -- 172.21.15.57:0/1363524511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 msgr2=0x55bf89c0f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70e8c52700 1 --2- 172.21.15.57:0/1363524511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 0x55bf89c0f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70e8c52700 1 -- 172.21.15.57:0/1363524511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89bf16b0 msgr2=0x55bf89bc9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70e8c52700 1 --2- 172.21.15.57:0/1363524511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89bf16b0 0x55bf89bc9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70e8c52700 1 -- 172.21.15.57:0/1363524511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf8990f7e0 con 0x55bf89c0efa0 2023-11-28T12:05:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70e3fff700 1 --2- 172.21.15.57:0/1363524511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 0x55bf89c0f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.711+0000 7f70e9453700 1 --2- 172.21.15.57:0/1363524511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89bf16b0 0x55bf89bc9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.712+0000 7f70e8c52700 1 --2- 172.21.15.57:0/1363524511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89bc8a40 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f70d4005ac0 tx=0x7f70d4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efd230ade1d2065 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.712+0000 7f70ca7fc700 1 -- 172.21.15.57:0/1363524511 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4005e80 con 0x55bf89c0efa0 2023-11-28T12:05:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.712+0000 7f70ca7fc700 1 -- 172.21.15.57:0/1363524511 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d4001d40 con 0x55bf89c0efa0 2023-11-28T12:05:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.712+0000 7f70ca7fc700 1 -- 172.21.15.57:0/1363524511 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d400d850 con 0x55bf89c0efa0 2023-11-28T12:05:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.712+0000 7f70f7055b80 1 -- 172.21.15.57:0/1363524511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 msgr2=0x55bf89bc8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.712+0000 7f70f7055b80 1 --2- 172.21.15.57:0/1363524511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89bc8a40 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f70d4005ac0 tx=0x7f70d4010000 comp rx=0 tx=0).stop 2023-11-28T12:05:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.712+0000 7f70f7055b80 1 -- 172.21.15.57:0/1363524511 shutdown_connections 2023-11-28T12:05:06.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.713+0000 7f70f7055b80 1 --2- 172.21.15.57:0/1363524511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89bf16b0 0x55bf89bc9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.713+0000 7f70f7055b80 1 --2- 172.21.15.57:0/1363524511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89bc8a40 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.713+0000 7f70f7055b80 1 --2- 172.21.15.57:0/1363524511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 0x55bf89c0f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.713+0000 7f70f7055b80 1 -- 172.21.15.57:0/1363524511 >> 172.21.15.57:0/1363524511 conn(0x55bf89bbed70 msgr2=0x55bf89bc7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.713+0000 7f70f7055b80 1 -- 172.21.15.57:0/1363524511 shutdown_connections 2023-11-28T12:05:06.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.713+0000 7f70f7055b80 1 -- 172.21.15.57:0/1363524511 wait complete. 2023-11-28T12:05:06.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.713+0000 7f70f7055b80 1 -- start start 2023-11-28T12:05:06.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.714+0000 7f70f7055b80 1 Processor -- start 2023-11-28T12:05:06.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.714+0000 7f70f7055b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89bf16b0 0x55bf89c07e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.714+0000 7f70f7055b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89c05b10 0x55bf89c09770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.714+0000 7f70f7055b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89c09cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.714+0000 7f70f7055b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf89b1b170 con 0x55bf89c0efa0 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.714+0000 7f70f7055b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf899103e0 con 0x55bf89c05b10 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.714+0000 7f70f7055b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf89ab3390 con 0x55bf89bf16b0 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.714+0000 7f70f7055b80 10 client.0 _ll_register_callbacks cb 0x55bf89ab9450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.714+0000 7f70e9453700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89c09cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70e9453700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89c09cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47144/0 (socket says 172.21.15.57:47144) 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70e9453700 1 -- 172.21.15.57:0/3310510481 learned_addr learned my addr 172.21.15.57:0/3310510481 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70e8c52700 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89bf16b0 0x55bf89c07e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70e3fff700 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89c05b10 0x55bf89c09770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70e9453700 1 -- 172.21.15.57:0/3310510481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89bf16b0 msgr2=0x55bf89c07e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70e9453700 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89bf16b0 0x55bf89c07e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70e9453700 1 -- 172.21.15.57:0/3310510481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89c05b10 msgr2=0x55bf89c09770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70e9453700 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89c05b10 0x55bf89c09770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70e9453700 1 -- 172.21.15.57:0/3310510481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70cc0035b0 con 0x55bf89c0efa0 2023-11-28T12:05:06.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70e3fff700 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89c05b10 0x55bf89c09770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:06.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70e8c52700 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89bf16b0 0x55bf89c07e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:06.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70e9453700 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89c09cb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f70dc053120 tx=0x7f70dc0563f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fcd84987c4311ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:05:06.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70c37fe700 1 -- 172.21.15.57:0/3310510481 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc05ad80 con 0x55bf89c0efa0 2023-11-28T12:05:06.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70c37fe700 1 -- 172.21.15.57:0/3310510481 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70dc06dc20 con 0x55bf89c0efa0 2023-11-28T12:05:06.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70f7055b80 1 -- 172.21.15.57:0/3310510481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf89b1f080 con 0x55bf89c0efa0 2023-11-28T12:05:06.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70c37fe700 1 -- 172.21.15.57:0/3310510481 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70dc06dde0 con 0x55bf89c0efa0 2023-11-28T12:05:06.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70f7055b80 10 client.6447 fetch_fsmap learned FSMap version 32 2023-11-28T12:05:06.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70f7055b80 1 -- 172.21.15.57:0/3310510481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55bf899fc0f0 con 0x55bf89c0efa0 2023-11-28T12:05:06.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.715+0000 7f70c37fe700 1 -- 172.21.15.57:0/3310510481 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf89b1f080 con 0x55bf89c0efa0 2023-11-28T12:05:06.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.716+0000 7f70c37fe700 1 -- 172.21.15.57:0/3310510481 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f70dc052a10 con 0x55bf89c0efa0 2023-11-28T12:05:06.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.716+0000 7f70f7055b80 10 client.6447 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T12:05:06.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.716+0000 7f70f7055b80 10 client.6447 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.716+0000 7f70f7055b80 20 client.6447 populate_metadata read hostname 'smithi057' 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.716+0000 7f70f7055b80 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55bf89bf2df0 0x55bf89c0ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.716+0000 7f70f7055b80 4 client.6447 mds_command: new command op to 6219 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.716+0000 7f70f7055b80 1 -- 172.21.15.57:0/3310510481 --> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf8996af40 con 0x55bf89bf2df0 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.716+0000 7f70e8c52700 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55bf89bf2df0 0x55bf89c0ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.716+0000 7f70c37fe700 10 client.6447 ms_handle_connect on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.716+0000 7f70e8c52700 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55bf89bf2df0 0x55bf89c0ba80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec8faf5c6848cb07 server_cookie=bbba689e276e2712 in_seq=0 out_seq=0 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70c37fe700 1 -- 172.21.15.57:0/3310510481 <== mds.0 v2:172.21.15.57:6833/685888843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55bf8996af40 con 0x55bf89bf2df0 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70c37fe700 10 client.6447 handle_command_reply: tid=0 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70f7055b80 1 client.6447 shutdown 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70f7055b80 1 -- 172.21.15.57:0/3310510481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 msgr2=0x55bf89c09cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70f7055b80 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89c09cb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f70dc053120 tx=0x7f70dc0563f0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70f7055b80 1 -- 172.21.15.57:0/3310510481 shutdown_connections 2023-11-28T12:05:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70f7055b80 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55bf89bf2df0 0x55bf89c0ba80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70f7055b80 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89c09cb0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70f7055b80 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89bf16b0 0x55bf89c07e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70f7055b80 1 --2- 172.21.15.57:0/3310510481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89c05b10 0x55bf89c09770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70c37fe700 0 client.6447 ms_handle_reset on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70f7055b80 1 -- 172.21.15.57:0/3310510481 >> 172.21.15.57:0/3310510481 conn(0x55bf89bbed70 msgr2=0x55bf89bc46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70f7055b80 1 -- 172.21.15.57:0/3310510481 shutdown_connections 2023-11-28T12:05:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70f7055b80 1 -- 172.21.15.57:0/3310510481 wait complete. 2023-11-28T12:05:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.717+0000 7f70f7055b80 20 client.6447 trim_cache size 0 max 16384 2023-11-28T12:05:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.734+0000 7f70f7055b80 1 Processor -- start 2023-11-28T12:05:06.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.734+0000 7f70f7055b80 1 -- start start 2023-11-28T12:05:06.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.735+0000 7f70f7055b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89bf16b0 0x55bf89c087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.735+0000 7f70f7055b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 0x55bf89bf33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.735+0000 7f70f7055b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89bf3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.735+0000 7f70f7055b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf89bef370 con 0x55bf89c0efa0 2023-11-28T12:05:06.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.735+0000 7f70f7055b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf89bc6d90 con 0x55bf89bf16b0 2023-11-28T12:05:06.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.735+0000 7f70f7055b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf89bf2960 con 0x55bf89c05b10 2023-11-28T12:05:06.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.735+0000 7f70e9453700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89bf3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.735+0000 7f70e9453700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89bf3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47152/0 (socket says 172.21.15.57:47152) 2023-11-28T12:05:06.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.735+0000 7f70e9453700 1 -- 172.21.15.57:0/675127814 learned_addr learned my addr 172.21.15.57:0/675127814 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70e9453700 1 -- 172.21.15.57:0/675127814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 msgr2=0x55bf89bf33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70e8c52700 1 --2- 172.21.15.57:0/675127814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89bf16b0 0x55bf89c087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70e3fff700 1 --2- 172.21.15.57:0/675127814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 0x55bf89bf33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70e9453700 1 --2- 172.21.15.57:0/675127814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 0x55bf89bf33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70e9453700 1 -- 172.21.15.57:0/675127814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89bf16b0 msgr2=0x55bf89c087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70e9453700 1 --2- 172.21.15.57:0/675127814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89bf16b0 0x55bf89c087f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70e9453700 1 -- 172.21.15.57:0/675127814 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf89ae2330 con 0x55bf89c0efa0 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70e9453700 1 --2- 172.21.15.57:0/675127814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89bf3920 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f70dc053640 tx=0x7f70dc056890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8ad33de03d2b32 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70caffd700 1 -- 172.21.15.57:0/675127814 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc06f070 con 0x55bf89c0efa0 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70caffd700 1 -- 172.21.15.57:0/675127814 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc06a080 con 0x55bf89c0efa0 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70caffd700 1 -- 172.21.15.57:0/675127814 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc05ab80 con 0x55bf89c0efa0 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70f7055b80 1 -- 172.21.15.57:0/675127814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 msgr2=0x55bf89bf3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70f7055b80 1 --2- 172.21.15.57:0/675127814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89bf3920 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f70dc053640 tx=0x7f70dc056890 comp rx=0 tx=0).stop 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70f7055b80 1 -- 172.21.15.57:0/675127814 shutdown_connections 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70f7055b80 1 --2- 172.21.15.57:0/675127814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89c0efa0 0x55bf89bf3920 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70f7055b80 1 --2- 172.21.15.57:0/675127814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89bf16b0 0x55bf89c087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70f7055b80 1 --2- 172.21.15.57:0/675127814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 0x55bf89bf33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70f7055b80 1 -- 172.21.15.57:0/675127814 >> 172.21.15.57:0/675127814 conn(0x55bf89bbed70 msgr2=0x55bf89bf0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.736+0000 7f70f7055b80 1 -- 172.21.15.57:0/675127814 shutdown_connections 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.737+0000 7f70f7055b80 1 -- 172.21.15.57:0/675127814 wait complete. 2023-11-28T12:05:06.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.737+0000 7f70f7055b80 1 -- start start 2023-11-28T12:05:06.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.737+0000 7f70f7055b80 1 Processor -- start 2023-11-28T12:05:06.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.738+0000 7f70f7055b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89bf16b0 0x55bf89bef520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.738+0000 7f70f7055b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 0x55bf89c07180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.738+0000 7f70f7055b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89c0efa0 0x55bf89c076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.738+0000 7f70f7055b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf89c73d90 con 0x55bf89bf16b0 2023-11-28T12:05:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.738+0000 7f70f7055b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf89c05820 con 0x55bf89c0efa0 2023-11-28T12:05:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.738+0000 7f70f7055b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf89befbd0 con 0x55bf89c05b10 2023-11-28T12:05:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.738+0000 7f70f7055b80 10 client.0 _ll_register_callbacks cb 0x55bf89ab9450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:05:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.738+0000 7f70e8c52700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89bf16b0 0x55bf89bef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.738+0000 7f70e8c52700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89bf16b0 0x55bf89bef520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47164/0 (socket says 172.21.15.57:47164) 2023-11-28T12:05:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.738+0000 7f70e8c52700 1 -- 172.21.15.57:0/3955286824 learned_addr learned my addr 172.21.15.57:0/3955286824 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.738+0000 7f70e3fff700 1 --2- 172.21.15.57:0/3955286824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 0x55bf89c07180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.738+0000 7f70e9453700 1 --2- 172.21.15.57:0/3955286824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89c0efa0 0x55bf89c076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70e8c52700 1 -- 172.21.15.57:0/3955286824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 msgr2=0x55bf89c07180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70e8c52700 1 --2- 172.21.15.57:0/3955286824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 0x55bf89c07180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70e8c52700 1 -- 172.21.15.57:0/3955286824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89c0efa0 msgr2=0x55bf89c076c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:06.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70e8c52700 1 --2- 172.21.15.57:0/3955286824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89c0efa0 0x55bf89c076c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70e8c52700 1 -- 172.21.15.57:0/3955286824 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf899fc0f0 con 0x55bf89bf16b0 2023-11-28T12:05:06.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:05:06.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70e8c52700 1 --2- 172.21.15.57:0/3955286824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89bf16b0 0x55bf89bef520 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f70d4000f60 tx=0x7f70d4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b291f8d2546916 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:06.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70c37fe700 1 -- 172.21.15.57:0/3955286824 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4004a70 con 0x55bf89bf16b0 2023-11-28T12:05:06.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70c37fe700 1 -- 172.21.15.57:0/3955286824 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70d402f850 con 0x55bf89bf16b0 2023-11-28T12:05:06.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70f7055b80 1 -- 172.21.15.57:0/3955286824 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70d4001d40 con 0x55bf89bf16b0 2023-11-28T12:05:06.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70c37fe700 1 -- 172.21.15.57:0/3955286824 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70d4005e80 con 0x55bf89bf16b0 2023-11-28T12:05:06.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70f7055b80 10 client.6453 fetch_fsmap learned FSMap version 32 2023-11-28T12:05:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70f7055b80 1 -- 172.21.15.57:0/3955286824 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55bf89970220 con 0x55bf89bf16b0 2023-11-28T12:05:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70c37fe700 1 -- 172.21.15.57:0/3955286824 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d4001d40 con 0x55bf89bf16b0 2023-11-28T12:05:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70c37fe700 1 -- 172.21.15.57:0/3955286824 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f70d402fa10 con 0x55bf89bf16b0 2023-11-28T12:05:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70f7055b80 10 client.6453 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T12:05:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70f7055b80 10 client.6453 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T12:05:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.739+0000 7f70f7055b80 20 client.6453 populate_metadata read hostname 'smithi057' 2023-11-28T12:05:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70f7055b80 1 --2- 172.21.15.57:0/3955286824 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55bf89bea300 0x55bf89c04890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70f7055b80 4 client.6453 mds_command: new command op to 6219 tid=0[{"prefix": "scrub status"}] 2023-11-28T12:05:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70f7055b80 1 -- 172.21.15.57:0/3955286824 --> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bf899a0090 con 0x55bf89bea300 2023-11-28T12:05:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70e3fff700 1 --2- 172.21.15.57:0/3955286824 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55bf89bea300 0x55bf89c04890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:06.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70c37fe700 10 client.6453 ms_handle_connect on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:06.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70e3fff700 1 --2- 172.21.15.57:0/3955286824 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55bf89bea300 0x55bf89c04890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0be8121133f4f1 server_cookie=e9529034b1ac4e24 in_seq=0 out_seq=0 2023-11-28T12:05:06.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70c37fe700 1 -- 172.21.15.57:0/3955286824 <== mds.0 v2:172.21.15.57:6833/685888843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x55bf899a0090 con 0x55bf89bea300 2023-11-28T12:05:06.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70c37fe700 10 client.6453 handle_command_reply: tid=0 2023-11-28T12:05:06.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70f7055b80 1 client.6453 shutdown 2023-11-28T12:05:06.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70f7055b80 1 -- 172.21.15.57:0/3955286824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89bf16b0 msgr2=0x55bf89bef520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70f7055b80 1 --2- 172.21.15.57:0/3955286824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89bf16b0 0x55bf89bef520 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f70d4000f60 tx=0x7f70d4000ed0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70f7055b80 1 -- 172.21.15.57:0/3955286824 shutdown_connections 2023-11-28T12:05:06.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70f7055b80 1 --2- 172.21.15.57:0/3955286824 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55bf89bea300 0x55bf89c04890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.740+0000 7f70f7055b80 1 --2- 172.21.15.57:0/3955286824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bf89c0efa0 0x55bf89c076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70f7055b80 1 --2- 172.21.15.57:0/3955286824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf89bf16b0 0x55bf89bef520 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70c37fe700 0 client.6453 ms_handle_reset on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:06.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70f7055b80 1 --2- 172.21.15.57:0/3955286824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55bf89c05b10 0x55bf89c07180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70f7055b80 1 -- 172.21.15.57:0/3955286824 >> 172.21.15.57:0/3955286824 conn(0x55bf89bbed70 msgr2=0x55bf89c09770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70f7055b80 1 -- 172.21.15.57:0/3955286824 shutdown_connections 2023-11-28T12:05:06.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70f7055b80 1 -- 172.21.15.57:0/3955286824 wait complete. 2023-11-28T12:05:06.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70f7055b80 20 client.6453 trim_cache size 0 max 16384 2023-11-28T12:05:06.749 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:05:06.750 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T12:05:06.750 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-11-28T12:05:06.750 INFO:teuthology.orchestra.run.smithi057.stdout: "b4d60452-6a11-4833-acda-f42833416fec": { 2023-11-28T12:05:06.750 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-11-28T12:05:06.750 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "b4d60452-6a11-4833-acda-f42833416fec", 2023-11-28T12:05:06.750 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive" 2023-11-28T12:05:06.750 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:05:06.750 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:05:06.750 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:05:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70eaeb6700 1 -- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f70d00682f0 msgr2=0x7f70d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70eaeb6700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f70d00682f0 0x7f70d006a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70d40011e0 tx=0x7f70d4009040 comp rx=0 tx=0).stop 2023-11-28T12:05:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70eaeb6700 1 -- 172.21.15.57:0/1942304272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e415a5f0 msgr2=0x7f70e40c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70eaeb6700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e415a5f0 0x7f70e40c7e00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00efc0 tx=0x7f70dc00c3a0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70eaeb6700 1 -- 172.21.15.57:0/1942304272 shutdown_connections 2023-11-28T12:05:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70eaeb6700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f70d00682f0 0x7f70d006a7a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70eaeb6700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e415a5f0 0x7f70e40c7e00 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70eaeb6700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e40b2ee0 0x7f70e40ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70eaeb6700 1 --2- 172.21.15.57:0/1942304272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e40cc200 0x7f70e406c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70eaeb6700 1 -- 172.21.15.57:0/1942304272 >> 172.21.15.57:0/1942304272 conn(0x7f70e406ea80 msgr2=0x7f70e40ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70eaeb6700 1 -- 172.21.15.57:0/1942304272 shutdown_connections 2023-11-28T12:05:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:06.741+0000 7f70eaeb6700 1 -- 172.21.15.57:0/1942304272 wait complete. 2023-11-28T12:05:06.756 INFO:tasks.cephfs.filesystem:scrub status for tag:b4d60452-6a11-4833-acda-f42833416fec - {'path': '/', 'tag': 'b4d60452-6a11-4833-acda-f42833416fec', 'options': 'recursive'} 2023-11-28T12:05:08.579 INFO:tasks.ceph.mon.a.smithi057.stderr:2023-11-28T12:05:08.578+0000 7fb777740700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-28T12:05:16.631 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:05:16.634 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:05:36.757 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T12:05:36.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.903+0000 7fb27ca1a700 1 Processor -- start 2023-11-28T12:05:36.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.903+0000 7fb27ca1a700 1 -- start start 2023-11-28T12:05:36.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb27ca1a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb27815a5f0 0x7fb2780d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb27ca1a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb2780cc200 0x7fb2780b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb27ca1a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27815a9c0 con 0x7fb2780b2d10 2023-11-28T12:05:36.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb27ca1a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2780b3190 con 0x7fb27815a5f0 2023-11-28T12:05:36.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb27ca1a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2780c7b80 con 0x7fb2780cc200 2023-11-28T12:05:36.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb27659c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb27815a5f0 0x7fb2780d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb276d9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fb2780b2d10 0x7fb27815e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46844/0 (socket says 172.21.15.57:46844) 2023-11-28T12:05:36.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb276d9d700 1 -- 172.21.15.57:0/1050472941 learned_addr learned my addr 172.21.15.57:0/1050472941 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:05:36.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb275d9b700 1 --2- 172.21.15.57:0/1050472941 >> v2:172.21.15.181:3300/0 conn(0x7fb2780cc200 0x7fb2780b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb27659c700 1 -- 172.21.15.57:0/1050472941 >> v2:172.21.15.181:3300/0 conn(0x7fb2780cc200 msgr2=0x7fb2780b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:36.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb27659c700 1 --2- 172.21.15.57:0/1050472941 >> v2:172.21.15.181:3300/0 conn(0x7fb2780cc200 0x7fb2780b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb27659c700 1 -- 172.21.15.57:0/1050472941 >> v1:172.21.15.57:6789/0 conn(0x7fb2780b2d10 legacy=0x7fb27815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:36.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb27659c700 1 -- 172.21.15.57:0/1050472941 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2780cc990 con 0x7fb27815a5f0 2023-11-28T12:05:36.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb27559a700 1 -- 172.21.15.57:0/1050472941 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747074694 0 0) 0x7fb27815a9c0 con 0x7fb2780b2d10 2023-11-28T12:05:36.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.904+0000 7fb275d9b700 1 --2- 172.21.15.57:0/1050472941 >> v2:172.21.15.181:3300/0 conn(0x7fb2780cc200 0x7fb2780b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:36.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.905+0000 7fb27659c700 1 --2- 172.21.15.57:0/1050472941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb27815a5f0 0x7fb2780d0ad0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb27806d8d0 tx=0x7fb26c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a03b34ae9d19104 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:36.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.905+0000 7fb27559a700 1 -- 172.21.15.57:0/1050472941 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c011830 con 0x7fb27815a5f0 2023-11-28T12:05:36.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.905+0000 7fb27559a700 1 -- 172.21.15.57:0/1050472941 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c011990 con 0x7fb27815a5f0 2023-11-28T12:05:36.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.905+0000 7fb27559a700 1 -- 172.21.15.57:0/1050472941 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c011d20 con 0x7fb27815a5f0 2023-11-28T12:05:36.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.905+0000 7fb27ca1a700 1 -- 172.21.15.57:0/1050472941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb27815a5f0 msgr2=0x7fb2780d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:36.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.905+0000 7fb27ca1a700 1 --2- 172.21.15.57:0/1050472941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb27815a5f0 0x7fb2780d0ad0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb27806d8d0 tx=0x7fb26c009b20 comp rx=0 tx=0).stop 2023-11-28T12:05:36.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.905+0000 7fb27ca1a700 1 -- 172.21.15.57:0/1050472941 shutdown_connections 2023-11-28T12:05:36.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.905+0000 7fb27ca1a700 1 --2- 172.21.15.57:0/1050472941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb27815a5f0 0x7fb2780d0ad0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.905+0000 7fb27ca1a700 1 --2- 172.21.15.57:0/1050472941 >> v2:172.21.15.181:3300/0 conn(0x7fb2780cc200 0x7fb2780b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.905+0000 7fb27ca1a700 1 -- 172.21.15.57:0/1050472941 >> 172.21.15.57:0/1050472941 conn(0x7fb27806ea80 msgr2=0x7fb27806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:36.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.905+0000 7fb27ca1a700 1 -- 172.21.15.57:0/1050472941 shutdown_connections 2023-11-28T12:05:36.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.905+0000 7fb27ca1a700 1 -- 172.21.15.57:0/1050472941 wait complete. 2023-11-28T12:05:36.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.907+0000 7fb27ca1a700 1 Processor -- start 2023-11-28T12:05:36.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.907+0000 7fb27ca1a700 1 -- start start 2023-11-28T12:05:36.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27ca1a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2780b2d10 0x7fb27814b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27ca1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2780cc200 0x7fb2781f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27ca1a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb27815a5f0 0x7fb2780b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27ca1a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2780c9ab0 con 0x7fb2780b2d10 2023-11-28T12:05:36.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27ca1a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2781f2d90 con 0x7fb2780cc200 2023-11-28T12:05:36.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27ca1a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2781f3070 con 0x7fb27815a5f0 2023-11-28T12:05:36.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2780b2d10 0x7fb27814b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2780b2d10 0x7fb27814b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37952/0 (socket says 172.21.15.57:37952) 2023-11-28T12:05:36.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27659c700 1 -- 172.21.15.57:0/1138721432 learned_addr learned my addr 172.21.15.57:0/1138721432 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:36.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb275d9b700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2780cc200 0x7fb2781f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb276d9d700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb27815a5f0 0x7fb2780b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27659c700 1 -- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb27815a5f0 msgr2=0x7fb2780b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:36.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27659c700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb27815a5f0 0x7fb2780b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27659c700 1 -- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2780cc200 msgr2=0x7fb2781f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:36.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27659c700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2780cc200 0x7fb2781f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27659c700 1 -- 172.21.15.57:0/1138721432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c0094a0 con 0x7fb2780b2d10 2023-11-28T12:05:36.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.908+0000 7fb27659c700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2780b2d10 0x7fb27814b0c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0090b0 tx=0x7fb26c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:36.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.909+0000 7fb266ffd700 1 -- 172.21.15.57:0/1138721432 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0036a0 con 0x7fb2780b2d10 2023-11-28T12:05:36.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.909+0000 7fb266ffd700 1 -- 172.21.15.57:0/1138721432 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c01f070 con 0x7fb2780b2d10 2023-11-28T12:05:36.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.909+0000 7fb27ca1a700 1 -- 172.21.15.57:0/1138721432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2780ccf50 con 0x7fb2780b2d10 2023-11-28T12:05:36.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.909+0000 7fb266ffd700 1 -- 172.21.15.57:0/1138721432 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c003a00 con 0x7fb2780b2d10 2023-11-28T12:05:36.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.909+0000 7fb27ca1a700 1 -- 172.21.15.57:0/1138721432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2780ccc70 con 0x7fb2780b2d10 2023-11-28T12:05:36.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.910+0000 7fb266ffd700 1 -- 172.21.15.57:0/1138721432 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb26c00f050 con 0x7fb2780b2d10 2023-11-28T12:05:36.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.911+0000 7fb266ffd700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb258068320 0x7fb25806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.911+0000 7fb266ffd700 1 -- 172.21.15.57:0/1138721432 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fb26c021070 con 0x7fb2780b2d10 2023-11-28T12:05:36.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.911+0000 7fb275d9b700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb258068320 0x7fb25806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.912+0000 7fb275d9b700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb258068320 0x7fb25806a7d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb2780cf340 tx=0x7fb268008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:36.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.913+0000 7fb288bb9b80 1 Processor -- start 2023-11-28T12:05:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.913+0000 7fb288bb9b80 1 -- start start 2023-11-28T12:05:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.914+0000 7fb288bb9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd8357efa0 0x55fd83538a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.914+0000 7fb288bb9b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd83575b10 0x55fd8357f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.914+0000 7fb288bb9b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 0x55fd83539bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.914+0000 7fb288bb9b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd834b6e30 con 0x55fd835616b0 2023-11-28T12:05:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.914+0000 7fb288bb9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd83423190 con 0x55fd8357efa0 2023-11-28T12:05:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.914+0000 7fb288bb9b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd83491e00 con 0x55fd83575b10 2023-11-28T12:05:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.914+0000 7fb276d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 0x55fd83539bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.914+0000 7fb276d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 0x55fd83539bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37960/0 (socket says 172.21.15.57:37960) 2023-11-28T12:05:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.914+0000 7fb276d9d700 1 -- 172.21.15.57:0/2075727883 learned_addr learned my addr 172.21.15.57:0/2075727883 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.914+0000 7fb27659c700 1 --2- 172.21.15.57:0/2075727883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd8357efa0 0x55fd83538a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.914+0000 7fb275d9b700 1 --2- 172.21.15.57:0/2075727883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd83575b10 0x55fd8357f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.915+0000 7fb276d9d700 1 -- 172.21.15.57:0/2075727883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd83575b10 msgr2=0x55fd8357f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.915+0000 7fb276d9d700 1 --2- 172.21.15.57:0/2075727883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd83575b10 0x55fd8357f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.915+0000 7fb276d9d700 1 -- 172.21.15.57:0/2075727883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd8357efa0 msgr2=0x55fd83538a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.915+0000 7fb276d9d700 1 --2- 172.21.15.57:0/2075727883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd8357efa0 0x55fd83538a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.915+0000 7fb276d9d700 1 -- 172.21.15.57:0/2075727883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd8327f7e0 con 0x55fd835616b0 2023-11-28T12:05:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.915+0000 7fb27659c700 1 --2- 172.21.15.57:0/2075727883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd8357efa0 0x55fd83538a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:05:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.915+0000 7fb276d9d700 1 --2- 172.21.15.57:0/2075727883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 0x55fd83539bb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb2700021a0 tx=0x7fb270007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f048ad8ba5d4556e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.915+0000 7fb257fff700 1 -- 172.21.15.57:0/2075727883 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb270007a40 con 0x55fd835616b0 2023-11-28T12:05:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.915+0000 7fb257fff700 1 -- 172.21.15.57:0/2075727883 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb270014480 con 0x55fd835616b0 2023-11-28T12:05:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.915+0000 7fb257fff700 1 -- 172.21.15.57:0/2075727883 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb270014830 con 0x55fd835616b0 2023-11-28T12:05:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.916+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2075727883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 msgr2=0x55fd83539bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.916+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2075727883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 0x55fd83539bb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb2700021a0 tx=0x7fb270007330 comp rx=0 tx=0).stop 2023-11-28T12:05:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.916+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2075727883 shutdown_connections 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.916+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2075727883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 0x55fd83539bb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.916+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2075727883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd8357efa0 0x55fd83538a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.916+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2075727883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd83575b10 0x55fd8357f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.916+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2075727883 >> 172.21.15.57:0/2075727883 conn(0x55fd8352ed70 msgr2=0x55fd83537f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.916+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2075727883 shutdown_connections 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.916+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2075727883 wait complete. 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.916+0000 7fb288bb9b80 1 -- start start 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.916+0000 7fb288bb9b80 1 Processor -- start 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.917+0000 7fb288bb9b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 0x55fd83561bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.917+0000 7fb288bb9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd83575b10 0x55fd83577df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.917+0000 7fb288bb9b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 0x55fd83578330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.917+0000 7fb288bb9b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8348b170 con 0x55fd835616b0 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.917+0000 7fb288bb9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd832803e0 con 0x55fd83575b10 2023-11-28T12:05:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.917+0000 7fb288bb9b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd83423390 con 0x55fd8357efa0 2023-11-28T12:05:36.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb288bb9b80 10 client.0 _ll_register_callbacks cb 0x55fd83429450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:05:36.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb27659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 0x55fd83561bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb27659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 0x55fd83561bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37972/0 (socket says 172.21.15.57:37972) 2023-11-28T12:05:36.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb27659c700 1 -- 172.21.15.57:0/2769818625 learned_addr learned my addr 172.21.15.57:0/2769818625 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:36.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb276d9d700 1 --2- 172.21.15.57:0/2769818625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 0x55fd83578330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb275d9b700 1 --2- 172.21.15.57:0/2769818625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd83575b10 0x55fd83577df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb27659c700 1 -- 172.21.15.57:0/2769818625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 msgr2=0x55fd83578330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:36.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb27659c700 1 --2- 172.21.15.57:0/2769818625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 0x55fd83578330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb27659c700 1 -- 172.21.15.57:0/2769818625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd83575b10 msgr2=0x55fd83577df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:36.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb27659c700 1 --2- 172.21.15.57:0/2769818625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd83575b10 0x55fd83577df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb27659c700 1 -- 172.21.15.57:0/2769818625 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27000f040 con 0x55fd835616b0 2023-11-28T12:05:36.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb27659c700 1 --2- 172.21.15.57:0/2769818625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 0x55fd83561bb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb26c057ac0 tx=0x7fb26c057600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e55625c76042501 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb2557fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:05:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb2557fa700 1 -- 172.21.15.57:0/2769818625 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c003b60 con 0x55fd835616b0 2023-11-28T12:05:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb2557fa700 1 -- 172.21.15.57:0/2769818625 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fb26c003cc0 con 0x55fd835616b0 2023-11-28T12:05:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2769818625 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd8348f080 con 0x55fd835616b0 2023-11-28T12:05:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb2557fa700 1 -- 172.21.15.57:0/2769818625 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb26c003e80 con 0x55fd835616b0 2023-11-28T12:05:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.918+0000 7fb288bb9b80 10 client.6468 fetch_fsmap learned FSMap version 33 2023-11-28T12:05:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.919+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2769818625 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55fd8336c0f0 con 0x55fd835616b0 2023-11-28T12:05:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.919+0000 7fb2557fa700 1 -- 172.21.15.57:0/2769818625 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fd8348f080 con 0x55fd835616b0 2023-11-28T12:05:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.919+0000 7fb2557fa700 1 -- 172.21.15.57:0/2769818625 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb26c05baf0 con 0x55fd835616b0 2023-11-28T12:05:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.919+0000 7fb288bb9b80 10 client.6468 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T12:05:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.919+0000 7fb288bb9b80 10 client.6468 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T12:05:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.919+0000 7fb288bb9b80 20 client.6468 populate_metadata read hostname 'smithi057' 2023-11-28T12:05:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.919+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2769818625 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55fd83567ed0 0x55fd83576470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.919+0000 7fb288bb9b80 4 client.6468 mds_command: new command op to 6219 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T12:05:36.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.919+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2769818625 --> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd832daf40 con 0x55fd83567ed0 2023-11-28T12:05:36.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.919+0000 7fb275d9b700 1 --2- 172.21.15.57:0/2769818625 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55fd83567ed0 0x55fd83576470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.919+0000 7fb2557fa700 10 client.6468 ms_handle_connect on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:36.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.919+0000 7fb275d9b700 1 --2- 172.21.15.57:0/2769818625 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55fd83567ed0 0x55fd83576470 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4b89c30554d1026 server_cookie=c9a5f46d360fcdd6 in_seq=0 out_seq=0 2023-11-28T12:05:36.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb2557fa700 1 -- 172.21.15.57:0/2769818625 <== mds.0 v2:172.21.15.57:6833/685888843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55fd832daf40 con 0x55fd83567ed0 2023-11-28T12:05:36.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb2557fa700 10 client.6468 handle_command_reply: tid=0 2023-11-28T12:05:36.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb288bb9b80 1 client.6468 shutdown 2023-11-28T12:05:36.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2769818625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 msgr2=0x55fd83561bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:36.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2769818625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 0x55fd83561bb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb26c057ac0 tx=0x7fb26c057600 comp rx=0 tx=0).stop 2023-11-28T12:05:36.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2769818625 shutdown_connections 2023-11-28T12:05:36.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2769818625 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55fd83567ed0 0x55fd83576470 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2769818625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 0x55fd83578330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2769818625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd835616b0 0x55fd83561bb0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2769818625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd83575b10 0x55fd83577df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2769818625 >> 172.21.15.57:0/2769818625 conn(0x55fd8352ed70 msgr2=0x55fd83533f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:36.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb2557fa700 0 client.6468 ms_handle_reset on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:36.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2769818625 shutdown_connections 2023-11-28T12:05:36.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2769818625 wait complete. 2023-11-28T12:05:36.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.920+0000 7fb288bb9b80 20 client.6468 trim_cache size 0 max 16384 2023-11-28T12:05:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.938+0000 7fb288bb9b80 1 Processor -- start 2023-11-28T12:05:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.938+0000 7fb288bb9b80 1 -- start start 2023-11-28T12:05:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.938+0000 7fb288bb9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd835616b0 0x55fd83582c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb288bb9b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 0x55fd8358d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb288bb9b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 0x55fd83563710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb288bb9b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd835653f0 con 0x55fd83575b10 2023-11-28T12:05:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb288bb9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8353b670 con 0x55fd835616b0 2023-11-28T12:05:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb288bb9b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd8355ef40 con 0x55fd8357efa0 2023-11-28T12:05:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb275d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 0x55fd8358d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb275d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 0x55fd8358d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37978/0 (socket says 172.21.15.57:37978) 2023-11-28T12:05:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb275d9b700 1 -- 172.21.15.57:0/4265878356 learned_addr learned my addr 172.21.15.57:0/4265878356 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb276d9d700 1 --2- 172.21.15.57:0/4265878356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 0x55fd83563710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb27659c700 1 --2- 172.21.15.57:0/4265878356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd835616b0 0x55fd83582c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb275d9b700 1 -- 172.21.15.57:0/4265878356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 msgr2=0x55fd83563710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb275d9b700 1 --2- 172.21.15.57:0/4265878356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 0x55fd83563710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb275d9b700 1 -- 172.21.15.57:0/4265878356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd835616b0 msgr2=0x55fd83582c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb275d9b700 1 --2- 172.21.15.57:0/4265878356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd835616b0 0x55fd83582c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb275d9b700 1 -- 172.21.15.57:0/4265878356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd83452330 con 0x55fd83575b10 2023-11-28T12:05:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb275d9b700 1 --2- 172.21.15.57:0/4265878356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 0x55fd8358d7c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x55fd8353bc70 tx=0x7fb268016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5901678d7266fb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb264ff9700 1 -- 172.21.15.57:0/4265878356 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268029070 con 0x55fd83575b10 2023-11-28T12:05:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb264ff9700 1 -- 172.21.15.57:0/4265878356 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26802a250 con 0x55fd83575b10 2023-11-28T12:05:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.939+0000 7fb264ff9700 1 -- 172.21.15.57:0/4265878356 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2680157d0 con 0x55fd83575b10 2023-11-28T12:05:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.940+0000 7fb288bb9b80 1 -- 172.21.15.57:0/4265878356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 msgr2=0x55fd8358d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.940+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/4265878356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 0x55fd8358d7c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x55fd8353bc70 tx=0x7fb268016000 comp rx=0 tx=0).stop 2023-11-28T12:05:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.940+0000 7fb288bb9b80 1 -- 172.21.15.57:0/4265878356 shutdown_connections 2023-11-28T12:05:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.940+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/4265878356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 0x55fd83563710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.940+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/4265878356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd835616b0 0x55fd83582c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.940+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/4265878356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 0x55fd8358d7c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.940+0000 7fb288bb9b80 1 -- 172.21.15.57:0/4265878356 >> 172.21.15.57:0/4265878356 conn(0x55fd8352ed70 msgr2=0x55fd83561bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.940+0000 7fb288bb9b80 1 -- 172.21.15.57:0/4265878356 shutdown_connections 2023-11-28T12:05:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.940+0000 7fb288bb9b80 1 -- 172.21.15.57:0/4265878356 wait complete. 2023-11-28T12:05:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.940+0000 7fb288bb9b80 1 -- start start 2023-11-28T12:05:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.941+0000 7fb288bb9b80 1 Processor -- start 2023-11-28T12:05:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb288bb9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd835616b0 0x55fd835c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb288bb9b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 0x55fd835ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb288bb9b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 0x55fd83590310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb288bb9b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd835e5450 con 0x55fd83575b10 2023-11-28T12:05:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb288bb9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd83590b40 con 0x55fd835616b0 2023-11-28T12:05:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb288bb9b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd835c9fa0 con 0x55fd8357efa0 2023-11-28T12:05:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb288bb9b80 10 client.0 _ll_register_callbacks cb 0x55fd83429450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:05:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb275d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 0x55fd835ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb275d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 0x55fd835ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37984/0 (socket says 172.21.15.57:37984) 2023-11-28T12:05:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb275d9b700 1 -- 172.21.15.57:0/2879692692 learned_addr learned my addr 172.21.15.57:0/2879692692 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb276d9d700 1 --2- 172.21.15.57:0/2879692692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 0x55fd83590310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb27659c700 1 --2- 172.21.15.57:0/2879692692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd835616b0 0x55fd835c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb275d9b700 1 -- 172.21.15.57:0/2879692692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 msgr2=0x55fd83590310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb275d9b700 1 --2- 172.21.15.57:0/2879692692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 0x55fd83590310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb275d9b700 1 -- 172.21.15.57:0/2879692692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd835616b0 msgr2=0x55fd835c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb275d9b700 1 --2- 172.21.15.57:0/2879692692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd835616b0 0x55fd835c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb275d9b700 1 -- 172.21.15.57:0/2879692692 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26800e040 con 0x55fd83575b10 2023-11-28T12:05:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb275d9b700 1 --2- 172.21.15.57:0/2879692692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 0x55fd835ca430 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb268000c00 tx=0x7fb2680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d667bc0f8f3bb765 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb2557fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:05:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb2557fa700 1 -- 172.21.15.57:0/2879692692 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2680157d0 con 0x55fd83575b10 2023-11-28T12:05:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb2557fa700 1 -- 172.21.15.57:0/2879692692 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fb268015930 con 0x55fd83575b10 2023-11-28T12:05:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.942+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2879692692 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb270014480 con 0x55fd83575b10 2023-11-28T12:05:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb2557fa700 1 -- 172.21.15.57:0/2879692692 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb268015af0 con 0x55fd83575b10 2023-11-28T12:05:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb288bb9b80 10 client.6474 fetch_fsmap learned FSMap version 33 2023-11-28T12:05:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2879692692 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55fd832e0220 con 0x55fd83575b10 2023-11-28T12:05:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb2557fa700 1 -- 172.21.15.57:0/2879692692 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb270014480 con 0x55fd83575b10 2023-11-28T12:05:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb2557fa700 1 -- 172.21.15.57:0/2879692692 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb268015c50 con 0x55fd83575b10 2023-11-28T12:05:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb288bb9b80 10 client.6474 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T12:05:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb288bb9b80 10 client.6474 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T12:05:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb288bb9b80 20 client.6474 populate_metadata read hostname 'smithi057' 2023-11-28T12:05:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2879692692 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55fd835e4670 0x55fd835e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb288bb9b80 4 client.6474 mds_command: new command op to 6219 tid=0[{"prefix": "scrub status"}] 2023-11-28T12:05:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2879692692 --> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fd83310090 con 0x55fd835e4670 2023-11-28T12:05:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb27659c700 1 --2- 172.21.15.57:0/2879692692 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55fd835e4670 0x55fd835e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb2557fa700 10 client.6474 ms_handle_connect on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.943+0000 7fb27659c700 1 --2- 172.21.15.57:0/2879692692 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55fd835e4670 0x55fd835e4a40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f5827978bd9787 server_cookie=3c49f88b4793c79 in_seq=0 out_seq=0 2023-11-28T12:05:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb2557fa700 1 -- 172.21.15.57:0/2879692692 <== mds.0 v2:172.21.15.57:6833/685888843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fd83310090 con 0x55fd835e4670 2023-11-28T12:05:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb2557fa700 10 client.6474 handle_command_reply: tid=0 2023-11-28T12:05:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb288bb9b80 1 client.6474 shutdown 2023-11-28T12:05:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2879692692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 msgr2=0x55fd835ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2879692692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 0x55fd835ca430 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb268000c00 tx=0x7fb2680047c0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2879692692 shutdown_connections 2023-11-28T12:05:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2879692692 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x55fd835e4670 0x55fd835e4a40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2879692692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fd8357efa0 0x55fd83590310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2879692692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fd835616b0 0x55fd835c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb2557fa700 0 client.6474 ms_handle_reset on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb288bb9b80 1 --2- 172.21.15.57:0/2879692692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fd83575b10 0x55fd835ca430 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2879692692 >> 172.21.15.57:0/2879692692 conn(0x55fd8352ed70 msgr2=0x55fd83567390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:36.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2879692692 shutdown_connections 2023-11-28T12:05:36.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb288bb9b80 1 -- 172.21.15.57:0/2879692692 wait complete. 2023-11-28T12:05:36.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.944+0000 7fb288bb9b80 20 client.6474 trim_cache size 0 max 16384 2023-11-28T12:05:36.952 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:05:36.952 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2023-11-28T12:05:36.952 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2023-11-28T12:05:36.952 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:05:36.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.945+0000 7fb27ca1a700 1 -- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb258068320 msgr2=0x7fb25806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:36.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.945+0000 7fb27ca1a700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb258068320 0x7fb25806a7d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb2780cf340 tx=0x7fb268008040 comp rx=0 tx=0).stop 2023-11-28T12:05:36.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.945+0000 7fb27ca1a700 1 -- 172.21.15.57:0/1138721432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2780b2d10 msgr2=0x7fb27814b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:36.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.945+0000 7fb27ca1a700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2780b2d10 0x7fb27814b0c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0090b0 tx=0x7fb26c004750 comp rx=0 tx=0).stop 2023-11-28T12:05:36.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.945+0000 7fb27ca1a700 1 -- 172.21.15.57:0/1138721432 shutdown_connections 2023-11-28T12:05:36.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.945+0000 7fb27ca1a700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb258068320 0x7fb25806a7d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.945+0000 7fb27ca1a700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb27815a5f0 0x7fb2780b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.945+0000 7fb27ca1a700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2780b2d10 0x7fb27814b0c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.945+0000 7fb27ca1a700 1 --2- 172.21.15.57:0/1138721432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2780cc200 0x7fb2781f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:36.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.945+0000 7fb27ca1a700 1 -- 172.21.15.57:0/1138721432 >> 172.21.15.57:0/1138721432 conn(0x7fb27806ea80 msgr2=0x7fb2780b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:36.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.945+0000 7fb27ca1a700 1 -- 172.21.15.57:0/1138721432 shutdown_connections 2023-11-28T12:05:36.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:36.945+0000 7fb27ca1a700 1 -- 172.21.15.57:0/1138721432 wait complete. 2023-11-28T12:05:36.960 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T12:05:36.960 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:05:37.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.094+0000 7f0980c7b700 1 Processor -- start 2023-11-28T12:05:37.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.094+0000 7f0980c7b700 1 -- start start 2023-11-28T12:05:37.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f0980c7b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f097c0b0880 0x7f097c06b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f0980c7b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f097c0ad1f0 0x7f097c0ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f0980c7b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097c0b43f0 con 0x7f097c072180 2023-11-28T12:05:37.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f0980c7b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f097c0adbb0 con 0x7f097c0ad1f0 2023-11-28T12:05:37.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f0980c7b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097c0b46d0 con 0x7f097c0b0880 2023-11-28T12:05:37.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f097ad9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f097c0ad1f0 0x7f097c0ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f097ad9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f097c0ad1f0 0x7f097c0ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37994/0 (socket says 172.21.15.57:37994) 2023-11-28T12:05:37.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f097ad9d700 1 -- 172.21.15.57:0/2970376090 learned_addr learned my addr 172.21.15.57:0/2970376090 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:37.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f097a59c700 1 --2- 172.21.15.57:0/2970376090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f097c0b0880 0x7f097c06b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f097ad9d700 1 -- 172.21.15.57:0/2970376090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f097c0b0880 msgr2=0x7f097c06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f097ad9d700 1 --2- 172.21.15.57:0/2970376090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f097c0b0880 0x7f097c06b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f097ad9d700 1 -- 172.21.15.57:0/2970376090 >> v1:172.21.15.181:6789/0 conn(0x7f097c072180 legacy=0x7f097c0aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f097ad9d700 1 -- 172.21.15.57:0/2970376090 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c0b4d00 con 0x7f097c0ad1f0 2023-11-28T12:05:37.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.095+0000 7f097a59c700 1 --2- 172.21.15.57:0/2970376090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f097c0b0880 0x7f097c06b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:05:37.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.096+0000 7f097ad9d700 1 --2- 172.21.15.57:0/2970376090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c0ad1f0 0x7f097c0ad670 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0974009e50 tx=0x7f097400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f96adc75c7d0ce0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:37.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.096+0000 7f097959a700 1 -- 172.21.15.57:0/2970376090 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097400e070 con 0x7f097c0ad1f0 2023-11-28T12:05:37.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.096+0000 7f097959a700 1 -- 172.21.15.57:0/2970376090 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0974003a20 con 0x7f097c0ad1f0 2023-11-28T12:05:37.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.096+0000 7f097959a700 1 -- 172.21.15.57:0/2970376090 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974003dd0 con 0x7f097c0ad1f0 2023-11-28T12:05:37.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.096+0000 7f0980c7b700 1 -- 172.21.15.57:0/2970376090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c0ad1f0 msgr2=0x7f097c0ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.096+0000 7f0980c7b700 1 --2- 172.21.15.57:0/2970376090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c0ad1f0 0x7f097c0ad670 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0974009e50 tx=0x7f097400b330 comp rx=0 tx=0).stop 2023-11-28T12:05:37.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.096+0000 7f0980c7b700 1 -- 172.21.15.57:0/2970376090 shutdown_connections 2023-11-28T12:05:37.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.096+0000 7f0980c7b700 1 --2- 172.21.15.57:0/2970376090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c0ad1f0 0x7f097c0ad670 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.096+0000 7f0980c7b700 1 --2- 172.21.15.57:0/2970376090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f097c0b0880 0x7f097c06b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.096+0000 7f0980c7b700 1 -- 172.21.15.57:0/2970376090 >> 172.21.15.57:0/2970376090 conn(0x7f097c0b2fc0 msgr2=0x7f097c071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:37.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.096+0000 7f0980c7b700 1 -- 172.21.15.57:0/2970376090 shutdown_connections 2023-11-28T12:05:37.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.096+0000 7f0980c7b700 1 -- 172.21.15.57:0/2970376090 wait complete. 2023-11-28T12:05:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.098+0000 7f0980c7b700 1 Processor -- start 2023-11-28T12:05:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.098+0000 7f0980c7b700 1 -- start start 2023-11-28T12:05:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f0980c7b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c072180 0x7f097c1f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f0980c7b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f097c0ad1f0 0x7f097c15f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f0980c7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f097c0b0880 0x7f097c15f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f0980c7b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c15ec80 con 0x7f097c072180 2023-11-28T12:05:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f0980c7b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c0c6aa0 con 0x7f097c0b0880 2023-11-28T12:05:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f0980c7b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097c158760 con 0x7f097c0ad1f0 2023-11-28T12:05:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f097a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c072180 0x7f097c1f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f097a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c072180 0x7f097c1f1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38010/0 (socket says 172.21.15.57:38010) 2023-11-28T12:05:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f097a59c700 1 -- 172.21.15.57:0/1030857527 learned_addr learned my addr 172.21.15.57:0/1030857527 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f097ad9d700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f097c0b0880 0x7f097c15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f0979d9b700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f097c0ad1f0 0x7f097c15f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f097a59c700 1 -- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f097c0ad1f0 msgr2=0x7f097c15f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f097a59c700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f097c0ad1f0 0x7f097c15f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f097a59c700 1 -- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f097c0b0880 msgr2=0x7f097c15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f097a59c700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f097c0b0880 0x7f097c15f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f097a59c700 1 -- 172.21.15.57:0/1030857527 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0974009b30 con 0x7f097c072180 2023-11-28T12:05:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f097a59c700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c072180 0x7f097c1f1fc0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f096c00c750 tx=0x7f096c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f096affd700 1 -- 172.21.15.57:0/1030857527 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c011800 con 0x7f097c072180 2023-11-28T12:05:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f096affd700 1 -- 172.21.15.57:0/1030857527 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096c011960 con 0x7f097c072180 2023-11-28T12:05:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.099+0000 7f0980c7b700 1 -- 172.21.15.57:0/1030857527 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097c0725f0 con 0x7f097c072180 2023-11-28T12:05:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.100+0000 7f096affd700 1 -- 172.21.15.57:0/1030857527 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c011cd0 con 0x7f097c072180 2023-11-28T12:05:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.100+0000 7f0980c7b700 1 -- 172.21.15.57:0/1030857527 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097c0b4fe0 con 0x7f097c072180 2023-11-28T12:05:37.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.101+0000 7f0980c7b700 1 -- 172.21.15.57:0/1030857527 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097c0b49b0 con 0x7f097c072180 2023-11-28T12:05:37.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.101+0000 7f096affd700 1 -- 172.21.15.57:0/1030857527 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f096c020070 con 0x7f097c072180 2023-11-28T12:05:37.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.101+0000 7f096affd700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f095c0682f0 0x7f095c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.102+0000 7f096affd700 1 -- 172.21.15.57:0/1030857527 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f096c085a80 con 0x7f097c072180 2023-11-28T12:05:37.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.102+0000 7f0979d9b700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f095c0682f0 0x7f095c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.102+0000 7f0979d9b700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f095c0682f0 0x7f095c06a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f097c0661c0 tx=0x7f09700090e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:37.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.106+0000 7f096affd700 1 -- 172.21.15.57:0/1030857527 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f096c052260 con 0x7f097c072180 2023-11-28T12:05:37.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.352+0000 7f0980c7b700 1 -- 172.21.15.57:0/1030857527 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f097c0743f0 con 0x7f097c072180 2023-11-28T12:05:37.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.353+0000 7f096affd700 1 -- 172.21.15.57:0/1030857527 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4652 (secure 0 0 0) 0x7f096c055880 con 0x7f097c072180 2023-11-28T12:05:37.354 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:05:37.354 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2317025916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2317025916},{"type":"v1","addr":"172.21.15.181:6836","nonce":2317025916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3486085199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3486085199},{"type":"v1","addr":"172.21.15.181:6837","nonce":3486085199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/4272259116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":4272259116},{"type":"v1","addr":"172.21.15.57:6834","nonce":4272259116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:04:52.360416+0000","modified":"2023-11-28T12:05:08.579934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.57:6835/685888843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":685888843},{"type":"v1","addr":"172.21.15.57:6835","nonce":685888843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T12:05:37.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.355+0000 7f0980c7b700 1 -- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f095c0682f0 msgr2=0x7f095c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.355+0000 7f0980c7b700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f095c0682f0 0x7f095c06a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f097c0661c0 tx=0x7f09700090e0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.355+0000 7f0980c7b700 1 -- 172.21.15.57:0/1030857527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c072180 msgr2=0x7f097c1f1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.355+0000 7f0980c7b700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c072180 0x7f097c1f1fc0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f096c00c750 tx=0x7f096c00ca60 comp rx=0 tx=0).stop 2023-11-28T12:05:37.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.355+0000 7f0980c7b700 1 -- 172.21.15.57:0/1030857527 shutdown_connections 2023-11-28T12:05:37.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.355+0000 7f0980c7b700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f095c0682f0 0x7f095c06a7a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.355+0000 7f0980c7b700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f097c0b0880 0x7f097c15f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.355+0000 7f0980c7b700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c072180 0x7f097c1f1fc0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.355+0000 7f0980c7b700 1 --2- 172.21.15.57:0/1030857527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f097c0ad1f0 0x7f097c15f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.355+0000 7f0980c7b700 1 -- 172.21.15.57:0/1030857527 >> 172.21.15.57:0/1030857527 conn(0x7f097c0b2fc0 msgr2=0x7f097c15c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:37.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.355+0000 7f0980c7b700 1 -- 172.21.15.57:0/1030857527 shutdown_connections 2023-11-28T12:05:37.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.355+0000 7f0980c7b700 1 -- 172.21.15.57:0/1030857527 wait complete. 2023-11-28T12:05:37.358 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2023-11-28T12:05:37.369 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2023-11-28T12:05:37.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.505+0000 7f08f2a5e700 1 Processor -- start 2023-11-28T12:05:37.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.505+0000 7f08f2a5e700 1 -- start start 2023-11-28T12:05:37.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.506+0000 7f08f2a5e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f08ec160cb0 0x7f08ec06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.506+0000 7f08f2a5e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08ec06b050 0x7f08ec0d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.506+0000 7f08f2a5e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec0d2d30 con 0x7f08ec15c8c0 2023-11-28T12:05:37.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.506+0000 7f08f2a5e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec06b4d0 con 0x7f08ec160cb0 2023-11-28T12:05:37.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.506+0000 7f08f2a5e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0d0b40 con 0x7f08ec06b050 2023-11-28T12:05:37.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.506+0000 7f08eb7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f08ec160cb0 0x7f08ec06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.506+0000 7f08ebfff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f08ec15c8c0 0x7f08ec0b5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46858/0 (socket says 172.21.15.57:46858) 2023-11-28T12:05:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.506+0000 7f08ebfff700 1 -- 172.21.15.57:0/10314198 learned_addr learned my addr 172.21.15.57:0/10314198 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:05:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.506+0000 7f08f0ffb700 1 --2- 172.21.15.57:0/10314198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08ec06b050 0x7f08ec0d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.507+0000 7f08eb7fe700 1 -- 172.21.15.57:0/10314198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08ec06b050 msgr2=0x7f08ec0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.507+0000 7f08eb7fe700 1 --2- 172.21.15.57:0/10314198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08ec06b050 0x7f08ec0d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.507+0000 7f08eb7fe700 1 -- 172.21.15.57:0/10314198 >> v1:172.21.15.57:6789/0 conn(0x7f08ec15c8c0 legacy=0x7f08ec0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.507+0000 7f08eb7fe700 1 -- 172.21.15.57:0/10314198 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec0d2a50 con 0x7f08ec160cb0 2023-11-28T12:05:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.507+0000 7f08f0ffb700 1 --2- 172.21.15.57:0/10314198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08ec06b050 0x7f08ec0d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.507+0000 7f08eb7fe700 1 --2- 172.21.15.57:0/10314198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ec160cb0 0x7f08ec06ab10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009840 tx=0x7f08e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26787a8cc5c1a2a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.507+0000 7f08eaffd700 1 -- 172.21.15.57:0/10314198 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0011830 con 0x7f08ec160cb0 2023-11-28T12:05:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.507+0000 7f08eaffd700 1 -- 172.21.15.57:0/10314198 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e0011990 con 0x7f08ec160cb0 2023-11-28T12:05:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.507+0000 7f08eaffd700 1 -- 172.21.15.57:0/10314198 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0011d20 con 0x7f08ec160cb0 2023-11-28T12:05:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.508+0000 7f08f2a5e700 1 -- 172.21.15.57:0/10314198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ec160cb0 msgr2=0x7f08ec06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.508+0000 7f08f2a5e700 1 --2- 172.21.15.57:0/10314198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ec160cb0 0x7f08ec06ab10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009840 tx=0x7f08e0009b50 comp rx=0 tx=0).stop 2023-11-28T12:05:37.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.508+0000 7f08f2a5e700 1 -- 172.21.15.57:0/10314198 shutdown_connections 2023-11-28T12:05:37.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.508+0000 7f08f2a5e700 1 --2- 172.21.15.57:0/10314198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08ec06b050 0x7f08ec0d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.508+0000 7f08f2a5e700 1 --2- 172.21.15.57:0/10314198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ec160cb0 0x7f08ec06ab10 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.508+0000 7f08f2a5e700 1 -- 172.21.15.57:0/10314198 >> 172.21.15.57:0/10314198 conn(0x7f08ec06c520 msgr2=0x7f08ec06c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:37.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.508+0000 7f08f2a5e700 1 -- 172.21.15.57:0/10314198 shutdown_connections 2023-11-28T12:05:37.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.508+0000 7f08f2a5e700 1 -- 172.21.15.57:0/10314198 wait complete. 2023-11-28T12:05:37.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.509+0000 7f08f2a5e700 1 Processor -- start 2023-11-28T12:05:37.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.509+0000 7f08f2a5e700 1 -- start start 2023-11-28T12:05:37.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08f2a5e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec06b050 0x7f08ec14df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08f2a5e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08ec15c8c0 0x7f08ec0ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08f2a5e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ec160cb0 0x7f08ec0b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08f2a5e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec14ee10 con 0x7f08ec160cb0 2023-11-28T12:05:37.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08f2a5e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0caed0 con 0x7f08ec06b050 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08f2a5e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec14e570 con 0x7f08ec15c8c0 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08f0ffb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ec160cb0 0x7f08ec0b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08f0ffb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ec160cb0 0x7f08ec0b7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38030/0 (socket says 172.21.15.57:38030) 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08f0ffb700 1 -- 172.21.15.57:0/316470477 learned_addr learned my addr 172.21.15.57:0/316470477 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08eb7fe700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08ec15c8c0 0x7f08ec0ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08ebfff700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec06b050 0x7f08ec14df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08f0ffb700 1 -- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08ec15c8c0 msgr2=0x7f08ec0ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08f0ffb700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08ec15c8c0 0x7f08ec0ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08f0ffb700 1 -- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec06b050 msgr2=0x7f08ec14df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.510+0000 7f08f0ffb700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec06b050 0x7f08ec14df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.511+0000 7f08f0ffb700 1 -- 172.21.15.57:0/316470477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e00094a0 con 0x7f08ec160cb0 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.511+0000 7f08eb7fe700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08ec15c8c0 0x7f08ec0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.511+0000 7f08f0ffb700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ec160cb0 0x7f08ec0b7cd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f08e4007df0 tx=0x7f08e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:37.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.511+0000 7f08e8ff9700 1 -- 172.21.15.57:0/316470477 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400ddd0 con 0x7f08ec160cb0 2023-11-28T12:05:37.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.511+0000 7f08e8ff9700 1 -- 172.21.15.57:0/316470477 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e400f040 con 0x7f08ec160cb0 2023-11-28T12:05:37.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.511+0000 7f08f2a5e700 1 -- 172.21.15.57:0/316470477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec0cda40 con 0x7f08ec160cb0 2023-11-28T12:05:37.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.511+0000 7f08e8ff9700 1 -- 172.21.15.57:0/316470477 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4003c60 con 0x7f08ec160cb0 2023-11-28T12:05:37.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.511+0000 7f08f2a5e700 1 -- 172.21.15.57:0/316470477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec0cd760 con 0x7f08ec160cb0 2023-11-28T12:05:37.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.512+0000 7f08e8ff9700 1 -- 172.21.15.57:0/316470477 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f08e401c030 con 0x7f08ec160cb0 2023-11-28T12:05:37.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.513+0000 7f08e8ff9700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f08d80682f0 0x7f08d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.513+0000 7f08e8ff9700 1 -- 172.21.15.57:0/316470477 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f08e4085e90 con 0x7f08ec160cb0 2023-11-28T12:05:37.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.513+0000 7f08ebfff700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f08d80682f0 0x7f08d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.514+0000 7f08ebfff700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f08d80682f0 0x7f08d806a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f08dc005910 tx=0x7f08dc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:37.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.515+0000 7f08febfdb80 1 Processor -- start 2023-11-28T12:05:37.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.515+0000 7f08febfdb80 1 -- start start 2023-11-28T12:05:37.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08febfdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f9924ff0 0x5647f98dea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08febfdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f991bb60 0x5647f99253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08febfdb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9907700 0x5647f98dfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08febfdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f985ce80 con 0x5647f991bb60 2023-11-28T12:05:37.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08febfdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f97c91e0 con 0x5647f9924ff0 2023-11-28T12:05:37.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08febfdb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647f9837e50 con 0x5647f9907700 2023-11-28T12:05:37.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08eb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f991bb60 0x5647f99253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08eb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f991bb60 0x5647f99253c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38032/0 (socket says 172.21.15.57:38032) 2023-11-28T12:05:37.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08eb7fe700 1 -- 172.21.15.57:0/2927350845 learned_addr learned my addr 172.21.15.57:0/2927350845 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:37.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08ebfff700 1 --2- 172.21.15.57:0/2927350845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f9924ff0 0x5647f98dea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08f0ffb700 1 --2- 172.21.15.57:0/2927350845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9907700 0x5647f98dfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08eb7fe700 1 -- 172.21.15.57:0/2927350845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9907700 msgr2=0x5647f98dfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08eb7fe700 1 --2- 172.21.15.57:0/2927350845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9907700 0x5647f98dfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08eb7fe700 1 -- 172.21.15.57:0/2927350845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f9924ff0 msgr2=0x5647f98dea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.516+0000 7f08eb7fe700 1 --2- 172.21.15.57:0/2927350845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f9924ff0 0x5647f98dea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.517+0000 7f08eb7fe700 1 -- 172.21.15.57:0/2927350845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647f9625830 con 0x5647f991bb60 2023-11-28T12:05:37.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.517+0000 7f08eb7fe700 1 --2- 172.21.15.57:0/2927350845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f991bb60 0x5647f99253c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0d0af0 tx=0x7f08e0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b964632d75c7bf02 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:37.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.517+0000 7f08d1ffb700 1 -- 172.21.15.57:0/2927350845 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0020070 con 0x5647f991bb60 2023-11-28T12:05:37.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.517+0000 7f08d1ffb700 1 -- 172.21.15.57:0/2927350845 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e00117d0 con 0x5647f991bb60 2023-11-28T12:05:37.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.517+0000 7f08d1ffb700 1 -- 172.21.15.57:0/2927350845 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0011d20 con 0x5647f991bb60 2023-11-28T12:05:37.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.517+0000 7f08febfdb80 1 -- 172.21.15.57:0/2927350845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f991bb60 msgr2=0x5647f99253c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.517+0000 7f08febfdb80 1 --2- 172.21.15.57:0/2927350845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f991bb60 0x5647f99253c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0d0af0 tx=0x7f08e0010000 comp rx=0 tx=0).stop 2023-11-28T12:05:37.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.517+0000 7f08febfdb80 1 -- 172.21.15.57:0/2927350845 shutdown_connections 2023-11-28T12:05:37.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.517+0000 7f08febfdb80 1 --2- 172.21.15.57:0/2927350845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9907700 0x5647f98dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.517+0000 7f08febfdb80 1 --2- 172.21.15.57:0/2927350845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f9924ff0 0x5647f98dea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.518+0000 7f08febfdb80 1 --2- 172.21.15.57:0/2927350845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f991bb60 0x5647f99253c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.518+0000 7f08febfdb80 1 -- 172.21.15.57:0/2927350845 >> 172.21.15.57:0/2927350845 conn(0x5647f98d4dc0 msgr2=0x5647f98ddfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:37.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.518+0000 7f08febfdb80 1 -- 172.21.15.57:0/2927350845 shutdown_connections 2023-11-28T12:05:37.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.518+0000 7f08febfdb80 1 -- 172.21.15.57:0/2927350845 wait complete. 2023-11-28T12:05:37.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.518+0000 7f08febfdb80 1 -- start start 2023-11-28T12:05:37.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.518+0000 7f08febfdb80 1 Processor -- start 2023-11-28T12:05:37.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.519+0000 7f08febfdb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9907700 0x5647f991d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.519+0000 7f08febfdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 0x5647f990d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.519+0000 7f08febfdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9924ff0 0x5647f9923f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.519+0000 7f08febfdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f98311c0 con 0x5647f9924ff0 2023-11-28T12:05:37.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.519+0000 7f08febfdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f9626430 con 0x5647f991bb60 2023-11-28T12:05:37.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.519+0000 7f08febfdb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647f97c93e0 con 0x5647f9907700 2023-11-28T12:05:37.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.519+0000 7f08febfdb80 10 client.0 _ll_register_callbacks cb 0x5647f97cf4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:05:37.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.519+0000 7f08f0ffb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9924ff0 0x5647f9923f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.519+0000 7f08f0ffb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9924ff0 0x5647f9923f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38036/0 (socket says 172.21.15.57:38036) 2023-11-28T12:05:37.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.519+0000 7f08f0ffb700 1 -- 172.21.15.57:0/2844218864 learned_addr learned my addr 172.21.15.57:0/2844218864 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:37.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08ebfff700 1 --2- 172.21.15.57:0/2844218864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9907700 0x5647f991d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08eb7fe700 1 --2- 172.21.15.57:0/2844218864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 0x5647f990d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08f0ffb700 1 -- 172.21.15.57:0/2844218864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9907700 msgr2=0x5647f991d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08f0ffb700 1 --2- 172.21.15.57:0/2844218864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9907700 0x5647f991d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08f0ffb700 1 -- 172.21.15.57:0/2844218864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 msgr2=0x5647f990d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08f0ffb700 1 --2- 172.21.15.57:0/2844218864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 0x5647f990d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08f0ffb700 1 -- 172.21.15.57:0/2844218864 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec0d2a50 con 0x5647f9924ff0 2023-11-28T12:05:37.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08f0ffb700 1 --2- 172.21.15.57:0/2844218864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9924ff0 0x5647f9923f00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f08e4055f50 tx=0x7f08e4055a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526cdc570839914b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:37.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08caffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:05:37.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08caffd700 1 -- 172.21.15.57:0/2844218864 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4055200 con 0x5647f9924ff0 2023-11-28T12:05:37.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08caffd700 1 -- 172.21.15.57:0/2844218864 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f08e4055360 con 0x5647f9924ff0 2023-11-28T12:05:37.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08febfdb80 1 -- 172.21.15.57:0/2844218864 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647f98350d0 con 0x5647f9924ff0 2023-11-28T12:05:37.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08caffd700 1 -- 172.21.15.57:0/2844218864 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08e4055520 con 0x5647f9924ff0 2023-11-28T12:05:37.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08febfdb80 10 client.6495 fetch_fsmap learned FSMap version 33 2023-11-28T12:05:37.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08febfdb80 1 -- 172.21.15.57:0/2844218864 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5647f9712140 con 0x5647f9924ff0 2023-11-28T12:05:37.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.520+0000 7f08caffd700 1 -- 172.21.15.57:0/2844218864 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e4062020 con 0x5647f9924ff0 2023-11-28T12:05:37.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.521+0000 7f08caffd700 1 -- 172.21.15.57:0/2844218864 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f08e4069a70 con 0x5647f9924ff0 2023-11-28T12:05:37.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.521+0000 7f08febfdb80 10 client.6495 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T12:05:37.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.521+0000 7f08febfdb80 10 client.6495 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T12:05:37.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.521+0000 7f08febfdb80 20 client.6495 populate_metadata read hostname 'smithi057' 2023-11-28T12:05:37.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.521+0000 7f08febfdb80 1 --2- 172.21.15.57:0/2844218864 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5647f98f70a0 0x5647f98f7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.521+0000 7f08febfdb80 4 client.6495 mds_command: new command op to 6219 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T12:05:37.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.521+0000 7f08febfdb80 1 -- 172.21.15.57:0/2844218864 --> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647f9680f90 con 0x5647f98f70a0 2023-11-28T12:05:37.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.521+0000 7f08ebfff700 1 --2- 172.21.15.57:0/2844218864 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5647f98f70a0 0x5647f98f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.521+0000 7f08caffd700 10 client.6495 ms_handle_connect on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:37.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.521+0000 7f08ebfff700 1 --2- 172.21.15.57:0/2844218864 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5647f98f70a0 0x5647f98f7470 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6954cd90d7f0062 server_cookie=3aa326f8f01b6e6c in_seq=0 out_seq=0 2023-11-28T12:05:37.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08caffd700 1 -- 172.21.15.57:0/2844218864 <== mds.0 v2:172.21.15.57:6833/685888843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5647f9680f90 con 0x5647f98f70a0 2023-11-28T12:05:37.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08caffd700 10 client.6495 handle_command_reply: tid=0 2023-11-28T12:05:37.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08febfdb80 1 client.6495 shutdown 2023-11-28T12:05:37.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08febfdb80 1 -- 172.21.15.57:0/2844218864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9924ff0 msgr2=0x5647f9923f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08febfdb80 1 --2- 172.21.15.57:0/2844218864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9924ff0 0x5647f9923f00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f08e4055f50 tx=0x7f08e4055a90 comp rx=0 tx=0).stop 2023-11-28T12:05:37.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08febfdb80 1 -- 172.21.15.57:0/2844218864 shutdown_connections 2023-11-28T12:05:37.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08febfdb80 1 --2- 172.21.15.57:0/2844218864 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5647f98f70a0 0x5647f98f7470 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08febfdb80 1 --2- 172.21.15.57:0/2844218864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9924ff0 0x5647f9923f00 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08febfdb80 1 --2- 172.21.15.57:0/2844218864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9907700 0x5647f991d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08caffd700 0 client.6495 ms_handle_reset on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:37.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08febfdb80 1 --2- 172.21.15.57:0/2844218864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 0x5647f990d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08febfdb80 1 -- 172.21.15.57:0/2844218864 >> 172.21.15.57:0/2844218864 conn(0x5647f98d4dc0 msgr2=0x5647f9906dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:37.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08febfdb80 1 -- 172.21.15.57:0/2844218864 shutdown_connections 2023-11-28T12:05:37.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08febfdb80 1 -- 172.21.15.57:0/2844218864 wait complete. 2023-11-28T12:05:37.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.522+0000 7f08febfdb80 20 client.6495 trim_cache size 0 max 16384 2023-11-28T12:05:37.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.535+0000 7f08febfdb80 1 Processor -- start 2023-11-28T12:05:37.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.536+0000 7f08febfdb80 1 -- start start 2023-11-28T12:05:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.536+0000 7f08febfdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 0x5647f9922b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.536+0000 7f08febfdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 0x5647f9923040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.536+0000 7f08febfdb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9924ff0 0x5647f98d9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.536+0000 7f08febfdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f98e1660 con 0x5647f9907700 2023-11-28T12:05:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.536+0000 7f08febfdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f9933790 con 0x5647f991bb60 2023-11-28T12:05:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.536+0000 7f08febfdb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647f98dd790 con 0x5647f9924ff0 2023-11-28T12:05:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.536+0000 7f08ebfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 0x5647f9922b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.536+0000 7f08ebfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 0x5647f9922b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38038/0 (socket says 172.21.15.57:38038) 2023-11-28T12:05:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.536+0000 7f08ebfff700 1 -- 172.21.15.57:0/1510376287 learned_addr learned my addr 172.21.15.57:0/1510376287 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.537+0000 7f08f0ffb700 1 --2- 172.21.15.57:0/1510376287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9924ff0 0x5647f98d9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.537+0000 7f08eb7fe700 1 --2- 172.21.15.57:0/1510376287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 0x5647f9923040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.537+0000 7f08ebfff700 1 -- 172.21.15.57:0/1510376287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9924ff0 msgr2=0x5647f98d9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.537+0000 7f08ebfff700 1 --2- 172.21.15.57:0/1510376287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9924ff0 0x5647f98d9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.537+0000 7f08ebfff700 1 -- 172.21.15.57:0/1510376287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 msgr2=0x5647f9923040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.537+0000 7f08ebfff700 1 --2- 172.21.15.57:0/1510376287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 0x5647f9923040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.537+0000 7f08ebfff700 1 -- 172.21.15.57:0/1510376287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647f97f8380 con 0x5647f9907700 2023-11-28T12:05:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.537+0000 7f08ebfff700 1 --2- 172.21.15.57:0/1510376287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 0x5647f9922b00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f08dc007fd0 tx=0x7f08dc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcd88be4a656ffa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.537+0000 7f08d27fc700 1 -- 172.21.15.57:0/1510376287 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0261a0 con 0x5647f9907700 2023-11-28T12:05:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.537+0000 7f08d27fc700 1 -- 172.21.15.57:0/1510376287 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc026300 con 0x5647f9907700 2023-11-28T12:05:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.537+0000 7f08d27fc700 1 -- 172.21.15.57:0/1510376287 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc026470 con 0x5647f9907700 2023-11-28T12:05:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.538+0000 7f08febfdb80 1 -- 172.21.15.57:0/1510376287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 msgr2=0x5647f9922b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.538+0000 7f08febfdb80 1 --2- 172.21.15.57:0/1510376287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 0x5647f9922b00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f08dc007fd0 tx=0x7f08dc012000 comp rx=0 tx=0).stop 2023-11-28T12:05:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.538+0000 7f08febfdb80 1 -- 172.21.15.57:0/1510376287 shutdown_connections 2023-11-28T12:05:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.538+0000 7f08febfdb80 1 --2- 172.21.15.57:0/1510376287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9924ff0 0x5647f98d9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.538+0000 7f08febfdb80 1 --2- 172.21.15.57:0/1510376287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 0x5647f9922b00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.538+0000 7f08febfdb80 1 --2- 172.21.15.57:0/1510376287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 0x5647f9923040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.538+0000 7f08febfdb80 1 -- 172.21.15.57:0/1510376287 >> 172.21.15.57:0/1510376287 conn(0x5647f98d4dc0 msgr2=0x5647f9906dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.538+0000 7f08febfdb80 1 -- 172.21.15.57:0/1510376287 shutdown_connections 2023-11-28T12:05:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.538+0000 7f08febfdb80 1 -- 172.21.15.57:0/1510376287 wait complete. 2023-11-28T12:05:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.538+0000 7f08febfdb80 1 -- start start 2023-11-28T12:05:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.538+0000 7f08febfdb80 1 Processor -- start 2023-11-28T12:05:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.539+0000 7f08febfdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 0x5647f9989b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.539+0000 7f08febfdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 0x5647f991e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.539+0000 7f08febfdb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9924ff0 0x5647f991e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.539+0000 7f08febfdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f998ae90 con 0x5647f9907700 2023-11-28T12:05:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.539+0000 7f08febfdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f998a080 con 0x5647f991bb60 2023-11-28T12:05:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.539+0000 7f08febfdb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647f98daaa0 con 0x5647f9924ff0 2023-11-28T12:05:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.539+0000 7f08febfdb80 10 client.0 _ll_register_callbacks cb 0x5647f97cf4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T12:05:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.539+0000 7f08ebfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 0x5647f9989b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.539+0000 7f08ebfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 0x5647f9989b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38044/0 (socket says 172.21.15.57:38044) 2023-11-28T12:05:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.539+0000 7f08ebfff700 1 -- 172.21.15.57:0/3142117186 learned_addr learned my addr 172.21.15.57:0/3142117186 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08eb7fe700 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 0x5647f991e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08ebfff700 1 -- 172.21.15.57:0/3142117186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9924ff0 msgr2=0x5647f991e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08f0ffb700 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9924ff0 0x5647f991e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08ebfff700 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9924ff0 0x5647f991e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08ebfff700 1 -- 172.21.15.57:0/3142117186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 msgr2=0x5647f991e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08ebfff700 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 0x5647f991e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08ebfff700 1 -- 172.21.15.57:0/3142117186 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc00f040 con 0x5647f9907700 2023-11-28T12:05:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08eb7fe700 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 0x5647f991e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:05:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08ebfff700 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 0x5647f9989b40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f08dc029040 tx=0x7f08dc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2174d2d4752525 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08caffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08caffd700 1 -- 172.21.15.57:0/3142117186 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc005be0 con 0x5647f9907700 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08caffd700 1 -- 172.21.15.57:0/3142117186 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f08dc024420 con 0x5647f9907700 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08febfdb80 1 -- 172.21.15.57:0/3142117186 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08e00117d0 con 0x5647f9907700 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08caffd700 1 -- 172.21.15.57:0/3142117186 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08dc0245e0 con 0x5647f9907700 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08febfdb80 10 client.6501 fetch_fsmap learned FSMap version 33 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08febfdb80 1 -- 172.21.15.57:0/3142117186 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5647f9686270 con 0x5647f9907700 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.540+0000 7f08caffd700 1 -- 172.21.15.57:0/3142117186 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e00117d0 con 0x5647f9907700 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08caffd700 1 -- 172.21.15.57:0/3142117186 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f08dc024990 con 0x5647f9907700 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08febfdb80 10 client.6501 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08febfdb80 10 client.6501 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08febfdb80 20 client.6501 populate_metadata read hostname 'smithi057' 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08febfdb80 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5647f998db90 0x5647f998df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08febfdb80 4 client.6501 mds_command: new command op to 6219 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-28T12:05:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08febfdb80 1 -- 172.21.15.57:0/3142117186 --> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5647f96b60e0 con 0x5647f998db90 2023-11-28T12:05:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08eb7fe700 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5647f998db90 0x5647f998df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08caffd700 10 client.6501 ms_handle_connect on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08eb7fe700 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5647f998db90 0x5647f998df60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8a688ed1838acb2 server_cookie=db80838f367be7a0 in_seq=0 out_seq=0 2023-11-28T12:05:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08caffd700 1 -- 172.21.15.57:0/3142117186 <== mds.0 v2:172.21.15.57:6833/685888843 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x5647f96b60e0 con 0x5647f998db90 2023-11-28T12:05:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08caffd700 10 client.6501 handle_command_reply: tid=0 2023-11-28T12:05:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08febfdb80 1 client.6501 shutdown 2023-11-28T12:05:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08febfdb80 1 -- 172.21.15.57:0/3142117186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 msgr2=0x5647f9989b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08febfdb80 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 0x5647f9989b40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f08dc029040 tx=0x7f08dc000ed0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08febfdb80 1 -- 172.21.15.57:0/3142117186 shutdown_connections 2023-11-28T12:05:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.541+0000 7f08febfdb80 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.57:6833/685888843,v1:172.21.15.57:6835/685888843] conn(0x5647f998db90 0x5647f998df60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08febfdb80 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5647f9924ff0 0x5647f991e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08febfdb80 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647f9907700 0x5647f9989b40 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08febfdb80 1 --2- 172.21.15.57:0/3142117186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5647f991bb60 0x5647f991e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08febfdb80 1 -- 172.21.15.57:0/3142117186 >> 172.21.15.57:0/3142117186 conn(0x5647f98d4dc0 msgr2=0x5647f9922b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:37.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08caffd700 0 client.6501 ms_handle_reset on v2:172.21.15.57:6833/685888843 2023-11-28T12:05:37.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08febfdb80 1 -- 172.21.15.57:0/3142117186 shutdown_connections 2023-11-28T12:05:37.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08febfdb80 1 -- 172.21.15.57:0/3142117186 wait complete. 2023-11-28T12:05:37.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08febfdb80 20 client.6501 trim_cache size 0 max 16384 2023-11-28T12:05:37.550 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:05:37.550 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2023-11-28T12:05:37.551 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-11-28T12:05:37.551 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-11-28T12:05:37.551 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1313936422, 2023-11-28T12:05:37.551 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511627777, 2023-11-28T12:05:37.551 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/dir_test/file" 2023-11-28T12:05:37.551 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-11-28T12:05:37.551 INFO:teuthology.orchestra.run.smithi057.stdout:] 2023-11-28T12:05:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08f2a5e700 1 -- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f08d80682f0 msgr2=0x7f08d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08f2a5e700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f08d80682f0 0x7f08d806a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f08dc005910 tx=0x7f08dc009040 comp rx=0 tx=0).stop 2023-11-28T12:05:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08f2a5e700 1 -- 172.21.15.57:0/316470477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ec160cb0 msgr2=0x7f08ec0b7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08f2a5e700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ec160cb0 0x7f08ec0b7cd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f08e4007df0 tx=0x7f08e400d3b0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08f2a5e700 1 -- 172.21.15.57:0/316470477 shutdown_connections 2023-11-28T12:05:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08f2a5e700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f08d80682f0 0x7f08d806a7a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08f2a5e700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08ec160cb0 0x7f08ec0b7cd0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08f2a5e700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec06b050 0x7f08ec14df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08f2a5e700 1 --2- 172.21.15.57:0/316470477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08ec15c8c0 0x7f08ec0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.542+0000 7f08f2a5e700 1 -- 172.21.15.57:0/316470477 >> 172.21.15.57:0/316470477 conn(0x7f08ec06c520 msgr2=0x7f08ec06c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:37.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.543+0000 7f08f2a5e700 1 -- 172.21.15.57:0/316470477 shutdown_connections 2023-11-28T12:05:37.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.543+0000 7f08f2a5e700 1 -- 172.21.15.57:0/316470477 wait complete. 2023-11-28T12:05:37.556 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:05:37.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.689+0000 7f90dc003700 1 Processor -- start 2023-11-28T12:05:37.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.689+0000 7f90dc003700 1 -- start start 2023-11-28T12:05:37.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.690+0000 7f90dc003700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d415e010 0x7f90d40ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.690+0000 7f90dc003700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d415af30 con 0x7f90d406e400 2023-11-28T12:05:37.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.690+0000 7f90dc003700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d415ef40 con 0x7f90d40d15f0 2023-11-28T12:05:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.690+0000 7f90dc003700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d415a9d0 con 0x7f90d415e010 2023-11-28T12:05:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.690+0000 7f90d9d9f700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f90d406e400 0x7f90d406e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46860/0 (socket says 172.21.15.57:46860) 2023-11-28T12:05:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.690+0000 7f90d9d9f700 1 -- 172.21.15.57:0/1162989556 learned_addr learned my addr 172.21.15.57:0/1162989556 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:05:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.690+0000 7f90da5a0700 1 --2- 172.21.15.57:0/1162989556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d415e010 0x7f90d40ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.690+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034315184 0 0) 0x7f90d415af30 con 0x7f90d406e400 2023-11-28T12:05:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.690+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90c00035b0 con 0x7f90d406e400 2023-11-28T12:05:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3028359133 0 0) 0x7f90c00035b0 con 0x7f90d406e400 2023-11-28T12:05:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d415e010 msgr2=0x7f90d40ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d8d9d700 1 --2- 172.21.15.57:0/1162989556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d415e010 0x7f90d40ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 >> v1:172.21.15.181:6789/0 conn(0x7f90d40d15f0 legacy=0x7f90d415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d415e980 con 0x7f90d406e400 2023-11-28T12:05:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f90c4001670 con 0x7f90d406e400 2023-11-28T12:05:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 >> v1:172.21.15.57:6789/0 conn(0x7f90d406e400 legacy=0x7f90d406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d8d9d700 1 --2- 172.21.15.57:0/1162989556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d415e010 0x7f90c0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d8d9d700 1 --2- 172.21.15.57:0/1162989556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90c0003fb0 0x7f90c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d8d9d700 1 --2- 172.21.15.57:0/1162989556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90c0004d90 0x7f90c0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c000bac0 con 0x7f90c0004d90 2023-11-28T12:05:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c000bc30 con 0x7f90c0003fb0 2023-11-28T12:05:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c000bdf0 con 0x7f90d415e010 2023-11-28T12:05:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d9d9f700 1 --2- 172.21.15.57:0/1162989556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d415e010 0x7f90c0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90da5a0700 1 --2- 172.21.15.57:0/1162989556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90c0004d90 0x7f90c0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d959e700 1 --2- 172.21.15.57:0/1162989556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90c0003fb0 0x7f90c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d9d9f700 1 -- 172.21.15.57:0/1162989556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90c0003fb0 msgr2=0x7f90c0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d9d9f700 1 --2- 172.21.15.57:0/1162989556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90c0003fb0 0x7f90c0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d9d9f700 1 -- 172.21.15.57:0/1162989556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90c0004d90 msgr2=0x7f90c0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d9d9f700 1 --2- 172.21.15.57:0/1162989556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90c0004d90 0x7f90c0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.691+0000 7f90d9d9f700 1 -- 172.21.15.57:0/1162989556 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c00035b0 con 0x7f90d415e010 2023-11-28T12:05:37.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.692+0000 7f90d9d9f700 1 --2- 172.21.15.57:0/1162989556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d415e010 0x7f90c0003a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f90c400c210 tx=0x7f90c0005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c80e1d8f0d41516 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:37.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.692+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4007c10 con 0x7f90d415e010 2023-11-28T12:05:37.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.692+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c400cdd0 con 0x7f90d415e010 2023-11-28T12:05:37.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.692+0000 7f90d8d9d700 1 -- 172.21.15.57:0/1162989556 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4005ba0 con 0x7f90d415e010 2023-11-28T12:05:37.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.692+0000 7f90dc003700 1 -- 172.21.15.57:0/1162989556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d415e010 msgr2=0x7f90c0003a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.692+0000 7f90dc003700 1 --2- 172.21.15.57:0/1162989556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d415e010 0x7f90c0003a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f90c400c210 tx=0x7f90c0005880 comp rx=0 tx=0).stop 2023-11-28T12:05:37.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.692+0000 7f90da5a0700 1 -- 172.21.15.57:0/1162989556 reap_dead start 2023-11-28T12:05:37.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.692+0000 7f90dc003700 1 -- 172.21.15.57:0/1162989556 shutdown_connections 2023-11-28T12:05:37.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.692+0000 7f90dc003700 1 -- 172.21.15.57:0/1162989556 >> 172.21.15.57:0/1162989556 conn(0x7f90d406a270 msgr2=0x7f90d406a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:37.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.692+0000 7f90dc003700 1 -- 172.21.15.57:0/1162989556 shutdown_connections 2023-11-28T12:05:37.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.692+0000 7f90dc003700 1 -- 172.21.15.57:0/1162989556 wait complete. 2023-11-28T12:05:37.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.694+0000 7f90dc003700 1 Processor -- start 2023-11-28T12:05:37.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.694+0000 7f90dc003700 1 -- start start 2023-11-28T12:05:37.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90dc003700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90d415e010 0x7f90d4159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90dc003700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d41e4540 0x7f90d414c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90dc003700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90d41e49b0 0x7f90d414c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90dc003700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d40cf340 con 0x7f90d415e010 2023-11-28T12:05:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90dc003700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d40cf620 con 0x7f90d41e49b0 2023-11-28T12:05:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90dc003700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d40cf900 con 0x7f90d41e4540 2023-11-28T12:05:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90d9d9f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90d415e010 0x7f90d4159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90d9d9f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90d415e010 0x7f90d4159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38052/0 (socket says 172.21.15.57:38052) 2023-11-28T12:05:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90d9d9f700 1 -- 172.21.15.57:0/3515874633 learned_addr learned my addr 172.21.15.57:0/3515874633 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90d959e700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d41e4540 0x7f90d414c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90da5a0700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90d41e49b0 0x7f90d414c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90d9d9f700 1 -- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d41e4540 msgr2=0x7f90d414c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90d9d9f700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d41e4540 0x7f90d414c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90d9d9f700 1 -- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90d41e49b0 msgr2=0x7f90d414c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90d9d9f700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90d41e49b0 0x7f90d414c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90d9d9f700 1 -- 172.21.15.57:0/3515874633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d415abc0 con 0x7f90d415e010 2023-11-28T12:05:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90d959e700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d41e4540 0x7f90d414c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:05:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90da5a0700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90d41e49b0 0x7f90d414c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:05:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90d9d9f700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90d415e010 0x7f90d4159ad0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f90c400c800 tx=0x7f90c4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90ca7fc700 1 -- 172.21.15.57:0/3515874633 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4004c80 con 0x7f90d415e010 2023-11-28T12:05:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90ca7fc700 1 -- 172.21.15.57:0/3515874633 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c4004e00 con 0x7f90d415e010 2023-11-28T12:05:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90dc003700 1 -- 172.21.15.57:0/3515874633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d415ec60 con 0x7f90d415e010 2023-11-28T12:05:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90ca7fc700 1 -- 172.21.15.57:0/3515874633 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4007310 con 0x7f90d415e010 2023-11-28T12:05:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.695+0000 7f90dc003700 1 -- 172.21.15.57:0/3515874633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d415e4b0 con 0x7f90d415e010 2023-11-28T12:05:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.696+0000 7f90dc003700 1 -- 172.21.15.57:0/3515874633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d40743f0 con 0x7f90d415e010 2023-11-28T12:05:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.697+0000 7f90ca7fc700 1 -- 172.21.15.57:0/3515874633 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f90c4010070 con 0x7f90d415e010 2023-11-28T12:05:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.697+0000 7f90ca7fc700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f90b8068320 0x7f90b806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.697+0000 7f90ca7fc700 1 -- 172.21.15.57:0/3515874633 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f90c4086c40 con 0x7f90d415e010 2023-11-28T12:05:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.697+0000 7f90d959e700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f90b8068320 0x7f90b806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.698+0000 7f90d959e700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f90b8068320 0x7f90b806a7d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f90d415d040 tx=0x7f90d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.702+0000 7f90ca7fc700 1 -- 172.21.15.57:0/3515874633 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f90c40b85e0 con 0x7f90d415e010 2023-11-28T12:05:37.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.945+0000 7f90dc003700 1 -- 172.21.15.57:0/3515874633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f90d406dcd0 con 0x7f90d415e010 2023-11-28T12:05:37.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.946+0000 7f90ca7fc700 1 -- 172.21.15.57:0/3515874633 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f90c4053420 con 0x7f90d415e010 2023-11-28T12:05:37.947 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:05:37.947 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:05:37.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.948+0000 7f90dc003700 1 -- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f90b8068320 msgr2=0x7f90b806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.948+0000 7f90dc003700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f90b8068320 0x7f90b806a7d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f90d415d040 tx=0x7f90d0009040 comp rx=0 tx=0).stop 2023-11-28T12:05:37.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.948+0000 7f90dc003700 1 -- 172.21.15.57:0/3515874633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90d415e010 msgr2=0x7f90d4159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:37.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.948+0000 7f90dc003700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90d415e010 0x7f90d4159ad0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f90c400c800 tx=0x7f90c4001a00 comp rx=0 tx=0).stop 2023-11-28T12:05:37.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.948+0000 7f90dc003700 1 -- 172.21.15.57:0/3515874633 shutdown_connections 2023-11-28T12:05:37.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.948+0000 7f90dc003700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f90b8068320 0x7f90b806a7d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.948+0000 7f90dc003700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90d41e49b0 0x7f90d414c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.948+0000 7f90dc003700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90d415e010 0x7f90d4159ad0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.948+0000 7f90dc003700 1 --2- 172.21.15.57:0/3515874633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90d41e4540 0x7f90d414c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:37.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.948+0000 7f90dc003700 1 -- 172.21.15.57:0/3515874633 >> 172.21.15.57:0/3515874633 conn(0x7f90d406a270 msgr2=0x7f90d40cd990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:37.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.948+0000 7f90dc003700 1 -- 172.21.15.57:0/3515874633 shutdown_connections 2023-11-28T12:05:37.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:37.948+0000 7f90dc003700 1 -- 172.21.15.57:0/3515874633 wait complete. 2023-11-28T12:05:37.962 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:05:42.963 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:05:43.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.104+0000 7fc8e4934700 1 Processor -- start 2023-11-28T12:05:43.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.104+0000 7fc8e4934700 1 -- start start 2023-11-28T12:05:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.105+0000 7fc8e4934700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc8e006e400 0x7fc8e006e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.105+0000 7fc8e4934700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e015af30 con 0x7fc8e015e010 2023-11-28T12:05:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.105+0000 7fc8e4934700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e015ef40 con 0x7fc8e00d15f0 2023-11-28T12:05:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.105+0000 7fc8e4934700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e015a9d0 con 0x7fc8e006e400 2023-11-28T12:05:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.106+0000 7fc8ded9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc8e015e010 0x7fc8e00ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46868/0 (socket says 172.21.15.57:46868) 2023-11-28T12:05:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.106+0000 7fc8ded9d700 1 -- 172.21.15.57:0/2892636702 learned_addr learned my addr 172.21.15.57:0/2892636702 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:05:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.106+0000 7fc8de59c700 1 --2- 172.21.15.57:0/2892636702 >> v2:172.21.15.181:3300/0 conn(0x7fc8e006e400 0x7fc8e006e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.106+0000 7fc8dd59a700 1 -- 172.21.15.57:0/2892636702 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994190510 0 0) 0x7fc8e015af30 con 0x7fc8e015e010 2023-11-28T12:05:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.106+0000 7fc8dd59a700 1 -- 172.21.15.57:0/2892636702 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8c80035b0 con 0x7fc8e015e010 2023-11-28T12:05:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.106+0000 7fc8de59c700 1 -- 172.21.15.57:0/2892636702 >> v1:172.21.15.181:6789/0 conn(0x7fc8e00d15f0 legacy=0x7fc8e015b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:43.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.106+0000 7fc8de59c700 1 -- 172.21.15.57:0/2892636702 >> v1:172.21.15.57:6789/0 conn(0x7fc8e015e010 legacy=0x7fc8e00ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:43.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.106+0000 7fc8dd59a700 1 -- 172.21.15.57:0/2892636702 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226626734 0 0) 0x7fc8e015ef40 con 0x7fc8e00d15f0 2023-11-28T12:05:43.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.106+0000 7fc8de59c700 1 -- 172.21.15.57:0/2892636702 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e006ed10 con 0x7fc8e006e400 2023-11-28T12:05:43.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.106+0000 7fc8de59c700 1 --2- 172.21.15.57:0/2892636702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8e006e400 0x7fc8e006e7d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009840 tx=0x7fc8d0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d173d5ac51d670b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:43.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.107+0000 7fc8dd59a700 1 -- 172.21.15.57:0/2892636702 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d80019f0 con 0x7fc8e006e400 2023-11-28T12:05:43.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.107+0000 7fc8dd59a700 1 -- 172.21.15.57:0/2892636702 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d0004030 con 0x7fc8e006e400 2023-11-28T12:05:43.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.107+0000 7fc8dd59a700 1 -- 172.21.15.57:0/2892636702 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00043c0 con 0x7fc8e006e400 2023-11-28T12:05:43.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.107+0000 7fc8e4934700 1 -- 172.21.15.57:0/2892636702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8e006e400 msgr2=0x7fc8e006e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:43.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.107+0000 7fc8e4934700 1 --2- 172.21.15.57:0/2892636702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8e006e400 0x7fc8e006e7d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009840 tx=0x7fc8d0009b50 comp rx=0 tx=0).stop 2023-11-28T12:05:43.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.107+0000 7fc8e4934700 1 -- 172.21.15.57:0/2892636702 shutdown_connections 2023-11-28T12:05:43.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.107+0000 7fc8e4934700 1 --2- 172.21.15.57:0/2892636702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8e006e400 0x7fc8e006e7d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:43.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.107+0000 7fc8e4934700 1 -- 172.21.15.57:0/2892636702 >> 172.21.15.57:0/2892636702 conn(0x7fc8e006a270 msgr2=0x7fc8e006a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:43.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.107+0000 7fc8e4934700 1 -- 172.21.15.57:0/2892636702 shutdown_connections 2023-11-28T12:05:43.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.107+0000 7fc8e4934700 1 -- 172.21.15.57:0/2892636702 wait complete. 2023-11-28T12:05:43.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.109+0000 7fc8e4934700 1 Processor -- start 2023-11-28T12:05:43.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.109+0000 7fc8e4934700 1 -- start start 2023-11-28T12:05:43.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8e4934700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8e006e400 0x7fc8e0149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:43.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8e4934700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8e00d15f0 0x7fc8e0152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:43.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8e4934700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8e015e010 0x7fc8e00b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:43.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8e4934700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e0152cd0 con 0x7fc8e006e400 2023-11-28T12:05:43.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8e4934700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00b2960 con 0x7fc8e00d15f0 2023-11-28T12:05:43.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8e4934700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00b2c40 con 0x7fc8e015e010 2023-11-28T12:05:43.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8de59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8e006e400 0x7fc8e0149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:43.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8de59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8e006e400 0x7fc8e0149db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38062/0 (socket says 172.21.15.57:38062) 2023-11-28T12:05:43.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8de59c700 1 -- 172.21.15.57:0/2020937494 learned_addr learned my addr 172.21.15.57:0/2020937494 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:43.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8ddd9b700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8e00d15f0 0x7fc8e0152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:43.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8ded9d700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8e015e010 0x7fc8e00b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:43.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8de59c700 1 -- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8e015e010 msgr2=0x7fc8e00b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:43.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8de59c700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8e015e010 0x7fc8e00b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:43.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8de59c700 1 -- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8e00d15f0 msgr2=0x7fc8e0152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:43.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8de59c700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8e00d15f0 0x7fc8e0152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:43.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8de59c700 1 -- 172.21.15.57:0/2020937494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d00094a0 con 0x7fc8e006e400 2023-11-28T12:05:43.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8de59c700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8e006e400 0x7fc8e0149db0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00b6eb0 tx=0x7fc8d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:43.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8ceffd700 1 -- 172.21.15.57:0/2020937494 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00045b0 con 0x7fc8e006e400 2023-11-28T12:05:43.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8ceffd700 1 -- 172.21.15.57:0/2020937494 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d0010590 con 0x7fc8e006e400 2023-11-28T12:05:43.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.110+0000 7fc8e4934700 1 -- 172.21.15.57:0/2020937494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e015ec60 con 0x7fc8e006e400 2023-11-28T12:05:43.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.111+0000 7fc8ceffd700 1 -- 172.21.15.57:0/2020937494 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0010920 con 0x7fc8e006e400 2023-11-28T12:05:43.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.111+0000 7fc8e4934700 1 -- 172.21.15.57:0/2020937494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e015e980 con 0x7fc8e006e400 2023-11-28T12:05:43.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.112+0000 7fc8e4934700 1 -- 172.21.15.57:0/2020937494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e015abc0 con 0x7fc8e006e400 2023-11-28T12:05:43.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.112+0000 7fc8ceffd700 1 -- 172.21.15.57:0/2020937494 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc8d000f050 con 0x7fc8e006e400 2023-11-28T12:05:43.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.112+0000 7fc8ceffd700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc8c00682f0 0x7fc8c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:43.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.113+0000 7fc8ceffd700 1 -- 172.21.15.57:0/2020937494 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fc8d00873a0 con 0x7fc8e006e400 2023-11-28T12:05:43.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.113+0000 7fc8ddd9b700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc8c00682f0 0x7fc8c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:43.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.113+0000 7fc8ddd9b700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc8c00682f0 0x7fc8c006a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc8e01f3560 tx=0x7fc8d4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:43.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.117+0000 7fc8ceffd700 1 -- 172.21.15.57:0/2020937494 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc8d0053b20 con 0x7fc8e006e400 2023-11-28T12:05:43.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.363+0000 7fc8e4934700 1 -- 172.21.15.57:0/2020937494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc8e00743f0 con 0x7fc8e006e400 2023-11-28T12:05:43.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.363+0000 7fc8ceffd700 1 -- 172.21.15.57:0/2020937494 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fc8d0057140 con 0x7fc8e006e400 2023-11-28T12:05:43.365 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:05:43.365 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:05:43.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.365+0000 7fc8e4934700 1 -- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc8c00682f0 msgr2=0x7fc8c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:43.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.365+0000 7fc8e4934700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc8c00682f0 0x7fc8c006a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc8e01f3560 tx=0x7fc8d4009040 comp rx=0 tx=0).stop 2023-11-28T12:05:43.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.365+0000 7fc8e4934700 1 -- 172.21.15.57:0/2020937494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8e006e400 msgr2=0x7fc8e0149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:43.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.365+0000 7fc8e4934700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8e006e400 0x7fc8e0149db0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00b6eb0 tx=0x7fc8d0004750 comp rx=0 tx=0).stop 2023-11-28T12:05:43.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.365+0000 7fc8e4934700 1 -- 172.21.15.57:0/2020937494 shutdown_connections 2023-11-28T12:05:43.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.365+0000 7fc8e4934700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc8c00682f0 0x7fc8c006a7a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:43.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.365+0000 7fc8e4934700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8e015e010 0x7fc8e00b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:43.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.365+0000 7fc8e4934700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8e006e400 0x7fc8e0149db0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:43.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.365+0000 7fc8e4934700 1 --2- 172.21.15.57:0/2020937494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8e00d15f0 0x7fc8e0152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:43.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.365+0000 7fc8e4934700 1 -- 172.21.15.57:0/2020937494 >> 172.21.15.57:0/2020937494 conn(0x7fc8e006a270 msgr2=0x7fc8e0154570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:43.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.366+0000 7fc8e4934700 1 -- 172.21.15.57:0/2020937494 shutdown_connections 2023-11-28T12:05:43.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:43.366+0000 7fc8e4934700 1 -- 172.21.15.57:0/2020937494 wait complete. 2023-11-28T12:05:43.380 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:05:46.697 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:05:46.701 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:05:48.382 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:05:48.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.524+0000 7f8d54b98700 1 Processor -- start 2023-11-28T12:05:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.524+0000 7f8d54b98700 1 -- start start 2023-11-28T12:05:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.525+0000 7f8d54b98700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d5006a270 0x7f8d5006a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.525+0000 7f8d54b98700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8d5015cc80 0x7f8d5006de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.525+0000 7f8d54b98700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5006ab80 con 0x7f8d5006e3a0 2023-11-28T12:05:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.525+0000 7f8d54b98700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d500b2490 con 0x7f8d5015cc80 2023-11-28T12:05:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.525+0000 7f8d54b98700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500b1810 con 0x7f8d5006a270 2023-11-28T12:05:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.526+0000 7f8d4dd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8d5015cc80 0x7f8d5006de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.526+0000 7f8d4e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d5006a270 0x7f8d5006a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.526+0000 7f8d4ed9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8d5006e3a0 0x7f8d5006ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:58344/0 (socket says 172.21.15.57:58344) 2023-11-28T12:05:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.526+0000 7f8d4ed9d700 1 -- 172.21.15.57:0/2050997779 learned_addr learned my addr 172.21.15.57:0/2050997779 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:05:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.526+0000 7f8d4e59c700 1 -- 172.21.15.57:0/2050997779 >> v2:172.21.15.181:3300/0 conn(0x7f8d5015cc80 msgr2=0x7f8d5006de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.526+0000 7f8d4e59c700 1 --2- 172.21.15.57:0/2050997779 >> v2:172.21.15.181:3300/0 conn(0x7f8d5015cc80 0x7f8d5006de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.526+0000 7f8d4e59c700 1 -- 172.21.15.57:0/2050997779 >> v1:172.21.15.181:6789/0 conn(0x7f8d5006e3a0 legacy=0x7f8d5006ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.526+0000 7f8d4d59a700 1 -- 172.21.15.57:0/2050997779 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693284844 0 0) 0x7f8d5006ab80 con 0x7f8d5006e3a0 2023-11-28T12:05:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.526+0000 7f8d4e59c700 1 -- 172.21.15.57:0/2050997779 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d500b1af0 con 0x7f8d5006a270 2023-11-28T12:05:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.526+0000 7f8d4dd9b700 1 --2- 172.21.15.57:0/2050997779 >> v2:172.21.15.181:3300/0 conn(0x7f8d5015cc80 0x7f8d5006de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:05:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.526+0000 7f8d4e59c700 1 --2- 172.21.15.57:0/2050997779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d5006a270 0x7f8d5006a640 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009840 tx=0x7f8d40009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=163274523c1ec05 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.527+0000 7f8d4d59a700 1 -- 172.21.15.57:0/2050997779 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40011830 con 0x7f8d5006a270 2023-11-28T12:05:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.527+0000 7f8d4d59a700 1 -- 172.21.15.57:0/2050997779 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d40011990 con 0x7f8d5006a270 2023-11-28T12:05:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.527+0000 7f8d4d59a700 1 -- 172.21.15.57:0/2050997779 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40011d20 con 0x7f8d5006a270 2023-11-28T12:05:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.527+0000 7f8d54b98700 1 -- 172.21.15.57:0/2050997779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d5006a270 msgr2=0x7f8d5006a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.527+0000 7f8d54b98700 1 --2- 172.21.15.57:0/2050997779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d5006a270 0x7f8d5006a640 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009840 tx=0x7f8d40009b50 comp rx=0 tx=0).stop 2023-11-28T12:05:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.527+0000 7f8d54b98700 1 -- 172.21.15.57:0/2050997779 shutdown_connections 2023-11-28T12:05:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.527+0000 7f8d54b98700 1 --2- 172.21.15.57:0/2050997779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d5006a270 0x7f8d5006a640 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.527+0000 7f8d54b98700 1 --2- 172.21.15.57:0/2050997779 >> v2:172.21.15.181:3300/0 conn(0x7f8d5015cc80 0x7f8d5006de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.527+0000 7f8d54b98700 1 -- 172.21.15.57:0/2050997779 >> 172.21.15.57:0/2050997779 conn(0x7f8d5006c010 msgr2=0x7f8d500cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.527+0000 7f8d54b98700 1 -- 172.21.15.57:0/2050997779 shutdown_connections 2023-11-28T12:05:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.527+0000 7f8d54b98700 1 -- 172.21.15.57:0/2050997779 wait complete. 2023-11-28T12:05:48.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.529+0000 7f8d54b98700 1 Processor -- start 2023-11-28T12:05:48.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.529+0000 7f8d54b98700 1 -- start start 2023-11-28T12:05:48.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.530+0000 7f8d54b98700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d5006a270 0x7f8d50152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:48.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.530+0000 7f8d54b98700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d5006e3a0 0x7f8d5015b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:48.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.530+0000 7f8d54b98700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d5015cc80 0x7f8d5015bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:48.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.530+0000 7f8d54b98700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d501532b0 con 0x7f8d5006e3a0 2023-11-28T12:05:48.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.530+0000 7f8d54b98700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500c7fa0 con 0x7f8d5015cc80 2023-11-28T12:05:48.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.530+0000 7f8d54b98700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500c8280 con 0x7f8d5006a270 2023-11-28T12:05:48.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.530+0000 7f8d4dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d5006e3a0 0x7f8d5015b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:48.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.530+0000 7f8d4dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d5006e3a0 0x7f8d5015b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38824/0 (socket says 172.21.15.57:38824) 2023-11-28T12:05:48.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.530+0000 7f8d4dd9b700 1 -- 172.21.15.57:0/2874550475 learned_addr learned my addr 172.21.15.57:0/2874550475 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:48.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.530+0000 7f8d4e59c700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d5006a270 0x7f8d50152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:48.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.531+0000 7f8d4ed9d700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d5015cc80 0x7f8d5015bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:48.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.531+0000 7f8d4dd9b700 1 -- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d5006a270 msgr2=0x7f8d50152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:48.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.531+0000 7f8d4dd9b700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d5006a270 0x7f8d50152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:48.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.531+0000 7f8d4dd9b700 1 -- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d5015cc80 msgr2=0x7f8d5015bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:48.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.531+0000 7f8d4dd9b700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d5015cc80 0x7f8d5015bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:48.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.531+0000 7f8d4dd9b700 1 -- 172.21.15.57:0/2874550475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d440094a0 con 0x7f8d5006e3a0 2023-11-28T12:05:48.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.531+0000 7f8d4e59c700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d5006a270 0x7f8d50152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:48.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.531+0000 7f8d4dd9b700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d5006e3a0 0x7f8d5015b980 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400ebe0 tx=0x7f8d44009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:48.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.531+0000 7f8d3effd700 1 -- 172.21.15.57:0/2874550475 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400c8b0 con 0x7f8d5006e3a0 2023-11-28T12:05:48.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.531+0000 7f8d3effd700 1 -- 172.21.15.57:0/2874550475 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4400ca10 con 0x7f8d5006e3a0 2023-11-28T12:05:48.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.531+0000 7f8d3effd700 1 -- 172.21.15.57:0/2874550475 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400cda0 con 0x7f8d5006e3a0 2023-11-28T12:05:48.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.531+0000 7f8d54b98700 1 -- 172.21.15.57:0/2874550475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d400094a0 con 0x7f8d5006e3a0 2023-11-28T12:05:48.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.531+0000 7f8d54b98700 1 -- 172.21.15.57:0/2874550475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d500b20b0 con 0x7f8d5006e3a0 2023-11-28T12:05:48.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.532+0000 7f8d54b98700 1 -- 172.21.15.57:0/2874550475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d500b1dd0 con 0x7f8d5006e3a0 2023-11-28T12:05:48.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.533+0000 7f8d3effd700 1 -- 172.21.15.57:0/2874550475 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8d44019030 con 0x7f8d5006e3a0 2023-11-28T12:05:48.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.533+0000 7f8d3effd700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8d300682f0 0x7f8d3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:48.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.534+0000 7f8d3effd700 1 -- 172.21.15.57:0/2874550475 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f8d44086590 con 0x7f8d5006e3a0 2023-11-28T12:05:48.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.534+0000 7f8d4e59c700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8d300682f0 0x7f8d3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:48.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.534+0000 7f8d4e59c700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8d300682f0 0x7f8d3006a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8d40000c00 tx=0x7f8d4001c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:48.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.537+0000 7f8d3effd700 1 -- 172.21.15.57:0/2874550475 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8d44052d00 con 0x7f8d5006e3a0 2023-11-28T12:05:48.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.789+0000 7f8d54b98700 1 -- 172.21.15.57:0/2874550475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8d5006e7e0 con 0x7f8d5006e3a0 2023-11-28T12:05:48.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.789+0000 7f8d3effd700 1 -- 172.21.15.57:0/2874550475 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f8d44056320 con 0x7f8d5006e3a0 2023-11-28T12:05:48.790 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:05:48.791 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:05:48.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.792+0000 7f8d54b98700 1 -- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8d300682f0 msgr2=0x7f8d3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:48.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.792+0000 7f8d54b98700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8d300682f0 0x7f8d3006a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8d40000c00 tx=0x7f8d4001c040 comp rx=0 tx=0).stop 2023-11-28T12:05:48.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.792+0000 7f8d54b98700 1 -- 172.21.15.57:0/2874550475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d5006e3a0 msgr2=0x7f8d5015b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:48.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.792+0000 7f8d54b98700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d5006e3a0 0x7f8d5015b980 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400ebe0 tx=0x7f8d44009e20 comp rx=0 tx=0).stop 2023-11-28T12:05:48.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.792+0000 7f8d54b98700 1 -- 172.21.15.57:0/2874550475 shutdown_connections 2023-11-28T12:05:48.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.792+0000 7f8d54b98700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8d300682f0 0x7f8d3006a7a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:48.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.792+0000 7f8d54b98700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d5015cc80 0x7f8d5015bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:48.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.792+0000 7f8d54b98700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d5006a270 0x7f8d50152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:48.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.792+0000 7f8d54b98700 1 --2- 172.21.15.57:0/2874550475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d5006e3a0 0x7f8d5015b980 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:48.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.792+0000 7f8d54b98700 1 -- 172.21.15.57:0/2874550475 >> 172.21.15.57:0/2874550475 conn(0x7f8d5006c010 msgr2=0x7f8d500b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:48.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.792+0000 7f8d54b98700 1 -- 172.21.15.57:0/2874550475 shutdown_connections 2023-11-28T12:05:48.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:48.792+0000 7f8d54b98700 1 -- 172.21.15.57:0/2874550475 wait complete. 2023-11-28T12:05:48.806 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:05:53.807 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:05:53.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.950+0000 7fa39d71d700 1 Processor -- start 2023-11-28T12:05:53.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.950+0000 7fa39d71d700 1 -- start start 2023-11-28T12:05:53.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.951+0000 7fa39d71d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3980b22a0 0x7fa3980cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:53.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.951+0000 7fa39d71d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3980d1ee0 con 0x7fa39806ab10 2023-11-28T12:05:53.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.951+0000 7fa39d71d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3980cdb10 con 0x7fa3980cd5a0 2023-11-28T12:05:53.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.951+0000 7fa39d71d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980d1890 con 0x7fa3980b22a0 2023-11-28T12:05:53.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.951+0000 7fa396ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa39806ab10 0x7fa39806aee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:39730/0 (socket says 172.21.15.57:39730) 2023-11-28T12:05:53.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.951+0000 7fa396ffd700 1 -- 172.21.15.57:0/1829757721 learned_addr learned my addr 172.21.15.57:0/1829757721 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:05:53.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa3977fe700 1 --2- 172.21.15.57:0/1829757721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3980b22a0 0x7fa3980cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:53.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140262531 0 0) 0x7fa3980d1ee0 con 0x7fa39806ab10 2023-11-28T12:05:53.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3800035b0 con 0x7fa39806ab10 2023-11-28T12:05:53.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 165135183 0 0) 0x7fa3800035b0 con 0x7fa39806ab10 2023-11-28T12:05:53.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3980b22a0 msgr2=0x7fa3980cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:53.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 --2- 172.21.15.57:0/1829757721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3980b22a0 0x7fa3980cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:53.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 >> v1:172.21.15.181:6789/0 conn(0x7fa3980cd5a0 legacy=0x7fa3980cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:53.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3980d1b70 con 0x7fa39806ab10 2023-11-28T12:05:53.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fa388001670 con 0x7fa39806ab10 2023-11-28T12:05:53.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 >> v1:172.21.15.57:6789/0 conn(0x7fa39806ab10 legacy=0x7fa39806aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:53.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 --2- 172.21.15.57:0/1829757721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa380003a30 0x7fa380003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:53.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 --2- 172.21.15.57:0/1829757721 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380004380 0x7fa380004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:53.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 --2- 172.21.15.57:0/1829757721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800050f0 0x7fa38000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:53.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980d1890 con 0x7fa380004380 2023-11-28T12:05:53.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38000c000 con 0x7fa3800050f0 2023-11-28T12:05:53.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38000c1c0 con 0x7fa380003a30 2023-11-28T12:05:53.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa3977fe700 1 --2- 172.21.15.57:0/1829757721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3980b22a0 0x7fa3980cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:05:53.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa3967fc700 1 --2- 172.21.15.57:0/1829757721 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380004380 0x7fa380004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:53.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa396ffd700 1 --2- 172.21.15.57:0/1829757721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa380003a30 0x7fa380003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:53.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.952+0000 7fa3977fe700 1 --2- 172.21.15.57:0/1829757721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800050f0 0x7fa38000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:53.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.953+0000 7fa3967fc700 1 -- 172.21.15.57:0/1829757721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa380003a30 msgr2=0x7fa380003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:53.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.953+0000 7fa3967fc700 1 --2- 172.21.15.57:0/1829757721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa380003a30 0x7fa380003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:53.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.953+0000 7fa3967fc700 1 -- 172.21.15.57:0/1829757721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800050f0 msgr2=0x7fa38000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:53.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.953+0000 7fa3967fc700 1 --2- 172.21.15.57:0/1829757721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800050f0 0x7fa38000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:53.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.953+0000 7fa3967fc700 1 -- 172.21.15.57:0/1829757721 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3800035b0 con 0x7fa380004380 2023-11-28T12:05:53.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.953+0000 7fa3967fc700 1 --2- 172.21.15.57:0/1829757721 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380004380 0x7fa380004850 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa3980cda40 tx=0x7fa380005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f636b04f280e04b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:53.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.953+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c0013f0 con 0x7fa380004380 2023-11-28T12:05:53.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.953+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c006a60 con 0x7fa380004380 2023-11-28T12:05:53.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.953+0000 7fa395ffb700 1 -- 172.21.15.57:0/1829757721 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c006e10 con 0x7fa380004380 2023-11-28T12:05:53.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.953+0000 7fa39d71d700 1 -- 172.21.15.57:0/1829757721 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380004380 msgr2=0x7fa380004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:53.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.953+0000 7fa39d71d700 1 --2- 172.21.15.57:0/1829757721 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa380004380 0x7fa380004850 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa3980cda40 tx=0x7fa380005880 comp rx=0 tx=0).stop 2023-11-28T12:05:53.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.953+0000 7fa3977fe700 1 -- 172.21.15.57:0/1829757721 reap_dead start 2023-11-28T12:05:53.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.954+0000 7fa39d71d700 1 -- 172.21.15.57:0/1829757721 shutdown_connections 2023-11-28T12:05:53.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.954+0000 7fa39d71d700 1 -- 172.21.15.57:0/1829757721 >> 172.21.15.57:0/1829757721 conn(0x7fa39806c520 msgr2=0x7fa39806c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:53.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.954+0000 7fa39d71d700 1 -- 172.21.15.57:0/1829757721 shutdown_connections 2023-11-28T12:05:53.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.954+0000 7fa39d71d700 1 -- 172.21.15.57:0/1829757721 wait complete. 2023-11-28T12:05:53.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.955+0000 7fa39d71d700 1 Processor -- start 2023-11-28T12:05:53.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.955+0000 7fa39d71d700 1 -- start start 2023-11-28T12:05:53.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa39d71d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa380004380 0x7fa398154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:53.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa39d71d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa398159910 0x7fa39815b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:53.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa39d71d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa398159d80 0x7fa39815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:53.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa39d71d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39815bfb0 con 0x7fa398159d80 2023-11-28T12:05:53.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa39d71d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980cbe00 con 0x7fa380004380 2023-11-28T12:05:53.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa39d71d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980cc0e0 con 0x7fa398159910 2023-11-28T12:05:53.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa3977fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa398159d80 0x7fa39815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:53.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa3977fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa398159d80 0x7fa39815ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38836/0 (socket says 172.21.15.57:38836) 2023-11-28T12:05:53.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa3977fe700 1 -- 172.21.15.57:0/7293232 learned_addr learned my addr 172.21.15.57:0/7293232 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:53.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa3967fc700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa398159910 0x7fa39815b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:53.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa396ffd700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa380004380 0x7fa398154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:53.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa3977fe700 1 -- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa398159910 msgr2=0x7fa39815b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:53.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa3977fe700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa398159910 0x7fa39815b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:53.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa3977fe700 1 -- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa380004380 msgr2=0x7fa398154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:53.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa3977fe700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa380004380 0x7fa398154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:53.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa3977fe700 1 -- 172.21.15.57:0/7293232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3980b1ec0 con 0x7fa398159d80 2023-11-28T12:05:53.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa396ffd700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa380004380 0x7fa398154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:53.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa3967fc700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa398159910 0x7fa39815b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:05:53.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.956+0000 7fa3977fe700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa398159d80 0x7fa39815ba70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa39806bec0 tx=0x7fa39000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:53.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.957+0000 7fa37f7fe700 1 -- 172.21.15.57:0/7293232 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39000e040 con 0x7fa398159d80 2023-11-28T12:05:53.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.957+0000 7fa37f7fe700 1 -- 172.21.15.57:0/7293232 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39000f040 con 0x7fa398159d80 2023-11-28T12:05:53.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.957+0000 7fa39d71d700 1 -- 172.21.15.57:0/7293232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39815cad0 con 0x7fa398159d80 2023-11-28T12:05:53.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.957+0000 7fa37f7fe700 1 -- 172.21.15.57:0/7293232 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390003c60 con 0x7fa398159d80 2023-11-28T12:05:53.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.957+0000 7fa39d71d700 1 -- 172.21.15.57:0/7293232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3980b2050 con 0x7fa398159d80 2023-11-28T12:05:53.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.958+0000 7fa39d71d700 1 -- 172.21.15.57:0/7293232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3980743f0 con 0x7fa398159d80 2023-11-28T12:05:53.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.958+0000 7fa37f7fe700 1 -- 172.21.15.57:0/7293232 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa390020030 con 0x7fa398159d80 2023-11-28T12:05:53.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.959+0000 7fa37f7fe700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa3840682f0 0x7fa38406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:53.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.959+0000 7fa37f7fe700 1 -- 172.21.15.57:0/7293232 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fa390085a90 con 0x7fa398159d80 2023-11-28T12:05:53.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.959+0000 7fa396ffd700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa3840682f0 0x7fa38406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:53.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.960+0000 7fa396ffd700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa3840682f0 0x7fa38406a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa388000c00 tx=0x7fa388009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:53.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:53.963+0000 7fa37f7fe700 1 -- 172.21.15.57:0/7293232 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa390052270 con 0x7fa398159d80 2023-11-28T12:05:54.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.211+0000 7fa39d71d700 1 -- 172.21.15.57:0/7293232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa3980d2c80 con 0x7fa398159d80 2023-11-28T12:05:54.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.212+0000 7fa37f7fe700 1 -- 172.21.15.57:0/7293232 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fa390055890 con 0x7fa398159d80 2023-11-28T12:05:54.212 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:05:54.213 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:05:54.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.213+0000 7fa39d71d700 1 -- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa3840682f0 msgr2=0x7fa38406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:54.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.213+0000 7fa39d71d700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa3840682f0 0x7fa38406a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa388000c00 tx=0x7fa388009040 comp rx=0 tx=0).stop 2023-11-28T12:05:54.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.214+0000 7fa39d71d700 1 -- 172.21.15.57:0/7293232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa398159d80 msgr2=0x7fa39815ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:54.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.214+0000 7fa39d71d700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa398159d80 0x7fa39815ba70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa39806bec0 tx=0x7fa39000f770 comp rx=0 tx=0).stop 2023-11-28T12:05:54.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.214+0000 7fa39d71d700 1 -- 172.21.15.57:0/7293232 shutdown_connections 2023-11-28T12:05:54.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.214+0000 7fa39d71d700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa3840682f0 0x7fa38406a7a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:54.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.214+0000 7fa39d71d700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa398159d80 0x7fa39815ba70 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:54.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.214+0000 7fa39d71d700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa380004380 0x7fa398154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:54.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.214+0000 7fa39d71d700 1 --2- 172.21.15.57:0/7293232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa398159910 0x7fa39815b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:54.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.214+0000 7fa39d71d700 1 -- 172.21.15.57:0/7293232 >> 172.21.15.57:0/7293232 conn(0x7fa39806c520 msgr2=0x7fa398155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:54.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.214+0000 7fa39d71d700 1 -- 172.21.15.57:0/7293232 shutdown_connections 2023-11-28T12:05:54.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:54.214+0000 7fa39d71d700 1 -- 172.21.15.57:0/7293232 wait complete. 2023-11-28T12:05:54.227 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:05:59.228 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:05:59.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.369+0000 7f8385918700 1 Processor -- start 2023-11-28T12:05:59.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.369+0000 7f8385918700 1 -- start start 2023-11-28T12:05:59.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f8385918700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83800d0c60 0x7f83800b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:59.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f8385918700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f838006e400 0x7f83800d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:59.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f8385918700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83800d1570 con 0x7f8380160e70 2023-11-28T12:05:59.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f8385918700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838006e900 con 0x7f838006e400 2023-11-28T12:05:59.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f8385918700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8380160390 con 0x7f83800d0c60 2023-11-28T12:05:59.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f837e7fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f838006e400 0x7f83800d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:59.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f837e7fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f838006e400 0x7f83800d1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59658/0 (socket says 172.21.15.57:59658) 2023-11-28T12:05:59.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f837e7fc700 1 -- 172.21.15.57:0/1730921570 learned_addr learned my addr 172.21.15.57:0/1730921570 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:59.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f837effd700 1 --2- 172.21.15.57:0/1730921570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83800d0c60 0x7f83800b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:59.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f837e7fc700 1 -- 172.21.15.57:0/1730921570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83800d0c60 msgr2=0x7f83800b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:05:59.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f837e7fc700 1 --2- 172.21.15.57:0/1730921570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83800d0c60 0x7f83800b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:59.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f837e7fc700 1 -- 172.21.15.57:0/1730921570 >> v1:172.21.15.181:6789/0 conn(0x7f8380160e70 legacy=0x7f838006f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:59.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f837e7fc700 1 -- 172.21.15.57:0/1730921570 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8380160670 con 0x7f838006e400 2023-11-28T12:05:59.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f837e7fc700 1 --2- 172.21.15.57:0/1730921570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f838006e400 0x7f83800d1030 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f83740098f0 tx=0x7f8374009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4428038c2d63fda server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:59.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.370+0000 7f837dffb700 1 -- 172.21.15.57:0/1730921570 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837400b8a0 con 0x7f838006e400 2023-11-28T12:05:59.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.371+0000 7f837dffb700 1 -- 172.21.15.57:0/1730921570 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837400ba00 con 0x7f838006e400 2023-11-28T12:05:59.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.371+0000 7f837dffb700 1 -- 172.21.15.57:0/1730921570 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837400bd90 con 0x7f838006e400 2023-11-28T12:05:59.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.371+0000 7f8385918700 1 -- 172.21.15.57:0/1730921570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f838006e400 msgr2=0x7f83800d1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:59.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.371+0000 7f8385918700 1 --2- 172.21.15.57:0/1730921570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f838006e400 0x7f83800d1030 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f83740098f0 tx=0x7f8374009c00 comp rx=0 tx=0).stop 2023-11-28T12:05:59.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.371+0000 7f8385918700 1 -- 172.21.15.57:0/1730921570 shutdown_connections 2023-11-28T12:05:59.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.371+0000 7f8385918700 1 --2- 172.21.15.57:0/1730921570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83800d0c60 0x7f83800b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:59.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.371+0000 7f8385918700 1 --2- 172.21.15.57:0/1730921570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f838006e400 0x7f83800d1030 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:59.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.371+0000 7f8385918700 1 -- 172.21.15.57:0/1730921570 >> 172.21.15.57:0/1730921570 conn(0x7f838006a270 msgr2=0x7f838006a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:59.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.371+0000 7f8385918700 1 -- 172.21.15.57:0/1730921570 shutdown_connections 2023-11-28T12:05:59.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.371+0000 7f8385918700 1 -- 172.21.15.57:0/1730921570 wait complete. 2023-11-28T12:05:59.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.372+0000 7f8385918700 1 Processor -- start 2023-11-28T12:05:59.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.373+0000 7f8385918700 1 -- start start 2023-11-28T12:05:59.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.373+0000 7f8385918700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f838006e400 0x7f838014bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:59.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.373+0000 7f8385918700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83800d0c60 0x7f8380157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:59.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.373+0000 7f8385918700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8380160e70 0x7f8380151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:59.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.373+0000 7f8385918700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8380157960 con 0x7f83800d0c60 2023-11-28T12:05:59.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.373+0000 7f8385918700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8380158330 con 0x7f8380160e70 2023-11-28T12:05:59.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.373+0000 7f8385918700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8380156d50 con 0x7f838006e400 2023-11-28T12:05:59.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f837e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83800d0c60 0x7f8380157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:59.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f837e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83800d0c60 0x7f8380157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59664/0 (socket says 172.21.15.57:59664) 2023-11-28T12:05:59.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f837e7fc700 1 -- 172.21.15.57:0/918549574 learned_addr learned my addr 172.21.15.57:0/918549574 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:05:59.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f837effd700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f838006e400 0x7f838014bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:59.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f837f7fe700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8380160e70 0x7f8380151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:59.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f837e7fc700 1 -- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f838006e400 msgr2=0x7f838014bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:59.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f837e7fc700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f838006e400 0x7f838014bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:59.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f837e7fc700 1 -- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8380160e70 msgr2=0x7f8380151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:59.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f837e7fc700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8380160e70 0x7f8380151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:59.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f837e7fc700 1 -- 172.21.15.57:0/918549574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8374009550 con 0x7f83800d0c60 2023-11-28T12:05:59.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f837e7fc700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83800d0c60 0x7f8380157df0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8374009c40 tx=0x7f8374004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:59.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f8367fff700 1 -- 172.21.15.57:0/918549574 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740036a0 con 0x7f83800d0c60 2023-11-28T12:05:59.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f8367fff700 1 -- 172.21.15.57:0/918549574 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837401e070 con 0x7f83800d0c60 2023-11-28T12:05:59.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f8385918700 1 -- 172.21.15.57:0/918549574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83800d0740 con 0x7f83800d0c60 2023-11-28T12:05:59.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f8367fff700 1 -- 172.21.15.57:0/918549574 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8374003a60 con 0x7f83800d0c60 2023-11-28T12:05:59.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.374+0000 7f8385918700 1 -- 172.21.15.57:0/918549574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8380160950 con 0x7f83800d0c60 2023-11-28T12:05:59.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.375+0000 7f8385918700 1 -- 172.21.15.57:0/918549574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838015d980 con 0x7f83800d0c60 2023-11-28T12:05:59.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.376+0000 7f8367fff700 1 -- 172.21.15.57:0/918549574 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f837400f050 con 0x7f83800d0c60 2023-11-28T12:05:59.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.376+0000 7f8367fff700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f836c0682f0 0x7f836c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:05:59.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.376+0000 7f8367fff700 1 -- 172.21.15.57:0/918549574 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f8374020070 con 0x7f83800d0c60 2023-11-28T12:05:59.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.377+0000 7f837effd700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f836c0682f0 0x7f836c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:05:59.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.377+0000 7f837effd700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f836c0682f0 0x7f836c06a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8370009930 tx=0x7f8370008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:05:59.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.381+0000 7f8367fff700 1 -- 172.21.15.57:0/918549574 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8374053970 con 0x7f83800d0c60 2023-11-28T12:05:59.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.625+0000 7f8385918700 1 -- 172.21.15.57:0/918549574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f83800743f0 con 0x7f83800d0c60 2023-11-28T12:05:59.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.625+0000 7f8367fff700 1 -- 172.21.15.57:0/918549574 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f8374056f90 con 0x7f83800d0c60 2023-11-28T12:05:59.627 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:05:59.627 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:05:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.627+0000 7f8385918700 1 -- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f836c0682f0 msgr2=0x7f836c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.627+0000 7f8385918700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f836c0682f0 0x7f836c06a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8370009930 tx=0x7f8370008040 comp rx=0 tx=0).stop 2023-11-28T12:05:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.627+0000 7f8385918700 1 -- 172.21.15.57:0/918549574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83800d0c60 msgr2=0x7f8380157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:05:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.627+0000 7f8385918700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83800d0c60 0x7f8380157df0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8374009c40 tx=0x7f8374004750 comp rx=0 tx=0).stop 2023-11-28T12:05:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.627+0000 7f8385918700 1 -- 172.21.15.57:0/918549574 shutdown_connections 2023-11-28T12:05:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.627+0000 7f8385918700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f836c0682f0 0x7f836c06a7a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.627+0000 7f8385918700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8380160e70 0x7f8380151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.627+0000 7f8385918700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f838006e400 0x7f838014bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:59.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.627+0000 7f8385918700 1 --2- 172.21.15.57:0/918549574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83800d0c60 0x7f8380157df0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:05:59.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.627+0000 7f8385918700 1 -- 172.21.15.57:0/918549574 >> 172.21.15.57:0/918549574 conn(0x7f838006a270 msgr2=0x7f838015a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:05:59.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.628+0000 7f8385918700 1 -- 172.21.15.57:0/918549574 shutdown_connections 2023-11-28T12:05:59.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:05:59.628+0000 7f8385918700 1 -- 172.21.15.57:0/918549574 wait complete. 2023-11-28T12:05:59.642 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:06:04.643 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:06:04.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.786+0000 7f611c197700 1 Processor -- start 2023-11-28T12:06:04.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.786+0000 7f611c197700 1 -- start start 2023-11-28T12:06:04.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.787+0000 7f611c197700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140ad900 0x7f6114072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:04.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.787+0000 7f611c197700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61140726c0 con 0x7f611406b7b0 2023-11-28T12:06:04.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.787+0000 7f611c197700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61140b43f0 con 0x7f61140b0880 2023-11-28T12:06:04.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.787+0000 7f611c197700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611406bc50 con 0x7f61140ad900 2023-11-28T12:06:04.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.787+0000 7f6119732700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f611406b7b0 0x7f61140ad1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:44264/0 (socket says 172.21.15.57:44264) 2023-11-28T12:06:04.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.787+0000 7f6119732700 1 -- 172.21.15.57:0/130433975 learned_addr learned my addr 172.21.15.57:0/130433975 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:06:04.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.787+0000 7f611a734700 1 --2- 172.21.15.57:0/130433975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140ad900 0x7f6114072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:04.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.787+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402838631 0 0) 0x7f61140726c0 con 0x7f611406b7b0 2023-11-28T12:06:04.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.787+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61000035b0 con 0x7f611406b7b0 2023-11-28T12:06:04.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.787+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3892700383 0 0) 0x7f61000035b0 con 0x7f611406b7b0 2023-11-28T12:06:04.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140ad900 msgr2=0x7f6114072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:06:04.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6118f31700 1 --2- 172.21.15.57:0/130433975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140ad900 0x7f6114072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:04.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 >> v1:172.21.15.181:6789/0 conn(0x7f61140b0880 legacy=0x7f61140aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:04.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61140b46d0 con 0x7f611406b7b0 2023-11-28T12:06:04.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f6110001670 con 0x7f611406b7b0 2023-11-28T12:06:04.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 >> v1:172.21.15.57:6789/0 conn(0x7f611406b7b0 legacy=0x7f61140ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:04.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6118f31700 1 --2- 172.21.15.57:0/130433975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6100003a30 0x7f6100003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:04.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6118f31700 1 --2- 172.21.15.57:0/130433975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6100004420 0x7f61000048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:04.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6118f31700 1 --2- 172.21.15.57:0/130433975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140ad900 0x7f6100005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:04.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611406bc50 con 0x7f6100004420 2023-11-28T12:06:04.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610000ba40 con 0x7f6100003a30 2023-11-28T12:06:04.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f610000bc00 con 0x7f61140ad900 2023-11-28T12:06:04.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6119732700 1 --2- 172.21.15.57:0/130433975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6100004420 0x7f61000048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:04.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6119f33700 1 --2- 172.21.15.57:0/130433975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6100003a30 0x7f6100003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:04.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f611a734700 1 --2- 172.21.15.57:0/130433975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140ad900 0x7f6100005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:04.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6119732700 1 -- 172.21.15.57:0/130433975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140ad900 msgr2=0x7f6100005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:06:04.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6119732700 1 --2- 172.21.15.57:0/130433975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140ad900 0x7f6100005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:04.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6119732700 1 -- 172.21.15.57:0/130433975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6100003a30 msgr2=0x7f6100003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:06:04.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6119732700 1 --2- 172.21.15.57:0/130433975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6100003a30 0x7f6100003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:04.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6119732700 1 -- 172.21.15.57:0/130433975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61000035b0 con 0x7f6100004420 2023-11-28T12:06:04.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6119732700 1 --2- 172.21.15.57:0/130433975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6100004420 0x7f61000048f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f611000c210 tx=0x7f6100005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f5f714048c7115 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:04.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.788+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110007b70 con 0x7f6100004420 2023-11-28T12:06:04.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.789+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611000cdb0 con 0x7f6100004420 2023-11-28T12:06:04.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.789+0000 7f6118f31700 1 -- 172.21.15.57:0/130433975 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110005ba0 con 0x7f6100004420 2023-11-28T12:06:04.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.789+0000 7f611c197700 1 -- 172.21.15.57:0/130433975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6100004420 msgr2=0x7f61000048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:04.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.789+0000 7f611c197700 1 --2- 172.21.15.57:0/130433975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6100004420 0x7f61000048f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f611000c210 tx=0x7f6100005880 comp rx=0 tx=0).stop 2023-11-28T12:06:04.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.789+0000 7f611a734700 1 -- 172.21.15.57:0/130433975 reap_dead start 2023-11-28T12:06:04.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.789+0000 7f611c197700 1 -- 172.21.15.57:0/130433975 shutdown_connections 2023-11-28T12:06:04.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.789+0000 7f611c197700 1 -- 172.21.15.57:0/130433975 >> 172.21.15.57:0/130433975 conn(0x7f61140b2fc0 msgr2=0x7f6114071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:04.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.789+0000 7f611c197700 1 -- 172.21.15.57:0/130433975 shutdown_connections 2023-11-28T12:06:04.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.789+0000 7f611c197700 1 -- 172.21.15.57:0/130433975 wait complete. 2023-11-28T12:06:04.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.790+0000 7f611c197700 1 Processor -- start 2023-11-28T12:06:04.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.791+0000 7f611c197700 1 -- start start 2023-11-28T12:06:04.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.791+0000 7f611c197700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f611409cd50 0x7f61140c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:04.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.791+0000 7f611c197700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61140c7620 0x7f61140af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:04.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.791+0000 7f611c197700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140afc60 0x7f611415aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:04.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.791+0000 7f611c197700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61140b0100 con 0x7f61140c7620 2023-11-28T12:06:04.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.791+0000 7f611c197700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61140c8720 con 0x7f611409cd50 2023-11-28T12:06:04.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.791+0000 7f611c197700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61140c7b10 con 0x7f61140afc60 2023-11-28T12:06:04.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.791+0000 7f6119732700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61140c7620 0x7f61140af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:04.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.791+0000 7f6119732700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61140c7620 0x7f61140af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45260/0 (socket says 172.21.15.57:45260) 2023-11-28T12:06:04.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.791+0000 7f6119732700 1 -- 172.21.15.57:0/1942626622 learned_addr learned my addr 172.21.15.57:0/1942626622 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:04.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f611a734700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140afc60 0x7f611415aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:04.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f6119f33700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f611409cd50 0x7f61140c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:04.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f6119732700 1 -- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140afc60 msgr2=0x7f611415aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:04.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f6119732700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140afc60 0x7f611415aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:04.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f6119732700 1 -- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f611409cd50 msgr2=0x7f61140c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:04.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f6119732700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f611409cd50 0x7f61140c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:04.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f6119732700 1 -- 172.21.15.57:0/1942626622 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61140b4fe0 con 0x7f61140c7620 2023-11-28T12:06:04.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f611a734700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140afc60 0x7f611415aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:06:04.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f6119732700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61140c7620 0x7f61140af720 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6110005ab0 tx=0x7f6110001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:04.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f61067fc700 1 -- 172.21.15.57:0/1942626622 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110005e60 con 0x7f61140c7620 2023-11-28T12:06:04.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f61067fc700 1 -- 172.21.15.57:0/1942626622 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6110000ed0 con 0x7f61140c7620 2023-11-28T12:06:04.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f61067fc700 1 -- 172.21.15.57:0/1942626622 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61100072f0 con 0x7f61140c7620 2023-11-28T12:06:04.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f611c197700 1 -- 172.21.15.57:0/1942626622 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61140b4d00 con 0x7f61140c7620 2023-11-28T12:06:04.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.792+0000 7f611c197700 1 -- 172.21.15.57:0/1942626622 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61140b49b0 con 0x7f61140c7620 2023-11-28T12:06:04.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.793+0000 7f611c197700 1 -- 172.21.15.57:0/1942626622 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61140743f0 con 0x7f61140c7620 2023-11-28T12:06:04.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.794+0000 7f61067fc700 1 -- 172.21.15.57:0/1942626622 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6110010070 con 0x7f61140c7620 2023-11-28T12:06:04.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.794+0000 7f61067fc700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f60f8068320 0x7f60f806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:04.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.794+0000 7f61067fc700 1 -- 172.21.15.57:0/1942626622 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f6110086df0 con 0x7f61140c7620 2023-11-28T12:06:04.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.794+0000 7f6119f33700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f60f8068320 0x7f60f806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:04.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.795+0000 7f6119f33700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f60f8068320 0x7f60f806a7d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6108000c00 tx=0x7f6108009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:04.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:04.799+0000 7f61067fc700 1 -- 172.21.15.57:0/1942626622 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f61100534e0 con 0x7f61140c7620 2023-11-28T12:06:05.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.045+0000 7f611c197700 1 -- 172.21.15.57:0/1942626622 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6114073330 con 0x7f61140c7620 2023-11-28T12:06:05.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.045+0000 7f61067fc700 1 -- 172.21.15.57:0/1942626622 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f611002b0c0 con 0x7f61140c7620 2023-11-28T12:06:05.046 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:06:05.047 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:06:05.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.047+0000 7f611c197700 1 -- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f60f8068320 msgr2=0x7f60f806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:05.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.047+0000 7f611c197700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f60f8068320 0x7f60f806a7d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6108000c00 tx=0x7f6108009040 comp rx=0 tx=0).stop 2023-11-28T12:06:05.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.047+0000 7f611c197700 1 -- 172.21.15.57:0/1942626622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61140c7620 msgr2=0x7f61140af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:05.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.047+0000 7f611c197700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61140c7620 0x7f61140af720 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6110005ab0 tx=0x7f6110001930 comp rx=0 tx=0).stop 2023-11-28T12:06:05.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.047+0000 7f611c197700 1 -- 172.21.15.57:0/1942626622 shutdown_connections 2023-11-28T12:06:05.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.047+0000 7f611c197700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f60f8068320 0x7f60f806a7d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:05.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.047+0000 7f611c197700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61140afc60 0x7f611415aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:05.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.047+0000 7f611c197700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f611409cd50 0x7f61140c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:05.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.047+0000 7f611c197700 1 --2- 172.21.15.57:0/1942626622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61140c7620 0x7f61140af720 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:05.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.048+0000 7f611c197700 1 -- 172.21.15.57:0/1942626622 >> 172.21.15.57:0/1942626622 conn(0x7f61140b2fc0 msgr2=0x7f611415c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:05.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.048+0000 7f611c197700 1 -- 172.21.15.57:0/1942626622 shutdown_connections 2023-11-28T12:06:05.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:05.048+0000 7f611c197700 1 -- 172.21.15.57:0/1942626622 wait complete. 2023-11-28T12:06:05.062 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:06:10.063 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:06:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.206+0000 7f15ada4d700 1 Processor -- start 2023-11-28T12:06:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.206+0000 7f15ada4d700 1 -- start start 2023-11-28T12:06:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.207+0000 7f15ada4d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15a80cd5a0 0x7f15a80cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.207+0000 7f15ada4d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f15a80b22a0 0x7f15a80cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.207+0000 7f15ada4d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a80d1ee0 con 0x7f15a806ab10 2023-11-28T12:06:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.207+0000 7f15ada4d700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80cdb10 con 0x7f15a80b22a0 2023-11-28T12:06:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.207+0000 7f15ada4d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80d1890 con 0x7f15a80cd5a0 2023-11-28T12:06:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.207+0000 7f15a77fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f15a80b22a0 0x7f15a80cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.207+0000 7f15a77fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f15a80b22a0 0x7f15a80cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45272/0 (socket says 172.21.15.57:45272) 2023-11-28T12:06:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.207+0000 7f15a77fe700 1 -- 172.21.15.57:0/3595175933 learned_addr learned my addr 172.21.15.57:0/3595175933 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.207+0000 7f15a67fc700 1 --2- 172.21.15.57:0/3595175933 >> v2:172.21.15.181:3300/0 conn(0x7f15a80cd5a0 0x7f15a80cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.208+0000 7f15a77fe700 1 -- 172.21.15.57:0/3595175933 >> v2:172.21.15.181:3300/0 conn(0x7f15a80cd5a0 msgr2=0x7f15a80cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:06:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.208+0000 7f15a77fe700 1 --2- 172.21.15.57:0/3595175933 >> v2:172.21.15.181:3300/0 conn(0x7f15a80cd5a0 0x7f15a80cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.208+0000 7f15a77fe700 1 -- 172.21.15.57:0/3595175933 >> v1:172.21.15.181:6789/0 conn(0x7f15a806ab10 legacy=0x7f15a806aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.208+0000 7f15a77fe700 1 -- 172.21.15.57:0/3595175933 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80d1b70 con 0x7f15a80b22a0 2023-11-28T12:06:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.208+0000 7f15a77fe700 1 --2- 172.21.15.57:0/3595175933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a80b22a0 0x7f15a80cf980 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f15a80d0a60 tx=0x7f15a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abab9b8970350f87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.208+0000 7f15a5ffb700 1 -- 172.21.15.57:0/3595175933 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a000e070 con 0x7f15a80b22a0 2023-11-28T12:06:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.208+0000 7f15a5ffb700 1 -- 172.21.15.57:0/3595175933 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a000b9c0 con 0x7f15a80b22a0 2023-11-28T12:06:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.208+0000 7f15a5ffb700 1 -- 172.21.15.57:0/3595175933 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a000bd70 con 0x7f15a80b22a0 2023-11-28T12:06:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.209+0000 7f15ada4d700 1 -- 172.21.15.57:0/3595175933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a80b22a0 msgr2=0x7f15a80cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.209+0000 7f15ada4d700 1 --2- 172.21.15.57:0/3595175933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a80b22a0 0x7f15a80cf980 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f15a80d0a60 tx=0x7f15a000b330 comp rx=0 tx=0).stop 2023-11-28T12:06:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.209+0000 7f15ada4d700 1 -- 172.21.15.57:0/3595175933 shutdown_connections 2023-11-28T12:06:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.209+0000 7f15ada4d700 1 --2- 172.21.15.57:0/3595175933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a80b22a0 0x7f15a80cf980 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.209+0000 7f15ada4d700 1 --2- 172.21.15.57:0/3595175933 >> v2:172.21.15.181:3300/0 conn(0x7f15a80cd5a0 0x7f15a80cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.209+0000 7f15ada4d700 1 -- 172.21.15.57:0/3595175933 >> 172.21.15.57:0/3595175933 conn(0x7f15a806c520 msgr2=0x7f15a806c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:10.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.209+0000 7f15ada4d700 1 -- 172.21.15.57:0/3595175933 shutdown_connections 2023-11-28T12:06:10.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.209+0000 7f15ada4d700 1 -- 172.21.15.57:0/3595175933 wait complete. 2023-11-28T12:06:10.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.210+0000 7f15ada4d700 1 Processor -- start 2023-11-28T12:06:10.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.210+0000 7f15ada4d700 1 -- start start 2023-11-28T12:06:10.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15ada4d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a806ab10 0x7f15a81f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15ada4d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a80b22a0 0x7f15a81f1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15ada4d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a80cd5a0 0x7f15a8159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15ada4d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8158cf0 con 0x7f15a80b22a0 2023-11-28T12:06:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15ada4d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8159b10 con 0x7f15a806ab10 2023-11-28T12:06:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15ada4d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81f66c0 con 0x7f15a80cd5a0 2023-11-28T12:06:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15a67fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a80b22a0 0x7f15a81f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15a67fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a80b22a0 0x7f15a81f1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45276/0 (socket says 172.21.15.57:45276) 2023-11-28T12:06:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15a67fc700 1 -- 172.21.15.57:0/4060109137 learned_addr learned my addr 172.21.15.57:0/4060109137 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15a77fe700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a80cd5a0 0x7f15a8159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15a6ffd700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a806ab10 0x7f15a81f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15a67fc700 1 -- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a80cd5a0 msgr2=0x7f15a8159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15a67fc700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a80cd5a0 0x7f15a8159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:10.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15a67fc700 1 -- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a806ab10 msgr2=0x7f15a81f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:10.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15a67fc700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a806ab10 0x7f15a81f6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:10.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.211+0000 7f15a67fc700 1 -- 172.21.15.57:0/4060109137 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a0009b30 con 0x7f15a80b22a0 2023-11-28T12:06:10.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.212+0000 7f15a67fc700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a80b22a0 0x7f15a81f1990 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f159000d7d0 tx=0x7f159000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:10.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.212+0000 7f159ffff700 1 -- 172.21.15.57:0/4060109137 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15900125f0 con 0x7f15a80b22a0 2023-11-28T12:06:10.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.212+0000 7f159ffff700 1 -- 172.21.15.57:0/4060109137 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1590012750 con 0x7f15a80b22a0 2023-11-28T12:06:10.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.212+0000 7f15ada4d700 1 -- 172.21.15.57:0/4060109137 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a80b1ec0 con 0x7f15a80b22a0 2023-11-28T12:06:10.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.212+0000 7f159ffff700 1 -- 172.21.15.57:0/4060109137 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590012ae0 con 0x7f15a80b22a0 2023-11-28T12:06:10.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.212+0000 7f15ada4d700 1 -- 172.21.15.57:0/4060109137 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a815cad0 con 0x7f15a80b22a0 2023-11-28T12:06:10.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.213+0000 7f15ada4d700 1 -- 172.21.15.57:0/4060109137 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a80b2050 con 0x7f15a80b22a0 2023-11-28T12:06:10.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.213+0000 7f159ffff700 1 -- 172.21.15.57:0/4060109137 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1590012c40 con 0x7f15a80b22a0 2023-11-28T12:06:10.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.214+0000 7f159ffff700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15880682f0 0x7f158806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:10.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.214+0000 7f159ffff700 1 -- 172.21.15.57:0/4060109137 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f1590085ec0 con 0x7f15a80b22a0 2023-11-28T12:06:10.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.214+0000 7f15a6ffd700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15880682f0 0x7f158806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:10.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.215+0000 7f15a6ffd700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15880682f0 0x7f158806a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1598007fd0 tx=0x7f1598009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:10.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.218+0000 7f159ffff700 1 -- 172.21.15.57:0/4060109137 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f15900526a0 con 0x7f15a80b22a0 2023-11-28T12:06:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.462+0000 7f15ada4d700 1 -- 172.21.15.57:0/4060109137 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f15a80d2c80 con 0x7f15a80b22a0 2023-11-28T12:06:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.462+0000 7f159ffff700 1 -- 172.21.15.57:0/4060109137 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f1590012ed0 con 0x7f15a80b22a0 2023-11-28T12:06:10.463 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:06:10.464 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:06:10.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.464+0000 7f15ada4d700 1 -- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15880682f0 msgr2=0x7f158806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:10.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.464+0000 7f15ada4d700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15880682f0 0x7f158806a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1598007fd0 tx=0x7f1598009040 comp rx=0 tx=0).stop 2023-11-28T12:06:10.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.464+0000 7f15ada4d700 1 -- 172.21.15.57:0/4060109137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a80b22a0 msgr2=0x7f15a81f1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:10.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.464+0000 7f15ada4d700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a80b22a0 0x7f15a81f1990 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f159000d7d0 tx=0x7f159000dae0 comp rx=0 tx=0).stop 2023-11-28T12:06:10.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.464+0000 7f15ada4d700 1 -- 172.21.15.57:0/4060109137 shutdown_connections 2023-11-28T12:06:10.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.464+0000 7f15ada4d700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15880682f0 0x7f158806a7a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:10.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.464+0000 7f15ada4d700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a80cd5a0 0x7f15a8159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:10.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.464+0000 7f15ada4d700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a806ab10 0x7f15a81f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:10.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.464+0000 7f15ada4d700 1 --2- 172.21.15.57:0/4060109137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a80b22a0 0x7f15a81f1990 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:10.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.464+0000 7f15ada4d700 1 -- 172.21.15.57:0/4060109137 >> 172.21.15.57:0/4060109137 conn(0x7f15a806c520 msgr2=0x7f15a80b8570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:10.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.464+0000 7f15ada4d700 1 -- 172.21.15.57:0/4060109137 shutdown_connections 2023-11-28T12:06:10.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:10.464+0000 7f15ada4d700 1 -- 172.21.15.57:0/4060109137 wait complete. 2023-11-28T12:06:10.479 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:06:15.481 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:06:15.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.622+0000 7fa0192eb700 1 Processor -- start 2023-11-28T12:06:15.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.622+0000 7fa0192eb700 1 -- start start 2023-11-28T12:06:15.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.623+0000 7fa0192eb700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa0140d15f0 0x7fa01415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:15.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.623+0000 7fa0192eb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01415e010 0x7fa0140ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:15.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.623+0000 7fa0192eb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01415af30 con 0x7fa01406e400 2023-11-28T12:06:15.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.623+0000 7fa0192eb700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01415a9d0 con 0x7fa0140d15f0 2023-11-28T12:06:15.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.623+0000 7fa0192eb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0141584e0 con 0x7fa01415e010 2023-11-28T12:06:15.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.623+0000 7fa0127fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa0140d15f0 0x7fa01415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:15.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.623+0000 7fa0127fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa0140d15f0 0x7fa01415b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53934/0 (socket says 172.21.15.57:53934) 2023-11-28T12:06:15.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.623+0000 7fa0127fc700 1 -- 172.21.15.57:0/3974230856 learned_addr learned my addr 172.21.15.57:0/3974230856 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:15.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.623+0000 7fa0137fe700 1 --2- 172.21.15.57:0/3974230856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01415e010 0x7fa0140ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:15.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.624+0000 7fa0127fc700 1 -- 172.21.15.57:0/3974230856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01415e010 msgr2=0x7fa0140ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:06:15.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.624+0000 7fa0127fc700 1 --2- 172.21.15.57:0/3974230856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01415e010 0x7fa0140ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:15.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.624+0000 7fa0127fc700 1 -- 172.21.15.57:0/3974230856 >> v1:172.21.15.181:6789/0 conn(0x7fa01406e400 legacy=0x7fa01406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:15.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.624+0000 7fa0127fc700 1 -- 172.21.15.57:0/3974230856 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01415ef40 con 0x7fa0140d15f0 2023-11-28T12:06:15.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.624+0000 7fa011ffb700 1 -- 172.21.15.57:0/3974230856 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407642760 0 0) 0x7fa01415af30 con 0x7fa01406e400 2023-11-28T12:06:15.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.624+0000 7fa0127fc700 1 --2- 172.21.15.57:0/3974230856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0140d15f0 0x7fa01415b3c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa0040098f0 tx=0x7fa004009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843bf84d69678070 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:15.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.624+0000 7fa011ffb700 1 -- 172.21.15.57:0/3974230856 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004011830 con 0x7fa0140d15f0 2023-11-28T12:06:15.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.624+0000 7fa011ffb700 1 -- 172.21.15.57:0/3974230856 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa004011990 con 0x7fa0140d15f0 2023-11-28T12:06:15.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.624+0000 7fa011ffb700 1 -- 172.21.15.57:0/3974230856 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004011d20 con 0x7fa0140d15f0 2023-11-28T12:06:15.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.625+0000 7fa0192eb700 1 -- 172.21.15.57:0/3974230856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0140d15f0 msgr2=0x7fa01415b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:15.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.625+0000 7fa0192eb700 1 --2- 172.21.15.57:0/3974230856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0140d15f0 0x7fa01415b3c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa0040098f0 tx=0x7fa004009c00 comp rx=0 tx=0).stop 2023-11-28T12:06:15.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.625+0000 7fa0192eb700 1 -- 172.21.15.57:0/3974230856 shutdown_connections 2023-11-28T12:06:15.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.625+0000 7fa0192eb700 1 --2- 172.21.15.57:0/3974230856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01415e010 0x7fa0140ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:15.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.625+0000 7fa0192eb700 1 --2- 172.21.15.57:0/3974230856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0140d15f0 0x7fa01415b3c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:15.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.625+0000 7fa0192eb700 1 -- 172.21.15.57:0/3974230856 >> 172.21.15.57:0/3974230856 conn(0x7fa01406a270 msgr2=0x7fa01406a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:15.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.625+0000 7fa0192eb700 1 -- 172.21.15.57:0/3974230856 shutdown_connections 2023-11-28T12:06:15.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.625+0000 7fa0192eb700 1 -- 172.21.15.57:0/3974230856 wait complete. 2023-11-28T12:06:15.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.626+0000 7fa0192eb700 1 Processor -- start 2023-11-28T12:06:15.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.626+0000 7fa0192eb700 1 -- start start 2023-11-28T12:06:15.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.627+0000 7fa0192eb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406e400 0x7fa0140b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:15.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.627+0000 7fa0192eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0140d15f0 0x7fa01415b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:15.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.627+0000 7fa0192eb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa01415e010 0x7fa01415bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:15.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.627+0000 7fa0192eb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa014157370 con 0x7fa01415e010 2023-11-28T12:06:15.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.627+0000 7fa0192eb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140b2c40 con 0x7fa0140d15f0 2023-11-28T12:06:15.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.627+0000 7fa0192eb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140b0ca0 con 0x7fa01406e400 2023-11-28T12:06:15.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.627+0000 7fa0137fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa01415e010 0x7fa01415bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:15.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.627+0000 7fa0137fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa01415e010 0x7fa01415bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53946/0 (socket says 172.21.15.57:53946) 2023-11-28T12:06:15.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.627+0000 7fa0137fe700 1 -- 172.21.15.57:0/2439724928 learned_addr learned my addr 172.21.15.57:0/2439724928 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:15.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.627+0000 7fa012ffd700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406e400 0x7fa0140b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:15.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.627+0000 7fa0127fc700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0140d15f0 0x7fa01415b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:15.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.628+0000 7fa0137fe700 1 -- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406e400 msgr2=0x7fa0140b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:15.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.628+0000 7fa0137fe700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406e400 0x7fa0140b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:15.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.628+0000 7fa0137fe700 1 -- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0140d15f0 msgr2=0x7fa01415b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:15.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.628+0000 7fa0137fe700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0140d15f0 0x7fa01415b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:15.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.628+0000 7fa0137fe700 1 -- 172.21.15.57:0/2439724928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa004009550 con 0x7fa01415e010 2023-11-28T12:06:15.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.628+0000 7fa0127fc700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0140d15f0 0x7fa01415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:06:15.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.628+0000 7fa0137fe700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa01415e010 0x7fa01415bed0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00efc0 tx=0x7fa00c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:15.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.628+0000 7fa00b7fe700 1 -- 172.21.15.57:0/2439724928 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c009ba0 con 0x7fa01415e010 2023-11-28T12:06:15.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.628+0000 7fa00b7fe700 1 -- 172.21.15.57:0/2439724928 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c00f040 con 0x7fa01415e010 2023-11-28T12:06:15.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.628+0000 7fa0192eb700 1 -- 172.21.15.57:0/2439724928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01415ec60 con 0x7fa01415e010 2023-11-28T12:06:15.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.628+0000 7fa00b7fe700 1 -- 172.21.15.57:0/2439724928 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c014480 con 0x7fa01415e010 2023-11-28T12:06:15.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.628+0000 7fa0192eb700 1 -- 172.21.15.57:0/2439724928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01415e980 con 0x7fa01415e010 2023-11-28T12:06:15.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.629+0000 7fa0192eb700 1 -- 172.21.15.57:0/2439724928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01415abc0 con 0x7fa01415e010 2023-11-28T12:06:15.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.630+0000 7fa00b7fe700 1 -- 172.21.15.57:0/2439724928 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa00c0145e0 con 0x7fa01415e010 2023-11-28T12:06:15.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.630+0000 7fa00b7fe700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ff40682f0 0x7f9ff406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:15.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.630+0000 7fa00b7fe700 1 -- 172.21.15.57:0/2439724928 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fa00c0866a0 con 0x7fa01415e010 2023-11-28T12:06:15.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.630+0000 7fa012ffd700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ff40682f0 0x7f9ff406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:15.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.631+0000 7fa012ffd700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ff40682f0 0x7f9ff406a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc000f60 tx=0x7f9ffc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:15.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.635+0000 7fa00b7fe700 1 -- 172.21.15.57:0/2439724928 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa00c052e80 con 0x7fa01415e010 2023-11-28T12:06:15.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.882+0000 7fa0192eb700 1 -- 172.21.15.57:0/2439724928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa0140743f0 con 0x7fa01415e010 2023-11-28T12:06:15.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.882+0000 7fa00b7fe700 1 -- 172.21.15.57:0/2439724928 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fa00c0564a0 con 0x7fa01415e010 2023-11-28T12:06:15.883 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:06:15.883 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:06:15.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.884+0000 7fa0192eb700 1 -- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ff40682f0 msgr2=0x7f9ff406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:15.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.884+0000 7fa0192eb700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ff40682f0 0x7f9ff406a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc000f60 tx=0x7f9ffc009040 comp rx=0 tx=0).stop 2023-11-28T12:06:15.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.884+0000 7fa0192eb700 1 -- 172.21.15.57:0/2439724928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa01415e010 msgr2=0x7fa01415bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:15.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.884+0000 7fa0192eb700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa01415e010 0x7fa01415bed0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00efc0 tx=0x7fa00c00c3a0 comp rx=0 tx=0).stop 2023-11-28T12:06:15.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.884+0000 7fa0192eb700 1 -- 172.21.15.57:0/2439724928 shutdown_connections 2023-11-28T12:06:15.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.884+0000 7fa0192eb700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9ff40682f0 0x7f9ff406a7a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:15.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.884+0000 7fa0192eb700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa01415e010 0x7fa01415bed0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:15.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.884+0000 7fa0192eb700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406e400 0x7fa0140b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:15.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.884+0000 7fa0192eb700 1 --2- 172.21.15.57:0/2439724928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0140d15f0 0x7fa01415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:15.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.884+0000 7fa0192eb700 1 -- 172.21.15.57:0/2439724928 >> 172.21.15.57:0/2439724928 conn(0x7fa01406a270 msgr2=0x7fa01406a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:15.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.884+0000 7fa0192eb700 1 -- 172.21.15.57:0/2439724928 shutdown_connections 2023-11-28T12:06:15.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:15.884+0000 7fa0192eb700 1 -- 172.21.15.57:0/2439724928 wait complete. 2023-11-28T12:06:15.901 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:06:16.764 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:06:16.767 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:06:20.902 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:06:21.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.045+0000 7fd18afde700 1 Processor -- start 2023-11-28T12:06:21.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.045+0000 7fd18afde700 1 -- start start 2023-11-28T12:06:21.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.046+0000 7fd18afde700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd18406ab10 0x7fd18406aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:21.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.046+0000 7fd18afde700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1840cd5a0 0x7fd184160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:21.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.046+0000 7fd18afde700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1840d1ee0 con 0x7fd1840b22a0 2023-11-28T12:06:21.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.046+0000 7fd18afde700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18406b420 con 0x7fd18406ab10 2023-11-28T12:06:21.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.046+0000 7fd18afde700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1840cdb10 con 0x7fd1840cd5a0 2023-11-28T12:06:21.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.046+0000 7fd18957b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd1840b22a0 0x7fd184160840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36956/0 (socket says 172.21.15.57:36956) 2023-11-28T12:06:21.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.046+0000 7fd18957b700 1 -- 172.21.15.57:0/3099223700 learned_addr learned my addr 172.21.15.57:0/3099223700 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:06:21.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.046+0000 7fd183fff700 1 --2- 172.21.15.57:0/3099223700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1840cd5a0 0x7fd184160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:21.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.046+0000 7fd188d7a700 1 --2- 172.21.15.57:0/3099223700 >> v2:172.21.15.181:3300/0 conn(0x7fd18406ab10 0x7fd18406aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:21.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.047+0000 7fd1837fe700 1 -- 172.21.15.57:0/3099223700 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215007608 0 0) 0x7fd1840d1ee0 con 0x7fd1840b22a0 2023-11-28T12:06:21.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.047+0000 7fd1837fe700 1 -- 172.21.15.57:0/3099223700 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1700035b0 con 0x7fd1840b22a0 2023-11-28T12:06:21.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.047+0000 7fd183fff700 1 -- 172.21.15.57:0/3099223700 >> v2:172.21.15.181:3300/0 conn(0x7fd18406ab10 msgr2=0x7fd18406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:06:21.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.047+0000 7fd183fff700 1 --2- 172.21.15.57:0/3099223700 >> v2:172.21.15.181:3300/0 conn(0x7fd18406ab10 0x7fd18406aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:21.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.047+0000 7fd1837fe700 1 -- 172.21.15.57:0/3099223700 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 972707778 0 0) 0x7fd1700035b0 con 0x7fd1840b22a0 2023-11-28T12:06:21.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.047+0000 7fd183fff700 1 -- 172.21.15.57:0/3099223700 >> v1:172.21.15.57:6789/0 conn(0x7fd1840b22a0 legacy=0x7fd184160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:21.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.047+0000 7fd183fff700 1 -- 172.21.15.57:0/3099223700 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1840d1890 con 0x7fd1840cd5a0 2023-11-28T12:06:21.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.047+0000 7fd188d7a700 1 --2- 172.21.15.57:0/3099223700 >> v2:172.21.15.181:3300/0 conn(0x7fd18406ab10 0x7fd18406aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:06:21.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.047+0000 7fd183fff700 1 --2- 172.21.15.57:0/3099223700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1840cd5a0 0x7fd184160300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009840 tx=0x7fd16c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a5f44439fa1cb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:21.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.047+0000 7fd1837fe700 1 -- 172.21.15.57:0/3099223700 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0019f0 con 0x7fd1840cd5a0 2023-11-28T12:06:21.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.047+0000 7fd1837fe700 1 -- 172.21.15.57:0/3099223700 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16c004030 con 0x7fd1840cd5a0 2023-11-28T12:06:21.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.047+0000 7fd1837fe700 1 -- 172.21.15.57:0/3099223700 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c0043c0 con 0x7fd1840cd5a0 2023-11-28T12:06:21.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.048+0000 7fd18afde700 1 -- 172.21.15.57:0/3099223700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1840cd5a0 msgr2=0x7fd184160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:21.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.048+0000 7fd18afde700 1 --2- 172.21.15.57:0/3099223700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1840cd5a0 0x7fd184160300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009840 tx=0x7fd16c009b50 comp rx=0 tx=0).stop 2023-11-28T12:06:21.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.048+0000 7fd18afde700 1 -- 172.21.15.57:0/3099223700 shutdown_connections 2023-11-28T12:06:21.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.048+0000 7fd18afde700 1 --2- 172.21.15.57:0/3099223700 >> v2:172.21.15.181:3300/0 conn(0x7fd18406ab10 0x7fd18406aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:21.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.048+0000 7fd18afde700 1 --2- 172.21.15.57:0/3099223700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1840cd5a0 0x7fd184160300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:21.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.048+0000 7fd18afde700 1 -- 172.21.15.57:0/3099223700 >> 172.21.15.57:0/3099223700 conn(0x7fd18406c520 msgr2=0x7fd18406c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:21.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.048+0000 7fd18afde700 1 -- 172.21.15.57:0/3099223700 shutdown_connections 2023-11-28T12:06:21.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.048+0000 7fd18afde700 1 -- 172.21.15.57:0/3099223700 wait complete. 2023-11-28T12:06:21.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.049+0000 7fd18afde700 1 Processor -- start 2023-11-28T12:06:21.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.049+0000 7fd18afde700 1 -- start start 2023-11-28T12:06:21.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.050+0000 7fd18afde700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd18406ab10 0x7fd18415b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:21.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.050+0000 7fd18afde700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1840b22a0 0x7fd1841589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:21.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.050+0000 7fd18afde700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1840cd5a0 0x7fd184155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:21.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.050+0000 7fd18afde700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd184159e10 con 0x7fd18406ab10 2023-11-28T12:06:21.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.050+0000 7fd18afde700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd184158ef0 con 0x7fd1840cd5a0 2023-11-28T12:06:21.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.050+0000 7fd18afde700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd184159190 con 0x7fd1840b22a0 2023-11-28T12:06:21.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.050+0000 7fd188d7a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd18406ab10 0x7fd18415b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:21.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.050+0000 7fd188d7a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd18406ab10 0x7fd18415b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53954/0 (socket says 172.21.15.57:53954) 2023-11-28T12:06:21.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.050+0000 7fd188d7a700 1 -- 172.21.15.57:0/126493602 learned_addr learned my addr 172.21.15.57:0/126493602 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:21.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.051+0000 7fd183fff700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1840b22a0 0x7fd1841589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:21.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.051+0000 7fd18957b700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1840cd5a0 0x7fd184155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:21.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.051+0000 7fd188d7a700 1 -- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1840b22a0 msgr2=0x7fd1841589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:21.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.051+0000 7fd188d7a700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1840b22a0 0x7fd1841589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:21.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.051+0000 7fd188d7a700 1 -- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1840cd5a0 msgr2=0x7fd184155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:21.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.051+0000 7fd188d7a700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1840cd5a0 0x7fd184155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:21.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.051+0000 7fd188d7a700 1 -- 172.21.15.57:0/126493602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1740094a0 con 0x7fd18406ab10 2023-11-28T12:06:21.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.051+0000 7fd188d7a700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd18406ab10 0x7fd18415b720 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd17400ebe0 tx=0x7fd17400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:21.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.051+0000 7fd1817fa700 1 -- 172.21.15.57:0/126493602 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174009760 con 0x7fd18406ab10 2023-11-28T12:06:21.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.051+0000 7fd1817fa700 1 -- 172.21.15.57:0/126493602 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17400c850 con 0x7fd18406ab10 2023-11-28T12:06:21.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.051+0000 7fd18afde700 1 -- 172.21.15.57:0/126493602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16c0094a0 con 0x7fd18406ab10 2023-11-28T12:06:21.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.051+0000 7fd1817fa700 1 -- 172.21.15.57:0/126493602 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17400cbe0 con 0x7fd18406ab10 2023-11-28T12:06:21.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.051+0000 7fd18afde700 1 -- 172.21.15.57:0/126493602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18415cad0 con 0x7fd18406ab10 2023-11-28T12:06:21.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.052+0000 7fd18afde700 1 -- 172.21.15.57:0/126493602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1840d1b70 con 0x7fd18406ab10 2023-11-28T12:06:21.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.053+0000 7fd1817fa700 1 -- 172.21.15.57:0/126493602 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd174019030 con 0x7fd18406ab10 2023-11-28T12:06:21.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.053+0000 7fd1817fa700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1640682f0 0x7fd16406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:21.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.053+0000 7fd1817fa700 1 -- 172.21.15.57:0/126493602 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fd174085c90 con 0x7fd18406ab10 2023-11-28T12:06:21.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.053+0000 7fd183fff700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1640682f0 0x7fd16406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:21.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.054+0000 7fd183fff700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1640682f0 0x7fd16406a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0090b0 tx=0x7fd16c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:21.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.058+0000 7fd1817fa700 1 -- 172.21.15.57:0/126493602 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd174052380 con 0x7fd18406ab10 2023-11-28T12:06:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.305+0000 7fd18afde700 1 -- 172.21.15.57:0/126493602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd1840b1ec0 con 0x7fd18406ab10 2023-11-28T12:06:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.305+0000 7fd1817fa700 1 -- 172.21.15.57:0/126493602 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fd1740559a0 con 0x7fd18406ab10 2023-11-28T12:06:21.307 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:06:21.307 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:06:21.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.307+0000 7fd18afde700 1 -- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1640682f0 msgr2=0x7fd16406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.307+0000 7fd18afde700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1640682f0 0x7fd16406a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0090b0 tx=0x7fd16c008030 comp rx=0 tx=0).stop 2023-11-28T12:06:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.307+0000 7fd18afde700 1 -- 172.21.15.57:0/126493602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd18406ab10 msgr2=0x7fd18415b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.307+0000 7fd18afde700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd18406ab10 0x7fd18415b720 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd17400ebe0 tx=0x7fd17400eef0 comp rx=0 tx=0).stop 2023-11-28T12:06:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.307+0000 7fd18afde700 1 -- 172.21.15.57:0/126493602 shutdown_connections 2023-11-28T12:06:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.307+0000 7fd18afde700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd1640682f0 0x7fd16406a7a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.307+0000 7fd18afde700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1840cd5a0 0x7fd184155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.307+0000 7fd18afde700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd18406ab10 0x7fd18415b720 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.307+0000 7fd18afde700 1 --2- 172.21.15.57:0/126493602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1840b22a0 0x7fd1841589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.307+0000 7fd18afde700 1 -- 172.21.15.57:0/126493602 >> 172.21.15.57:0/126493602 conn(0x7fd18406c520 msgr2=0x7fd1840b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.307+0000 7fd18afde700 1 -- 172.21.15.57:0/126493602 shutdown_connections 2023-11-28T12:06:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:21.307+0000 7fd18afde700 1 -- 172.21.15.57:0/126493602 wait complete. 2023-11-28T12:06:21.322 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:06:26.323 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:06:26.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.465+0000 7f456ec74700 1 Processor -- start 2023-11-28T12:06:26.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.465+0000 7f456ec74700 1 -- start start 2023-11-28T12:06:26.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.466+0000 7f456ec74700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f456806e400 0x7f45680b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:26.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.466+0000 7f456ec74700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4568160e70 0x7f456815c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:26.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.466+0000 7f456ec74700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456806e900 con 0x7f45680d0c60 2023-11-28T12:06:26.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.466+0000 7f456ec74700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4568160390 con 0x7f456806e400 2023-11-28T12:06:26.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.466+0000 7f456ec74700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4568160670 con 0x7f4568160e70 2023-11-28T12:06:26.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.466+0000 7f4567fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f456806e400 0x7f45680b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:26.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.466+0000 7f456d211700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4568160e70 0x7f456815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:26.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.466+0000 7f456ca10700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f45680d0c60 0x7f45680d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:39128/0 (socket says 172.21.15.57:39128) 2023-11-28T12:06:26.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.467+0000 7f456ca10700 1 -- 172.21.15.57:0/3823387954 learned_addr learned my addr 172.21.15.57:0/3823387954 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:06:26.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.467+0000 7f4567fff700 1 -- 172.21.15.57:0/3823387954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4568160e70 msgr2=0x7f456815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:06:26.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.467+0000 7f4567fff700 1 --2- 172.21.15.57:0/3823387954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4568160e70 0x7f456815c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:26.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.467+0000 7f4567fff700 1 -- 172.21.15.57:0/3823387954 >> v1:172.21.15.181:6789/0 conn(0x7f45680d0c60 legacy=0x7f45680d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:26.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.467+0000 7f4567fff700 1 -- 172.21.15.57:0/3823387954 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4568160950 con 0x7f456806e400 2023-11-28T12:06:26.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.467+0000 7f45677fe700 1 -- 172.21.15.57:0/3823387954 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794009047 0 0) 0x7f456806e900 con 0x7f45680d0c60 2023-11-28T12:06:26.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.467+0000 7f456d211700 1 --2- 172.21.15.57:0/3823387954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4568160e70 0x7f456815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:06:26.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.467+0000 7f4567fff700 1 --2- 172.21.15.57:0/3823387954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f456806e400 0x7f45680b5be0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4554009840 tx=0x7f4554009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10be92e25c40afcd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:26.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.467+0000 7f45677fe700 1 -- 172.21.15.57:0/3823387954 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554011830 con 0x7f456806e400 2023-11-28T12:06:26.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.467+0000 7f45677fe700 1 -- 172.21.15.57:0/3823387954 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4554011990 con 0x7f456806e400 2023-11-28T12:06:26.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.467+0000 7f45677fe700 1 -- 172.21.15.57:0/3823387954 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554011d20 con 0x7f456806e400 2023-11-28T12:06:26.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.468+0000 7f456ec74700 1 -- 172.21.15.57:0/3823387954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f456806e400 msgr2=0x7f45680b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:26.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.468+0000 7f456ec74700 1 --2- 172.21.15.57:0/3823387954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f456806e400 0x7f45680b5be0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4554009840 tx=0x7f4554009b50 comp rx=0 tx=0).stop 2023-11-28T12:06:26.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.468+0000 7f456ec74700 1 -- 172.21.15.57:0/3823387954 shutdown_connections 2023-11-28T12:06:26.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.468+0000 7f456ec74700 1 --2- 172.21.15.57:0/3823387954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4568160e70 0x7f456815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:26.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.468+0000 7f456ec74700 1 --2- 172.21.15.57:0/3823387954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f456806e400 0x7f45680b5be0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:26.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.468+0000 7f456ec74700 1 -- 172.21.15.57:0/3823387954 >> 172.21.15.57:0/3823387954 conn(0x7f456806a270 msgr2=0x7f456806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:26.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.468+0000 7f456ec74700 1 -- 172.21.15.57:0/3823387954 shutdown_connections 2023-11-28T12:06:26.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.468+0000 7f456ec74700 1 -- 172.21.15.57:0/3823387954 wait complete. 2023-11-28T12:06:26.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.469+0000 7f456ec74700 1 Processor -- start 2023-11-28T12:06:26.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.469+0000 7f456ec74700 1 -- start start 2023-11-28T12:06:26.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.470+0000 7f456ec74700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f456806e400 0x7f45680b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:26.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.470+0000 7f456ec74700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45680d0c60 0x7f45680b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:26.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.470+0000 7f456ec74700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4568160e70 0x7f4568151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:26.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.470+0000 7f456ec74700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45680b0f00 con 0x7f4568160e70 2023-11-28T12:06:26.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.470+0000 7f456ec74700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45680b11e0 con 0x7f456806e400 2023-11-28T12:06:26.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.470+0000 7f456ec74700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45680b14c0 con 0x7f45680d0c60 2023-11-28T12:06:26.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.470+0000 7f456d211700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4568160e70 0x7f4568151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:26.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.470+0000 7f456d211700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4568160e70 0x7f4568151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35762/0 (socket says 172.21.15.57:35762) 2023-11-28T12:06:26.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.470+0000 7f456d211700 1 -- 172.21.15.57:0/1651616211 learned_addr learned my addr 172.21.15.57:0/1651616211 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:26.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.470+0000 7f4567fff700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45680d0c60 0x7f45680b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:26.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.471+0000 7f456ca10700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f456806e400 0x7f45680b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:26.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.471+0000 7f456d211700 1 -- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45680d0c60 msgr2=0x7f45680b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:26.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.471+0000 7f456d211700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45680d0c60 0x7f45680b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:26.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.471+0000 7f456d211700 1 -- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f456806e400 msgr2=0x7f45680b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:26.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.471+0000 7f456d211700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f456806e400 0x7f45680b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:26.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.471+0000 7f456d211700 1 -- 172.21.15.57:0/1651616211 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4560009a70 con 0x7f4568160e70 2023-11-28T12:06:26.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.471+0000 7f456d211700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4568160e70 0x7f4568151ce0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f456000efb0 tx=0x7f456000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:26.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.471+0000 7f45657fa700 1 -- 172.21.15.57:0/1651616211 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456000cc90 con 0x7f4568160e70 2023-11-28T12:06:26.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.471+0000 7f45657fa700 1 -- 172.21.15.57:0/1651616211 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456000f040 con 0x7f4568160e70 2023-11-28T12:06:26.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.471+0000 7f456ec74700 1 -- 172.21.15.57:0/1651616211 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45540094a0 con 0x7f4568160e70 2023-11-28T12:06:26.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.471+0000 7f45657fa700 1 -- 172.21.15.57:0/1651616211 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560014480 con 0x7f4568160e70 2023-11-28T12:06:26.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.471+0000 7f456ec74700 1 -- 172.21.15.57:0/1651616211 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456815ceb0 con 0x7f4568160e70 2023-11-28T12:06:26.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.472+0000 7f456ec74700 1 -- 172.21.15.57:0/1651616211 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45680d0740 con 0x7f4568160e70 2023-11-28T12:06:26.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.473+0000 7f45657fa700 1 -- 172.21.15.57:0/1651616211 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f45600145e0 con 0x7f4568160e70 2023-11-28T12:06:26.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.474+0000 7f45657fa700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f454c0682f0 0x7f454c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:26.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.474+0000 7f45657fa700 1 -- 172.21.15.57:0/1651616211 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f4560086ff0 con 0x7f4568160e70 2023-11-28T12:06:26.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.474+0000 7f456ca10700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f454c0682f0 0x7f454c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:26.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.475+0000 7f456ca10700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f454c0682f0 0x7f454c06a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f455c007fd0 tx=0x7f455c007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:26.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.477+0000 7f45657fa700 1 -- 172.21.15.57:0/1651616211 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4560053760 con 0x7f4568160e70 2023-11-28T12:06:26.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.721+0000 7f456ec74700 1 -- 172.21.15.57:0/1651616211 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f456815d040 con 0x7f4568160e70 2023-11-28T12:06:26.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.721+0000 7f45657fa700 1 -- 172.21.15.57:0/1651616211 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f45600180c0 con 0x7f4568160e70 2023-11-28T12:06:26.722 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:06:26.723 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:06:26.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.723+0000 7f456ec74700 1 -- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f454c0682f0 msgr2=0x7f454c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.723+0000 7f456ec74700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f454c0682f0 0x7f454c06a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f455c007fd0 tx=0x7f455c007de0 comp rx=0 tx=0).stop 2023-11-28T12:06:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.723+0000 7f456ec74700 1 -- 172.21.15.57:0/1651616211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4568160e70 msgr2=0x7f4568151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.723+0000 7f456ec74700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4568160e70 0x7f4568151ce0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f456000efb0 tx=0x7f456000c3a0 comp rx=0 tx=0).stop 2023-11-28T12:06:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.723+0000 7f456ec74700 1 -- 172.21.15.57:0/1651616211 shutdown_connections 2023-11-28T12:06:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.723+0000 7f456ec74700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f454c0682f0 0x7f454c06a7a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.723+0000 7f456ec74700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4568160e70 0x7f4568151ce0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.723+0000 7f456ec74700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f456806e400 0x7f45680b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.723+0000 7f456ec74700 1 --2- 172.21.15.57:0/1651616211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45680d0c60 0x7f45680b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.723+0000 7f456ec74700 1 -- 172.21.15.57:0/1651616211 >> 172.21.15.57:0/1651616211 conn(0x7f456806a270 msgr2=0x7f456815ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.723+0000 7f456ec74700 1 -- 172.21.15.57:0/1651616211 shutdown_connections 2023-11-28T12:06:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:26.723+0000 7f456ec74700 1 -- 172.21.15.57:0/1651616211 wait complete. 2023-11-28T12:06:26.738 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:06:31.739 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:06:31.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.882+0000 7f83167b7700 1 Processor -- start 2023-11-28T12:06:31.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.882+0000 7f83167b7700 1 -- start start 2023-11-28T12:06:31.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.883+0000 7f83167b7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f831006e400 0x7f831006e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:31.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.883+0000 7f83167b7700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f831015e010 0x7f83100ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:31.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.883+0000 7f83167b7700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831015af30 con 0x7f83100d15f0 2023-11-28T12:06:31.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.883+0000 7f83167b7700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831015a9d0 con 0x7f831015e010 2023-11-28T12:06:31.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.883+0000 7f83167b7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83101584e0 con 0x7f831006e400 2023-11-28T12:06:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.883+0000 7f8314d54700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f831015e010 0x7f83100ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.883+0000 7f830f7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f83100d15f0 0x7f831015b3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:52926/0 (socket says 172.21.15.57:52926) 2023-11-28T12:06:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.883+0000 7f8314d54700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f831015e010 0x7f83100ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35764/0 (socket says 172.21.15.57:35764) 2023-11-28T12:06:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.883+0000 7f830f7fe700 1 -- 172.21.15.57:0/1960053530 learned_addr learned my addr 172.21.15.57:0/1960053530 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:06:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.884+0000 7f830ffff700 1 --2- 172.21.15.57:0/1960053530 >> v2:172.21.15.181:3300/0 conn(0x7f831006e400 0x7f831006e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.884+0000 7f8314d54700 1 -- 172.21.15.57:0/1960053530 >> v2:172.21.15.181:3300/0 conn(0x7f831006e400 msgr2=0x7f831006e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:06:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.884+0000 7f8314d54700 1 --2- 172.21.15.57:0/1960053530 >> v2:172.21.15.181:3300/0 conn(0x7f831006e400 0x7f831006e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.884+0000 7f8314d54700 1 -- 172.21.15.57:0/1960053530 >> v1:172.21.15.57:6789/0 conn(0x7f83100d15f0 legacy=0x7f831015b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.884+0000 7f8314d54700 1 -- 172.21.15.57:0/1960053530 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831015ef40 con 0x7f831015e010 2023-11-28T12:06:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.884+0000 7f830effd700 1 -- 172.21.15.57:0/1960053530 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145034783 0 0) 0x7f831015af30 con 0x7f83100d15f0 2023-11-28T12:06:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.884+0000 7f8314d54700 1 --2- 172.21.15.57:0/1960053530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f831015e010 0x7f83100ce970 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f83100cd900 tx=0x7f830800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bad3f96e79f6729 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.884+0000 7f830effd700 1 -- 172.21.15.57:0/1960053530 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830800e070 con 0x7f831015e010 2023-11-28T12:06:31.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.884+0000 7f830effd700 1 -- 172.21.15.57:0/1960053530 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830800bbd0 con 0x7f831015e010 2023-11-28T12:06:31.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.884+0000 7f830effd700 1 -- 172.21.15.57:0/1960053530 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308012430 con 0x7f831015e010 2023-11-28T12:06:31.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.885+0000 7f83167b7700 1 -- 172.21.15.57:0/1960053530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f831015e010 msgr2=0x7f83100ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:31.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.885+0000 7f83167b7700 1 --2- 172.21.15.57:0/1960053530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f831015e010 0x7f83100ce970 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f83100cd900 tx=0x7f830800b330 comp rx=0 tx=0).stop 2023-11-28T12:06:31.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.885+0000 7f83167b7700 1 -- 172.21.15.57:0/1960053530 shutdown_connections 2023-11-28T12:06:31.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.885+0000 7f83167b7700 1 --2- 172.21.15.57:0/1960053530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f831015e010 0x7f83100ce970 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:31.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.885+0000 7f83167b7700 1 --2- 172.21.15.57:0/1960053530 >> v2:172.21.15.181:3300/0 conn(0x7f831006e400 0x7f831006e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.885+0000 7f83167b7700 1 -- 172.21.15.57:0/1960053530 >> 172.21.15.57:0/1960053530 conn(0x7f831006a270 msgr2=0x7f831006a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.885+0000 7f83167b7700 1 -- 172.21.15.57:0/1960053530 shutdown_connections 2023-11-28T12:06:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.885+0000 7f83167b7700 1 -- 172.21.15.57:0/1960053530 wait complete. 2023-11-28T12:06:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.886+0000 7f83167b7700 1 Processor -- start 2023-11-28T12:06:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.886+0000 7f83167b7700 1 -- start start 2023-11-28T12:06:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f83167b7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f831006e400 0x7f83100cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f83167b7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83100d15f0 0x7f83100cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f83167b7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f831015e010 0x7f83100b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f83167b7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83100b3000 con 0x7f831015e010 2023-11-28T12:06:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f83167b7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83100cfef0 con 0x7f83100d15f0 2023-11-28T12:06:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f83167b7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83100d01d0 con 0x7f831006e400 2023-11-28T12:06:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f8314d54700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f831015e010 0x7f83100b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:31.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f8314d54700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f831015e010 0x7f83100b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35776/0 (socket says 172.21.15.57:35776) 2023-11-28T12:06:31.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f8314d54700 1 -- 172.21.15.57:0/3171162734 learned_addr learned my addr 172.21.15.57:0/3171162734 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:31.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f830f7fe700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83100d15f0 0x7f83100cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:31.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f830ffff700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f831006e400 0x7f83100cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:31.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f8314d54700 1 -- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f831006e400 msgr2=0x7f83100cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:31.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f8314d54700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f831006e400 0x7f83100cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:31.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f8314d54700 1 -- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83100d15f0 msgr2=0x7f83100cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:31.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f8314d54700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83100d15f0 0x7f83100cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:31.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.887+0000 7f8314d54700 1 -- 172.21.15.57:0/3171162734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8308009a70 con 0x7f831015e010 2023-11-28T12:06:31.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.888+0000 7f830ffff700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f831006e400 0x7f83100cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:06:31.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.888+0000 7f830f7fe700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83100d15f0 0x7f83100cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:06:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.888+0000 7f8314d54700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f831015e010 0x7f83100b0760 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8308014040 tx=0x7f8308004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.888+0000 7f830cff9700 1 -- 172.21.15.57:0/3171162734 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308012430 con 0x7f831015e010 2023-11-28T12:06:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.888+0000 7f830cff9700 1 -- 172.21.15.57:0/3171162734 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8308020070 con 0x7f831015e010 2023-11-28T12:06:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.888+0000 7f83167b7700 1 -- 172.21.15.57:0/3171162734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831015e980 con 0x7f831015e010 2023-11-28T12:06:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.888+0000 7f830cff9700 1 -- 172.21.15.57:0/3171162734 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308004280 con 0x7f831015e010 2023-11-28T12:06:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.888+0000 7f83167b7700 1 -- 172.21.15.57:0/3171162734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831006ed10 con 0x7f831015e010 2023-11-28T12:06:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.889+0000 7f83167b7700 1 -- 172.21.15.57:0/3171162734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831015ec60 con 0x7f831015e010 2023-11-28T12:06:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.889+0000 7f830cff9700 1 -- 172.21.15.57:0/3171162734 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f830802e070 con 0x7f831015e010 2023-11-28T12:06:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.890+0000 7f830cff9700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82fc0682f0 0x7f82fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.890+0000 7f830cff9700 1 -- 172.21.15.57:0/3171162734 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f8308022070 con 0x7f831015e010 2023-11-28T12:06:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.890+0000 7f830ffff700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82fc0682f0 0x7f82fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:31.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.891+0000 7f830ffff700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82fc0682f0 0x7f82fc06a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8300009a00 tx=0x7f8300008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:31.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:31.894+0000 7f830cff9700 1 -- 172.21.15.57:0/3171162734 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8308089e60 con 0x7f831015e010 2023-11-28T12:06:32.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.140+0000 7f83167b7700 1 -- 172.21.15.57:0/3171162734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f83100743f0 con 0x7f831015e010 2023-11-28T12:06:32.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.140+0000 7f830cff9700 1 -- 172.21.15.57:0/3171162734 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f8308054390 con 0x7f831015e010 2023-11-28T12:06:32.141 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:06:32.141 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:06:32.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.142+0000 7f83167b7700 1 -- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82fc0682f0 msgr2=0x7f82fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:32.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.142+0000 7f83167b7700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82fc0682f0 0x7f82fc06a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8300009a00 tx=0x7f8300008040 comp rx=0 tx=0).stop 2023-11-28T12:06:32.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.142+0000 7f83167b7700 1 -- 172.21.15.57:0/3171162734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f831015e010 msgr2=0x7f83100b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:32.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.142+0000 7f83167b7700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f831015e010 0x7f83100b0760 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8308014040 tx=0x7f8308004ad0 comp rx=0 tx=0).stop 2023-11-28T12:06:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.142+0000 7f83167b7700 1 -- 172.21.15.57:0/3171162734 shutdown_connections 2023-11-28T12:06:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.142+0000 7f83167b7700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f82fc0682f0 0x7f82fc06a7a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.142+0000 7f83167b7700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f831015e010 0x7f83100b0760 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.142+0000 7f83167b7700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f831006e400 0x7f83100cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.142+0000 7f83167b7700 1 --2- 172.21.15.57:0/3171162734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83100d15f0 0x7f83100cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.142+0000 7f83167b7700 1 -- 172.21.15.57:0/3171162734 >> 172.21.15.57:0/3171162734 conn(0x7f831006a270 msgr2=0x7f83101553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.142+0000 7f83167b7700 1 -- 172.21.15.57:0/3171162734 shutdown_connections 2023-11-28T12:06:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:32.142+0000 7f83167b7700 1 -- 172.21.15.57:0/3171162734 wait complete. 2023-11-28T12:06:32.157 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:06:37.157 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:06:37.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.300+0000 7f2701efb700 1 Processor -- start 2023-11-28T12:06:37.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.300+0000 7f2701efb700 1 -- start start 2023-11-28T12:06:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.301+0000 7f2701efb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f26fc0d15f0 0x7f26fc15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.301+0000 7f2701efb700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26fc15af30 con 0x7f26fc15e010 2023-11-28T12:06:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.301+0000 7f2701efb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26fc15ef40 con 0x7f26fc06e400 2023-11-28T12:06:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.301+0000 7f2701efb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26fc15a9d0 con 0x7f26fc0d15f0 2023-11-28T12:06:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.301+0000 7f26fbfff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f26fc15e010 0x7f26fc0ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:59014/0 (socket says 172.21.15.57:59014) 2023-11-28T12:06:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.301+0000 7f26fbfff700 1 -- 172.21.15.57:0/3836058906 learned_addr learned my addr 172.21.15.57:0/3836058906 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:06:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.301+0000 7f26faffd700 1 --2- 172.21.15.57:0/3836058906 >> v2:172.21.15.181:3300/0 conn(0x7f26fc0d15f0 0x7f26fc15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:37.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.301+0000 7f26fa7fc700 1 -- 172.21.15.57:0/3836058906 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773385980 0 0) 0x7f26fc15af30 con 0x7f26fc15e010 2023-11-28T12:06:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.301+0000 7f26fa7fc700 1 -- 172.21.15.57:0/3836058906 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e80035b0 con 0x7f26fc15e010 2023-11-28T12:06:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.302+0000 7f26fa7fc700 1 -- 172.21.15.57:0/3836058906 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658669995 0 0) 0x7f26fc15ef40 con 0x7f26fc06e400 2023-11-28T12:06:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.302+0000 7f26fa7fc700 1 -- 172.21.15.57:0/3836058906 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26fc15af30 con 0x7f26fc06e400 2023-11-28T12:06:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.302+0000 7f26faffd700 1 -- 172.21.15.57:0/3836058906 >> v1:172.21.15.181:6789/0 conn(0x7f26fc06e400 legacy=0x7f26fc06e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.302+0000 7f26faffd700 1 -- 172.21.15.57:0/3836058906 >> v1:172.21.15.57:6789/0 conn(0x7f26fc15e010 legacy=0x7f26fc0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.302+0000 7f26fa7fc700 1 -- 172.21.15.57:0/3836058906 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3214552788 0 0) 0x7f26e80035b0 con 0x7f26fc15e010 2023-11-28T12:06:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.302+0000 7f26faffd700 1 -- 172.21.15.57:0/3836058906 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc15e980 con 0x7f26fc0d15f0 2023-11-28T12:06:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.302+0000 7f26faffd700 1 --2- 172.21.15.57:0/3836058906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc0d15f0 0x7f26fc15b3c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009840 tx=0x7f26e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf8b0b1a24e2e8e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:37.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.302+0000 7f26fa7fc700 1 -- 172.21.15.57:0/3836058906 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4011830 con 0x7f26fc0d15f0 2023-11-28T12:06:37.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.302+0000 7f26fa7fc700 1 -- 172.21.15.57:0/3836058906 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e4011990 con 0x7f26fc0d15f0 2023-11-28T12:06:37.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.302+0000 7f26fa7fc700 1 -- 172.21.15.57:0/3836058906 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4011d20 con 0x7f26fc0d15f0 2023-11-28T12:06:37.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.303+0000 7f2701efb700 1 -- 172.21.15.57:0/3836058906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc0d15f0 msgr2=0x7f26fc15b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:37.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.303+0000 7f2701efb700 1 --2- 172.21.15.57:0/3836058906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc0d15f0 0x7f26fc15b3c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009840 tx=0x7f26e4009b50 comp rx=0 tx=0).stop 2023-11-28T12:06:37.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.303+0000 7f2701efb700 1 -- 172.21.15.57:0/3836058906 shutdown_connections 2023-11-28T12:06:37.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.303+0000 7f2701efb700 1 --2- 172.21.15.57:0/3836058906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc0d15f0 0x7f26fc15b3c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:37.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.303+0000 7f2701efb700 1 -- 172.21.15.57:0/3836058906 >> 172.21.15.57:0/3836058906 conn(0x7f26fc06a270 msgr2=0x7f26fc06a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:37.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.303+0000 7f2701efb700 1 -- 172.21.15.57:0/3836058906 shutdown_connections 2023-11-28T12:06:37.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.303+0000 7f2701efb700 1 -- 172.21.15.57:0/3836058906 wait complete. 2023-11-28T12:06:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.304+0000 7f2701efb700 1 Processor -- start 2023-11-28T12:06:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.305+0000 7f2701efb700 1 -- start start 2023-11-28T12:06:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f2701efb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26fc06e400 0x7f26fc0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f2701efb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc0d15f0 0x7f26fc15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f2701efb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26fc15e010 0x7f26fc0cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f2701efb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc15c600 con 0x7f26fc06e400 2023-11-28T12:06:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f2701efb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc15c8e0 con 0x7f26fc0d15f0 2023-11-28T12:06:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f2701efb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc15cbc0 con 0x7f26fc15e010 2023-11-28T12:06:37.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26fb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26fc06e400 0x7f26fc0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26fb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26fc06e400 0x7f26fc0b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48322/0 (socket says 172.21.15.57:48322) 2023-11-28T12:06:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26fb7fe700 1 -- 172.21.15.57:0/3350059065 learned_addr learned my addr 172.21.15.57:0/3350059065 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26fbfff700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26fc15e010 0x7f26fc0cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26faffd700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc0d15f0 0x7f26fc15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26fb7fe700 1 -- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26fc15e010 msgr2=0x7f26fc0cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26fb7fe700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26fc15e010 0x7f26fc0cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26fb7fe700 1 -- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc0d15f0 msgr2=0x7f26fc15d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:37.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26fb7fe700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc0d15f0 0x7f26fc15d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26fb7fe700 1 -- 172.21.15.57:0/3350059065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e40094a0 con 0x7f26fc06e400 2023-11-28T12:06:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26faffd700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc0d15f0 0x7f26fc15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:06:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26fb7fe700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26fc06e400 0x7f26fc0b1c10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00c520 tx=0x7f26ec00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26f3fff700 1 -- 172.21.15.57:0/3350059065 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0052d0 con 0x7f26fc06e400 2023-11-28T12:06:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26f3fff700 1 -- 172.21.15.57:0/3350059065 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec005430 con 0x7f26fc06e400 2023-11-28T12:06:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f2701efb700 1 -- 172.21.15.57:0/3350059065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26fc15abc0 con 0x7f26fc06e400 2023-11-28T12:06:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f26f3fff700 1 -- 172.21.15.57:0/3350059065 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0038c0 con 0x7f26fc06e400 2023-11-28T12:06:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.306+0000 7f2701efb700 1 -- 172.21.15.57:0/3350059065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc15ec60 con 0x7f26fc06e400 2023-11-28T12:06:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.308+0000 7f2701efb700 1 -- 172.21.15.57:0/3350059065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26fc0743f0 con 0x7f26fc06e400 2023-11-28T12:06:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.308+0000 7f26f3fff700 1 -- 172.21.15.57:0/3350059065 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f26ec019030 con 0x7f26fc06e400 2023-11-28T12:06:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.308+0000 7f26f3fff700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f26dc0682f0 0x7f26dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.308+0000 7f26f3fff700 1 -- 172.21.15.57:0/3350059065 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f26ec085150 con 0x7f26fc06e400 2023-11-28T12:06:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.309+0000 7f26faffd700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f26dc0682f0 0x7f26dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.309+0000 7f26faffd700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f26dc0682f0 0x7f26dc06a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0d1a00 tx=0x7f26e401c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.313+0000 7f26f3fff700 1 -- 172.21.15.57:0/3350059065 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f26ec0518d0 con 0x7f26fc06e400 2023-11-28T12:06:37.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.559+0000 7f2701efb700 1 -- 172.21.15.57:0/3350059065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f26fc06dcd0 con 0x7f26fc06e400 2023-11-28T12:06:37.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.559+0000 7f26f3fff700 1 -- 172.21.15.57:0/3350059065 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f26ec054ef0 con 0x7f26fc06e400 2023-11-28T12:06:37.560 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:06:37.560 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:06:37.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.561+0000 7f2701efb700 1 -- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f26dc0682f0 msgr2=0x7f26dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:37.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.561+0000 7f2701efb700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f26dc0682f0 0x7f26dc06a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0d1a00 tx=0x7f26e401c040 comp rx=0 tx=0).stop 2023-11-28T12:06:37.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.561+0000 7f2701efb700 1 -- 172.21.15.57:0/3350059065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26fc06e400 msgr2=0x7f26fc0b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:37.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.561+0000 7f2701efb700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26fc06e400 0x7f26fc0b1c10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00c520 tx=0x7f26ec00c8e0 comp rx=0 tx=0).stop 2023-11-28T12:06:37.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.561+0000 7f2701efb700 1 -- 172.21.15.57:0/3350059065 shutdown_connections 2023-11-28T12:06:37.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.561+0000 7f2701efb700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f26dc0682f0 0x7f26dc06a7a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:37.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.561+0000 7f2701efb700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26fc15e010 0x7f26fc0cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:37.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.561+0000 7f2701efb700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26fc06e400 0x7f26fc0b1c10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:37.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.561+0000 7f2701efb700 1 --2- 172.21.15.57:0/3350059065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26fc0d15f0 0x7f26fc15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:37.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.561+0000 7f2701efb700 1 -- 172.21.15.57:0/3350059065 >> 172.21.15.57:0/3350059065 conn(0x7f26fc06a270 msgr2=0x7f26fc155e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:37.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.562+0000 7f2701efb700 1 -- 172.21.15.57:0/3350059065 shutdown_connections 2023-11-28T12:06:37.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:37.562+0000 7f2701efb700 1 -- 172.21.15.57:0/3350059065 wait complete. 2023-11-28T12:06:37.576 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:06:42.577 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:06:42.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.722+0000 7fec03895700 1 Processor -- start 2023-11-28T12:06:42.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.722+0000 7fec03895700 1 -- start start 2023-11-28T12:06:42.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.723+0000 7fec03895700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b220 0x7febfc164f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:42.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.723+0000 7fec03895700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7febfc0d1570 0x7febfc0d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:42.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.723+0000 7fec03895700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febfc160ee0 con 0x7febfc160b10 2023-11-28T12:06:42.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.723+0000 7fec03895700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febfc1654d0 con 0x7febfc0d1570 2023-11-28T12:06:42.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.723+0000 7fec03895700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febfc0d0b20 con 0x7febfc06b220 2023-11-28T12:06:42.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.723+0000 7fec00e30700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b220 0x7febfc164f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:42.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.723+0000 7fec01e32700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7febfc0d1570 0x7febfc0d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:42.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.723+0000 7fec00e30700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b220 0x7febfc164f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:59990/0 (socket says 172.21.15.57:59990) 2023-11-28T12:06:42.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.723+0000 7fec01e32700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7febfc0d1570 0x7febfc0d19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:58184/0 (socket says 172.21.15.57:58184) 2023-11-28T12:06:42.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.723+0000 7fec00e30700 1 -- 172.21.15.57:0/1051500077 learned_addr learned my addr 172.21.15.57:0/1051500077 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:42.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.723+0000 7fec01631700 1 --1- 172.21.15.57:0/1051500077 >> v1:172.21.15.181:6789/0 conn(0x7febfc160b10 0x7febfc06ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:49916/0 (socket says 172.21.15.57:49916) 2023-11-28T12:06:42.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.724+0000 7fec00e30700 1 -- 172.21.15.57:0/1051500077 >> v2:172.21.15.181:3300/0 conn(0x7febfc0d1570 msgr2=0x7febfc0d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:06:42.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.724+0000 7fec00e30700 1 --2- 172.21.15.57:0/1051500077 >> v2:172.21.15.181:3300/0 conn(0x7febfc0d1570 0x7febfc0d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:42.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.724+0000 7fec00e30700 1 -- 172.21.15.57:0/1051500077 >> v1:172.21.15.181:6789/0 conn(0x7febfc160b10 legacy=0x7febfc06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:42.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.724+0000 7febf3fff700 1 -- 172.21.15.57:0/1051500077 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774183632 0 0) 0x7febf8001110 con 0x7febfc160b10 2023-11-28T12:06:42.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.724+0000 7fec00e30700 1 -- 172.21.15.57:0/1051500077 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc0cf2e0 con 0x7febfc06b220 2023-11-28T12:06:42.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.724+0000 7fec01e32700 1 --2- 172.21.15.57:0/1051500077 >> v2:172.21.15.181:3300/0 conn(0x7febfc0d1570 0x7febfc0d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:06:42.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.724+0000 7fec00e30700 1 --2- 172.21.15.57:0/1051500077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b220 0x7febfc164f90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7febec0098f0 tx=0x7febec009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4bfdfc70efeec1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:42.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.724+0000 7febf3fff700 1 -- 172.21.15.57:0/1051500077 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec011830 con 0x7febfc06b220 2023-11-28T12:06:42.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.724+0000 7febf3fff700 1 -- 172.21.15.57:0/1051500077 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec011990 con 0x7febfc06b220 2023-11-28T12:06:42.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.724+0000 7febf3fff700 1 -- 172.21.15.57:0/1051500077 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec011d20 con 0x7febfc06b220 2023-11-28T12:06:42.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.725+0000 7fec03895700 1 -- 172.21.15.57:0/1051500077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b220 msgr2=0x7febfc164f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:42.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.725+0000 7fec03895700 1 --2- 172.21.15.57:0/1051500077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b220 0x7febfc164f90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7febec0098f0 tx=0x7febec009c00 comp rx=0 tx=0).stop 2023-11-28T12:06:42.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.725+0000 7fec03895700 1 -- 172.21.15.57:0/1051500077 shutdown_connections 2023-11-28T12:06:42.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.725+0000 7fec03895700 1 --2- 172.21.15.57:0/1051500077 >> v2:172.21.15.181:3300/0 conn(0x7febfc0d1570 0x7febfc0d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:42.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.725+0000 7fec03895700 1 --2- 172.21.15.57:0/1051500077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b220 0x7febfc164f90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:42.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.725+0000 7fec03895700 1 -- 172.21.15.57:0/1051500077 >> 172.21.15.57:0/1051500077 conn(0x7febfc06c520 msgr2=0x7febfc06c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:42.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.725+0000 7fec03895700 1 -- 172.21.15.57:0/1051500077 shutdown_connections 2023-11-28T12:06:42.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.725+0000 7fec03895700 1 -- 172.21.15.57:0/1051500077 wait complete. 2023-11-28T12:06:42.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.726+0000 7fec03895700 1 Processor -- start 2023-11-28T12:06:42.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.726+0000 7fec03895700 1 -- start start 2023-11-28T12:06:42.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.727+0000 7fec03895700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febfc06b220 0x7febfc158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:42.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.727+0000 7fec03895700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc0d1570 0x7febfc1ea380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:42.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.727+0000 7fec03895700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc160b10 0x7febfc159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:42.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.727+0000 7fec03895700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc1eae20 con 0x7febfc06b220 2023-11-28T12:06:42.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.727+0000 7fec03895700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc1eb100 con 0x7febfc160b10 2023-11-28T12:06:42.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.727+0000 7fec03895700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febfc1eb3e0 con 0x7febfc0d1570 2023-11-28T12:06:42.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.727+0000 7fec01631700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febfc06b220 0x7febfc158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:42.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.727+0000 7fec01631700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febfc06b220 0x7febfc158180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48338/0 (socket says 172.21.15.57:48338) 2023-11-28T12:06:42.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.727+0000 7fec01631700 1 -- 172.21.15.57:0/2870637031 learned_addr learned my addr 172.21.15.57:0/2870637031 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:42.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.727+0000 7fec01e32700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc160b10 0x7febfc159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:42.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.728+0000 7fec00e30700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc0d1570 0x7febfc1ea380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:42.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.728+0000 7fec01631700 1 -- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc0d1570 msgr2=0x7febfc1ea380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:42.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.728+0000 7fec01631700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc0d1570 0x7febfc1ea380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:42.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.728+0000 7fec01631700 1 -- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc160b10 msgr2=0x7febfc159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:42.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.728+0000 7fec01631700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc160b10 0x7febfc159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:42.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.728+0000 7fec01631700 1 -- 172.21.15.57:0/2870637031 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf4009a70 con 0x7febfc06b220 2023-11-28T12:06:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.728+0000 7fec01e32700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc160b10 0x7febfc159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:06:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.728+0000 7fec01631700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febfc06b220 0x7febfc158180 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7febf800c730 tx=0x7febf800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.728+0000 7febf1ffb700 1 -- 172.21.15.57:0/2870637031 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf8007560 con 0x7febfc06b220 2023-11-28T12:06:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.728+0000 7febf1ffb700 1 -- 172.21.15.57:0/2870637031 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf80076e0 con 0x7febfc06b220 2023-11-28T12:06:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.728+0000 7fec03895700 1 -- 172.21.15.57:0/2870637031 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febec009550 con 0x7febfc06b220 2023-11-28T12:06:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.728+0000 7febf1ffb700 1 -- 172.21.15.57:0/2870637031 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf8004ab0 con 0x7febfc06b220 2023-11-28T12:06:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.729+0000 7fec03895700 1 -- 172.21.15.57:0/2870637031 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febfc0d0d70 con 0x7febfc06b220 2023-11-28T12:06:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.729+0000 7fec03895700 1 -- 172.21.15.57:0/2870637031 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febfc0cf5c0 con 0x7febfc06b220 2023-11-28T12:06:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.730+0000 7febf1ffb700 1 -- 172.21.15.57:0/2870637031 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7febf801a020 con 0x7febfc06b220 2023-11-28T12:06:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.730+0000 7febf1ffb700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7febe00682f0 0x7febe006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.731+0000 7febf1ffb700 1 -- 172.21.15.57:0/2870637031 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7febf8085760 con 0x7febfc06b220 2023-11-28T12:06:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.731+0000 7fec00e30700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7febe00682f0 0x7febe006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:42.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.731+0000 7fec00e30700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7febe00682f0 0x7febe006a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7febfc06b610 tx=0x7febec008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:42.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.735+0000 7febf1ffb700 1 -- 172.21.15.57:0/2870637031 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7febf8051f40 con 0x7febfc06b220 2023-11-28T12:06:42.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.981+0000 7fec03895700 1 -- 172.21.15.57:0/2870637031 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7febfc0d1050 con 0x7febfc06b220 2023-11-28T12:06:42.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.981+0000 7febf1ffb700 1 -- 172.21.15.57:0/2870637031 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7febf8055560 con 0x7febfc06b220 2023-11-28T12:06:42.982 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:06:42.983 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:06:42.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.983+0000 7fec03895700 1 -- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7febe00682f0 msgr2=0x7febe006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:42.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.983+0000 7fec03895700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7febe00682f0 0x7febe006a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7febfc06b610 tx=0x7febec008030 comp rx=0 tx=0).stop 2023-11-28T12:06:42.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.983+0000 7fec03895700 1 -- 172.21.15.57:0/2870637031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febfc06b220 msgr2=0x7febfc158180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:42.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.983+0000 7fec03895700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febfc06b220 0x7febfc158180 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7febf800c730 tx=0x7febf800ca40 comp rx=0 tx=0).stop 2023-11-28T12:06:42.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.983+0000 7fec03895700 1 -- 172.21.15.57:0/2870637031 shutdown_connections 2023-11-28T12:06:42.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.983+0000 7fec03895700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7febe00682f0 0x7febe006a7a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:42.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.983+0000 7fec03895700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc160b10 0x7febfc159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:42.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.983+0000 7fec03895700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febfc06b220 0x7febfc158180 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:42.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.983+0000 7fec03895700 1 --2- 172.21.15.57:0/2870637031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc0d1570 0x7febfc1ea380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:42.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.983+0000 7fec03895700 1 -- 172.21.15.57:0/2870637031 >> 172.21.15.57:0/2870637031 conn(0x7febfc06c520 msgr2=0x7febfc0d2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:42.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.983+0000 7fec03895700 1 -- 172.21.15.57:0/2870637031 shutdown_connections 2023-11-28T12:06:42.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:42.983+0000 7fec03895700 1 -- 172.21.15.57:0/2870637031 wait complete. 2023-11-28T12:06:42.998 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:06:46.844 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:06:46.847 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:06:47.999 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:06:48.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.141+0000 7f8adfaa8700 1 Processor -- start 2023-11-28T12:06:48.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.141+0000 7f8adfaa8700 1 -- start start 2023-11-28T12:06:48.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.142+0000 7f8adfaa8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8ad806a270 0x7f8ad806a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:48.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.142+0000 7f8adfaa8700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8ad8070d90 0x7f8ad80711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:48.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.142+0000 7f8adfaa8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad8151a50 con 0x7f8ad806da40 2023-11-28T12:06:48.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.142+0000 7f8adfaa8700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad815f0c0 con 0x7f8ad8070d90 2023-11-28T12:06:48.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.142+0000 7f8adfaa8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad815b440 con 0x7f8ad806a270 2023-11-28T12:06:48.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.142+0000 7f8add043700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8ad8070d90 0x7f8ad80711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:48.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.143+0000 7f8add043700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8ad8070d90 0x7f8ad80711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38946/0 (socket says 172.21.15.57:38946) 2023-11-28T12:06:48.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.143+0000 7f8add043700 1 -- 172.21.15.57:0/2998389959 learned_addr learned my addr 172.21.15.57:0/2998389959 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:48.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.143+0000 7f8add844700 1 --2- 172.21.15.57:0/2998389959 >> v2:172.21.15.181:3300/0 conn(0x7f8ad806a270 0x7f8ad806a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:48.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.143+0000 7f8add043700 1 -- 172.21.15.57:0/2998389959 >> v2:172.21.15.181:3300/0 conn(0x7f8ad806a270 msgr2=0x7f8ad806a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:06:48.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.143+0000 7f8add043700 1 --2- 172.21.15.57:0/2998389959 >> v2:172.21.15.181:3300/0 conn(0x7f8ad806a270 0x7f8ad806a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:48.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.143+0000 7f8add043700 1 -- 172.21.15.57:0/2998389959 >> v1:172.21.15.181:6789/0 conn(0x7f8ad806da40 legacy=0x7f8ad806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:48.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.143+0000 7f8add043700 1 -- 172.21.15.57:0/2998389959 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad806ab80 con 0x7f8ad8070d90 2023-11-28T12:06:48.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.143+0000 7f8add043700 1 --2- 172.21.15.57:0/2998389959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad8070d90 0x7f8ad80711e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80cd250 tx=0x7f8acc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56eeaa4dc2248451 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:48.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.143+0000 7f8adc842700 1 -- 172.21.15.57:0/2998389959 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00b870 con 0x7f8ad8070d90 2023-11-28T12:06:48.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.143+0000 7f8adc842700 1 -- 172.21.15.57:0/2998389959 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc00b9d0 con 0x7f8ad8070d90 2023-11-28T12:06:48.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.143+0000 7f8adc842700 1 -- 172.21.15.57:0/2998389959 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00bd60 con 0x7f8ad8070d90 2023-11-28T12:06:48.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.144+0000 7f8adfaa8700 1 -- 172.21.15.57:0/2998389959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad8070d90 msgr2=0x7f8ad80711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:48.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.144+0000 7f8adfaa8700 1 --2- 172.21.15.57:0/2998389959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad8070d90 0x7f8ad80711e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80cd250 tx=0x7f8acc009bd0 comp rx=0 tx=0).stop 2023-11-28T12:06:48.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.144+0000 7f8adfaa8700 1 -- 172.21.15.57:0/2998389959 shutdown_connections 2023-11-28T12:06:48.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.144+0000 7f8adfaa8700 1 --2- 172.21.15.57:0/2998389959 >> v2:172.21.15.181:3300/0 conn(0x7f8ad806a270 0x7f8ad806a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:48.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.144+0000 7f8adfaa8700 1 --2- 172.21.15.57:0/2998389959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad8070d90 0x7f8ad80711e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:48.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.144+0000 7f8adfaa8700 1 -- 172.21.15.57:0/2998389959 >> 172.21.15.57:0/2998389959 conn(0x7f8ad806c010 msgr2=0x7f8ad80cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:48.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.144+0000 7f8adfaa8700 1 -- 172.21.15.57:0/2998389959 shutdown_connections 2023-11-28T12:06:48.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.144+0000 7f8adfaa8700 1 -- 172.21.15.57:0/2998389959 wait complete. 2023-11-28T12:06:48.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.145+0000 7f8adfaa8700 1 Processor -- start 2023-11-28T12:06:48.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.145+0000 7f8adfaa8700 1 -- start start 2023-11-28T12:06:48.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.146+0000 7f8adfaa8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ad806a270 0x7f8ad81509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:48.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.146+0000 7f8adfaa8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad806da40 0x7f8ad80cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:48.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.146+0000 7f8adfaa8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ad8070d90 0x7f8ad8159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:48.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.146+0000 7f8adfaa8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad80cf4c0 con 0x7f8ad806da40 2023-11-28T12:06:48.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.146+0000 7f8adfaa8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad80ce8b0 con 0x7f8ad8070d90 2023-11-28T12:06:48.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.146+0000 7f8adfaa8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad80ceb90 con 0x7f8ad806a270 2023-11-28T12:06:48.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.146+0000 7f8add043700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad806da40 0x7f8ad80cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:48.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.146+0000 7f8add043700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad806da40 0x7f8ad80cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38952/0 (socket says 172.21.15.57:38952) 2023-11-28T12:06:48.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.146+0000 7f8add043700 1 -- 172.21.15.57:0/715580029 learned_addr learned my addr 172.21.15.57:0/715580029 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:48.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.146+0000 7f8add844700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ad806a270 0x7f8ad81509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:48.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.146+0000 7f8ade045700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ad8070d90 0x7f8ad8159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:48.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.147+0000 7f8add043700 1 -- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ad806a270 msgr2=0x7f8ad81509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:48.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.147+0000 7f8add043700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ad806a270 0x7f8ad81509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:48.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.147+0000 7f8add043700 1 -- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ad8070d90 msgr2=0x7f8ad8159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:48.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.147+0000 7f8add043700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ad8070d90 0x7f8ad8159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:48.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.147+0000 7f8add043700 1 -- 172.21.15.57:0/715580029 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc009550 con 0x7f8ad806da40 2023-11-28T12:06:48.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.147+0000 7f8add043700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad806da40 0x7f8ad80cded0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8acc000c00 tx=0x7f8acc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:48.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.147+0000 7f8aca7fc700 1 -- 172.21.15.57:0/715580029 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc004310 con 0x7f8ad806da40 2023-11-28T12:06:48.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.147+0000 7f8aca7fc700 1 -- 172.21.15.57:0/715580029 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc01e070 con 0x7f8ad806da40 2023-11-28T12:06:48.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.147+0000 7f8adfaa8700 1 -- 172.21.15.57:0/715580029 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad815adf0 con 0x7f8ad806da40 2023-11-28T12:06:48.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.147+0000 7f8aca7fc700 1 -- 172.21.15.57:0/715580029 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0036a0 con 0x7f8ad806da40 2023-11-28T12:06:48.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.147+0000 7f8adfaa8700 1 -- 172.21.15.57:0/715580029 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad8071ac0 con 0x7f8ad806da40 2023-11-28T12:06:48.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.148+0000 7f8adfaa8700 1 -- 172.21.15.57:0/715580029 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad815b0d0 con 0x7f8ad806da40 2023-11-28T12:06:48.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.149+0000 7f8aca7fc700 1 -- 172.21.15.57:0/715580029 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8acc029020 con 0x7f8ad806da40 2023-11-28T12:06:48.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.149+0000 7f8aca7fc700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8abc0682f0 0x7f8abc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:48.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.149+0000 7f8aca7fc700 1 -- 172.21.15.57:0/715580029 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f8acc086e00 con 0x7f8ad806da40 2023-11-28T12:06:48.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.149+0000 7f8add844700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8abc0682f0 0x7f8abc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:48.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.150+0000 7f8add844700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8abc0682f0 0x7f8abc06a7a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4006b20 tx=0x7f8ad4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:48.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.153+0000 7f8aca7fc700 1 -- 172.21.15.57:0/715580029 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8acc053560 con 0x7f8ad806da40 2023-11-28T12:06:48.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.399+0000 7f8adfaa8700 1 -- 172.21.15.57:0/715580029 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8ad80743f0 con 0x7f8ad806da40 2023-11-28T12:06:48.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.399+0000 7f8aca7fc700 1 -- 172.21.15.57:0/715580029 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f8acc056b80 con 0x7f8ad806da40 2023-11-28T12:06:48.401 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:06:48.401 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:06:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.401+0000 7f8adfaa8700 1 -- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8abc0682f0 msgr2=0x7f8abc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.401+0000 7f8adfaa8700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8abc0682f0 0x7f8abc06a7a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4006b20 tx=0x7f8ad4008040 comp rx=0 tx=0).stop 2023-11-28T12:06:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.401+0000 7f8adfaa8700 1 -- 172.21.15.57:0/715580029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad806da40 msgr2=0x7f8ad80cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.401+0000 7f8adfaa8700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad806da40 0x7f8ad80cded0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8acc000c00 tx=0x7f8acc004750 comp rx=0 tx=0).stop 2023-11-28T12:06:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.401+0000 7f8adfaa8700 1 -- 172.21.15.57:0/715580029 shutdown_connections 2023-11-28T12:06:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.401+0000 7f8adfaa8700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f8abc0682f0 0x7f8abc06a7a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.401+0000 7f8adfaa8700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ad8070d90 0x7f8ad8159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.401+0000 7f8adfaa8700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ad806a270 0x7f8ad81509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.401+0000 7f8adfaa8700 1 --2- 172.21.15.57:0/715580029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ad806da40 0x7f8ad80cded0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:48.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.401+0000 7f8adfaa8700 1 -- 172.21.15.57:0/715580029 >> 172.21.15.57:0/715580029 conn(0x7f8ad806c010 msgr2=0x7f8ad80cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:48.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.402+0000 7f8adfaa8700 1 -- 172.21.15.57:0/715580029 shutdown_connections 2023-11-28T12:06:48.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:48.402+0000 7f8adfaa8700 1 -- 172.21.15.57:0/715580029 wait complete. 2023-11-28T12:06:48.416 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:06:53.416 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:06:53.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.558+0000 7f3b6a8eb700 1 Processor -- start 2023-11-28T12:06:53.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.558+0000 7f3b6a8eb700 1 -- start start 2023-11-28T12:06:53.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b6a8eb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b6406feb0 0x7f3b640cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:53.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b6a8eb700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3b640b4b00 0x7f3b640cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:53.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b6a8eb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b640b4fc0 con 0x7f3b640ced70 2023-11-28T12:06:53.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b6a8eb700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b64159c30 con 0x7f3b640b4b00 2023-11-28T12:06:53.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b6a8eb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640b05e0 con 0x7f3b6406feb0 2023-11-28T12:06:53.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b68e88700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3b640b4b00 0x7f3b640cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:53.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b68e88700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3b640b4b00 0x7f3b640cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38968/0 (socket says 172.21.15.57:38968) 2023-11-28T12:06:53.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b68e88700 1 -- 172.21.15.57:0/3803202580 learned_addr learned my addr 172.21.15.57:0/3803202580 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:53.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b637fe700 1 --2- 172.21.15.57:0/3803202580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b6406feb0 0x7f3b640cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:53.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b68e88700 1 -- 172.21.15.57:0/3803202580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b6406feb0 msgr2=0x7f3b640cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:06:53.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b68e88700 1 --2- 172.21.15.57:0/3803202580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b6406feb0 0x7f3b640cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:53.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b68e88700 1 -- 172.21.15.57:0/3803202580 >> v1:172.21.15.181:6789/0 conn(0x7f3b640ced70 legacy=0x7f3b640b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:53.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b68e88700 1 -- 172.21.15.57:0/3803202580 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b640b08c0 con 0x7f3b640b4b00 2023-11-28T12:06:53.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.559+0000 7f3b637fe700 1 --2- 172.21.15.57:0/3803202580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b6406feb0 0x7f3b640cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:06:53.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.560+0000 7f3b68e88700 1 --2- 172.21.15.57:0/3803202580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b640b4b00 0x7f3b640cd710 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009e50 tx=0x7f3b5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad9c121f0c6f97e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:53.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.560+0000 7f3b62ffd700 1 -- 172.21.15.57:0/3803202580 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00e070 con 0x7f3b640b4b00 2023-11-28T12:06:53.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.560+0000 7f3b62ffd700 1 -- 172.21.15.57:0/3803202580 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5c00b9c0 con 0x7f3b640b4b00 2023-11-28T12:06:53.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.560+0000 7f3b62ffd700 1 -- 172.21.15.57:0/3803202580 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00bd70 con 0x7f3b640b4b00 2023-11-28T12:06:53.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.560+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3803202580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b640b4b00 msgr2=0x7f3b640cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:53.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.560+0000 7f3b6a8eb700 1 --2- 172.21.15.57:0/3803202580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b640b4b00 0x7f3b640cd710 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009e50 tx=0x7f3b5c00b330 comp rx=0 tx=0).stop 2023-11-28T12:06:53.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.560+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3803202580 shutdown_connections 2023-11-28T12:06:53.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.560+0000 7f3b6a8eb700 1 --2- 172.21.15.57:0/3803202580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b640b4b00 0x7f3b640cd710 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:53.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.560+0000 7f3b6a8eb700 1 --2- 172.21.15.57:0/3803202580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b6406feb0 0x7f3b640cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:53.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.560+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3803202580 >> 172.21.15.57:0/3803202580 conn(0x7f3b640b2fc0 msgr2=0x7f3b64071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:53.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.560+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3803202580 shutdown_connections 2023-11-28T12:06:53.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.560+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3803202580 wait complete. 2023-11-28T12:06:53.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.562+0000 7f3b6a8eb700 1 Processor -- start 2023-11-28T12:06:53.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.562+0000 7f3b6a8eb700 1 -- start start 2023-11-28T12:06:53.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.562+0000 7f3b6a8eb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b6406feb0 0x7f3b641582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:53.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b6a8eb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b640b4b00 0x7f3b641e6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:53.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b6a8eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b640ced70 0x7f3b641e6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:53.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b6a8eb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640c4600 con 0x7f3b6406feb0 2023-11-28T12:06:53.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b6a8eb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640acf40 con 0x7f3b640ced70 2023-11-28T12:06:53.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b6a8eb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640ad220 con 0x7f3b640b4b00 2023-11-28T12:06:53.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b63fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b6406feb0 0x7f3b641582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:53.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b63fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b6406feb0 0x7f3b641582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38974/0 (socket says 172.21.15.57:38974) 2023-11-28T12:06:53.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b63fff700 1 -- 172.21.15.57:0/3268949197 learned_addr learned my addr 172.21.15.57:0/3268949197 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:53.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b68e88700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b640ced70 0x7f3b641e6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:53.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b637fe700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b640b4b00 0x7f3b641e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:53.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b63fff700 1 -- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b640b4b00 msgr2=0x7f3b641e6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:53.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b63fff700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b640b4b00 0x7f3b641e6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:53.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b63fff700 1 -- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b640ced70 msgr2=0x7f3b641e6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:53.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b63fff700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b640ced70 0x7f3b641e6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:53.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b63fff700 1 -- 172.21.15.57:0/3268949197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c009b30 con 0x7f3b6406feb0 2023-11-28T12:06:53.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b63fff700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b6406feb0 0x7f3b641582e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00c450 tx=0x7f3b4c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:53.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b60ff9700 1 -- 172.21.15.57:0/3268949197 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c0052d0 con 0x7f3b6406feb0 2023-11-28T12:06:53.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b60ff9700 1 -- 172.21.15.57:0/3268949197 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c005430 con 0x7f3b6406feb0 2023-11-28T12:06:53.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.563+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3268949197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b640b5260 con 0x7f3b6406feb0 2023-11-28T12:06:53.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.564+0000 7f3b60ff9700 1 -- 172.21.15.57:0/3268949197 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c0038c0 con 0x7f3b6406feb0 2023-11-28T12:06:53.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.564+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3268949197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b640b0ba0 con 0x7f3b6406feb0 2023-11-28T12:06:53.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.564+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3268949197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6406b280 con 0x7f3b6406feb0 2023-11-28T12:06:53.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.565+0000 7f3b60ff9700 1 -- 172.21.15.57:0/3268949197 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3b4c019030 con 0x7f3b6406feb0 2023-11-28T12:06:53.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.565+0000 7f3b60ff9700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3b440682f0 0x7f3b4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:53.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.566+0000 7f3b60ff9700 1 -- 172.21.15.57:0/3268949197 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f3b4c084ee0 con 0x7f3b6406feb0 2023-11-28T12:06:53.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.566+0000 7f3b637fe700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3b440682f0 0x7f3b4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:53.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.566+0000 7f3b637fe700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3b440682f0 0x7f3b4406a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3b640adab0 tx=0x7f3b54008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:53.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.570+0000 7f3b60ff9700 1 -- 172.21.15.57:0/3268949197 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3b4c0516c0 con 0x7f3b6406feb0 2023-11-28T12:06:53.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.813+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3268949197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3b64067370 con 0x7f3b6406feb0 2023-11-28T12:06:53.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.813+0000 7f3b60ff9700 1 -- 172.21.15.57:0/3268949197 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f3b4c0170c0 con 0x7f3b6406feb0 2023-11-28T12:06:53.815 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:06:53.815 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:06:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.815+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3b440682f0 msgr2=0x7f3b4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.815+0000 7f3b6a8eb700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3b440682f0 0x7f3b4406a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3b640adab0 tx=0x7f3b54008040 comp rx=0 tx=0).stop 2023-11-28T12:06:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.815+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3268949197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b6406feb0 msgr2=0x7f3b641582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.815+0000 7f3b6a8eb700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b6406feb0 0x7f3b641582e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00c450 tx=0x7f3b4c00c760 comp rx=0 tx=0).stop 2023-11-28T12:06:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.815+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3268949197 shutdown_connections 2023-11-28T12:06:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.815+0000 7f3b6a8eb700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3b440682f0 0x7f3b4406a7a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.815+0000 7f3b6a8eb700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b640ced70 0x7f3b641e6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.815+0000 7f3b6a8eb700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b6406feb0 0x7f3b641582e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.815+0000 7f3b6a8eb700 1 --2- 172.21.15.57:0/3268949197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b640b4b00 0x7f3b641e6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:53.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.815+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3268949197 >> 172.21.15.57:0/3268949197 conn(0x7f3b640b2fc0 msgr2=0x7f3b6406a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:53.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.816+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3268949197 shutdown_connections 2023-11-28T12:06:53.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:53.816+0000 7f3b6a8eb700 1 -- 172.21.15.57:0/3268949197 wait complete. 2023-11-28T12:06:53.831 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:06:58.832 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:06:58.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.975+0000 7f5e2094c700 1 Processor -- start 2023-11-28T12:06:58.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.975+0000 7f5e2094c700 1 -- start start 2023-11-28T12:06:58.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.976+0000 7f5e2094c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5e1c0ce040 0x7f5e1c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:58.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.976+0000 7f5e2094c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e1c0b1770 0x7f5e1c0b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:58.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.976+0000 7f5e2094c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1c13fc90 con 0x7f5e1c142db0 2023-11-28T12:06:58.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.976+0000 7f5e2094c700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e1c13f480 con 0x7f5e1c0ce040 2023-11-28T12:06:58.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.976+0000 7f5e2094c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e1c0b20d0 con 0x7f5e1c0b1770 2023-11-28T12:06:58.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.976+0000 7f5e1a59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5e1c0ce040 0x7f5e1c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:58.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.976+0000 7f5e1a59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5e1c0ce040 0x7f5e1c15d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44220/0 (socket says 172.21.15.57:44220) 2023-11-28T12:06:58.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.976+0000 7f5e1a59c700 1 -- 172.21.15.57:0/3122946320 learned_addr learned my addr 172.21.15.57:0/3122946320 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:58.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.976+0000 7f5e1ad9d700 1 --2- 172.21.15.57:0/3122946320 >> v2:172.21.15.181:3300/0 conn(0x7f5e1c0b1770 0x7f5e1c0b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:58.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.977+0000 7f5e1a59c700 1 -- 172.21.15.57:0/3122946320 >> v2:172.21.15.181:3300/0 conn(0x7f5e1c0b1770 msgr2=0x7f5e1c0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:06:58.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.977+0000 7f5e1a59c700 1 --2- 172.21.15.57:0/3122946320 >> v2:172.21.15.181:3300/0 conn(0x7f5e1c0b1770 0x7f5e1c0b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:58.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.977+0000 7f5e1a59c700 1 -- 172.21.15.57:0/3122946320 >> v1:172.21.15.181:6789/0 conn(0x7f5e1c142db0 legacy=0x7f5e1c13ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:58.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.977+0000 7f5e1a59c700 1 -- 172.21.15.57:0/3122946320 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c13f6d0 con 0x7f5e1c0ce040 2023-11-28T12:06:58.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.977+0000 7f5e1a59c700 1 --2- 172.21.15.57:0/3122946320 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e1c0ce040 0x7f5e1c15d900 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c142cd0 tx=0x7f5e0c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67043d10f27b0bd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:58.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.977+0000 7f5e1959a700 1 -- 172.21.15.57:0/3122946320 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c00dc70 con 0x7f5e1c0ce040 2023-11-28T12:06:58.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.977+0000 7f5e1959a700 1 -- 172.21.15.57:0/3122946320 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0c00ddd0 con 0x7f5e1c0ce040 2023-11-28T12:06:58.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.977+0000 7f5e1959a700 1 -- 172.21.15.57:0/3122946320 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c010690 con 0x7f5e1c0ce040 2023-11-28T12:06:58.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.977+0000 7f5e2094c700 1 -- 172.21.15.57:0/3122946320 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e1c0ce040 msgr2=0x7f5e1c15d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:58.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.977+0000 7f5e2094c700 1 --2- 172.21.15.57:0/3122946320 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e1c0ce040 0x7f5e1c15d900 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c142cd0 tx=0x7f5e0c00be70 comp rx=0 tx=0).stop 2023-11-28T12:06:58.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.978+0000 7f5e2094c700 1 -- 172.21.15.57:0/3122946320 shutdown_connections 2023-11-28T12:06:58.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.978+0000 7f5e2094c700 1 --2- 172.21.15.57:0/3122946320 >> v2:172.21.15.181:3300/0 conn(0x7f5e1c0b1770 0x7f5e1c0b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:58.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.978+0000 7f5e2094c700 1 --2- 172.21.15.57:0/3122946320 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e1c0ce040 0x7f5e1c15d900 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:58.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.978+0000 7f5e2094c700 1 -- 172.21.15.57:0/3122946320 >> 172.21.15.57:0/3122946320 conn(0x7f5e1c06c010 msgr2=0x7f5e1c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:58.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.978+0000 7f5e2094c700 1 -- 172.21.15.57:0/3122946320 shutdown_connections 2023-11-28T12:06:58.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.978+0000 7f5e2094c700 1 -- 172.21.15.57:0/3122946320 wait complete. 2023-11-28T12:06:58.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.980+0000 7f5e2094c700 1 Processor -- start 2023-11-28T12:06:58.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.980+0000 7f5e2094c700 1 -- start start 2023-11-28T12:06:58.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e2094c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c0b1770 0x7f5e1c0c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:58.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e2094c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e1c0ce040 0x7f5e1c14a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:58.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e2094c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e1c142db0 0x7f5e1c1f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:58.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e2094c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c149590 con 0x7f5e1c142db0 2023-11-28T12:06:58.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e2094c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c1f24d0 con 0x7f5e1c0b1770 2023-11-28T12:06:58.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e2094c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c1f27b0 con 0x7f5e1c0ce040 2023-11-28T12:06:58.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e1ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e1c142db0 0x7f5e1c1f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:58.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e1a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c0b1770 0x7f5e1c0c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e1ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e1c142db0 0x7f5e1c1f1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44236/0 (socket says 172.21.15.57:44236) 2023-11-28T12:06:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e1ad9d700 1 -- 172.21.15.57:0/3791039048 learned_addr learned my addr 172.21.15.57:0/3791039048 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:06:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e19d9b700 1 --2- 172.21.15.57:0/3791039048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e1c0ce040 0x7f5e1c14a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e1ad9d700 1 -- 172.21.15.57:0/3791039048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e1c0ce040 msgr2=0x7f5e1c14a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e1ad9d700 1 --2- 172.21.15.57:0/3791039048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e1c0ce040 0x7f5e1c14a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e1ad9d700 1 -- 172.21.15.57:0/3791039048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c0b1770 msgr2=0x7f5e1c0c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e1ad9d700 1 --2- 172.21.15.57:0/3791039048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c0b1770 0x7f5e1c0c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e1ad9d700 1 -- 172.21.15.57:0/3791039048 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0c00b7f0 con 0x7f5e1c142db0 2023-11-28T12:06:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e1ad9d700 1 --2- 172.21.15.57:0/3791039048 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e1c142db0 0x7f5e1c1f1a10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400efc0 tx=0x7f5e1400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.981+0000 7f5e12ffd700 1 -- 172.21.15.57:0/3791039048 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1400cad0 con 0x7f5e1c142db0 2023-11-28T12:06:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.982+0000 7f5e12ffd700 1 -- 172.21.15.57:0/3791039048 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1400f040 con 0x7f5e1c142db0 2023-11-28T12:06:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.982+0000 7f5e12ffd700 1 -- 172.21.15.57:0/3791039048 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1400ce70 con 0x7f5e1c142db0 2023-11-28T12:06:58.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.982+0000 7f5e2094c700 1 -- 172.21.15.57:0/3791039048 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1c15d100 con 0x7f5e1c142db0 2023-11-28T12:06:58.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.982+0000 7f5e2094c700 1 -- 172.21.15.57:0/3791039048 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1c13f9b0 con 0x7f5e1c142db0 2023-11-28T12:06:58.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.983+0000 7f5e2094c700 1 -- 172.21.15.57:0/3791039048 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1c15d3e0 con 0x7f5e1c142db0 2023-11-28T12:06:58.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.983+0000 7f5e12ffd700 1 -- 172.21.15.57:0/3791039048 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5e14014480 con 0x7f5e1c142db0 2023-11-28T12:06:58.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.984+0000 7f5e12ffd700 1 --2- 172.21.15.57:0/3791039048 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5dfc0682f0 0x7f5dfc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:06:58.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.984+0000 7f5e12ffd700 1 -- 172.21.15.57:0/3791039048 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f5e140868b0 con 0x7f5e1c142db0 2023-11-28T12:06:58.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.984+0000 7f5e1a59c700 1 --2- 172.21.15.57:0/3791039048 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5dfc0682f0 0x7f5dfc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:06:58.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.984+0000 7f5e1a59c700 1 --2- 172.21.15.57:0/3791039048 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5dfc0682f0 0x7f5dfc06a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00beb0 tx=0x7f5e0c013000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:06:58.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:58.988+0000 7f5e12ffd700 1 -- 172.21.15.57:0/3791039048 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5e14053030 con 0x7f5e1c142db0 2023-11-28T12:06:59.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.233+0000 7f5e2094c700 1 -- 172.21.15.57:0/3791039048 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5e1c0743f0 con 0x7f5e1c142db0 2023-11-28T12:06:59.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.233+0000 7f5e12ffd700 1 -- 172.21.15.57:0/3791039048 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f5e14056650 con 0x7f5e1c142db0 2023-11-28T12:06:59.234 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:06:59.234 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:06:59.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.236+0000 7f5e2094c700 1 -- 172.21.15.57:0/3791039048 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5dfc0682f0 msgr2=0x7f5dfc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:59.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.236+0000 7f5e2094c700 1 --2- 172.21.15.57:0/3791039048 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5dfc0682f0 0x7f5dfc06a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00beb0 tx=0x7f5e0c013000 comp rx=0 tx=0).stop 2023-11-28T12:06:59.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.236+0000 7f5e2094c700 1 -- 172.21.15.57:0/3791039048 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e1c142db0 msgr2=0x7f5e1c1f1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:06:59.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.236+0000 7f5e2094c700 1 --2- 172.21.15.57:0/3791039048 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e1c142db0 0x7f5e1c1f1a10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400efc0 tx=0x7f5e1400c0c0 comp rx=0 tx=0).stop 2023-11-28T12:06:59.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.236+0000 7f5e2094c700 1 -- 172.21.15.57:0/3791039048 shutdown_connections 2023-11-28T12:06:59.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.236+0000 7f5e2094c700 1 --2- 172.21.15.57:0/3791039048 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5dfc0682f0 0x7f5dfc06a7a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:59.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.236+0000 7f5e2094c700 1 --2- 172.21.15.57:0/3791039048 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e1c142db0 0x7f5e1c1f1a10 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:59.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.236+0000 7f5e2094c700 1 --2- 172.21.15.57:0/3791039048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c0b1770 0x7f5e1c0c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:59.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.236+0000 7f5e2094c700 1 --2- 172.21.15.57:0/3791039048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e1c0ce040 0x7f5e1c14a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:06:59.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.236+0000 7f5e2094c700 1 -- 172.21.15.57:0/3791039048 >> 172.21.15.57:0/3791039048 conn(0x7f5e1c06c010 msgr2=0x7f5e1c13ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:06:59.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.236+0000 7f5e2094c700 1 -- 172.21.15.57:0/3791039048 shutdown_connections 2023-11-28T12:06:59.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:06:59.236+0000 7f5e2094c700 1 -- 172.21.15.57:0/3791039048 wait complete. 2023-11-28T12:06:59.249 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:07:04.250 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:07:04.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.391+0000 7f4c140b4700 1 Processor -- start 2023-11-28T12:07:04.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.391+0000 7f4c140b4700 1 -- start start 2023-11-28T12:07:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.392+0000 7f4c140b4700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4c0c06a270 0x7f4c0c06a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.392+0000 7f4c140b4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c0c15cc80 0x7f4c0c06de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.392+0000 7f4c140b4700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0c06ab80 con 0x7f4c0c06e3a0 2023-11-28T12:07:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.392+0000 7f4c140b4700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c0c0b2490 con 0x7f4c0c06a270 2023-11-28T12:07:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.392+0000 7f4c140b4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c0c0b1810 con 0x7f4c0c15cc80 2023-11-28T12:07:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.392+0000 7f4c11e50700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4c0c06a270 0x7f4c0c06a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.392+0000 7f4c11e50700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4c0c06a270 0x7f4c0c06a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44252/0 (socket says 172.21.15.57:44252) 2023-11-28T12:07:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.392+0000 7f4c12651700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4c0c06e3a0 0x7f4c0c06ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38132/0 (socket says 172.21.15.57:38132) 2023-11-28T12:07:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.392+0000 7f4c11e50700 1 -- 172.21.15.57:0/3951814116 learned_addr learned my addr 172.21.15.57:0/3951814116 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.392+0000 7f4c1164f700 1 --2- 172.21.15.57:0/3951814116 >> v2:172.21.15.181:3300/0 conn(0x7f4c0c15cc80 0x7f4c0c06de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.393+0000 7f4c11e50700 1 -- 172.21.15.57:0/3951814116 >> v2:172.21.15.181:3300/0 conn(0x7f4c0c15cc80 msgr2=0x7f4c0c06de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.393+0000 7f4c11e50700 1 --2- 172.21.15.57:0/3951814116 >> v2:172.21.15.181:3300/0 conn(0x7f4c0c15cc80 0x7f4c0c06de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.393+0000 7f4c11e50700 1 -- 172.21.15.57:0/3951814116 >> v1:172.21.15.57:6789/0 conn(0x7f4c0c06e3a0 legacy=0x7f4c0c06ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.393+0000 7f4c11e50700 1 -- 172.21.15.57:0/3951814116 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c0b1af0 con 0x7f4c0c06a270 2023-11-28T12:07:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.393+0000 7f4c10e4e700 1 -- 172.21.15.57:0/3951814116 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561991698 0 0) 0x7f4c0c06ab80 con 0x7f4c0c06e3a0 2023-11-28T12:07:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.393+0000 7f4c11e50700 1 --2- 172.21.15.57:0/3951814116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c06a270 0x7f4c0c06a640 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c06c6c0 tx=0x7f4c08009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4ea002289da51e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.393+0000 7f4c10e4e700 1 -- 172.21.15.57:0/3951814116 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08011830 con 0x7f4c0c06a270 2023-11-28T12:07:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.393+0000 7f4c10e4e700 1 -- 172.21.15.57:0/3951814116 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c08011990 con 0x7f4c0c06a270 2023-11-28T12:07:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.393+0000 7f4c10e4e700 1 -- 172.21.15.57:0/3951814116 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08011d20 con 0x7f4c0c06a270 2023-11-28T12:07:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.394+0000 7f4c140b4700 1 -- 172.21.15.57:0/3951814116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c06a270 msgr2=0x7f4c0c06a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.394+0000 7f4c140b4700 1 --2- 172.21.15.57:0/3951814116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c06a270 0x7f4c0c06a640 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c06c6c0 tx=0x7f4c08009bd0 comp rx=0 tx=0).stop 2023-11-28T12:07:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.394+0000 7f4c140b4700 1 -- 172.21.15.57:0/3951814116 shutdown_connections 2023-11-28T12:07:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.394+0000 7f4c140b4700 1 --2- 172.21.15.57:0/3951814116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c06a270 0x7f4c0c06a640 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.394+0000 7f4c140b4700 1 --2- 172.21.15.57:0/3951814116 >> v2:172.21.15.181:3300/0 conn(0x7f4c0c15cc80 0x7f4c0c06de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.394+0000 7f4c140b4700 1 -- 172.21.15.57:0/3951814116 >> 172.21.15.57:0/3951814116 conn(0x7f4c0c06c010 msgr2=0x7f4c0c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.394+0000 7f4c140b4700 1 -- 172.21.15.57:0/3951814116 shutdown_connections 2023-11-28T12:07:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.394+0000 7f4c140b4700 1 -- 172.21.15.57:0/3951814116 wait complete. 2023-11-28T12:07:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.395+0000 7f4c140b4700 1 Processor -- start 2023-11-28T12:07:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.395+0000 7f4c140b4700 1 -- start start 2023-11-28T12:07:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c140b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c0c06a270 0x7f4c0c0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c140b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c0c06e3a0 0x7f4c0c0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c140b4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c15cc80 0x7f4c0c0b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c140b4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1e82b0 con 0x7f4c0c15cc80 2023-11-28T12:07:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c140b4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1e8590 con 0x7f4c0c06e3a0 2023-11-28T12:07:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c140b4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1e8870 con 0x7f4c0c06a270 2023-11-28T12:07:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c12651700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c15cc80 0x7f4c0c0b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c12651700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c15cc80 0x7f4c0c0b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44254/0 (socket says 172.21.15.57:44254) 2023-11-28T12:07:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c12651700 1 -- 172.21.15.57:0/184303835 learned_addr learned my addr 172.21.15.57:0/184303835 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c11e50700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c0c06a270 0x7f4c0c0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c1164f700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c0c06e3a0 0x7f4c0c0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c12651700 1 -- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c0c06a270 msgr2=0x7f4c0c0c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c12651700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c0c06a270 0x7f4c0c0c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c12651700 1 -- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c0c06e3a0 msgr2=0x7f4c0c0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c12651700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c0c06e3a0 0x7f4c0c0b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c12651700 1 -- 172.21.15.57:0/184303835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c08009550 con 0x7f4c0c15cc80 2023-11-28T12:07:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.396+0000 7f4c12651700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c15cc80 0x7f4c0c0b02e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400cba0 tx=0x7f4c0400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.397+0000 7f4c027fc700 1 -- 172.21.15.57:0/184303835 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0400aaa0 con 0x7f4c0c15cc80 2023-11-28T12:07:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.397+0000 7f4c027fc700 1 -- 172.21.15.57:0/184303835 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0400ac20 con 0x7f4c0c15cc80 2023-11-28T12:07:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.397+0000 7f4c140b4700 1 -- 172.21.15.57:0/184303835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0c0b20b0 con 0x7f4c0c15cc80 2023-11-28T12:07:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.397+0000 7f4c027fc700 1 -- 172.21.15.57:0/184303835 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c040070d0 con 0x7f4c0c15cc80 2023-11-28T12:07:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.397+0000 7f4c140b4700 1 -- 172.21.15.57:0/184303835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0c0b1dd0 con 0x7f4c0c15cc80 2023-11-28T12:07:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.398+0000 7f4c140b4700 1 -- 172.21.15.57:0/184303835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0c06e7c0 con 0x7f4c0c15cc80 2023-11-28T12:07:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.398+0000 7f4c027fc700 1 -- 172.21.15.57:0/184303835 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4c04010070 con 0x7f4c0c15cc80 2023-11-28T12:07:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.399+0000 7f4c027fc700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4bf00682f0 0x7f4bf006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.399+0000 7f4c027fc700 1 -- 172.21.15.57:0/184303835 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f4c04085660 con 0x7f4c0c15cc80 2023-11-28T12:07:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.399+0000 7f4c11e50700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4bf00682f0 0x7f4bf006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.399+0000 7f4c11e50700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4bf00682f0 0x7f4bf006a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4c08014040 tx=0x7f4c08008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:04.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.403+0000 7f4c027fc700 1 -- 172.21.15.57:0/184303835 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4c04051e40 con 0x7f4c0c15cc80 2023-11-28T12:07:04.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.645+0000 7f4c140b4700 1 -- 172.21.15.57:0/184303835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4c0c0743f0 con 0x7f4c0c15cc80 2023-11-28T12:07:04.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.646+0000 7f4c027fc700 1 -- 172.21.15.57:0/184303835 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f4c040180c0 con 0x7f4c0c15cc80 2023-11-28T12:07:04.647 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:04.647 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:07:04.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.648+0000 7f4c140b4700 1 -- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4bf00682f0 msgr2=0x7f4bf006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:04.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.648+0000 7f4c140b4700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4bf00682f0 0x7f4bf006a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4c08014040 tx=0x7f4c08008030 comp rx=0 tx=0).stop 2023-11-28T12:07:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.648+0000 7f4c140b4700 1 -- 172.21.15.57:0/184303835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c15cc80 msgr2=0x7f4c0c0b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.648+0000 7f4c140b4700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c15cc80 0x7f4c0c0b02e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400cba0 tx=0x7f4c0400ceb0 comp rx=0 tx=0).stop 2023-11-28T12:07:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.648+0000 7f4c140b4700 1 -- 172.21.15.57:0/184303835 shutdown_connections 2023-11-28T12:07:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.648+0000 7f4c140b4700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4bf00682f0 0x7f4bf006a7a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.648+0000 7f4c140b4700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c15cc80 0x7f4c0c0b02e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.648+0000 7f4c140b4700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c0c06a270 0x7f4c0c0c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.648+0000 7f4c140b4700 1 --2- 172.21.15.57:0/184303835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c0c06e3a0 0x7f4c0c0b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.648+0000 7f4c140b4700 1 -- 172.21.15.57:0/184303835 >> 172.21.15.57:0/184303835 conn(0x7f4c0c06c010 msgr2=0x7f4c0c0adb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.648+0000 7f4c140b4700 1 -- 172.21.15.57:0/184303835 shutdown_connections 2023-11-28T12:07:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:04.648+0000 7f4c140b4700 1 -- 172.21.15.57:0/184303835 wait complete. 2023-11-28T12:07:04.663 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:07:09.664 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:07:09.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.805+0000 7f6572ea4700 1 Processor -- start 2023-11-28T12:07:09.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.805+0000 7f6572ea4700 1 -- start start 2023-11-28T12:07:09.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f6572ea4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656c0d15f0 0x7f656c15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:09.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f6572ea4700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c15af30 con 0x7f656c15e010 2023-11-28T12:07:09.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f6572ea4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c15ef40 con 0x7f656c06e400 2023-11-28T12:07:09.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f6572ea4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c15a9d0 con 0x7f656c0d15f0 2023-11-28T12:07:09.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f6571441700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f656c15e010 0x7f656c0ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:34460/0 (socket says 172.21.15.57:34460) 2023-11-28T12:07:09.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f6571441700 1 -- 172.21.15.57:0/2401579250 learned_addr learned my addr 172.21.15.57:0/2401579250 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:09.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f656bfff700 1 --2- 172.21.15.57:0/2401579250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656c0d15f0 0x7f656c15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:09.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f656b7fe700 1 -- 172.21.15.57:0/2401579250 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516559789 0 0) 0x7f656c15af30 con 0x7f656c15e010 2023-11-28T12:07:09.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f656b7fe700 1 -- 172.21.15.57:0/2401579250 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f655c0035b0 con 0x7f656c15e010 2023-11-28T12:07:09.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f656b7fe700 1 -- 172.21.15.57:0/2401579250 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552355446 0 0) 0x7f656c15ef40 con 0x7f656c06e400 2023-11-28T12:07:09.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f656b7fe700 1 -- 172.21.15.57:0/2401579250 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f656c15af30 con 0x7f656c06e400 2023-11-28T12:07:09.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f656bfff700 1 -- 172.21.15.57:0/2401579250 >> v1:172.21.15.181:6789/0 conn(0x7f656c06e400 legacy=0x7f656c06e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:09.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f656bfff700 1 -- 172.21.15.57:0/2401579250 >> v1:172.21.15.57:6789/0 conn(0x7f656c15e010 legacy=0x7f656c0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:09.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f656b7fe700 1 -- 172.21.15.57:0/2401579250 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2081085555 0 0) 0x7f655c0035b0 con 0x7f656c15e010 2023-11-28T12:07:09.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.806+0000 7f656bfff700 1 -- 172.21.15.57:0/2401579250 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c15e980 con 0x7f656c0d15f0 2023-11-28T12:07:09.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.807+0000 7f656bfff700 1 --2- 172.21.15.57:0/2401579250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656c0d15f0 0x7f656c15b3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6558009870 tx=0x7f6558009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b17e7b6907bbbb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:09.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.807+0000 7f656b7fe700 1 -- 172.21.15.57:0/2401579250 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65640019f0 con 0x7f656c0d15f0 2023-11-28T12:07:09.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.807+0000 7f656b7fe700 1 -- 172.21.15.57:0/2401579250 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6558004030 con 0x7f656c0d15f0 2023-11-28T12:07:09.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.807+0000 7f656b7fe700 1 -- 172.21.15.57:0/2401579250 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65580043c0 con 0x7f656c0d15f0 2023-11-28T12:07:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.807+0000 7f6572ea4700 1 -- 172.21.15.57:0/2401579250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656c0d15f0 msgr2=0x7f656c15b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.807+0000 7f6572ea4700 1 --2- 172.21.15.57:0/2401579250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656c0d15f0 0x7f656c15b3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6558009870 tx=0x7f6558009b80 comp rx=0 tx=0).stop 2023-11-28T12:07:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.807+0000 7f6572ea4700 1 -- 172.21.15.57:0/2401579250 shutdown_connections 2023-11-28T12:07:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.807+0000 7f6572ea4700 1 --2- 172.21.15.57:0/2401579250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656c0d15f0 0x7f656c15b3c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.807+0000 7f6572ea4700 1 -- 172.21.15.57:0/2401579250 >> 172.21.15.57:0/2401579250 conn(0x7f656c06a270 msgr2=0x7f656c06a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.807+0000 7f6572ea4700 1 -- 172.21.15.57:0/2401579250 shutdown_connections 2023-11-28T12:07:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.807+0000 7f6572ea4700 1 -- 172.21.15.57:0/2401579250 wait complete. 2023-11-28T12:07:09.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.809+0000 7f6572ea4700 1 Processor -- start 2023-11-28T12:07:09.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.809+0000 7f6572ea4700 1 -- start start 2023-11-28T12:07:09.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6572ea4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f656c06e400 0x7f656c1e5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:09.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6572ea4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656c0d15f0 0x7f656c1410a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:09.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6572ea4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f656c15e010 0x7f656c1415e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:09.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6572ea4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c153a60 con 0x7f656c15e010 2023-11-28T12:07:09.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6572ea4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c1591b0 con 0x7f656c06e400 2023-11-28T12:07:09.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6572ea4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c159490 con 0x7f656c0d15f0 2023-11-28T12:07:09.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6571441700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f656c15e010 0x7f656c1415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:09.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6571441700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f656c15e010 0x7f656c1415e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34486/0 (socket says 172.21.15.57:34486) 2023-11-28T12:07:09.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6571441700 1 -- 172.21.15.57:0/1516854218 learned_addr learned my addr 172.21.15.57:0/1516854218 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:09.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f656bfff700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656c0d15f0 0x7f656c1410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:09.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6570c40700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f656c06e400 0x7f656c1e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:09.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6571441700 1 -- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656c0d15f0 msgr2=0x7f656c1410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:09.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6571441700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656c0d15f0 0x7f656c1410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:09.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6571441700 1 -- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f656c06e400 msgr2=0x7f656c1e5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:09.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6571441700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f656c06e400 0x7f656c1e5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:09.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6571441700 1 -- 172.21.15.57:0/1516854218 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65580094a0 con 0x7f656c15e010 2023-11-28T12:07:09.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.810+0000 7f6571441700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f656c15e010 0x7f656c1415e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f656400cc00 tx=0x7f656400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:09.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.811+0000 7f65697fa700 1 -- 172.21.15.57:0/1516854218 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656400aa80 con 0x7f656c15e010 2023-11-28T12:07:09.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.811+0000 7f65697fa700 1 -- 172.21.15.57:0/1516854218 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656400abe0 con 0x7f656c15e010 2023-11-28T12:07:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.811+0000 7f6572ea4700 1 -- 172.21.15.57:0/1516854218 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656c15abc0 con 0x7f656c15e010 2023-11-28T12:07:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.811+0000 7f65697fa700 1 -- 172.21.15.57:0/1516854218 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65640070d0 con 0x7f656c15e010 2023-11-28T12:07:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.811+0000 7f6572ea4700 1 -- 172.21.15.57:0/1516854218 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c15ec60 con 0x7f656c15e010 2023-11-28T12:07:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.812+0000 7f6572ea4700 1 -- 172.21.15.57:0/1516854218 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656c15b900 con 0x7f656c15e010 2023-11-28T12:07:09.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.813+0000 7f65697fa700 1 -- 172.21.15.57:0/1516854218 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f656401a030 con 0x7f656c15e010 2023-11-28T12:07:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.813+0000 7f65697fa700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65500682f0 0x7f655006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.813+0000 7f65697fa700 1 -- 172.21.15.57:0/1516854218 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f6564085c00 con 0x7f656c15e010 2023-11-28T12:07:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.813+0000 7f6570c40700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65500682f0 0x7f655006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.814+0000 7f6570c40700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65500682f0 0x7f655006a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f65600011e0 tx=0x7f6560009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:09.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:09.817+0000 7f65697fa700 1 -- 172.21.15.57:0/1516854218 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6564052480 con 0x7f656c15e010 2023-11-28T12:07:10.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.061+0000 7f6572ea4700 1 -- 172.21.15.57:0/1516854218 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f656c0743f0 con 0x7f656c15e010 2023-11-28T12:07:10.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.062+0000 7f65697fa700 1 -- 172.21.15.57:0/1516854218 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f6564018080 con 0x7f656c15e010 2023-11-28T12:07:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:10.064 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:07:10.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.064+0000 7f6572ea4700 1 -- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65500682f0 msgr2=0x7f655006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:10.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.064+0000 7f6572ea4700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65500682f0 0x7f655006a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f65600011e0 tx=0x7f6560009040 comp rx=0 tx=0).stop 2023-11-28T12:07:10.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.064+0000 7f6572ea4700 1 -- 172.21.15.57:0/1516854218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f656c15e010 msgr2=0x7f656c1415e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:10.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.064+0000 7f6572ea4700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f656c15e010 0x7f656c1415e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f656400cc00 tx=0x7f656400cf10 comp rx=0 tx=0).stop 2023-11-28T12:07:10.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.064+0000 7f6572ea4700 1 -- 172.21.15.57:0/1516854218 shutdown_connections 2023-11-28T12:07:10.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.064+0000 7f6572ea4700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65500682f0 0x7f655006a7a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:10.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.064+0000 7f6572ea4700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f656c15e010 0x7f656c1415e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:10.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.064+0000 7f6572ea4700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f656c06e400 0x7f656c1e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:10.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.064+0000 7f6572ea4700 1 --2- 172.21.15.57:0/1516854218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656c0d15f0 0x7f656c1410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:10.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.064+0000 7f6572ea4700 1 -- 172.21.15.57:0/1516854218 >> 172.21.15.57:0/1516854218 conn(0x7f656c06a270 msgr2=0x7f656c06c260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:10.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.065+0000 7f6572ea4700 1 -- 172.21.15.57:0/1516854218 shutdown_connections 2023-11-28T12:07:10.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:10.065+0000 7f6572ea4700 1 -- 172.21.15.57:0/1516854218 wait complete. 2023-11-28T12:07:10.079 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:07:15.080 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:07:15.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.223+0000 7feb5df47700 1 Processor -- start 2023-11-28T12:07:15.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.223+0000 7feb5df47700 1 -- start start 2023-11-28T12:07:15.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.224+0000 7feb5df47700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb5815cc80 0x7feb5806de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:15.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.224+0000 7feb5df47700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb580b1810 con 0x7feb5806a270 2023-11-28T12:07:15.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.224+0000 7feb5df47700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb580b1af0 con 0x7feb5806e3a0 2023-11-28T12:07:15.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.224+0000 7feb5df47700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb580b2490 con 0x7feb5815cc80 2023-11-28T12:07:15.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.224+0000 7feb577fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7feb5806a270 0x7feb5806a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:56684/0 (socket says 172.21.15.57:56684) 2023-11-28T12:07:15.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.224+0000 7feb577fe700 1 -- 172.21.15.57:0/4187531435 learned_addr learned my addr 172.21.15.57:0/4187531435 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:15.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.224+0000 7feb56ffd700 1 --2- 172.21.15.57:0/4187531435 >> v2:172.21.15.181:3300/0 conn(0x7feb5815cc80 0x7feb5806de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:15.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.224+0000 7feb567fc700 1 -- 172.21.15.57:0/4187531435 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410798605 0 0) 0x7feb580b1810 con 0x7feb5806a270 2023-11-28T12:07:15.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.224+0000 7feb567fc700 1 -- 172.21.15.57:0/4187531435 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb440035b0 con 0x7feb5806a270 2023-11-28T12:07:15.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.225+0000 7feb567fc700 1 -- 172.21.15.57:0/4187531435 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2944784778 0 0) 0x7feb440035b0 con 0x7feb5806a270 2023-11-28T12:07:15.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.225+0000 7feb56ffd700 1 -- 172.21.15.57:0/4187531435 >> v1:172.21.15.181:6789/0 conn(0x7feb5806e3a0 legacy=0x7feb58070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:15.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.225+0000 7feb56ffd700 1 -- 172.21.15.57:0/4187531435 >> v1:172.21.15.57:6789/0 conn(0x7feb5806a270 legacy=0x7feb5806a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:15.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.225+0000 7feb56ffd700 1 -- 172.21.15.57:0/4187531435 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb580b1dd0 con 0x7feb5815cc80 2023-11-28T12:07:15.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.225+0000 7feb56ffd700 1 --2- 172.21.15.57:0/4187531435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5815cc80 0x7feb5806de60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7feb40009840 tx=0x7feb40009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f0ee3cbd8e000f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:15.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.225+0000 7feb567fc700 1 -- 172.21.15.57:0/4187531435 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48001670 con 0x7feb5815cc80 2023-11-28T12:07:15.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.225+0000 7feb567fc700 1 -- 172.21.15.57:0/4187531435 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb40004030 con 0x7feb5815cc80 2023-11-28T12:07:15.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.225+0000 7feb567fc700 1 -- 172.21.15.57:0/4187531435 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb400043a0 con 0x7feb5815cc80 2023-11-28T12:07:15.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.226+0000 7feb5df47700 1 -- 172.21.15.57:0/4187531435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5815cc80 msgr2=0x7feb5806de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:15.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.226+0000 7feb5df47700 1 --2- 172.21.15.57:0/4187531435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5815cc80 0x7feb5806de60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7feb40009840 tx=0x7feb40009b50 comp rx=0 tx=0).stop 2023-11-28T12:07:15.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.226+0000 7feb5df47700 1 -- 172.21.15.57:0/4187531435 shutdown_connections 2023-11-28T12:07:15.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.226+0000 7feb5df47700 1 --2- 172.21.15.57:0/4187531435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5815cc80 0x7feb5806de60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:15.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.226+0000 7feb5df47700 1 -- 172.21.15.57:0/4187531435 >> 172.21.15.57:0/4187531435 conn(0x7feb5806c010 msgr2=0x7feb580cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:15.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.226+0000 7feb5df47700 1 -- 172.21.15.57:0/4187531435 shutdown_connections 2023-11-28T12:07:15.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.226+0000 7feb5df47700 1 -- 172.21.15.57:0/4187531435 wait complete. 2023-11-28T12:07:15.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.227+0000 7feb5df47700 1 Processor -- start 2023-11-28T12:07:15.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.227+0000 7feb5df47700 1 -- start start 2023-11-28T12:07:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb5df47700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5806a270 0x7feb58153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb5df47700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5806e3a0 0x7feb581f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb5df47700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb5815cc80 0x7feb58070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb5df47700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb580c4fd0 con 0x7feb5815cc80 2023-11-28T12:07:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb5df47700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb580c4c30 con 0x7feb5806a270 2023-11-28T12:07:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb5df47700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb581f78b0 con 0x7feb5806e3a0 2023-11-28T12:07:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb57fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb5815cc80 0x7feb58070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb57fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb5815cc80 0x7feb58070c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54446/0 (socket says 172.21.15.57:54446) 2023-11-28T12:07:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb57fff700 1 -- 172.21.15.57:0/1885896246 learned_addr learned my addr 172.21.15.57:0/1885896246 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb56ffd700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5806e3a0 0x7feb581f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb577fe700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5806a270 0x7feb58153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb57fff700 1 -- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5806e3a0 msgr2=0x7feb581f6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb57fff700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5806e3a0 0x7feb581f6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb57fff700 1 -- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5806a270 msgr2=0x7feb58153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb57fff700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5806a270 0x7feb58153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb57fff700 1 -- 172.21.15.57:0/1885896246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400094a0 con 0x7feb5815cc80 2023-11-28T12:07:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.228+0000 7feb577fe700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5806a270 0x7feb58153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.229+0000 7feb57fff700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb5815cc80 0x7feb58070c20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7feb5000c7f0 tx=0x7feb5000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.229+0000 7feb4ffff700 1 -- 172.21.15.57:0/1885896246 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50003e30 con 0x7feb5815cc80 2023-11-28T12:07:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.229+0000 7feb4ffff700 1 -- 172.21.15.57:0/1885896246 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb50005070 con 0x7feb5815cc80 2023-11-28T12:07:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.229+0000 7feb5df47700 1 -- 172.21.15.57:0/1885896246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5806e7c0 con 0x7feb5815cc80 2023-11-28T12:07:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.229+0000 7feb4ffff700 1 -- 172.21.15.57:0/1885896246 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50005400 con 0x7feb5815cc80 2023-11-28T12:07:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.229+0000 7feb5df47700 1 -- 172.21.15.57:0/1885896246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb580b20b0 con 0x7feb5815cc80 2023-11-28T12:07:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.230+0000 7feb5df47700 1 -- 172.21.15.57:0/1885896246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5806e950 con 0x7feb5815cc80 2023-11-28T12:07:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.230+0000 7feb4ffff700 1 -- 172.21.15.57:0/1885896246 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feb50010070 con 0x7feb5815cc80 2023-11-28T12:07:15.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.231+0000 7feb4ffff700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb380682f0 0x7feb3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:15.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.231+0000 7feb4ffff700 1 -- 172.21.15.57:0/1885896246 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7feb500858b0 con 0x7feb5815cc80 2023-11-28T12:07:15.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.231+0000 7feb577fe700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb380682f0 0x7feb3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:15.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.231+0000 7feb577fe700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb380682f0 0x7feb3806a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feb48005a70 tx=0x7feb48001c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:15.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.235+0000 7feb4ffff700 1 -- 172.21.15.57:0/1885896246 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feb5806e950 con 0x7feb5815cc80 2023-11-28T12:07:15.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.482+0000 7feb5df47700 1 -- 172.21.15.57:0/1885896246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7feb580c5110 con 0x7feb5815cc80 2023-11-28T12:07:15.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.483+0000 7feb4ffff700 1 -- 172.21.15.57:0/1885896246 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7feb580c5110 con 0x7feb5815cc80 2023-11-28T12:07:15.484 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:15.484 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:07:15.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.485+0000 7feb5df47700 1 -- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb380682f0 msgr2=0x7feb3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:15.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.485+0000 7feb5df47700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb380682f0 0x7feb3806a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feb48005a70 tx=0x7feb48001c00 comp rx=0 tx=0).stop 2023-11-28T12:07:15.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.485+0000 7feb5df47700 1 -- 172.21.15.57:0/1885896246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb5815cc80 msgr2=0x7feb58070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.485+0000 7feb5df47700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb5815cc80 0x7feb58070c20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7feb5000c7f0 tx=0x7feb5000cb00 comp rx=0 tx=0).stop 2023-11-28T12:07:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.485+0000 7feb5df47700 1 -- 172.21.15.57:0/1885896246 shutdown_connections 2023-11-28T12:07:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.485+0000 7feb5df47700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feb380682f0 0x7feb3806a7a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.485+0000 7feb5df47700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb5815cc80 0x7feb58070c20 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.485+0000 7feb5df47700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5806a270 0x7feb58153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.485+0000 7feb5df47700 1 --2- 172.21.15.57:0/1885896246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb5806e3a0 0x7feb581f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.485+0000 7feb5df47700 1 -- 172.21.15.57:0/1885896246 >> 172.21.15.57:0/1885896246 conn(0x7feb5806c010 msgr2=0x7feb580af730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.485+0000 7feb5df47700 1 -- 172.21.15.57:0/1885896246 shutdown_connections 2023-11-28T12:07:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:15.485+0000 7feb5df47700 1 -- 172.21.15.57:0/1885896246 wait complete. 2023-11-28T12:07:15.499 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:07:16.920 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:07:16.923 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:07:20.500 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:07:20.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.642+0000 7fd358c05700 1 Processor -- start 2023-11-28T12:07:20.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.642+0000 7fd358c05700 1 -- start start 2023-11-28T12:07:20.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.643+0000 7fd358c05700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd35406ab10 0x7fd35406aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:20.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.643+0000 7fd358c05700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3540d0450 0x7fd354160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:20.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.643+0000 7fd358c05700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3540d1ee0 con 0x7fd3540cd5a0 2023-11-28T12:07:20.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.643+0000 7fd358c05700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35406b420 con 0x7fd35406ab10 2023-11-28T12:07:20.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.643+0000 7fd358c05700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540cdb10 con 0x7fd3540d0450 2023-11-28T12:07:20.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.644+0000 7fd351d9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd3540cd5a0 0x7fd354160300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:56694/0 (socket says 172.21.15.57:56694) 2023-11-28T12:07:20.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.644+0000 7fd351d9b700 1 -- 172.21.15.57:0/4015015656 learned_addr learned my addr 172.21.15.57:0/4015015656 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:20.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.644+0000 7fd352d9d700 1 --2- 172.21.15.57:0/4015015656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3540d0450 0x7fd354160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:20.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.644+0000 7fd35259c700 1 --2- 172.21.15.57:0/4015015656 >> v2:172.21.15.181:3300/0 conn(0x7fd35406ab10 0x7fd35406aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:20.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.644+0000 7fd35159a700 1 -- 172.21.15.57:0/4015015656 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189628851 0 0) 0x7fd3540d1ee0 con 0x7fd3540cd5a0 2023-11-28T12:07:20.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.644+0000 7fd35159a700 1 -- 172.21.15.57:0/4015015656 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3400035b0 con 0x7fd3540cd5a0 2023-11-28T12:07:20.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.644+0000 7fd352d9d700 1 -- 172.21.15.57:0/4015015656 >> v2:172.21.15.181:3300/0 conn(0x7fd35406ab10 msgr2=0x7fd35406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:20.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.644+0000 7fd352d9d700 1 --2- 172.21.15.57:0/4015015656 >> v2:172.21.15.181:3300/0 conn(0x7fd35406ab10 0x7fd35406aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:20.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.644+0000 7fd352d9d700 1 -- 172.21.15.57:0/4015015656 >> v1:172.21.15.57:6789/0 conn(0x7fd3540cd5a0 legacy=0x7fd354160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:20.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.644+0000 7fd35159a700 1 -- 172.21.15.57:0/4015015656 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 350895834 0 0) 0x7fd3400035b0 con 0x7fd3540cd5a0 2023-11-28T12:07:20.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.644+0000 7fd352d9d700 1 -- 172.21.15.57:0/4015015656 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3540d1890 con 0x7fd3540d0450 2023-11-28T12:07:20.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.644+0000 7fd35259c700 1 --2- 172.21.15.57:0/4015015656 >> v2:172.21.15.181:3300/0 conn(0x7fd35406ab10 0x7fd35406aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:07:20.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.645+0000 7fd352d9d700 1 --2- 172.21.15.57:0/4015015656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3540d0450 0x7fd354160a10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009d90 tx=0x7fd34c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc6f87438a77727d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:20.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.645+0000 7fd35159a700 1 -- 172.21.15.57:0/4015015656 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00e070 con 0x7fd3540d0450 2023-11-28T12:07:20.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.645+0000 7fd35159a700 1 -- 172.21.15.57:0/4015015656 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c003a20 con 0x7fd3540d0450 2023-11-28T12:07:20.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.645+0000 7fd35159a700 1 -- 172.21.15.57:0/4015015656 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c001670 con 0x7fd3540d0450 2023-11-28T12:07:20.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.645+0000 7fd358c05700 1 -- 172.21.15.57:0/4015015656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3540d0450 msgr2=0x7fd354160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:20.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.645+0000 7fd358c05700 1 --2- 172.21.15.57:0/4015015656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3540d0450 0x7fd354160a10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009d90 tx=0x7fd34c00b330 comp rx=0 tx=0).stop 2023-11-28T12:07:20.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.645+0000 7fd358c05700 1 -- 172.21.15.57:0/4015015656 shutdown_connections 2023-11-28T12:07:20.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.645+0000 7fd358c05700 1 --2- 172.21.15.57:0/4015015656 >> v2:172.21.15.181:3300/0 conn(0x7fd35406ab10 0x7fd35406aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:20.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.645+0000 7fd358c05700 1 --2- 172.21.15.57:0/4015015656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3540d0450 0x7fd354160a10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.645+0000 7fd358c05700 1 -- 172.21.15.57:0/4015015656 >> 172.21.15.57:0/4015015656 conn(0x7fd35406c520 msgr2=0x7fd35406c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.645+0000 7fd358c05700 1 -- 172.21.15.57:0/4015015656 shutdown_connections 2023-11-28T12:07:20.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.645+0000 7fd358c05700 1 -- 172.21.15.57:0/4015015656 wait complete. 2023-11-28T12:07:20.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.647+0000 7fd358c05700 1 Processor -- start 2023-11-28T12:07:20.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.647+0000 7fd358c05700 1 -- start start 2023-11-28T12:07:20.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd358c05700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd35406ab10 0x7fd3540cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:20.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd358c05700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3540cd5a0 0x7fd354158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:20.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd358c05700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3540d0450 0x7fd354143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:20.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd358c05700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540c6a80 con 0x7fd3540cd5a0 2023-11-28T12:07:20.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd358c05700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540c6cd0 con 0x7fd3540d0450 2023-11-28T12:07:20.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd358c05700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540c6fb0 con 0x7fd35406ab10 2023-11-28T12:07:20.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd351d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3540cd5a0 0x7fd354158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:20.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd351d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3540cd5a0 0x7fd354158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54454/0 (socket says 172.21.15.57:54454) 2023-11-28T12:07:20.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd351d9b700 1 -- 172.21.15.57:0/2749418036 learned_addr learned my addr 172.21.15.57:0/2749418036 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:20.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd35259c700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd35406ab10 0x7fd3540cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:20.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd352d9d700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3540d0450 0x7fd354143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:20.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd351d9b700 1 -- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd35406ab10 msgr2=0x7fd3540cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:20.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd351d9b700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd35406ab10 0x7fd3540cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:20.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd351d9b700 1 -- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3540d0450 msgr2=0x7fd354143ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:20.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd351d9b700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3540d0450 0x7fd354143ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:20.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd351d9b700 1 -- 172.21.15.57:0/2749418036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3440094a0 con 0x7fd3540cd5a0 2023-11-28T12:07:20.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd35259c700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd35406ab10 0x7fd3540cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:07:20.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd351d9b700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3540cd5a0 0x7fd354158d10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c730 tx=0x7fd33c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:20.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.648+0000 7fd34affd700 1 -- 172.21.15.57:0/2749418036 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c004c80 con 0x7fd3540cd5a0 2023-11-28T12:07:20.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.649+0000 7fd34affd700 1 -- 172.21.15.57:0/2749418036 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c004de0 con 0x7fd3540cd5a0 2023-11-28T12:07:20.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.649+0000 7fd358c05700 1 -- 172.21.15.57:0/2749418036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34c009a70 con 0x7fd3540cd5a0 2023-11-28T12:07:20.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.649+0000 7fd34affd700 1 -- 172.21.15.57:0/2749418036 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c007310 con 0x7fd3540cd5a0 2023-11-28T12:07:20.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.649+0000 7fd358c05700 1 -- 172.21.15.57:0/2749418036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35415cad0 con 0x7fd3540cd5a0 2023-11-28T12:07:20.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.650+0000 7fd358c05700 1 -- 172.21.15.57:0/2749418036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3540d1b70 con 0x7fd3540cd5a0 2023-11-28T12:07:20.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.650+0000 7fd34affd700 1 -- 172.21.15.57:0/2749418036 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd33c01a020 con 0x7fd3540cd5a0 2023-11-28T12:07:20.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.651+0000 7fd34affd700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd3340682f0 0x7fd33406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:20.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.651+0000 7fd34affd700 1 -- 172.21.15.57:0/2749418036 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fd33c085a30 con 0x7fd3540cd5a0 2023-11-28T12:07:20.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.651+0000 7fd35259c700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd3340682f0 0x7fd33406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:20.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.651+0000 7fd35259c700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd3340682f0 0x7fd33406a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd344011380 tx=0x7fd344009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:20.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.655+0000 7fd34affd700 1 -- 172.21.15.57:0/2749418036 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd33c052190 con 0x7fd3540cd5a0 2023-11-28T12:07:20.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.900+0000 7fd358c05700 1 -- 172.21.15.57:0/2749418036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd3540b1ec0 con 0x7fd3540cd5a0 2023-11-28T12:07:20.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.900+0000 7fd34affd700 1 -- 172.21.15.57:0/2749418036 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fd33c0557b0 con 0x7fd3540cd5a0 2023-11-28T12:07:20.902 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:20.902 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:07:20.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.903+0000 7fd358c05700 1 -- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd3340682f0 msgr2=0x7fd33406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:20.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.903+0000 7fd358c05700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd3340682f0 0x7fd33406a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd344011380 tx=0x7fd344009210 comp rx=0 tx=0).stop 2023-11-28T12:07:20.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.903+0000 7fd358c05700 1 -- 172.21.15.57:0/2749418036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3540cd5a0 msgr2=0x7fd354158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:20.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.903+0000 7fd358c05700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3540cd5a0 0x7fd354158d10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c730 tx=0x7fd33c00ca40 comp rx=0 tx=0).stop 2023-11-28T12:07:20.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.903+0000 7fd358c05700 1 -- 172.21.15.57:0/2749418036 shutdown_connections 2023-11-28T12:07:20.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.903+0000 7fd358c05700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd3340682f0 0x7fd33406a7a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:20.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.903+0000 7fd358c05700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3540d0450 0x7fd354143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:20.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.903+0000 7fd358c05700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd35406ab10 0x7fd3540cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:20.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.903+0000 7fd358c05700 1 --2- 172.21.15.57:0/2749418036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3540cd5a0 0x7fd354158d10 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:20.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.903+0000 7fd358c05700 1 -- 172.21.15.57:0/2749418036 >> 172.21.15.57:0/2749418036 conn(0x7fd35406c520 msgr2=0x7fd3540b7230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:20.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.903+0000 7fd358c05700 1 -- 172.21.15.57:0/2749418036 shutdown_connections 2023-11-28T12:07:20.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:20.903+0000 7fd358c05700 1 -- 172.21.15.57:0/2749418036 wait complete. 2023-11-28T12:07:20.917 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T12:07:25.919 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T12:07:26.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.060+0000 7f887b34c700 1 Processor -- start 2023-11-28T12:07:26.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.061+0000 7f887b34c700 1 -- start start 2023-11-28T12:07:26.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.061+0000 7f887b34c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f887406e400 0x7f887406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.061+0000 7f887b34c700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887415af30 con 0x7f88740d15f0 2023-11-28T12:07:26.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.061+0000 7f887b34c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887415ef40 con 0x7f887415e010 2023-11-28T12:07:26.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.061+0000 7f887b34c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887415a9d0 con 0x7f887406e400 2023-11-28T12:07:26.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f88788e7700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f88740d15f0 0x7f887415b3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49800/0 (socket says 172.21.15.57:49800) 2023-11-28T12:07:26.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f88788e7700 1 -- 172.21.15.57:0/3612188817 learned_addr learned my addr 172.21.15.57:0/3612188817 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:26.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f88790e8700 1 --2- 172.21.15.57:0/3612188817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f887406e400 0x7f887406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897072754 0 0) 0x7f887415af30 con 0x7f88740d15f0 2023-11-28T12:07:26.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88600035b0 con 0x7f88740d15f0 2023-11-28T12:07:26.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895804620 0 0) 0x7f887415ef40 con 0x7f887415e010 2023-11-28T12:07:26.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f887415af30 con 0x7f887415e010 2023-11-28T12:07:26.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 208532699 0 0) 0x7f88600035b0 con 0x7f88740d15f0 2023-11-28T12:07:26.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f887406e400 msgr2=0x7f887406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:26.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 --2- 172.21.15.57:0/3612188817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f887406e400 0x7f887406e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 >> v1:172.21.15.181:6789/0 conn(0x7f887415e010 legacy=0x7f88740ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:26.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887406ed10 con 0x7f88740d15f0 2023-11-28T12:07:26.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f8864001670 con 0x7f88740d15f0 2023-11-28T12:07:26.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 >> v1:172.21.15.57:6789/0 conn(0x7f88740d15f0 legacy=0x7f887415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:26.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 --2- 172.21.15.57:0/3612188817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8860003bc0 0x7f8860003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 --2- 172.21.15.57:0/3612188817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8860004850 0x7f8860004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 --2- 172.21.15.57:0/3612188817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f887406e400 0x7f8860005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886000bac0 con 0x7f8860004850 2023-11-28T12:07:26.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886000bc00 con 0x7f8860003bc0 2023-11-28T12:07:26.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886000bd70 con 0x7f887406e400 2023-11-28T12:07:26.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.062+0000 7f88788e7700 1 --2- 172.21.15.57:0/3612188817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8860004850 0x7f8860004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.063+0000 7f88790e8700 1 --2- 172.21.15.57:0/3612188817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8860003bc0 0x7f8860003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.063+0000 7f88798e9700 1 --2- 172.21.15.57:0/3612188817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f887406e400 0x7f8860005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.063+0000 7f88788e7700 1 -- 172.21.15.57:0/3612188817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f887406e400 msgr2=0x7f8860005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:26.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.063+0000 7f88788e7700 1 --2- 172.21.15.57:0/3612188817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f887406e400 0x7f8860005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.063+0000 7f88788e7700 1 -- 172.21.15.57:0/3612188817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8860003bc0 msgr2=0x7f8860003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:26.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.063+0000 7f88788e7700 1 --2- 172.21.15.57:0/3612188817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8860003bc0 0x7f8860003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.063+0000 7f88788e7700 1 -- 172.21.15.57:0/3612188817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88600035b0 con 0x7f8860004850 2023-11-28T12:07:26.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.063+0000 7f88788e7700 1 --2- 172.21.15.57:0/3612188817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8860004850 0x7f8860004cc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f88740d1a00 tx=0x7f8860005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b3c1436dde2d88 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:26.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.063+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886400cd20 con 0x7f8860004850 2023-11-28T12:07:26.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.063+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8864001af0 con 0x7f8860004850 2023-11-28T12:07:26.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.063+0000 7f886bfff700 1 -- 172.21.15.57:0/3612188817 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886400cea0 con 0x7f8860004850 2023-11-28T12:07:26.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.064+0000 7f887b34c700 1 -- 172.21.15.57:0/3612188817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8860004850 msgr2=0x7f8860004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:26.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.064+0000 7f887b34c700 1 --2- 172.21.15.57:0/3612188817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8860004850 0x7f8860004cc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f88740d1a00 tx=0x7f8860005880 comp rx=0 tx=0).stop 2023-11-28T12:07:26.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.064+0000 7f88798e9700 1 -- 172.21.15.57:0/3612188817 reap_dead start 2023-11-28T12:07:26.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.064+0000 7f887b34c700 1 -- 172.21.15.57:0/3612188817 shutdown_connections 2023-11-28T12:07:26.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.064+0000 7f887b34c700 1 -- 172.21.15.57:0/3612188817 >> 172.21.15.57:0/3612188817 conn(0x7f887406a270 msgr2=0x7f887406a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:26.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.064+0000 7f887b34c700 1 -- 172.21.15.57:0/3612188817 shutdown_connections 2023-11-28T12:07:26.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.064+0000 7f887b34c700 1 -- 172.21.15.57:0/3612188817 wait complete. 2023-11-28T12:07:26.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.065+0000 7f887b34c700 1 Processor -- start 2023-11-28T12:07:26.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.065+0000 7f887b34c700 1 -- start start 2023-11-28T12:07:26.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f887b34c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8860004850 0x7f88740b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f887b34c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88740d0470 0x7f88741570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f887b34c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88740b3120 0x7f8874159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f887b34c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88740b2c90 con 0x7f88740b3120 2023-11-28T12:07:26.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f887b34c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88740b23d0 con 0x7f88740d0470 2023-11-28T12:07:26.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f887b34c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88740b2640 con 0x7f8860004850 2023-11-28T12:07:26.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f88798e9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88740b3120 0x7f8874159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f88798e9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88740b3120 0x7f8874159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38846/0 (socket says 172.21.15.57:38846) 2023-11-28T12:07:26.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f88798e9700 1 -- 172.21.15.57:0/676137415 learned_addr learned my addr 172.21.15.57:0/676137415 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:26.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f88788e7700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88740d0470 0x7f88741570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f88790e8700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8860004850 0x7f88740b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f88798e9700 1 -- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8860004850 msgr2=0x7f88740b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f88798e9700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8860004850 0x7f88740b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f88798e9700 1 -- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88740d0470 msgr2=0x7f88741570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f88798e9700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88740d0470 0x7f88741570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f88798e9700 1 -- 172.21.15.57:0/676137415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887415ec60 con 0x7f88740b3120 2023-11-28T12:07:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f88788e7700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88740d0470 0x7f88741570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.066+0000 7f88790e8700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8860004850 0x7f88740b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.067+0000 7f88798e9700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88740b3120 0x7f8874159c10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f88740cd8b0 tx=0x7f886c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.067+0000 7f8869ffb700 1 -- 172.21.15.57:0/676137415 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c003c10 con 0x7f88740b3120 2023-11-28T12:07:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.067+0000 7f8869ffb700 1 -- 172.21.15.57:0/676137415 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c003d90 con 0x7f88740b3120 2023-11-28T12:07:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.067+0000 7f8869ffb700 1 -- 172.21.15.57:0/676137415 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00c950 con 0x7f88740b3120 2023-11-28T12:07:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.067+0000 7f887b34c700 1 -- 172.21.15.57:0/676137415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887415e980 con 0x7f88740b3120 2023-11-28T12:07:26.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.067+0000 7f887b34c700 1 -- 172.21.15.57:0/676137415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887415abc0 con 0x7f88740b3120 2023-11-28T12:07:26.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.068+0000 7f887b34c700 1 -- 172.21.15.57:0/676137415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88740b2e40 con 0x7f88740b3120 2023-11-28T12:07:26.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.068+0000 7f8869ffb700 1 -- 172.21.15.57:0/676137415 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f886c00cab0 con 0x7f88740b3120 2023-11-28T12:07:26.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.069+0000 7f8869ffb700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f88580682f0 0x7f885806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.069+0000 7f8869ffb700 1 -- 172.21.15.57:0/676137415 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f886c085fb0 con 0x7f88740b3120 2023-11-28T12:07:26.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.069+0000 7f88790e8700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f88580682f0 0x7f885806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.069+0000 7f88790e8700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f88580682f0 0x7f885806a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8870009b80 tx=0x7f8870008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:26.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.073+0000 7f8869ffb700 1 -- 172.21.15.57:0/676137415 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f886c052700 con 0x7f88740b3120 2023-11-28T12:07:26.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.316+0000 7f887b34c700 1 -- 172.21.15.57:0/676137415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f88740743f0 con 0x7f88740b3120 2023-11-28T12:07:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.317+0000 7f8869ffb700 1 -- 172.21.15.57:0/676137415 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f886c055d20 con 0x7f88740b3120 2023-11-28T12:07:26.318 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:26.318 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T12:07:26.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.319+0000 7f887b34c700 1 -- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f88580682f0 msgr2=0x7f885806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:26.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.319+0000 7f887b34c700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f88580682f0 0x7f885806a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8870009b80 tx=0x7f8870008040 comp rx=0 tx=0).stop 2023-11-28T12:07:26.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.319+0000 7f887b34c700 1 -- 172.21.15.57:0/676137415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88740b3120 msgr2=0x7f8874159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:26.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.319+0000 7f887b34c700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88740b3120 0x7f8874159c10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f88740cd8b0 tx=0x7f886c00ef20 comp rx=0 tx=0).stop 2023-11-28T12:07:26.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.319+0000 7f887b34c700 1 -- 172.21.15.57:0/676137415 shutdown_connections 2023-11-28T12:07:26.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.319+0000 7f887b34c700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f88580682f0 0x7f885806a7a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.319+0000 7f887b34c700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88740b3120 0x7f8874159c10 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.319+0000 7f887b34c700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8860004850 0x7f88740b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.319+0000 7f887b34c700 1 --2- 172.21.15.57:0/676137415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88740d0470 0x7f88741570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.319+0000 7f887b34c700 1 -- 172.21.15.57:0/676137415 >> 172.21.15.57:0/676137415 conn(0x7f887406a270 msgr2=0x7f88740b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:26.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.319+0000 7f887b34c700 1 -- 172.21.15.57:0/676137415 shutdown_connections 2023-11-28T12:07:26.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.319+0000 7f887b34c700 1 -- 172.21.15.57:0/676137415 wait complete. 2023-11-28T12:07:26.334 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T12:07:26.334 DEBUG:teuthology.orchestra.run.smithi057:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T12:07:26.385 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T12:07:26.439 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T12:07:26.440 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:07:26.440 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:07:26.463 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:07:26.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.647+0000 7feacaee8700 1 Processor -- start 2023-11-28T12:07:26.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.647+0000 7feacaee8700 1 -- start start 2023-11-28T12:07:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.648+0000 7feacaee8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feac406a270 0x7feac406a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.648+0000 7feacaee8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac406e570 0x7feac4070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.648+0000 7feacaee8700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac406ab80 con 0x7feac415cc80 2023-11-28T12:07:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.648+0000 7feacaee8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac40b2490 con 0x7feac406a270 2023-11-28T12:07:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.648+0000 7feacaee8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac40b1810 con 0x7feac406e570 2023-11-28T12:07:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.648+0000 7feac3fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7feac415cc80 0x7feac406de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49804/0 (socket says 172.21.15.57:49804) 2023-11-28T12:07:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.648+0000 7feac3fff700 1 -- 172.21.15.57:0/397462761 learned_addr learned my addr 172.21.15.57:0/397462761 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.648+0000 7feac8c84700 1 --2- 172.21.15.57:0/397462761 >> v2:172.21.15.181:3300/0 conn(0x7feac406a270 0x7feac406a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.648+0000 7feac9485700 1 --2- 172.21.15.57:0/397462761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac406e570 0x7feac4070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.649+0000 7feac37fe700 1 -- 172.21.15.57:0/397462761 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1775345055 0 0) 0x7feac406ab80 con 0x7feac415cc80 2023-11-28T12:07:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.649+0000 7feac37fe700 1 -- 172.21.15.57:0/397462761 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaac0035b0 con 0x7feac415cc80 2023-11-28T12:07:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.649+0000 7feac9485700 1 -- 172.21.15.57:0/397462761 >> v2:172.21.15.181:3300/0 conn(0x7feac406a270 msgr2=0x7feac406a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.649+0000 7feac9485700 1 --2- 172.21.15.57:0/397462761 >> v2:172.21.15.181:3300/0 conn(0x7feac406a270 0x7feac406a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.649+0000 7feac9485700 1 -- 172.21.15.57:0/397462761 >> v1:172.21.15.57:6789/0 conn(0x7feac415cc80 legacy=0x7feac406de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.649+0000 7feac9485700 1 -- 172.21.15.57:0/397462761 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac40b1af0 con 0x7feac406e570 2023-11-28T12:07:26.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.649+0000 7feac8c84700 1 --2- 172.21.15.57:0/397462761 >> v2:172.21.15.181:3300/0 conn(0x7feac406a270 0x7feac406a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:07:26.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.649+0000 7feac9485700 1 --2- 172.21.15.57:0/397462761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac406e570 0x7feac4070d20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feabc009d90 tx=0x7feabc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b00da191ddc305a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:26.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.649+0000 7feac37fe700 1 -- 172.21.15.57:0/397462761 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc00e070 con 0x7feac406e570 2023-11-28T12:07:26.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.649+0000 7feac37fe700 1 -- 172.21.15.57:0/397462761 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc00bb20 con 0x7feac406e570 2023-11-28T12:07:26.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.649+0000 7feac37fe700 1 -- 172.21.15.57:0/397462761 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc012430 con 0x7feac406e570 2023-11-28T12:07:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.650+0000 7feacaee8700 1 -- 172.21.15.57:0/397462761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac406e570 msgr2=0x7feac4070d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.650+0000 7feacaee8700 1 --2- 172.21.15.57:0/397462761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac406e570 0x7feac4070d20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feabc009d90 tx=0x7feabc00b330 comp rx=0 tx=0).stop 2023-11-28T12:07:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.650+0000 7feacaee8700 1 -- 172.21.15.57:0/397462761 shutdown_connections 2023-11-28T12:07:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.650+0000 7feacaee8700 1 --2- 172.21.15.57:0/397462761 >> v2:172.21.15.181:3300/0 conn(0x7feac406a270 0x7feac406a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.650+0000 7feacaee8700 1 --2- 172.21.15.57:0/397462761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac406e570 0x7feac4070d20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.650+0000 7feacaee8700 1 -- 172.21.15.57:0/397462761 >> 172.21.15.57:0/397462761 conn(0x7feac406c010 msgr2=0x7feac40cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.650+0000 7feacaee8700 1 -- 172.21.15.57:0/397462761 shutdown_connections 2023-11-28T12:07:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.650+0000 7feacaee8700 1 -- 172.21.15.57:0/397462761 wait complete. 2023-11-28T12:07:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.651+0000 7feacaee8700 1 Processor -- start 2023-11-28T12:07:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.651+0000 7feacaee8700 1 -- start start 2023-11-28T12:07:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.652+0000 7feacaee8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac406a270 0x7feac414c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.652+0000 7feacaee8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feac406e570 0x7feac40c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.652+0000 7feacaee8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feac415cc80 0x7feac40c4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.652+0000 7feacaee8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac415f570 con 0x7feac415cc80 2023-11-28T12:07:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.652+0000 7feacaee8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac415e960 con 0x7feac406e570 2023-11-28T12:07:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.652+0000 7feacaee8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac415ec40 con 0x7feac406a270 2023-11-28T12:07:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.652+0000 7feac9485700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feac415cc80 0x7feac40c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.652+0000 7feac9485700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feac415cc80 0x7feac40c4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38850/0 (socket says 172.21.15.57:38850) 2023-11-28T12:07:26.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.652+0000 7feac9485700 1 -- 172.21.15.57:0/160463949 learned_addr learned my addr 172.21.15.57:0/160463949 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:26.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feac8c84700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac406a270 0x7feac414c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feac3fff700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feac406e570 0x7feac40c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feac9485700 1 -- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac406a270 msgr2=0x7feac414c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:26.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feac9485700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac406a270 0x7feac414c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feac9485700 1 -- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feac406e570 msgr2=0x7feac40c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:26.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feac9485700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feac406e570 0x7feac40c3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feac9485700 1 -- 172.21.15.57:0/160463949 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab40094a0 con 0x7feac415cc80 2023-11-28T12:07:26.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feac3fff700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feac406e570 0x7feac40c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:26.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feac9485700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feac415cc80 0x7feac40c4480 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7feabc014040 tx=0x7feabc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:26.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feac17fa700 1 -- 172.21.15.57:0/160463949 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc012a70 con 0x7feac415cc80 2023-11-28T12:07:26.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feac17fa700 1 -- 172.21.15.57:0/160463949 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc020070 con 0x7feac415cc80 2023-11-28T12:07:26.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feacaee8700 1 -- 172.21.15.57:0/160463949 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feabc009a70 con 0x7feac415cc80 2023-11-28T12:07:26.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feac17fa700 1 -- 172.21.15.57:0/160463949 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc012e30 con 0x7feac415cc80 2023-11-28T12:07:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.653+0000 7feacaee8700 1 -- 172.21.15.57:0/160463949 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac40b20b0 con 0x7feac415cc80 2023-11-28T12:07:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.654+0000 7feacaee8700 1 -- 172.21.15.57:0/160463949 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac40b1dd0 con 0x7feac415cc80 2023-11-28T12:07:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.655+0000 7feac17fa700 1 -- 172.21.15.57:0/160463949 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feabc02e070 con 0x7feac415cc80 2023-11-28T12:07:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.655+0000 7feac17fa700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feab00682f0 0x7feab006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.655+0000 7feac17fa700 1 -- 172.21.15.57:0/160463949 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7feabc022070 con 0x7feac415cc80 2023-11-28T12:07:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.655+0000 7feac8c84700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feab00682f0 0x7feab006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:26.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.656+0000 7feac8c84700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feab00682f0 0x7feab006a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7feab4009c60 tx=0x7feab4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:26.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.660+0000 7feac17fa700 1 -- 172.21.15.57:0/160463949 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feabc054230 con 0x7feac415cc80 2023-11-28T12:07:26.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.862+0000 7feacaee8700 1 -- 172.21.15.57:0/160463949 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feac415daa0 con 0x7feac415cc80 2023-11-28T12:07:26.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.863+0000 7feac17fa700 1 -- 172.21.15.57:0/160463949 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18056 (secure 0 0 0) 0x7feabc057850 con 0x7feac415cc80 2023-11-28T12:07:26.865 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:26.865 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":65,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:05:34.795435+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:04:50.522537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T12:04:51.333855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.32","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:07:26.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.866+0000 7feacaee8700 1 -- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feab00682f0 msgr2=0x7feab006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:26.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.866+0000 7feacaee8700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feab00682f0 0x7feab006a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7feab4009c60 tx=0x7feab4008040 comp rx=0 tx=0).stop 2023-11-28T12:07:26.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.866+0000 7feacaee8700 1 -- 172.21.15.57:0/160463949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feac415cc80 msgr2=0x7feac40c4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:26.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.866+0000 7feacaee8700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feac415cc80 0x7feac40c4480 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7feabc014040 tx=0x7feabc004ad0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.866+0000 7feacaee8700 1 -- 172.21.15.57:0/160463949 shutdown_connections 2023-11-28T12:07:26.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.866+0000 7feacaee8700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7feab00682f0 0x7feab006a7a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.866+0000 7feacaee8700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feac415cc80 0x7feac40c4480 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.866+0000 7feacaee8700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac406a270 0x7feac414c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.866+0000 7feacaee8700 1 --2- 172.21.15.57:0/160463949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feac406e570 0x7feac40c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:26.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.866+0000 7feacaee8700 1 -- 172.21.15.57:0/160463949 >> 172.21.15.57:0/160463949 conn(0x7feac406c010 msgr2=0x7feac4153200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:26.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.866+0000 7feacaee8700 1 -- 172.21.15.57:0/160463949 shutdown_connections 2023-11-28T12:07:26.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:26.866+0000 7feacaee8700 1 -- 172.21.15.57:0/160463949 wait complete. 2023-11-28T12:07:26.883 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:04:50.522537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:04:51.333855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:07:26.884 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:07:27.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.021+0000 7fd06c3d5700 1 Processor -- start 2023-11-28T12:07:27.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.022+0000 7fd06c3d5700 1 -- start start 2023-11-28T12:07:27.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.022+0000 7fd06c3d5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd06406a980 0x7fd0640cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.022+0000 7fd06c3d5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd0640cdd50 0x7fd0640cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd06c3d5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06415e0d0 con 0x7fd06415a4d0 2023-11-28T12:07:27.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd06c3d5700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0640ce560 con 0x7fd06406a980 2023-11-28T12:07:27.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd06c3d5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06415e320 con 0x7fd0640cdd50 2023-11-28T12:07:27.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd069970700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd06406a980 0x7fd0640cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd069970700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd06406a980 0x7fd0640cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38854/0 (socket says 172.21.15.57:38854) 2023-11-28T12:07:27.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd069970700 1 -- 172.21.15.57:0/539350326 learned_addr learned my addr 172.21.15.57:0/539350326 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:27.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd06a972700 1 --2- 172.21.15.57:0/539350326 >> v2:172.21.15.181:3300/0 conn(0x7fd0640cdd50 0x7fd0640cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd069970700 1 -- 172.21.15.57:0/539350326 >> v2:172.21.15.181:3300/0 conn(0x7fd0640cdd50 msgr2=0x7fd0640cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:27.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd069970700 1 --2- 172.21.15.57:0/539350326 >> v2:172.21.15.181:3300/0 conn(0x7fd0640cdd50 0x7fd0640cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd069970700 1 -- 172.21.15.57:0/539350326 >> v1:172.21.15.181:6789/0 conn(0x7fd06415a4d0 legacy=0x7fd06406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd069970700 1 -- 172.21.15.57:0/539350326 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06415e600 con 0x7fd06406a980 2023-11-28T12:07:27.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd069970700 1 --2- 172.21.15.57:0/539350326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd06406a980 0x7fd0640cd810 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd0640cd6b0 tx=0x7fd058009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d8fdfa28b10dab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:27.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd06916f700 1 -- 172.21.15.57:0/539350326 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011830 con 0x7fd06406a980 2023-11-28T12:07:27.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd06916f700 1 -- 172.21.15.57:0/539350326 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd058011990 con 0x7fd06406a980 2023-11-28T12:07:27.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.023+0000 7fd06916f700 1 -- 172.21.15.57:0/539350326 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011d20 con 0x7fd06406a980 2023-11-28T12:07:27.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.024+0000 7fd06c3d5700 1 -- 172.21.15.57:0/539350326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd06406a980 msgr2=0x7fd0640cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.024+0000 7fd06c3d5700 1 --2- 172.21.15.57:0/539350326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd06406a980 0x7fd0640cd810 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd0640cd6b0 tx=0x7fd058009bd0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.024+0000 7fd06c3d5700 1 -- 172.21.15.57:0/539350326 shutdown_connections 2023-11-28T12:07:27.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.024+0000 7fd06c3d5700 1 --2- 172.21.15.57:0/539350326 >> v2:172.21.15.181:3300/0 conn(0x7fd0640cdd50 0x7fd0640cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.024+0000 7fd06c3d5700 1 --2- 172.21.15.57:0/539350326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd06406a980 0x7fd0640cd810 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.024+0000 7fd06c3d5700 1 -- 172.21.15.57:0/539350326 >> 172.21.15.57:0/539350326 conn(0x7fd06406c010 msgr2=0x7fd0640cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:27.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.024+0000 7fd06c3d5700 1 -- 172.21.15.57:0/539350326 shutdown_connections 2023-11-28T12:07:27.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.024+0000 7fd06c3d5700 1 -- 172.21.15.57:0/539350326 wait complete. 2023-11-28T12:07:27.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.025+0000 7fd06c3d5700 1 Processor -- start 2023-11-28T12:07:27.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.025+0000 7fd06c3d5700 1 -- start start 2023-11-28T12:07:27.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd06c3d5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06406a980 0x7fd0641f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd06c3d5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0640cdd50 0x7fd0640c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd06c3d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06415a4d0 0x7fd0640c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd06c3d5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0641f5170 con 0x7fd0640cdd50 2023-11-28T12:07:27.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd06c3d5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640c99e0 con 0x7fd06415a4d0 2023-11-28T12:07:27.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd06c3d5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640c6ff0 con 0x7fd06406a980 2023-11-28T12:07:27.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd069970700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0640cdd50 0x7fd0640c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd069970700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0640cdd50 0x7fd0640c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38866/0 (socket says 172.21.15.57:38866) 2023-11-28T12:07:27.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd069970700 1 -- 172.21.15.57:0/2881956488 learned_addr learned my addr 172.21.15.57:0/2881956488 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:27.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd06a972700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06415a4d0 0x7fd0640c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd06a171700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06406a980 0x7fd0641f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd069970700 1 -- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06406a980 msgr2=0x7fd0641f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd069970700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06406a980 0x7fd0641f4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd069970700 1 -- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06415a4d0 msgr2=0x7fd0640c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd069970700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06415a4d0 0x7fd0640c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.026+0000 7fd069970700 1 -- 172.21.15.57:0/2881956488 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd058009550 con 0x7fd0640cdd50 2023-11-28T12:07:27.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.027+0000 7fd06a972700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06415a4d0 0x7fd0640c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:27.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.027+0000 7fd06a171700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06406a980 0x7fd0641f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:27.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.027+0000 7fd069970700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0640cdd50 0x7fd0640c8f60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd058000c00 tx=0x7fd058004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:27.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.027+0000 7fd056ffd700 1 -- 172.21.15.57:0/2881956488 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058004570 con 0x7fd0640cdd50 2023-11-28T12:07:27.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.027+0000 7fd056ffd700 1 -- 172.21.15.57:0/2881956488 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05801f070 con 0x7fd0640cdd50 2023-11-28T12:07:27.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.027+0000 7fd06c3d5700 1 -- 172.21.15.57:0/2881956488 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0640ce1b0 con 0x7fd0640cdd50 2023-11-28T12:07:27.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.027+0000 7fd056ffd700 1 -- 172.21.15.57:0/2881956488 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0580038e0 con 0x7fd0640cdd50 2023-11-28T12:07:27.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.027+0000 7fd06c3d5700 1 -- 172.21.15.57:0/2881956488 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06415e8e0 con 0x7fd0640cdd50 2023-11-28T12:07:27.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.028+0000 7fd06c3d5700 1 -- 172.21.15.57:0/2881956488 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06415a8a0 con 0x7fd0640cdd50 2023-11-28T12:07:27.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.028+0000 7fd056ffd700 1 -- 172.21.15.57:0/2881956488 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd05802a020 con 0x7fd0640cdd50 2023-11-28T12:07:27.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.029+0000 7fd056ffd700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0480682f0 0x7fd04806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.029+0000 7fd056ffd700 1 -- 172.21.15.57:0/2881956488 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fd058087310 con 0x7fd0640cdd50 2023-11-28T12:07:27.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.029+0000 7fd06a171700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0480682f0 0x7fd04806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.030+0000 7fd06a171700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0480682f0 0x7fd04806a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd060000f60 tx=0x7fd060009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:27.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.033+0000 7fd056ffd700 1 -- 172.21.15.57:0/2881956488 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd058053af0 con 0x7fd0640cdd50 2023-11-28T12:07:27.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.232+0000 7fd06c3d5700 1 -- 172.21.15.57:0/2881956488 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd0640743f0 con 0x7fd0640cdd50 2023-11-28T12:07:27.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.232+0000 7fd056ffd700 1 -- 172.21.15.57:0/2881956488 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd058057110 con 0x7fd0640cdd50 2023-11-28T12:07:27.234 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:07:27.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.235+0000 7fd06c3d5700 1 -- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0480682f0 msgr2=0x7fd04806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.235+0000 7fd06c3d5700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0480682f0 0x7fd04806a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd060000f60 tx=0x7fd060009040 comp rx=0 tx=0).stop 2023-11-28T12:07:27.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.235+0000 7fd06c3d5700 1 -- 172.21.15.57:0/2881956488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0640cdd50 msgr2=0x7fd0640c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.235+0000 7fd06c3d5700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0640cdd50 0x7fd0640c8f60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd058000c00 tx=0x7fd058004750 comp rx=0 tx=0).stop 2023-11-28T12:07:27.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.235+0000 7fd06c3d5700 1 -- 172.21.15.57:0/2881956488 shutdown_connections 2023-11-28T12:07:27.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.235+0000 7fd06c3d5700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd0480682f0 0x7fd04806a7a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.235+0000 7fd06c3d5700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06415a4d0 0x7fd0640c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.235+0000 7fd06c3d5700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06406a980 0x7fd0641f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.235+0000 7fd06c3d5700 1 --2- 172.21.15.57:0/2881956488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0640cdd50 0x7fd0640c8f60 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.235+0000 7fd06c3d5700 1 -- 172.21.15.57:0/2881956488 >> 172.21.15.57:0/2881956488 conn(0x7fd06406c010 msgr2=0x7fd0641549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:27.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.235+0000 7fd06c3d5700 1 -- 172.21.15.57:0/2881956488 shutdown_connections 2023-11-28T12:07:27.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.235+0000 7fd06c3d5700 1 -- 172.21.15.57:0/2881956488 wait complete. 2023-11-28T12:07:27.249 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:07:27.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.385+0000 7f5ed4af1700 1 Processor -- start 2023-11-28T12:07:27.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.385+0000 7f5ed4af1700 1 -- start start 2023-11-28T12:07:27.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.386+0000 7f5ed4af1700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5ed00cf710 0x7f5ed00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.386+0000 7f5ed4af1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5ed006a980 0x7f5ed0071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.386+0000 7f5ed4af1700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed015cc10 con 0x7f5ed0071890 2023-11-28T12:07:27.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.386+0000 7f5ed4af1700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed006c9b0 con 0x7f5ed00cf710 2023-11-28T12:07:27.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.386+0000 7f5ed4af1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed006cc90 con 0x7f5ed006a980 2023-11-28T12:07:27.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.386+0000 7f5ece59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5ed00cf710 0x7f5ed00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.386+0000 7f5ecdd9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f5ed0071890 0x7f5ed006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49812/0 (socket says 172.21.15.57:49812) 2023-11-28T12:07:27.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.386+0000 7f5ecdd9b700 1 -- 172.21.15.57:0/2957353355 learned_addr learned my addr 172.21.15.57:0/2957353355 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:27.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.387+0000 7f5eced9d700 1 --2- 172.21.15.57:0/2957353355 >> v2:172.21.15.181:3300/0 conn(0x7f5ed006a980 0x7f5ed0071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.387+0000 7f5ece59c700 1 -- 172.21.15.57:0/2957353355 >> v2:172.21.15.181:3300/0 conn(0x7f5ed006a980 msgr2=0x7f5ed0071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:27.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.387+0000 7f5ece59c700 1 --2- 172.21.15.57:0/2957353355 >> v2:172.21.15.181:3300/0 conn(0x7f5ed006a980 0x7f5ed0071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.387+0000 7f5ece59c700 1 -- 172.21.15.57:0/2957353355 >> v1:172.21.15.57:6789/0 conn(0x7f5ed0071890 legacy=0x7f5ed006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.387+0000 7f5ecd59a700 1 -- 172.21.15.57:0/2957353355 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3258892335 0 0) 0x7f5ed015cc10 con 0x7f5ed0071890 2023-11-28T12:07:27.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.387+0000 7f5ece59c700 1 -- 172.21.15.57:0/2957353355 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed006cf70 con 0x7f5ed00cf710 2023-11-28T12:07:27.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.387+0000 7f5ece59c700 1 --2- 172.21.15.57:0/2957353355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ed00cf710 0x7f5ed00ce740 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009840 tx=0x7f5eb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd1e59b7b2e571c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:27.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.387+0000 7f5ecd59a700 1 -- 172.21.15.57:0/2957353355 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8011830 con 0x7f5ed00cf710 2023-11-28T12:07:27.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.387+0000 7f5ecd59a700 1 -- 172.21.15.57:0/2957353355 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb8011990 con 0x7f5ed00cf710 2023-11-28T12:07:27.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.387+0000 7f5ecd59a700 1 -- 172.21.15.57:0/2957353355 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8011d20 con 0x7f5ed00cf710 2023-11-28T12:07:27.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.388+0000 7f5ed4af1700 1 -- 172.21.15.57:0/2957353355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ed00cf710 msgr2=0x7f5ed00ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.388+0000 7f5ed4af1700 1 --2- 172.21.15.57:0/2957353355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ed00cf710 0x7f5ed00ce740 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009840 tx=0x7f5eb8009b50 comp rx=0 tx=0).stop 2023-11-28T12:07:27.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.388+0000 7f5ed4af1700 1 -- 172.21.15.57:0/2957353355 shutdown_connections 2023-11-28T12:07:27.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.388+0000 7f5ed4af1700 1 --2- 172.21.15.57:0/2957353355 >> v2:172.21.15.181:3300/0 conn(0x7f5ed006a980 0x7f5ed0071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.388+0000 7f5ed4af1700 1 --2- 172.21.15.57:0/2957353355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ed00cf710 0x7f5ed00ce740 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.388+0000 7f5ed4af1700 1 -- 172.21.15.57:0/2957353355 >> 172.21.15.57:0/2957353355 conn(0x7f5ed006c010 msgr2=0x7f5ed00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:27.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.388+0000 7f5ed4af1700 1 -- 172.21.15.57:0/2957353355 shutdown_connections 2023-11-28T12:07:27.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.388+0000 7f5ed4af1700 1 -- 172.21.15.57:0/2957353355 wait complete. 2023-11-28T12:07:27.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.389+0000 7f5ed4af1700 1 Processor -- start 2023-11-28T12:07:27.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.389+0000 7f5ed4af1700 1 -- start start 2023-11-28T12:07:27.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.390+0000 7f5ed4af1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ed006a980 0x7f5ed015fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.390+0000 7f5ed4af1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ed0071890 0x7f5ed00b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.390+0000 7f5ed4af1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed00cf710 0x7f5ed014dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.390+0000 7f5ed4af1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00c7050 con 0x7f5ed0071890 2023-11-28T12:07:27.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.390+0000 7f5ed4af1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00b4b90 con 0x7f5ed00cf710 2023-11-28T12:07:27.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.390+0000 7f5ed4af1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00b48b0 con 0x7f5ed006a980 2023-11-28T12:07:27.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.390+0000 7f5ecdd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ed0071890 0x7f5ed00b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.390+0000 7f5ecdd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ed0071890 0x7f5ed00b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38896/0 (socket says 172.21.15.57:38896) 2023-11-28T12:07:27.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.390+0000 7f5ecdd9b700 1 -- 172.21.15.57:0/152177071 learned_addr learned my addr 172.21.15.57:0/152177071 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:27.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.390+0000 7f5ece59c700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ed006a980 0x7f5ed015fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.390+0000 7f5eced9d700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed00cf710 0x7f5ed014dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.391+0000 7f5ecdd9b700 1 -- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ed006a980 msgr2=0x7f5ed015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.391+0000 7f5ecdd9b700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ed006a980 0x7f5ed015fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.391+0000 7f5ecdd9b700 1 -- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed00cf710 msgr2=0x7f5ed014dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.391+0000 7f5ecdd9b700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed00cf710 0x7f5ed014dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.391+0000 7f5ecdd9b700 1 -- 172.21.15.57:0/152177071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb80094a0 con 0x7f5ed0071890 2023-11-28T12:07:27.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.391+0000 7f5eced9d700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed00cf710 0x7f5ed014dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:27.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.391+0000 7f5ecdd9b700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ed0071890 0x7f5ed00b1a00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00011e0 tx=0x7f5ec000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:27.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.391+0000 7f5ec6ffd700 1 -- 172.21.15.57:0/152177071 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004c80 con 0x7f5ed0071890 2023-11-28T12:07:27.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.391+0000 7f5ec6ffd700 1 -- 172.21.15.57:0/152177071 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec0004e00 con 0x7f5ed0071890 2023-11-28T12:07:27.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.391+0000 7f5ec6ffd700 1 -- 172.21.15.57:0/152177071 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0007310 con 0x7f5ed0071890 2023-11-28T12:07:27.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.391+0000 7f5ed4af1700 1 -- 172.21.15.57:0/152177071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed006d530 con 0x7f5ed0071890 2023-11-28T12:07:27.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.391+0000 7f5ed4af1700 1 -- 172.21.15.57:0/152177071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed006d250 con 0x7f5ed0071890 2023-11-28T12:07:27.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.392+0000 7f5ed4af1700 1 -- 172.21.15.57:0/152177071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed0160480 con 0x7f5ed0071890 2023-11-28T12:07:27.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.392+0000 7f5ec6ffd700 1 -- 172.21.15.57:0/152177071 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5ec001a020 con 0x7f5ed0071890 2023-11-28T12:07:27.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.393+0000 7f5ec6ffd700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5eb00682f0 0x7f5eb006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.393+0000 7f5ec6ffd700 1 -- 172.21.15.57:0/152177071 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f5ec0085950 con 0x7f5ed0071890 2023-11-28T12:07:27.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.393+0000 7f5ece59c700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5eb00682f0 0x7f5eb006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.394+0000 7f5ece59c700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5eb00682f0 0x7f5eb006a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80090b0 tx=0x7f5eb801c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:27.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.397+0000 7f5ec6ffd700 1 -- 172.21.15.57:0/152177071 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5ec00521e0 con 0x7f5ed0071890 2023-11-28T12:07:27.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.595+0000 7f5ed4af1700 1 -- 172.21.15.57:0/152177071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ed00743f0 con 0x7f5ed0071890 2023-11-28T12:07:27.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.595+0000 7f5ec6ffd700 1 -- 172.21.15.57:0/152177071 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ec0055800 con 0x7f5ed0071890 2023-11-28T12:07:27.596 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:07:27.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.597+0000 7f5ed4af1700 1 -- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5eb00682f0 msgr2=0x7f5eb006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.597+0000 7f5ed4af1700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5eb00682f0 0x7f5eb006a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80090b0 tx=0x7f5eb801c040 comp rx=0 tx=0).stop 2023-11-28T12:07:27.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.597+0000 7f5ed4af1700 1 -- 172.21.15.57:0/152177071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ed0071890 msgr2=0x7f5ed00b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.597+0000 7f5ed4af1700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ed0071890 0x7f5ed00b1a00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00011e0 tx=0x7f5ec000cae0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.597+0000 7f5ed4af1700 1 -- 172.21.15.57:0/152177071 shutdown_connections 2023-11-28T12:07:27.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.597+0000 7f5ed4af1700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f5eb00682f0 0x7f5eb006a7a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.597+0000 7f5ed4af1700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed00cf710 0x7f5ed014dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.597+0000 7f5ed4af1700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ed006a980 0x7f5ed015fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.597+0000 7f5ed4af1700 1 --2- 172.21.15.57:0/152177071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ed0071890 0x7f5ed00b1a00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.597+0000 7f5ed4af1700 1 -- 172.21.15.57:0/152177071 >> 172.21.15.57:0/152177071 conn(0x7f5ed006c010 msgr2=0x7f5ed00b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:27.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.597+0000 7f5ed4af1700 1 -- 172.21.15.57:0/152177071 shutdown_connections 2023-11-28T12:07:27.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.597+0000 7f5ed4af1700 1 -- 172.21.15.57:0/152177071 wait complete. 2023-11-28T12:07:27.612 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:07:27.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.748+0000 7f65a837d700 1 Processor -- start 2023-11-28T12:07:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.748+0000 7f65a837d700 1 -- start start 2023-11-28T12:07:27.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.749+0000 7f65a837d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f65a00b1650 0x7f65a00b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.749+0000 7f65a837d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a01599b0 0x7f65a015d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.749+0000 7f65a837d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a0159ec0 con 0x7f65a00af0a0 2023-11-28T12:07:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.749+0000 7f65a837d700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a015aa60 con 0x7f65a00b1650 2023-11-28T12:07:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.749+0000 7f65a837d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a00b1f60 con 0x7f65a01599b0 2023-11-28T12:07:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.750+0000 7f65a6119700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f65a00b1650 0x7f65a00b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.750+0000 7f65a6119700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f65a00b1650 0x7f65a00b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38904/0 (socket says 172.21.15.57:38904) 2023-11-28T12:07:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.750+0000 7f65a6119700 1 -- 172.21.15.57:0/3147989043 learned_addr learned my addr 172.21.15.57:0/3147989043 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.750+0000 7f65a5918700 1 --2- 172.21.15.57:0/3147989043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a01599b0 0x7f65a015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.750+0000 7f65a6119700 1 -- 172.21.15.57:0/3147989043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a01599b0 msgr2=0x7f65a015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.750+0000 7f65a6119700 1 --2- 172.21.15.57:0/3147989043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a01599b0 0x7f65a015d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.750+0000 7f65a6119700 1 -- 172.21.15.57:0/3147989043 >> v1:172.21.15.181:6789/0 conn(0x7f65a00af0a0 legacy=0x7f65a015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.750+0000 7f65a6119700 1 -- 172.21.15.57:0/3147989043 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a015d8a0 con 0x7f65a00b1650 2023-11-28T12:07:27.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.750+0000 7f65a6119700 1 --2- 172.21.15.57:0/3147989043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a00b1650 0x7f65a00b1a20 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f659c009e50 tx=0x7f659c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7e1d03654ee50b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:27.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.750+0000 7f65a5117700 1 -- 172.21.15.57:0/3147989043 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00e070 con 0x7f65a00b1650 2023-11-28T12:07:27.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.750+0000 7f65a5117700 1 -- 172.21.15.57:0/3147989043 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c00b9c0 con 0x7f65a00b1650 2023-11-28T12:07:27.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.750+0000 7f65a5117700 1 -- 172.21.15.57:0/3147989043 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00bd70 con 0x7f65a00b1650 2023-11-28T12:07:27.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.751+0000 7f65a837d700 1 -- 172.21.15.57:0/3147989043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a00b1650 msgr2=0x7f65a00b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.751+0000 7f65a837d700 1 --2- 172.21.15.57:0/3147989043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a00b1650 0x7f65a00b1a20 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f659c009e50 tx=0x7f659c00b330 comp rx=0 tx=0).stop 2023-11-28T12:07:27.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.751+0000 7f65a837d700 1 -- 172.21.15.57:0/3147989043 shutdown_connections 2023-11-28T12:07:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.751+0000 7f65a837d700 1 --2- 172.21.15.57:0/3147989043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a00b1650 0x7f65a00b1a20 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.751+0000 7f65a837d700 1 --2- 172.21.15.57:0/3147989043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a01599b0 0x7f65a015d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.751+0000 7f65a837d700 1 -- 172.21.15.57:0/3147989043 >> 172.21.15.57:0/3147989043 conn(0x7f65a006c010 msgr2=0x7f65a00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.751+0000 7f65a837d700 1 -- 172.21.15.57:0/3147989043 shutdown_connections 2023-11-28T12:07:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.751+0000 7f65a837d700 1 -- 172.21.15.57:0/3147989043 wait complete. 2023-11-28T12:07:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.752+0000 7f65a837d700 1 Processor -- start 2023-11-28T12:07:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.752+0000 7f65a837d700 1 -- start start 2023-11-28T12:07:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.753+0000 7f65a837d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a00af0a0 0x7f65a00c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.753+0000 7f65a837d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a00b1650 0x7f65a00c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.753+0000 7f65a837d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a0145610 0x7f65a0147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.753+0000 7f65a837d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0145f90 con 0x7f65a00af0a0 2023-11-28T12:07:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.753+0000 7f65a837d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a00c71e0 con 0x7f65a0145610 2023-11-28T12:07:27.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.753+0000 7f65a837d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a00c4200 con 0x7f65a00b1650 2023-11-28T12:07:27.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.753+0000 7f65a6119700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a00af0a0 0x7f65a00c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.753+0000 7f65a6119700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a00af0a0 0x7f65a00c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38910/0 (socket says 172.21.15.57:38910) 2023-11-28T12:07:27.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.753+0000 7f65a6119700 1 -- 172.21.15.57:0/1698210249 learned_addr learned my addr 172.21.15.57:0/1698210249 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:27.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.753+0000 7f65a5918700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a00b1650 0x7f65a00c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.753+0000 7f65a691a700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a0145610 0x7f65a0147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.754+0000 7f65a6119700 1 -- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a00b1650 msgr2=0x7f65a00c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.754+0000 7f65a6119700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a00b1650 0x7f65a00c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.754+0000 7f65a6119700 1 -- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a0145610 msgr2=0x7f65a0147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.754+0000 7f65a6119700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a0145610 0x7f65a0147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.754+0000 7f65a6119700 1 -- 172.21.15.57:0/1698210249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c009b30 con 0x7f65a00af0a0 2023-11-28T12:07:27.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.754+0000 7f65a691a700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a0145610 0x7f65a0147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:27.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.754+0000 7f65a6119700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a00af0a0 0x7f65a00c6630 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f659c00b370 tx=0x7f659c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:27.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.754+0000 7f6596ffd700 1 -- 172.21.15.57:0/1698210249 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0048b0 con 0x7f65a00af0a0 2023-11-28T12:07:27.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.754+0000 7f6596ffd700 1 -- 172.21.15.57:0/1698210249 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c0092e0 con 0x7f65a00af0a0 2023-11-28T12:07:27.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.754+0000 7f6596ffd700 1 -- 172.21.15.57:0/1698210249 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c01b670 con 0x7f65a00af0a0 2023-11-28T12:07:27.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.754+0000 7f65a837d700 1 -- 172.21.15.57:0/1698210249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a015de60 con 0x7f65a00af0a0 2023-11-28T12:07:27.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.754+0000 7f65a837d700 1 -- 172.21.15.57:0/1698210249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a015db80 con 0x7f65a00af0a0 2023-11-28T12:07:27.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.755+0000 7f65a837d700 1 -- 172.21.15.57:0/1698210249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a015d590 con 0x7f65a00af0a0 2023-11-28T12:07:27.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.756+0000 7f6596ffd700 1 -- 172.21.15.57:0/1698210249 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f659c021030 con 0x7f65a00af0a0 2023-11-28T12:07:27.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.756+0000 7f6596ffd700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65840682f0 0x7f658406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:27.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.756+0000 7f6596ffd700 1 -- 172.21.15.57:0/1698210249 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f659c0871b0 con 0x7f65a00af0a0 2023-11-28T12:07:27.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.756+0000 7f65a5918700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65840682f0 0x7f658406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:27.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.757+0000 7f65a5918700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65840682f0 0x7f658406a7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f65a00c6590 tx=0x7f6590008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:27.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.760+0000 7f6596ffd700 1 -- 172.21.15.57:0/1698210249 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f659c053990 con 0x7f65a00af0a0 2023-11-28T12:07:27.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.959+0000 7f65a837d700 1 -- 172.21.15.57:0/1698210249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65a00743f0 con 0x7f65a00af0a0 2023-11-28T12:07:27.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.960+0000 7f6596ffd700 1 -- 172.21.15.57:0/1698210249 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7f659c056fb0 con 0x7f65a00af0a0 2023-11-28T12:07:27.961 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:07:27.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.962+0000 7f65a837d700 1 -- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65840682f0 msgr2=0x7f658406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.962+0000 7f65a837d700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65840682f0 0x7f658406a7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f65a00c6590 tx=0x7f6590008040 comp rx=0 tx=0).stop 2023-11-28T12:07:27.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.962+0000 7f65a837d700 1 -- 172.21.15.57:0/1698210249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a00af0a0 msgr2=0x7f65a00c6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:27.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.962+0000 7f65a837d700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a00af0a0 0x7f65a00c6630 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f659c00b370 tx=0x7f659c004ad0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.962+0000 7f65a837d700 1 -- 172.21.15.57:0/1698210249 shutdown_connections 2023-11-28T12:07:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.962+0000 7f65a837d700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f65840682f0 0x7f658406a7a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.962+0000 7f65a837d700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a0145610 0x7f65a0147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.962+0000 7f65a837d700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65a00af0a0 0x7f65a00c6630 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.962+0000 7f65a837d700 1 --2- 172.21.15.57:0/1698210249 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a00b1650 0x7f65a00c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.962+0000 7f65a837d700 1 -- 172.21.15.57:0/1698210249 >> 172.21.15.57:0/1698210249 conn(0x7f65a006c010 msgr2=0x7f65a00b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.962+0000 7f65a837d700 1 -- 172.21.15.57:0/1698210249 shutdown_connections 2023-11-28T12:07:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:27.962+0000 7f65a837d700 1 -- 172.21.15.57:0/1698210249 wait complete. 2023-11-28T12:07:27.976 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:07:28.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.114+0000 7fcd647c4700 1 Processor -- start 2023-11-28T12:07:28.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.114+0000 7fcd647c4700 1 -- start start 2023-11-28T12:07:28.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.115+0000 7fcd647c4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c06a980 0x7fcd5c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.115+0000 7fcd647c4700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5c15cc10 con 0x7fcd5c071890 2023-11-28T12:07:28.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.115+0000 7fcd647c4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5c06c9b0 con 0x7fcd5c0cf710 2023-11-28T12:07:28.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.115+0000 7fcd647c4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c06cc90 con 0x7fcd5c06a980 2023-11-28T12:07:28.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.115+0000 7fcd62d61700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcd5c071890 0x7fcd5c160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49828/0 (socket says 172.21.15.57:49828) 2023-11-28T12:07:28.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.115+0000 7fcd62d61700 1 -- 172.21.15.57:0/987018844 learned_addr learned my addr 172.21.15.57:0/987018844 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:28.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.115+0000 7fcd61d5f700 1 --2- 172.21.15.57:0/987018844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c06a980 0x7fcd5c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.115+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494538585 0 0) 0x7fcd5c15cc10 con 0x7fcd5c071890 2023-11-28T12:07:28.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.115+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd480035b0 con 0x7fcd5c071890 2023-11-28T12:07:28.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 482946276 0 0) 0x7fcd480035b0 con 0x7fcd5c071890 2023-11-28T12:07:28.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c06a980 msgr2=0x7fcd5c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 --2- 172.21.15.57:0/987018844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c06a980 0x7fcd5c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 >> v1:172.21.15.181:6789/0 conn(0x7fcd5c0cf710 legacy=0x7fcd5c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c06cf70 con 0x7fcd5c071890 2023-11-28T12:07:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fcd540019f0 con 0x7fcd5c071890 2023-11-28T12:07:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 >> v1:172.21.15.57:6789/0 conn(0x7fcd5c071890 legacy=0x7fcd5c160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 --2- 172.21.15.57:0/987018844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd48003a30 0x7fcd48003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 --2- 172.21.15.57:0/987018844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd48004420 0x7fcd480048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 --2- 172.21.15.57:0/987018844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c06a980 0x7fcd480051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c06cc90 con 0x7fcd48004420 2023-11-28T12:07:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4800bb20 con 0x7fcd48003a30 2023-11-28T12:07:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4800bce0 con 0x7fcd5c06a980 2023-11-28T12:07:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd61d5f700 1 --2- 172.21.15.57:0/987018844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd48004420 0x7fcd480048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd62560700 1 --2- 172.21.15.57:0/987018844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd48003a30 0x7fcd48003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd62d61700 1 --2- 172.21.15.57:0/987018844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c06a980 0x7fcd480051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd61d5f700 1 -- 172.21.15.57:0/987018844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c06a980 msgr2=0x7fcd480051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:28.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd61d5f700 1 --2- 172.21.15.57:0/987018844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c06a980 0x7fcd480051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd61d5f700 1 -- 172.21.15.57:0/987018844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd48003a30 msgr2=0x7fcd48003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:28.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd61d5f700 1 --2- 172.21.15.57:0/987018844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd48003a30 0x7fcd48003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd61d5f700 1 -- 172.21.15.57:0/987018844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd480035b0 con 0x7fcd48004420 2023-11-28T12:07:28.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd61d5f700 1 --2- 172.21.15.57:0/987018844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd48004420 0x7fcd480048f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c06ad70 tx=0x7fcd48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277a80b19944b39a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:28.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5800e6c0 con 0x7fcd48004420 2023-11-28T12:07:28.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5800e820 con 0x7fcd48004420 2023-11-28T12:07:28.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.116+0000 7fcd6155e700 1 -- 172.21.15.57:0/987018844 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5800ebb0 con 0x7fcd48004420 2023-11-28T12:07:28.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.117+0000 7fcd647c4700 1 -- 172.21.15.57:0/987018844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd48004420 msgr2=0x7fcd480048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:28.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.117+0000 7fcd647c4700 1 --2- 172.21.15.57:0/987018844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd48004420 0x7fcd480048f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c06ad70 tx=0x7fcd48005880 comp rx=0 tx=0).stop 2023-11-28T12:07:28.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.117+0000 7fcd62d61700 1 -- 172.21.15.57:0/987018844 reap_dead start 2023-11-28T12:07:28.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.117+0000 7fcd647c4700 1 -- 172.21.15.57:0/987018844 shutdown_connections 2023-11-28T12:07:28.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.117+0000 7fcd647c4700 1 -- 172.21.15.57:0/987018844 >> 172.21.15.57:0/987018844 conn(0x7fcd5c06c010 msgr2=0x7fcd5c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:28.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.117+0000 7fcd647c4700 1 -- 172.21.15.57:0/987018844 shutdown_connections 2023-11-28T12:07:28.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.117+0000 7fcd647c4700 1 -- 172.21.15.57:0/987018844 wait complete. 2023-11-28T12:07:28.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.118+0000 7fcd647c4700 1 Processor -- start 2023-11-28T12:07:28.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.118+0000 7fcd647c4700 1 -- start start 2023-11-28T12:07:28.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.119+0000 7fcd647c4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd5c15add0 0x7fcd5c15ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.119+0000 7fcd647c4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd5c15b1a0 0x7fcd5c0b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.119+0000 7fcd647c4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c0b4450 0x7fcd5c15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.119+0000 7fcd647c4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c0ca590 con 0x7fcd5c15add0 2023-11-28T12:07:28.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.119+0000 7fcd647c4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c1f7020 con 0x7fcd5c15b1a0 2023-11-28T12:07:28.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.119+0000 7fcd647c4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c0b48c0 con 0x7fcd5c0b4450 2023-11-28T12:07:28.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.119+0000 7fcd62560700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd5c15add0 0x7fcd5c15ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.119+0000 7fcd62560700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd5c15add0 0x7fcd5c15ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38922/0 (socket says 172.21.15.57:38922) 2023-11-28T12:07:28.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.119+0000 7fcd62560700 1 -- 172.21.15.57:0/555909611 learned_addr learned my addr 172.21.15.57:0/555909611 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:28.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd62d61700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c0b4450 0x7fcd5c15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd62560700 1 -- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c0b4450 msgr2=0x7fcd5c15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:28.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd61d5f700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd5c15b1a0 0x7fcd5c0b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd62560700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c0b4450 0x7fcd5c15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd62560700 1 -- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd5c15b1a0 msgr2=0x7fcd5c0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:28.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd62560700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd5c15b1a0 0x7fcd5c0b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd62560700 1 -- 172.21.15.57:0/555909611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c06d530 con 0x7fcd5c15add0 2023-11-28T12:07:28.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd62d61700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c0b4450 0x7fcd5c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:28.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd61d5f700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd5c15b1a0 0x7fcd5c0b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:28.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd62560700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd5c15add0 0x7fcd5c15ee30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00f820 tx=0x7fcd4c00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:28.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd52ffd700 1 -- 172.21.15.57:0/555909611 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0052e0 con 0x7fcd5c15add0 2023-11-28T12:07:28.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd52ffd700 1 -- 172.21.15.57:0/555909611 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c005440 con 0x7fcd5c15add0 2023-11-28T12:07:28.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd647c4700 1 -- 172.21.15.57:0/555909611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5c06d250 con 0x7fcd5c15add0 2023-11-28T12:07:28.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd52ffd700 1 -- 172.21.15.57:0/555909611 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0038c0 con 0x7fcd5c15add0 2023-11-28T12:07:28.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.120+0000 7fcd647c4700 1 -- 172.21.15.57:0/555909611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5c0ce740 con 0x7fcd5c15add0 2023-11-28T12:07:28.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.121+0000 7fcd647c4700 1 -- 172.21.15.57:0/555909611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5c15fa40 con 0x7fcd5c15add0 2023-11-28T12:07:28.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.122+0000 7fcd52ffd700 1 -- 172.21.15.57:0/555909611 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcd4c003a20 con 0x7fcd5c15add0 2023-11-28T12:07:28.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.122+0000 7fcd52ffd700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcd400682f0 0x7fcd4006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.122+0000 7fcd52ffd700 1 -- 172.21.15.57:0/555909611 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fcd4c085cd0 con 0x7fcd5c15add0 2023-11-28T12:07:28.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.122+0000 7fcd61d5f700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcd400682f0 0x7fcd4006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.123+0000 7fcd61d5f700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcd400682f0 0x7fcd4006a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c1f5fd0 tx=0x7fcd58008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:28.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.126+0000 7fcd52ffd700 1 -- 172.21.15.57:0/555909611 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcd4c0524b0 con 0x7fcd5c15add0 2023-11-28T12:07:28.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.325+0000 7fcd647c4700 1 -- 172.21.15.57:0/555909611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd5c0afd20 con 0x7fcd5c15add0 2023-11-28T12:07:28.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.326+0000 7fcd52ffd700 1 -- 172.21.15.57:0/555909611 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd4c003cb0 con 0x7fcd5c15add0 2023-11-28T12:07:28.327 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:07:28.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.328+0000 7fcd647c4700 1 -- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcd400682f0 msgr2=0x7fcd4006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:28.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.328+0000 7fcd647c4700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcd400682f0 0x7fcd4006a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c1f5fd0 tx=0x7fcd58008030 comp rx=0 tx=0).stop 2023-11-28T12:07:28.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.328+0000 7fcd647c4700 1 -- 172.21.15.57:0/555909611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd5c15add0 msgr2=0x7fcd5c15ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:28.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.328+0000 7fcd647c4700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd5c15add0 0x7fcd5c15ee30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00f820 tx=0x7fcd4c00fbe0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.328+0000 7fcd647c4700 1 -- 172.21.15.57:0/555909611 shutdown_connections 2023-11-28T12:07:28.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.328+0000 7fcd647c4700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcd400682f0 0x7fcd4006a7a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.328+0000 7fcd647c4700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd5c0b4450 0x7fcd5c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.328+0000 7fcd647c4700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd5c15add0 0x7fcd5c15ee30 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.328+0000 7fcd647c4700 1 --2- 172.21.15.57:0/555909611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd5c15b1a0 0x7fcd5c0b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.328+0000 7fcd647c4700 1 -- 172.21.15.57:0/555909611 >> 172.21.15.57:0/555909611 conn(0x7fcd5c06c010 msgr2=0x7fcd5c0af610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:28.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.328+0000 7fcd647c4700 1 -- 172.21.15.57:0/555909611 shutdown_connections 2023-11-28T12:07:28.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.328+0000 7fcd647c4700 1 -- 172.21.15.57:0/555909611 wait complete. 2023-11-28T12:07:28.343 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T12:07:28.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.478+0000 7f70e60d3700 1 Processor -- start 2023-11-28T12:07:28.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.478+0000 7f70e60d3700 1 -- start start 2023-11-28T12:07:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70e60d3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f70e015c8c0 0x7f70e0160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70e60d3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e00b5dc0 0x7f70e006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70e60d3700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f70e006b050 0x7f70e00d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70e60d3700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e006b4d0 con 0x7f70e006b050 2023-11-28T12:07:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70e60d3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e00d2d30 con 0x7f70e015c8c0 2023-11-28T12:07:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70e60d3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00d0b40 con 0x7f70e00b5dc0 2023-11-28T12:07:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70dffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f70e006b050 0x7f70e00d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70dffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f70e006b050 0x7f70e00d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38928/0 (socket says 172.21.15.57:38928) 2023-11-28T12:07:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70dffff700 1 -- 172.21.15.57:0/2585903250 learned_addr learned my addr 172.21.15.57:0/2585903250 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70deffd700 1 --2- 172.21.15.57:0/2585903250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e00b5dc0 0x7f70e006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70df7fe700 1 --2- 172.21.15.57:0/2585903250 >> v2:172.21.15.181:3300/0 conn(0x7f70e015c8c0 0x7f70e0160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70dffff700 1 -- 172.21.15.57:0/2585903250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e00b5dc0 msgr2=0x7f70e006ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70dffff700 1 --2- 172.21.15.57:0/2585903250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e00b5dc0 0x7f70e006ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70dffff700 1 -- 172.21.15.57:0/2585903250 >> v2:172.21.15.181:3300/0 conn(0x7f70e015c8c0 msgr2=0x7f70e0160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.479+0000 7f70dffff700 1 --2- 172.21.15.57:0/2585903250 >> v2:172.21.15.181:3300/0 conn(0x7f70e015c8c0 0x7f70e0160cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70dffff700 1 -- 172.21.15.57:0/2585903250 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e01611f0 con 0x7f70e006b050 2023-11-28T12:07:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70deffd700 1 --2- 172.21.15.57:0/2585903250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e00b5dc0 0x7f70e006ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70dffff700 1 --2- 172.21.15.57:0/2585903250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e006b050 0x7f70e00d0fd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f70e00d2ed0 tx=0x7f70d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d1949739a13fef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70de7fc700 1 -- 172.21.15.57:0/2585903250 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d800e070 con 0x7f70e006b050 2023-11-28T12:07:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70de7fc700 1 -- 172.21.15.57:0/2585903250 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d800b910 con 0x7f70e006b050 2023-11-28T12:07:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70de7fc700 1 -- 172.21.15.57:0/2585903250 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d800bcc0 con 0x7f70e006b050 2023-11-28T12:07:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70e60d3700 1 -- 172.21.15.57:0/2585903250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e006b050 msgr2=0x7f70e00d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:28.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70e60d3700 1 --2- 172.21.15.57:0/2585903250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e006b050 0x7f70e00d0fd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f70e00d2ed0 tx=0x7f70d800b330 comp rx=0 tx=0).stop 2023-11-28T12:07:28.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70e60d3700 1 -- 172.21.15.57:0/2585903250 shutdown_connections 2023-11-28T12:07:28.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70e60d3700 1 --2- 172.21.15.57:0/2585903250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e006b050 0x7f70e00d0fd0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70e60d3700 1 --2- 172.21.15.57:0/2585903250 >> v2:172.21.15.181:3300/0 conn(0x7f70e015c8c0 0x7f70e0160cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70e60d3700 1 --2- 172.21.15.57:0/2585903250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e00b5dc0 0x7f70e006ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70e60d3700 1 -- 172.21.15.57:0/2585903250 >> 172.21.15.57:0/2585903250 conn(0x7f70e006c520 msgr2=0x7f70e006c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:28.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.480+0000 7f70e60d3700 1 -- 172.21.15.57:0/2585903250 shutdown_connections 2023-11-28T12:07:28.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.481+0000 7f70e60d3700 1 -- 172.21.15.57:0/2585903250 wait complete. 2023-11-28T12:07:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.482+0000 7f70e60d3700 1 Processor -- start 2023-11-28T12:07:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.482+0000 7f70e60d3700 1 -- start start 2023-11-28T12:07:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70e60d3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e006b050 0x7f70e0155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70e60d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e00b5dc0 0x7f70e00cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70e60d3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e015c8c0 0x7f70e0155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70e60d3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00cce00 con 0x7f70e015c8c0 2023-11-28T12:07:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70e60d3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00cd0a0 con 0x7f70e00b5dc0 2023-11-28T12:07:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70e60d3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e0159420 con 0x7f70e006b050 2023-11-28T12:07:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70dffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e015c8c0 0x7f70e0155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70dffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e015c8c0 0x7f70e0155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38936/0 (socket says 172.21.15.57:38936) 2023-11-28T12:07:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70dffff700 1 -- 172.21.15.57:0/837470531 learned_addr learned my addr 172.21.15.57:0/837470531 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70df7fe700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e006b050 0x7f70e0155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70deffd700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e00b5dc0 0x7f70e00cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70dffff700 1 -- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e006b050 msgr2=0x7f70e0155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70dffff700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e006b050 0x7f70e0155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70dffff700 1 -- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e00b5dc0 msgr2=0x7f70e00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70dffff700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e00b5dc0 0x7f70e00cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70dffff700 1 -- 172.21.15.57:0/837470531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d8009b30 con 0x7f70e015c8c0 2023-11-28T12:07:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.483+0000 7f70dffff700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e015c8c0 0x7f70e0155650 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f70e00d0af0 tx=0x7f70d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.484+0000 7f70d7fff700 1 -- 172.21.15.57:0/837470531 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d80099b0 con 0x7f70e015c8c0 2023-11-28T12:07:28.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.484+0000 7f70d7fff700 1 -- 172.21.15.57:0/837470531 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d80092e0 con 0x7f70e015c8c0 2023-11-28T12:07:28.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.484+0000 7f70e60d3700 1 -- 172.21.15.57:0/837470531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e00cd760 con 0x7f70e015c8c0 2023-11-28T12:07:28.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.484+0000 7f70d7fff700 1 -- 172.21.15.57:0/837470531 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8004470 con 0x7f70e015c8c0 2023-11-28T12:07:28.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.484+0000 7f70e60d3700 1 -- 172.21.15.57:0/837470531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e00d2a50 con 0x7f70e015c8c0 2023-11-28T12:07:28.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.485+0000 7f70e60d3700 1 -- 172.21.15.57:0/837470531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e00cda40 con 0x7f70e015c8c0 2023-11-28T12:07:28.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.485+0000 7f70d7fff700 1 -- 172.21.15.57:0/837470531 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f70d802c070 con 0x7f70e015c8c0 2023-11-28T12:07:28.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.486+0000 7f70d7fff700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f70c00682f0 0x7f70c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:28.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.486+0000 7f70d7fff700 1 -- 172.21.15.57:0/837470531 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f70d8087260 con 0x7f70e015c8c0 2023-11-28T12:07:28.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.486+0000 7f70df7fe700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f70c00682f0 0x7f70c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:28.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.486+0000 7f70df7fe700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f70c00682f0 0x7f70c006a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f70e00cff20 tx=0x7f70d0009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:28.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.490+0000 7f70d7fff700 1 -- 172.21.15.57:0/837470531 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f70d8053a40 con 0x7f70e015c8c0 2023-11-28T12:07:28.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.708+0000 7f70e60d3700 1 -- 172.21.15.57:0/837470531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f70e0067d70 con 0x7f70e015c8c0 2023-11-28T12:07:28.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.708+0000 7f70d7fff700 1 -- 172.21.15.57:0/837470531 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7f70d8057060 con 0x7f70e015c8c0 2023-11-28T12:07:28.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.711+0000 7f70e60d3700 1 -- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f70c00682f0 msgr2=0x7f70c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:28.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.711+0000 7f70e60d3700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f70c00682f0 0x7f70c006a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f70e00cff20 tx=0x7f70d0009210 comp rx=0 tx=0).stop 2023-11-28T12:07:28.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.711+0000 7f70e60d3700 1 -- 172.21.15.57:0/837470531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e015c8c0 msgr2=0x7f70e0155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:28.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.711+0000 7f70e60d3700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e015c8c0 0x7f70e0155650 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f70e00d0af0 tx=0x7f70d8004ad0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.711+0000 7f70e60d3700 1 -- 172.21.15.57:0/837470531 shutdown_connections 2023-11-28T12:07:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.711+0000 7f70e60d3700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f70c00682f0 0x7f70c006a7a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.711+0000 7f70e60d3700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70e015c8c0 0x7f70e0155650 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.711+0000 7f70e60d3700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70e006b050 0x7f70e0155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.711+0000 7f70e60d3700 1 --2- 172.21.15.57:0/837470531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e00b5dc0 0x7f70e00cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.711+0000 7f70e60d3700 1 -- 172.21.15.57:0/837470531 >> 172.21.15.57:0/837470531 conn(0x7f70e006c520 msgr2=0x7f70e00d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.711+0000 7f70e60d3700 1 -- 172.21.15.57:0/837470531 shutdown_connections 2023-11-28T12:07:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:28.711+0000 7f70e60d3700 1 -- 172.21.15.57:0/837470531 wait complete. 2023-11-28T12:07:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:listed 0 entries 2023-11-28T12:07:28.725 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2023-11-28T12:07:28.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:07:28.726 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T12:07:28.765 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[145733]: fuse finished with error 0 and tester_r 0 2023-11-28T12:07:28.766 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:07:28.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:07:28.767 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T12:07:28.829 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:07:28.829 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:07:28.853 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251064k,nr_inodes=4062766,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14954 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253752k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T12:07:28.853 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:07:28.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:07:28.854 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T12:07:28.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:07:28.912 INFO:teuthology.orchestra.run.smithi181.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T12:07:28.912 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:07:28.913 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:07:28.935 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16254224k,nr_inodes=4063556,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15749 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3254388k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T12:07:28.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:07:28.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:07:28.936 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T12:07:28.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:07:28.999 INFO:teuthology.orchestra.run.smithi057.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T12:07:29.000 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:07:29.000 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:07:29.021 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251064k,nr_inodes=4062766,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14954 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253752k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T12:07:29.021 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:07:29.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:07:29.021 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T12:07:29.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:07:29.080 INFO:teuthology.orchestra.run.smithi181.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T12:07:29.080 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:07:29.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.222+0000 7fd98d859700 1 Processor -- start 2023-11-28T12:07:29.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.222+0000 7fd98d859700 1 -- start start 2023-11-28T12:07:29.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.223+0000 7fd98d859700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd98806ab10 0x7fd98806aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:29.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.223+0000 7fd98d859700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9880d1ee0 con 0x7fd9880cd5a0 2023-11-28T12:07:29.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.223+0000 7fd98d859700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd988161120 con 0x7fd9880d0450 2023-11-28T12:07:29.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.223+0000 7fd98d859700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98806b420 con 0x7fd98806ab10 2023-11-28T12:07:29.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.223+0000 7fd986ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd98806ab10 0x7fd98806aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:29.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.223+0000 7fd9867fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd9880cd5a0 0x7fd988160300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49836/0 (socket says 172.21.15.57:49836) 2023-11-28T12:07:29.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.223+0000 7fd9867fc700 1 -- 172.21.15.57:0/434307187 learned_addr learned my addr 172.21.15.57:0/434307187 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:29.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.223+0000 7fd986ffd700 1 -- 172.21.15.57:0/434307187 >> v1:172.21.15.181:6789/0 conn(0x7fd9880d0450 legacy=0x7fd988160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:29.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.223+0000 7fd986ffd700 1 -- 172.21.15.57:0/434307187 >> v1:172.21.15.57:6789/0 conn(0x7fd9880cd5a0 legacy=0x7fd988160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:29.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.223+0000 7fd986ffd700 1 -- 172.21.15.57:0/434307187 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9880cdb10 con 0x7fd98806ab10 2023-11-28T12:07:29.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.223+0000 7fd985ffb700 1 -- 172.21.15.57:0/434307187 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122528224 0 0) 0x7fd9880d1ee0 con 0x7fd9880cd5a0 2023-11-28T12:07:29.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.224+0000 7fd986ffd700 1 --2- 172.21.15.57:0/434307187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd98806ab10 0x7fd98806aee0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd978009840 tx=0x7fd978009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6d3bd927356246 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:29.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.224+0000 7fd985ffb700 1 -- 172.21.15.57:0/434307187 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978011830 con 0x7fd98806ab10 2023-11-28T12:07:29.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.224+0000 7fd985ffb700 1 -- 172.21.15.57:0/434307187 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd978011990 con 0x7fd98806ab10 2023-11-28T12:07:29.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.224+0000 7fd985ffb700 1 -- 172.21.15.57:0/434307187 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978011d20 con 0x7fd98806ab10 2023-11-28T12:07:29.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.224+0000 7fd98d859700 1 -- 172.21.15.57:0/434307187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd98806ab10 msgr2=0x7fd98806aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:29.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.224+0000 7fd98d859700 1 --2- 172.21.15.57:0/434307187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd98806ab10 0x7fd98806aee0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd978009840 tx=0x7fd978009b50 comp rx=0 tx=0).stop 2023-11-28T12:07:29.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.224+0000 7fd98d859700 1 -- 172.21.15.57:0/434307187 shutdown_connections 2023-11-28T12:07:29.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.224+0000 7fd98d859700 1 --2- 172.21.15.57:0/434307187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd98806ab10 0x7fd98806aee0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.224+0000 7fd98d859700 1 -- 172.21.15.57:0/434307187 >> 172.21.15.57:0/434307187 conn(0x7fd98806c520 msgr2=0x7fd98806c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:29.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.224+0000 7fd98d859700 1 -- 172.21.15.57:0/434307187 shutdown_connections 2023-11-28T12:07:29.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.224+0000 7fd98d859700 1 -- 172.21.15.57:0/434307187 wait complete. 2023-11-28T12:07:29.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.226+0000 7fd98d859700 1 Processor -- start 2023-11-28T12:07:29.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.226+0000 7fd98d859700 1 -- start start 2023-11-28T12:07:29.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd98d859700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd98806ab10 0x7fd9881513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:29.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd98d859700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9880cd5a0 0x7fd9880d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:29.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd98d859700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9880d0450 0x7fd9880d1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:29.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd98d859700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9881543e0 con 0x7fd9880d0450 2023-11-28T12:07:29.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd98d859700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd988151910 con 0x7fd9880cd5a0 2023-11-28T12:07:29.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd98d859700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd988158920 con 0x7fd98806ab10 2023-11-28T12:07:29.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd9877fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9880d0450 0x7fd9880d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:29.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd9877fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9880d0450 0x7fd9880d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38966/0 (socket says 172.21.15.57:38966) 2023-11-28T12:07:29.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd9877fe700 1 -- 172.21.15.57:0/829527918 learned_addr learned my addr 172.21.15.57:0/829527918 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:29.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd9867fc700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9880cd5a0 0x7fd9880d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:29.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd986ffd700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd98806ab10 0x7fd9881513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:29.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd9877fe700 1 -- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd98806ab10 msgr2=0x7fd9881513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:29.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd9877fe700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd98806ab10 0x7fd9881513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd9877fe700 1 -- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9880cd5a0 msgr2=0x7fd9880d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:29.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd9877fe700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9880cd5a0 0x7fd9880d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd9877fe700 1 -- 172.21.15.57:0/829527918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9780094a0 con 0x7fd9880d0450 2023-11-28T12:07:29.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd9877fe700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9880d0450 0x7fd9880d1060 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd98000cb50 tx=0x7fd98000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:29.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd97ffff700 1 -- 172.21.15.57:0/829527918 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980003c00 con 0x7fd9880d0450 2023-11-28T12:07:29.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd97ffff700 1 -- 172.21.15.57:0/829527918 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd980003d60 con 0x7fd9880d0450 2023-11-28T12:07:29.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.227+0000 7fd97ffff700 1 -- 172.21.15.57:0/829527918 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98000aa20 con 0x7fd9880d0450 2023-11-28T12:07:29.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.228+0000 7fd98d859700 1 -- 172.21.15.57:0/829527918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9880d1b70 con 0x7fd9880d0450 2023-11-28T12:07:29.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.228+0000 7fd98d859700 1 -- 172.21.15.57:0/829527918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9880d1890 con 0x7fd9880d0450 2023-11-28T12:07:29.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.229+0000 7fd98d859700 1 -- 172.21.15.57:0/829527918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd98815cad0 con 0x7fd9880d0450 2023-11-28T12:07:29.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.229+0000 7fd97ffff700 1 -- 172.21.15.57:0/829527918 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd980010050 con 0x7fd9880d0450 2023-11-28T12:07:29.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.229+0000 7fd97ffff700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9680682f0 0x7fd96806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:29.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.229+0000 7fd97ffff700 1 -- 172.21.15.57:0/829527918 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fd980085b80 con 0x7fd9880d0450 2023-11-28T12:07:29.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.230+0000 7fd986ffd700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9680682f0 0x7fd96806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:29.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.230+0000 7fd986ffd700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9680682f0 0x7fd96806a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd9780090b0 tx=0x7fd978008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:29.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.234+0000 7fd97ffff700 1 -- 172.21.15.57:0/829527918 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd980052360 con 0x7fd9880d0450 2023-11-28T12:07:29.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.478+0000 7fd98d859700 1 -- 172.21.15.57:0/829527918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9880743f0 con 0x7fd9880d0450 2023-11-28T12:07:29.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.479+0000 7fd97ffff700 1 -- 172.21.15.57:0/829527918 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4652 (secure 0 0 0) 0x7fd980055980 con 0x7fd9880d0450 2023-11-28T12:07:29.480 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:29.480 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2317025916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2317025916},{"type":"v1","addr":"172.21.15.181:6836","nonce":2317025916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3486085199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3486085199},{"type":"v1","addr":"172.21.15.181:6837","nonce":3486085199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/4272259116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":4272259116},{"type":"v1","addr":"172.21.15.57:6834","nonce":4272259116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:04:52.360416+0000","modified":"2023-11-28T12:05:08.579934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.57:6835/685888843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":685888843},{"type":"v1","addr":"172.21.15.57:6835","nonce":685888843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T12:07:29.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.481+0000 7fd98d859700 1 -- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9680682f0 msgr2=0x7fd96806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:29.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.481+0000 7fd98d859700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9680682f0 0x7fd96806a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd9780090b0 tx=0x7fd978008030 comp rx=0 tx=0).stop 2023-11-28T12:07:29.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.481+0000 7fd98d859700 1 -- 172.21.15.57:0/829527918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9880d0450 msgr2=0x7fd9880d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:29.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.481+0000 7fd98d859700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9880d0450 0x7fd9880d1060 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd98000cb50 tx=0x7fd98000ce60 comp rx=0 tx=0).stop 2023-11-28T12:07:29.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.481+0000 7fd98d859700 1 -- 172.21.15.57:0/829527918 shutdown_connections 2023-11-28T12:07:29.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.481+0000 7fd98d859700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd9680682f0 0x7fd96806a7a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.481+0000 7fd98d859700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9880d0450 0x7fd9880d1060 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.481+0000 7fd98d859700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd98806ab10 0x7fd9881513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.481+0000 7fd98d859700 1 --2- 172.21.15.57:0/829527918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9880cd5a0 0x7fd9880d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.481+0000 7fd98d859700 1 -- 172.21.15.57:0/829527918 >> 172.21.15.57:0/829527918 conn(0x7fd98806c520 msgr2=0x7fd9880b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:29.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.481+0000 7fd98d859700 1 -- 172.21.15.57:0/829527918 shutdown_connections 2023-11-28T12:07:29.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.481+0000 7fd98d859700 1 -- 172.21.15.57:0/829527918 wait complete. 2023-11-28T12:07:29.484 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2023-11-28T12:07:29.497 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:07:29.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.630+0000 7f2ee0b00700 1 Processor -- start 2023-11-28T12:07:29.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.630+0000 7f2ee0b00700 1 -- start start 2023-11-28T12:07:29.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.631+0000 7f2ee0b00700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2edc0ce040 0x7f2edc15d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:29.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.631+0000 7f2ee0b00700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2edc0b1770 0x7f2edc0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:29.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2ee0b00700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2edc13fc90 con 0x7f2edc142db0 2023-11-28T12:07:29.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2ee0b00700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2edc0b2130 con 0x7f2edc0b1770 2023-11-28T12:07:29.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2ee0b00700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc13f480 con 0x7f2edc0ce040 2023-11-28T12:07:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2edad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2edc0b1770 0x7f2edc0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2eda59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2edc0ce040 0x7f2edc15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2edad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2edc0b1770 0x7f2edc0b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:45600/0 (socket says 172.21.15.57:45600) 2023-11-28T12:07:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2ed9d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2edc142db0 0x7f2edc13ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:57812/0 (socket says 172.21.15.57:57812) 2023-11-28T12:07:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2edad9d700 1 -- 172.21.15.57:0/1742491219 learned_addr learned my addr 172.21.15.57:0/1742491219 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2edad9d700 1 -- 172.21.15.57:0/1742491219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2edc0ce040 msgr2=0x7f2edc15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2edad9d700 1 --2- 172.21.15.57:0/1742491219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2edc0ce040 0x7f2edc15d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2edad9d700 1 -- 172.21.15.57:0/1742491219 >> v1:172.21.15.181:6789/0 conn(0x7f2edc142db0 legacy=0x7f2edc13ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:29.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2edad9d700 1 -- 172.21.15.57:0/1742491219 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc13f6d0 con 0x7f2edc0b1770 2023-11-28T12:07:29.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2eda59c700 1 --2- 172.21.15.57:0/1742491219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2edc0ce040 0x7f2edc15d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:07:29.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.632+0000 7f2edad9d700 1 --2- 172.21.15.57:0/1742491219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2edc0b1770 0x7f2edc0b1bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009e50 tx=0x7f2ed400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dc0632ecc5e7b3a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.634+0000 7f2ed959a700 1 -- 172.21.15.57:0/1742491219 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400e070 con 0x7f2edc0b1770 2023-11-28T12:07:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.634+0000 7f2ed959a700 1 -- 172.21.15.57:0/1742491219 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed400b910 con 0x7f2edc0b1770 2023-11-28T12:07:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.634+0000 7f2ed959a700 1 -- 172.21.15.57:0/1742491219 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400bcc0 con 0x7f2edc0b1770 2023-11-28T12:07:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.634+0000 7f2ee0b00700 1 -- 172.21.15.57:0/1742491219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2edc0b1770 msgr2=0x7f2edc0b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.634+0000 7f2ee0b00700 1 --2- 172.21.15.57:0/1742491219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2edc0b1770 0x7f2edc0b1bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009e50 tx=0x7f2ed400b330 comp rx=0 tx=0).stop 2023-11-28T12:07:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.635+0000 7f2ee0b00700 1 -- 172.21.15.57:0/1742491219 shutdown_connections 2023-11-28T12:07:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.635+0000 7f2ee0b00700 1 --2- 172.21.15.57:0/1742491219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2edc0b1770 0x7f2edc0b1bf0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.635+0000 7f2ee0b00700 1 --2- 172.21.15.57:0/1742491219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2edc0ce040 0x7f2edc15d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.635+0000 7f2ee0b00700 1 -- 172.21.15.57:0/1742491219 >> 172.21.15.57:0/1742491219 conn(0x7f2edc06c010 msgr2=0x7f2edc0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.635+0000 7f2ee0b00700 1 -- 172.21.15.57:0/1742491219 shutdown_connections 2023-11-28T12:07:29.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.635+0000 7f2ee0b00700 1 -- 172.21.15.57:0/1742491219 wait complete. 2023-11-28T12:07:29.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.636+0000 7f2ee0b00700 1 Processor -- start 2023-11-28T12:07:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.636+0000 7f2ee0b00700 1 -- start start 2023-11-28T12:07:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.637+0000 7f2ee0b00700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2edc0b1770 0x7f2edc15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.637+0000 7f2ee0b00700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2edc0ce040 0x7f2edc15b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.637+0000 7f2ee0b00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2edc142db0 0x7f2edc0cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.637+0000 7f2ee0b00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc15bec0 con 0x7f2edc142db0 2023-11-28T12:07:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.637+0000 7f2ee0b00700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc0c43a0 con 0x7f2edc0b1770 2023-11-28T12:07:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.637+0000 7f2ee0b00700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc15abc0 con 0x7f2edc0ce040 2023-11-28T12:07:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.637+0000 7f2edad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2edc142db0 0x7f2edc0cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.637+0000 7f2edad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2edc142db0 0x7f2edc0cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38976/0 (socket says 172.21.15.57:38976) 2023-11-28T12:07:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.637+0000 7f2edad9d700 1 -- 172.21.15.57:0/3527999237 learned_addr learned my addr 172.21.15.57:0/3527999237 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.637+0000 7f2ed9d9b700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2edc0ce040 0x7f2edc15b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.637+0000 7f2eda59c700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2edc0b1770 0x7f2edc15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.638+0000 7f2edad9d700 1 -- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2edc0ce040 msgr2=0x7f2edc15b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.638+0000 7f2edad9d700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2edc0ce040 0x7f2edc15b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.638+0000 7f2edad9d700 1 -- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2edc0b1770 msgr2=0x7f2edc15a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.638+0000 7f2edad9d700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2edc0b1770 0x7f2edc15a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.638+0000 7f2edad9d700 1 -- 172.21.15.57:0/3527999237 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc0094a0 con 0x7f2edc142db0 2023-11-28T12:07:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.638+0000 7f2edad9d700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2edc142db0 0x7f2edc0cc9e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400b680 tx=0x7f2ed4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.638+0000 7f2ecaffd700 1 -- 172.21.15.57:0/3527999237 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4004830 con 0x7f2edc142db0 2023-11-28T12:07:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.638+0000 7f2ecaffd700 1 -- 172.21.15.57:0/3527999237 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed40099b0 con 0x7f2edc142db0 2023-11-28T12:07:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.638+0000 7f2ecaffd700 1 -- 172.21.15.57:0/3527999237 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed401b670 con 0x7f2edc142db0 2023-11-28T12:07:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.638+0000 7f2ee0b00700 1 -- 172.21.15.57:0/3527999237 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed4009b30 con 0x7f2edc142db0 2023-11-28T12:07:29.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.638+0000 7f2ee0b00700 1 -- 172.21.15.57:0/3527999237 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc15d100 con 0x7f2edc142db0 2023-11-28T12:07:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.639+0000 7f2ee0b00700 1 -- 172.21.15.57:0/3527999237 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2edc13f9b0 con 0x7f2edc142db0 2023-11-28T12:07:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.639+0000 7f2ecaffd700 1 -- 172.21.15.57:0/3527999237 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2ed402b070 con 0x7f2edc142db0 2023-11-28T12:07:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.640+0000 7f2ecaffd700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2ebc0682f0 0x7f2ebc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.640+0000 7f2ecaffd700 1 -- 172.21.15.57:0/3527999237 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f2ed4087410 con 0x7f2edc142db0 2023-11-28T12:07:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.640+0000 7f2eda59c700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2ebc0682f0 0x7f2ebc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.641+0000 7f2eda59c700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2ebc0682f0 0x7f2ebc06a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2edc157d00 tx=0x7f2ecc009210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:29.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.644+0000 7f2ecaffd700 1 -- 172.21.15.57:0/3527999237 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2ed4053bf0 con 0x7f2edc142db0 2023-11-28T12:07:29.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.843+0000 7f2ee0b00700 1 -- 172.21.15.57:0/3527999237 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2edc15d3e0 con 0x7f2edc142db0 2023-11-28T12:07:29.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.844+0000 7f2ecaffd700 1 -- 172.21.15.57:0/3527999237 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18056 (secure 0 0 0) 0x7f2ed4057210 con 0x7f2edc142db0 2023-11-28T12:07:29.845 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:29.846 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":65,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:05:34.795435+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:04:50.522537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T12:04:51.333855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.32","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:07:29.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.846+0000 7f2ee0b00700 1 -- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2ebc0682f0 msgr2=0x7f2ebc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:29.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.846+0000 7f2ee0b00700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2ebc0682f0 0x7f2ebc06a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2edc157d00 tx=0x7f2ecc009210 comp rx=0 tx=0).stop 2023-11-28T12:07:29.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.846+0000 7f2ee0b00700 1 -- 172.21.15.57:0/3527999237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2edc142db0 msgr2=0x7f2edc0cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:29.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.846+0000 7f2ee0b00700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2edc142db0 0x7f2edc0cc9e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400b680 tx=0x7f2ed4004ad0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.846+0000 7f2ee0b00700 1 -- 172.21.15.57:0/3527999237 shutdown_connections 2023-11-28T12:07:29.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.846+0000 7f2ee0b00700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2ebc0682f0 0x7f2ebc06a7a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.846+0000 7f2ee0b00700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2edc142db0 0x7f2edc0cc9e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.846+0000 7f2ee0b00700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2edc0b1770 0x7f2edc15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.846+0000 7f2ee0b00700 1 --2- 172.21.15.57:0/3527999237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2edc0ce040 0x7f2edc15b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:29.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.846+0000 7f2ee0b00700 1 -- 172.21.15.57:0/3527999237 >> 172.21.15.57:0/3527999237 conn(0x7f2edc06c010 msgr2=0x7f2edc158dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:29.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.846+0000 7f2ee0b00700 1 -- 172.21.15.57:0/3527999237 shutdown_connections 2023-11-28T12:07:29.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.846+0000 7f2ee0b00700 1 -- 172.21.15.57:0/3527999237 wait complete. 2023-11-28T12:07:29.861 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T12:01:37.189786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T12:01:37.712343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T12:04:50.522537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T12:04:51.333855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T12:07:29.862 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T12:07:30.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.998+0000 7f9a1a2ac700 1 Processor -- start 2023-11-28T12:07:30.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.998+0000 7f9a1a2ac700 1 -- start start 2023-11-28T12:07:30.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.999+0000 7f9a1a2ac700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9a140cf710 0x7f9a140ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.999+0000 7f9a1a2ac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a1406a980 0x7f9a14071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.999+0000 7f9a1a2ac700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1415cc10 con 0x7f9a14071890 2023-11-28T12:07:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.999+0000 7f9a1a2ac700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1406c9b0 con 0x7f9a140cf710 2023-11-28T12:07:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.999+0000 7f9a1a2ac700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1406cc90 con 0x7f9a1406a980 2023-11-28T12:07:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:29.999+0000 7f9a13fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9a140cf710 0x7f9a140ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.000+0000 7f9a137fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f9a14071890 0x7f9a1406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49852/0 (socket says 172.21.15.57:49852) 2023-11-28T12:07:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.000+0000 7f9a137fe700 1 -- 172.21.15.57:0/2891028247 learned_addr learned my addr 172.21.15.57:0/2891028247 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.000+0000 7f9a18849700 1 --2- 172.21.15.57:0/2891028247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a1406a980 0x7f9a14071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.000+0000 7f9a13fff700 1 -- 172.21.15.57:0/2891028247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a1406a980 msgr2=0x7f9a14071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:30.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.000+0000 7f9a13fff700 1 --2- 172.21.15.57:0/2891028247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a1406a980 0x7f9a14071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.000+0000 7f9a13fff700 1 -- 172.21.15.57:0/2891028247 >> v1:172.21.15.57:6789/0 conn(0x7f9a14071890 legacy=0x7f9a1406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.000+0000 7f9a13fff700 1 -- 172.21.15.57:0/2891028247 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1406cf70 con 0x7f9a140cf710 2023-11-28T12:07:30.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.000+0000 7f9a12ffd700 1 -- 172.21.15.57:0/2891028247 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008701693 0 0) 0x7f9a1415cc10 con 0x7f9a14071890 2023-11-28T12:07:30.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.000+0000 7f9a18849700 1 --2- 172.21.15.57:0/2891028247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a1406a980 0x7f9a14071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:30.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.000+0000 7f9a13fff700 1 --2- 172.21.15.57:0/2891028247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a140cf710 0x7f9a140ce740 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009870 tx=0x7f9a04009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8666b1f766c36d73 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:30.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.000+0000 7f9a12ffd700 1 -- 172.21.15.57:0/2891028247 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04011830 con 0x7f9a140cf710 2023-11-28T12:07:30.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.000+0000 7f9a12ffd700 1 -- 172.21.15.57:0/2891028247 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a04011990 con 0x7f9a140cf710 2023-11-28T12:07:30.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.000+0000 7f9a12ffd700 1 -- 172.21.15.57:0/2891028247 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04011d20 con 0x7f9a140cf710 2023-11-28T12:07:30.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.001+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/2891028247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a140cf710 msgr2=0x7f9a140ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.001+0000 7f9a1a2ac700 1 --2- 172.21.15.57:0/2891028247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a140cf710 0x7f9a140ce740 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009870 tx=0x7f9a04009b80 comp rx=0 tx=0).stop 2023-11-28T12:07:30.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.001+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/2891028247 shutdown_connections 2023-11-28T12:07:30.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.001+0000 7f9a1a2ac700 1 --2- 172.21.15.57:0/2891028247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a140cf710 0x7f9a140ce740 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.001+0000 7f9a1a2ac700 1 --2- 172.21.15.57:0/2891028247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a1406a980 0x7f9a14071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.001+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/2891028247 >> 172.21.15.57:0/2891028247 conn(0x7f9a1406c010 msgr2=0x7f9a140cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:30.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.001+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/2891028247 shutdown_connections 2023-11-28T12:07:30.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.001+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/2891028247 wait complete. 2023-11-28T12:07:30.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.002+0000 7f9a1a2ac700 1 Processor -- start 2023-11-28T12:07:30.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.002+0000 7f9a1a2ac700 1 -- start start 2023-11-28T12:07:30.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.003+0000 7f9a1a2ac700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a1406a980 0x7f9a140b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.003+0000 7f9a1a2ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a14071890 0x7f9a140c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.003+0000 7f9a1a2ac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a140cf710 0x7f9a1414b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.003+0000 7f9a1a2ac700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140b08b0 con 0x7f9a1406a980 2023-11-28T12:07:30.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.003+0000 7f9a1a2ac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140b0b90 con 0x7f9a14071890 2023-11-28T12:07:30.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.003+0000 7f9a1a2ac700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140b0e70 con 0x7f9a140cf710 2023-11-28T12:07:30.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.003+0000 7f9a13fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a1406a980 0x7f9a140b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.003+0000 7f9a13fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a1406a980 0x7f9a140b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38994/0 (socket says 172.21.15.57:38994) 2023-11-28T12:07:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.003+0000 7f9a13fff700 1 -- 172.21.15.57:0/1474739244 learned_addr learned my addr 172.21.15.57:0/1474739244 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.004+0000 7f9a18849700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a140cf710 0x7f9a1414b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.004+0000 7f9a137fe700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a14071890 0x7f9a140c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.004+0000 7f9a13fff700 1 -- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a140cf710 msgr2=0x7f9a1414b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.004+0000 7f9a13fff700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a140cf710 0x7f9a1414b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.004+0000 7f9a13fff700 1 -- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a14071890 msgr2=0x7f9a140c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.004+0000 7f9a13fff700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a14071890 0x7f9a140c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.004+0000 7f9a13fff700 1 -- 172.21.15.57:0/1474739244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a040094a0 con 0x7f9a1406a980 2023-11-28T12:07:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.004+0000 7f9a13fff700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a1406a980 0x7f9a140b1930 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9a04000c00 tx=0x7f9a04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.004+0000 7f9a10ff9700 1 -- 172.21.15.57:0/1474739244 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040036a0 con 0x7f9a1406a980 2023-11-28T12:07:30.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.004+0000 7f9a10ff9700 1 -- 172.21.15.57:0/1474739244 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0401e070 con 0x7f9a1406a980 2023-11-28T12:07:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.004+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/1474739244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1406d530 con 0x7f9a1406a980 2023-11-28T12:07:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.004+0000 7f9a10ff9700 1 -- 172.21.15.57:0/1474739244 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04003a60 con 0x7f9a1406a980 2023-11-28T12:07:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.004+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/1474739244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1406d250 con 0x7f9a1406a980 2023-11-28T12:07:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.005+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/1474739244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a14160480 con 0x7f9a1406a980 2023-11-28T12:07:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.006+0000 7f9a10ff9700 1 -- 172.21.15.57:0/1474739244 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9a0400f050 con 0x7f9a1406a980 2023-11-28T12:07:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.006+0000 7f9a10ff9700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9a000682f0 0x7f9a0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.006+0000 7f9a10ff9700 1 -- 172.21.15.57:0/1474739244 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f9a04087130 con 0x7f9a1406a980 2023-11-28T12:07:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.007+0000 7f9a137fe700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9a000682f0 0x7f9a0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.007+0000 7f9a137fe700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9a000682f0 0x7f9a0006a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9a08000f60 tx=0x7f9a08007be0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:30.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.011+0000 7f9a10ff9700 1 -- 172.21.15.57:0/1474739244 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9a040538a0 con 0x7f9a1406a980 2023-11-28T12:07:30.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.209+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/1474739244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9a140743f0 con 0x7f9a1406a980 2023-11-28T12:07:30.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.210+0000 7f9a10ff9700 1 -- 172.21.15.57:0/1474739244 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9a04056ec0 con 0x7f9a1406a980 2023-11-28T12:07:30.214 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 8 2023-11-28T12:07:30.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.212+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9a000682f0 msgr2=0x7f9a0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.212+0000 7f9a1a2ac700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9a000682f0 0x7f9a0006a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9a08000f60 tx=0x7f9a08007be0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.212+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/1474739244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a1406a980 msgr2=0x7f9a140b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.212+0000 7f9a1a2ac700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a1406a980 0x7f9a140b1930 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9a04000c00 tx=0x7f9a04004750 comp rx=0 tx=0).stop 2023-11-28T12:07:30.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.212+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/1474739244 shutdown_connections 2023-11-28T12:07:30.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.212+0000 7f9a1a2ac700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f9a000682f0 0x7f9a0006a7a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.212+0000 7f9a1a2ac700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a140cf710 0x7f9a1414b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.212+0000 7f9a1a2ac700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a1406a980 0x7f9a140b1930 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.212+0000 7f9a1a2ac700 1 --2- 172.21.15.57:0/1474739244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a14071890 0x7f9a140c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.212+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/1474739244 >> 172.21.15.57:0/1474739244 conn(0x7f9a1406c010 msgr2=0x7f9a1415ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:30.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.212+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/1474739244 shutdown_connections 2023-11-28T12:07:30.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.212+0000 7f9a1a2ac700 1 -- 172.21.15.57:0/1474739244 wait complete. 2023-11-28T12:07:30.227 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T12:07:30.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.363+0000 7efd9f7f7700 1 Processor -- start 2023-11-28T12:07:30.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.363+0000 7efd9f7f7700 1 -- start start 2023-11-28T12:07:30.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.364+0000 7efd9f7f7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd980b1650 0x7efd980b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.364+0000 7efd9f7f7700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd98159ec0 con 0x7efd980af0a0 2023-11-28T12:07:30.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.364+0000 7efd9f7f7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9815aa60 con 0x7efd981599b0 2023-11-28T12:07:30.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.364+0000 7efd9f7f7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd980b1f60 con 0x7efd980b1650 2023-11-28T12:07:30.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.364+0000 7efd9dd94700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7efd980af0a0 0x7efd9815a350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49868/0 (socket says 172.21.15.57:49868) 2023-11-28T12:07:30.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.364+0000 7efd9dd94700 1 -- 172.21.15.57:0/3473536972 learned_addr learned my addr 172.21.15.57:0/3473536972 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:30.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.365+0000 7efd9d593700 1 --2- 172.21.15.57:0/3473536972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd980b1650 0x7efd980b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.365+0000 7efd8bfff700 1 -- 172.21.15.57:0/3473536972 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729394099 0 0) 0x7efd98159ec0 con 0x7efd980af0a0 2023-11-28T12:07:30.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.365+0000 7efd8bfff700 1 -- 172.21.15.57:0/3473536972 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd840035b0 con 0x7efd980af0a0 2023-11-28T12:07:30.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.365+0000 7efd8bfff700 1 -- 172.21.15.57:0/3473536972 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2945704190 0 0) 0x7efd9815aa60 con 0x7efd981599b0 2023-11-28T12:07:30.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.365+0000 7efd8bfff700 1 -- 172.21.15.57:0/3473536972 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd98159ec0 con 0x7efd981599b0 2023-11-28T12:07:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.365+0000 7efd9d593700 1 -- 172.21.15.57:0/3473536972 >> v1:172.21.15.181:6789/0 conn(0x7efd981599b0 legacy=0x7efd9815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.365+0000 7efd8bfff700 1 -- 172.21.15.57:0/3473536972 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3450959897 0 0) 0x7efd840035b0 con 0x7efd980af0a0 2023-11-28T12:07:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.365+0000 7efd9d593700 1 -- 172.21.15.57:0/3473536972 >> v1:172.21.15.57:6789/0 conn(0x7efd980af0a0 legacy=0x7efd9815a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.365+0000 7efd9d593700 1 -- 172.21.15.57:0/3473536972 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9815d8a0 con 0x7efd980b1650 2023-11-28T12:07:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.365+0000 7efd9d593700 1 --2- 172.21.15.57:0/3473536972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd980b1650 0x7efd980b1a20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009870 tx=0x7efd8c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94d911d29ae04292 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.365+0000 7efd8bfff700 1 -- 172.21.15.57:0/3473536972 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd900019f0 con 0x7efd980b1650 2023-11-28T12:07:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.365+0000 7efd8bfff700 1 -- 172.21.15.57:0/3473536972 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c004030 con 0x7efd980b1650 2023-11-28T12:07:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.365+0000 7efd8bfff700 1 -- 172.21.15.57:0/3473536972 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0043c0 con 0x7efd980b1650 2023-11-28T12:07:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.366+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3473536972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd980b1650 msgr2=0x7efd980b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.366+0000 7efd9f7f7700 1 --2- 172.21.15.57:0/3473536972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd980b1650 0x7efd980b1a20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009870 tx=0x7efd8c009b80 comp rx=0 tx=0).stop 2023-11-28T12:07:30.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.366+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3473536972 shutdown_connections 2023-11-28T12:07:30.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.366+0000 7efd9f7f7700 1 --2- 172.21.15.57:0/3473536972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd980b1650 0x7efd980b1a20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.366+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3473536972 >> 172.21.15.57:0/3473536972 conn(0x7efd9806c010 msgr2=0x7efd980cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:30.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.366+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3473536972 shutdown_connections 2023-11-28T12:07:30.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.366+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3473536972 wait complete. 2023-11-28T12:07:30.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.367+0000 7efd9f7f7700 1 Processor -- start 2023-11-28T12:07:30.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.367+0000 7efd9f7f7700 1 -- start start 2023-11-28T12:07:30.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.368+0000 7efd9f7f7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd980af0a0 0x7efd98150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.368+0000 7efd9f7f7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd980b1650 0x7efd981e5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.368+0000 7efd9f7f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd98150e10 0x7efd981e3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.368+0000 7efd9f7f7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd981e5430 con 0x7efd98150e10 2023-11-28T12:07:30.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.368+0000 7efd9f7f7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd98150b70 con 0x7efd980b1650 2023-11-28T12:07:30.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.368+0000 7efd9f7f7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd981e47b0 con 0x7efd980af0a0 2023-11-28T12:07:30.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.368+0000 7efd9dd94700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd98150e10 0x7efd981e3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.368+0000 7efd9dd94700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd98150e10 0x7efd981e3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39006/0 (socket says 172.21.15.57:39006) 2023-11-28T12:07:30.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.368+0000 7efd9dd94700 1 -- 172.21.15.57:0/3329330394 learned_addr learned my addr 172.21.15.57:0/3329330394 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:30.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.368+0000 7efd9d593700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd980af0a0 0x7efd98150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.368+0000 7efd9cd92700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd980b1650 0x7efd981e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.369+0000 7efd9dd94700 1 -- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd980af0a0 msgr2=0x7efd98150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.369+0000 7efd9dd94700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd980af0a0 0x7efd98150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.369+0000 7efd9dd94700 1 -- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd980b1650 msgr2=0x7efd981e5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.369+0000 7efd9dd94700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd980b1650 0x7efd981e5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.369+0000 7efd9dd94700 1 -- 172.21.15.57:0/3329330394 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c0094a0 con 0x7efd98150e10 2023-11-28T12:07:30.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.369+0000 7efd9cd92700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd980b1650 0x7efd981e5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:30.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.369+0000 7efd9dd94700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd98150e10 0x7efd981e3ed0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7efd90001700 tx=0x7efd900021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.369+0000 7efd89ffb700 1 -- 172.21.15.57:0/3329330394 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9000ab50 con 0x7efd98150e10 2023-11-28T12:07:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.369+0000 7efd89ffb700 1 -- 172.21.15.57:0/3329330394 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9000acd0 con 0x7efd98150e10 2023-11-28T12:07:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.369+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3329330394 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9815de60 con 0x7efd98150e10 2023-11-28T12:07:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.369+0000 7efd89ffb700 1 -- 172.21.15.57:0/3329330394 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90007310 con 0x7efd98150e10 2023-11-28T12:07:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.369+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3329330394 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9815db80 con 0x7efd98150e10 2023-11-28T12:07:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.370+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3329330394 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd981e5570 con 0x7efd98150e10 2023-11-28T12:07:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.371+0000 7efd89ffb700 1 -- 172.21.15.57:0/3329330394 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efd9001a030 con 0x7efd98150e10 2023-11-28T12:07:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.371+0000 7efd89ffb700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efd7c0682f0 0x7efd7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.371+0000 7efd89ffb700 1 -- 172.21.15.57:0/3329330394 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7efd90085a40 con 0x7efd98150e10 2023-11-28T12:07:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.371+0000 7efd9d593700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efd7c0682f0 0x7efd7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.372+0000 7efd9d593700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efd7c0682f0 0x7efd7c06a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0097c0 tx=0x7efd8c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:30.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.375+0000 7efd89ffb700 1 -- 172.21.15.57:0/3329330394 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efd981e5570 con 0x7efd98150e10 2023-11-28T12:07:30.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.576+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3329330394 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd981e5570 con 0x7efd98150e10 2023-11-28T12:07:30.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.577+0000 7efd89ffb700 1 -- 172.21.15.57:0/3329330394 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd981e5570 con 0x7efd98150e10 2023-11-28T12:07:30.578 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-11-28T12:07:30.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.579+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efd7c0682f0 msgr2=0x7efd7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.579+0000 7efd9f7f7700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efd7c0682f0 0x7efd7c06a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0097c0 tx=0x7efd8c008030 comp rx=0 tx=0).stop 2023-11-28T12:07:30.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.579+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3329330394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd98150e10 msgr2=0x7efd981e3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.579+0000 7efd9f7f7700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd98150e10 0x7efd981e3ed0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7efd90001700 tx=0x7efd900021d0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.579+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3329330394 shutdown_connections 2023-11-28T12:07:30.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.579+0000 7efd9f7f7700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7efd7c0682f0 0x7efd7c06a7a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.579+0000 7efd9f7f7700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd98150e10 0x7efd981e3ed0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.579+0000 7efd9f7f7700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd980af0a0 0x7efd98150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.579+0000 7efd9f7f7700 1 --2- 172.21.15.57:0/3329330394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd980b1650 0x7efd981e5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.579+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3329330394 >> 172.21.15.57:0/3329330394 conn(0x7efd9806c010 msgr2=0x7efd9815c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:30.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.579+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3329330394 shutdown_connections 2023-11-28T12:07:30.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.579+0000 7efd9f7f7700 1 -- 172.21.15.57:0/3329330394 wait complete. 2023-11-28T12:07:30.594 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T12:07:30.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.731+0000 7f0ffebf4700 1 Processor -- start 2023-11-28T12:07:30.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.731+0000 7f0ffebf4700 1 -- start start 2023-11-28T12:07:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.732+0000 7f0ffebf4700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f0ff80d0ad0 0x7f0ff80cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.732+0000 7f0ffebf4700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff80cc740 con 0x7f0ff815a5f0 2023-11-28T12:07:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.732+0000 7f0ffebf4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff815a9c0 con 0x7f0ff80b2ee0 2023-11-28T12:07:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.732+0000 7f0ffebf4700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff80c7b80 con 0x7f0ff80d0ad0 2023-11-28T12:07:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.732+0000 7f0ffd191700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f0ff80d0ad0 0x7f0ff80cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.732+0000 7f0ffc990700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f0ff815a5f0 0x7f0ff80b27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49874/0 (socket says 172.21.15.57:49874) 2023-11-28T12:07:30.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.732+0000 7f0ffc990700 1 -- 172.21.15.57:0/483699821 learned_addr learned my addr 172.21.15.57:0/483699821 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.733+0000 7f0ffd191700 1 -- 172.21.15.57:0/483699821 >> v1:172.21.15.181:6789/0 conn(0x7f0ff80b2ee0 legacy=0x7f0ff80ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.733+0000 7f0ffd191700 1 -- 172.21.15.57:0/483699821 >> v1:172.21.15.57:6789/0 conn(0x7f0ff815a5f0 legacy=0x7f0ff80b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.733+0000 7f0ffd191700 1 -- 172.21.15.57:0/483699821 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff80cc990 con 0x7f0ff80d0ad0 2023-11-28T12:07:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.733+0000 7f0ffd191700 1 --2- 172.21.15.57:0/483699821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ff80d0ad0 0x7f0ff80cc200 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009d90 tx=0x7f0ff000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7e5fb987871b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.733+0000 7f0ff77fe700 1 -- 172.21.15.57:0/483699821 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff000e070 con 0x7f0ff80d0ad0 2023-11-28T12:07:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.733+0000 7f0ff77fe700 1 -- 172.21.15.57:0/483699821 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff000bbd0 con 0x7f0ff80d0ad0 2023-11-28T12:07:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.733+0000 7f0ff77fe700 1 -- 172.21.15.57:0/483699821 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0012430 con 0x7f0ff80d0ad0 2023-11-28T12:07:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.733+0000 7f0ffebf4700 1 -- 172.21.15.57:0/483699821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ff80d0ad0 msgr2=0x7f0ff80cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.733+0000 7f0ffebf4700 1 --2- 172.21.15.57:0/483699821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ff80d0ad0 0x7f0ff80cc200 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009d90 tx=0x7f0ff000b330 comp rx=0 tx=0).stop 2023-11-28T12:07:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.734+0000 7f0ffebf4700 1 -- 172.21.15.57:0/483699821 shutdown_connections 2023-11-28T12:07:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.734+0000 7f0ffebf4700 1 --2- 172.21.15.57:0/483699821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ff80d0ad0 0x7f0ff80cc200 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.734+0000 7f0ffebf4700 1 -- 172.21.15.57:0/483699821 >> 172.21.15.57:0/483699821 conn(0x7f0ff806ea80 msgr2=0x7f0ff806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:30.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.734+0000 7f0ffebf4700 1 -- 172.21.15.57:0/483699821 shutdown_connections 2023-11-28T12:07:30.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.734+0000 7f0ffebf4700 1 -- 172.21.15.57:0/483699821 wait complete. 2023-11-28T12:07:30.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.735+0000 7f0ffebf4700 1 Processor -- start 2023-11-28T12:07:30.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.735+0000 7f0ffebf4700 1 -- start start 2023-11-28T12:07:30.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffebf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ff80b2ee0 0x7f0ff814ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffebf4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ff80d0ad0 0x7f0ff80c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffebf4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ff815a5f0 0x7f0ff80c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffebf4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff81e3f70 con 0x7f0ff815a5f0 2023-11-28T12:07:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffebf4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff81e4250 con 0x7f0ff80b2ee0 2023-11-28T12:07:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffebf4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff80c49a0 con 0x7f0ff80d0ad0 2023-11-28T12:07:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffd191700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ff815a5f0 0x7f0ff80c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffd191700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ff815a5f0 0x7f0ff80c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39016/0 (socket says 172.21.15.57:39016) 2023-11-28T12:07:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffd191700 1 -- 172.21.15.57:0/374365101 learned_addr learned my addr 172.21.15.57:0/374365101 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffc990700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ff80b2ee0 0x7f0ff814ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ff7fff700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ff80d0ad0 0x7f0ff80c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffd191700 1 -- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ff80d0ad0 msgr2=0x7f0ff80c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffd191700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ff80d0ad0 0x7f0ff80c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffd191700 1 -- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ff80b2ee0 msgr2=0x7f0ff814ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffd191700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ff80b2ee0 0x7f0ff814ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffd191700 1 -- 172.21.15.57:0/374365101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0009a70 con 0x7f0ff815a5f0 2023-11-28T12:07:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.736+0000 7f0ffd191700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ff815a5f0 0x7f0ff80c6770 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0014040 tx=0x7f0ff0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.737+0000 7f0ff57fa700 1 -- 172.21.15.57:0/374365101 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00098f0 con 0x7f0ff815a5f0 2023-11-28T12:07:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.737+0000 7f0ff57fa700 1 -- 172.21.15.57:0/374365101 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff0020070 con 0x7f0ff815a5f0 2023-11-28T12:07:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.737+0000 7f0ffebf4700 1 -- 172.21.15.57:0/374365101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff80ccf50 con 0x7f0ff815a5f0 2023-11-28T12:07:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.737+0000 7f0ff57fa700 1 -- 172.21.15.57:0/374365101 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0004280 con 0x7f0ff815a5f0 2023-11-28T12:07:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.737+0000 7f0ffebf4700 1 -- 172.21.15.57:0/374365101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff80ccc70 con 0x7f0ff815a5f0 2023-11-28T12:07:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.738+0000 7f0ffebf4700 1 -- 172.21.15.57:0/374365101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff80cd230 con 0x7f0ff815a5f0 2023-11-28T12:07:30.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.738+0000 7f0ff57fa700 1 -- 172.21.15.57:0/374365101 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0ff002e070 con 0x7f0ff815a5f0 2023-11-28T12:07:30.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.738+0000 7f0ff57fa700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0fd80682f0 0x7f0fd806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:30.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.739+0000 7f0ff57fa700 1 -- 172.21.15.57:0/374365101 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f0ff0022070 con 0x7f0ff815a5f0 2023-11-28T12:07:30.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.739+0000 7f0ffc990700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0fd80682f0 0x7f0fd806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:30.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.739+0000 7f0ffc990700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0fd80682f0 0x7f0fd806a7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8005910 tx=0x7f0fe8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:30.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.743+0000 7f0ff57fa700 1 -- 172.21.15.57:0/374365101 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0ff00543c0 con 0x7f0ff815a5f0 2023-11-28T12:07:30.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.945+0000 7f0ffebf4700 1 -- 172.21.15.57:0/374365101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ff81e43c0 con 0x7f0ff815a5f0 2023-11-28T12:07:30.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.946+0000 7f0ff57fa700 1 -- 172.21.15.57:0/374365101 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0ff002c020 con 0x7f0ff815a5f0 2023-11-28T12:07:30.947 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-11-28T12:07:30.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.948+0000 7f0ffebf4700 1 -- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0fd80682f0 msgr2=0x7f0fd806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.948+0000 7f0ffebf4700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0fd80682f0 0x7f0fd806a7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8005910 tx=0x7f0fe8009040 comp rx=0 tx=0).stop 2023-11-28T12:07:30.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.948+0000 7f0ffebf4700 1 -- 172.21.15.57:0/374365101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ff815a5f0 msgr2=0x7f0ff80c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:30.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.948+0000 7f0ffebf4700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ff815a5f0 0x7f0ff80c6770 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0014040 tx=0x7f0ff0004ad0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.948+0000 7f0ffebf4700 1 -- 172.21.15.57:0/374365101 shutdown_connections 2023-11-28T12:07:30.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.948+0000 7f0ffebf4700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0fd80682f0 0x7f0fd806a7a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.948+0000 7f0ffebf4700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ff815a5f0 0x7f0ff80c6770 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.948+0000 7f0ffebf4700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ff80b2ee0 0x7f0ff814ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.948+0000 7f0ffebf4700 1 --2- 172.21.15.57:0/374365101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ff80d0ad0 0x7f0ff80c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:30.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.948+0000 7f0ffebf4700 1 -- 172.21.15.57:0/374365101 >> 172.21.15.57:0/374365101 conn(0x7f0ff806ea80 msgr2=0x7f0ff81553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:30.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.948+0000 7f0ffebf4700 1 -- 172.21.15.57:0/374365101 shutdown_connections 2023-11-28T12:07:30.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:30.948+0000 7f0ffebf4700 1 -- 172.21.15.57:0/374365101 wait complete. 2023-11-28T12:07:30.963 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T12:07:31.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.100+0000 7fb70f411700 1 Processor -- start 2023-11-28T12:07:31.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.100+0000 7fb70f411700 1 -- start start 2023-11-28T12:07:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.101+0000 7fb70f411700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7080b1650 0x7fb7080b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.101+0000 7fb70f411700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7081599b0 0x7fb70815d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.101+0000 7fb70f411700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb7080af0a0 0x7fb70815a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.101+0000 7fb70f411700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb708159ec0 con 0x7fb7080af0a0 2023-11-28T12:07:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.101+0000 7fb70f411700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7080b1f60 con 0x7fb7081599b0 2023-11-28T12:07:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.101+0000 7fb70f411700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70815d8a0 con 0x7fb7080b1650 2023-11-28T12:07:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.101+0000 7fb70d9ae700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb7080af0a0 0x7fb70815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.101+0000 7fb70d9ae700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb7080af0a0 0x7fb70815a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39026/0 (socket says 172.21.15.57:39026) 2023-11-28T12:07:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.101+0000 7fb70d9ae700 1 -- 172.21.15.57:0/3106335964 learned_addr learned my addr 172.21.15.57:0/3106335964 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.101+0000 7fb70c9ac700 1 --2- 172.21.15.57:0/3106335964 >> v2:172.21.15.181:3300/0 conn(0x7fb7081599b0 0x7fb70815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.101+0000 7fb70d1ad700 1 --2- 172.21.15.57:0/3106335964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7080b1650 0x7fb7080b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70d9ae700 1 -- 172.21.15.57:0/3106335964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7080b1650 msgr2=0x7fb7080b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:31.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70d9ae700 1 --2- 172.21.15.57:0/3106335964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7080b1650 0x7fb7080b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70d9ae700 1 -- 172.21.15.57:0/3106335964 >> v2:172.21.15.181:3300/0 conn(0x7fb7081599b0 msgr2=0x7fb70815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:31.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70d9ae700 1 --2- 172.21.15.57:0/3106335964 >> v2:172.21.15.181:3300/0 conn(0x7fb7081599b0 0x7fb70815d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70d9ae700 1 -- 172.21.15.57:0/3106335964 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70815db80 con 0x7fb7080af0a0 2023-11-28T12:07:31.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70d1ad700 1 --2- 172.21.15.57:0/3106335964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7080b1650 0x7fb7080b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:31.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70d9ae700 1 --2- 172.21.15.57:0/3106335964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7080af0a0 0x7fb70815a350 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb7080708e0 tx=0x7fb70000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8647a469f9da19ec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:31.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb6fffff700 1 -- 172.21.15.57:0/3106335964 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700010070 con 0x7fb7080af0a0 2023-11-28T12:07:31.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb6fffff700 1 -- 172.21.15.57:0/3106335964 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70000dd90 con 0x7fb7080af0a0 2023-11-28T12:07:31.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb6fffff700 1 -- 172.21.15.57:0/3106335964 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70000b310 con 0x7fb7080af0a0 2023-11-28T12:07:31.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70f411700 1 -- 172.21.15.57:0/3106335964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7080af0a0 msgr2=0x7fb70815a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70f411700 1 --2- 172.21.15.57:0/3106335964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7080af0a0 0x7fb70815a350 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb7080708e0 tx=0x7fb70000d430 comp rx=0 tx=0).stop 2023-11-28T12:07:31.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70f411700 1 -- 172.21.15.57:0/3106335964 shutdown_connections 2023-11-28T12:07:31.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70f411700 1 --2- 172.21.15.57:0/3106335964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7080b1650 0x7fb7080b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70f411700 1 --2- 172.21.15.57:0/3106335964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7080af0a0 0x7fb70815a350 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70f411700 1 --2- 172.21.15.57:0/3106335964 >> v2:172.21.15.181:3300/0 conn(0x7fb7081599b0 0x7fb70815d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.102+0000 7fb70f411700 1 -- 172.21.15.57:0/3106335964 >> 172.21.15.57:0/3106335964 conn(0x7fb70806c010 msgr2=0x7fb7080cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:31.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.103+0000 7fb70f411700 1 -- 172.21.15.57:0/3106335964 shutdown_connections 2023-11-28T12:07:31.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.103+0000 7fb70f411700 1 -- 172.21.15.57:0/3106335964 wait complete. 2023-11-28T12:07:31.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.104+0000 7fb70f411700 1 Processor -- start 2023-11-28T12:07:31.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.104+0000 7fb70f411700 1 -- start start 2023-11-28T12:07:31.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70f411700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7080af0a0 0x7fb7080caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70f411700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7080b1650 0x7fb7081f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70f411700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7080b3a50 0x7fb7080cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70f411700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080b4a90 con 0x7fb7080b3a50 2023-11-28T12:07:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70f411700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080b50f0 con 0x7fb7080b1650 2023-11-28T12:07:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70f411700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7081f28d0 con 0x7fb7080af0a0 2023-11-28T12:07:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70d9ae700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7080b3a50 0x7fb7080cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70d9ae700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7080b3a50 0x7fb7080cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39042/0 (socket says 172.21.15.57:39042) 2023-11-28T12:07:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70d9ae700 1 -- 172.21.15.57:0/335132502 learned_addr learned my addr 172.21.15.57:0/335132502 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70c9ac700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7080b1650 0x7fb7081f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70d1ad700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7080af0a0 0x7fb7080caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70d9ae700 1 -- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7080af0a0 msgr2=0x7fb7080caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70d9ae700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7080af0a0 0x7fb7080caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70d9ae700 1 -- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7080b1650 msgr2=0x7fb7081f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70d9ae700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7080b1650 0x7fb7081f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70d9ae700 1 -- 172.21.15.57:0/335132502 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70000bd40 con 0x7fb7080b3a50 2023-11-28T12:07:31.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70d9ae700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7080b3a50 0x7fb7080cb790 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb70000d830 tx=0x7fb700004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:31.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb6fdffb700 1 -- 172.21.15.57:0/335132502 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70000bbc0 con 0x7fb7080b3a50 2023-11-28T12:07:31.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb6fdffb700 1 -- 172.21.15.57:0/335132502 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70000b4c0 con 0x7fb7080b3a50 2023-11-28T12:07:31.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb70f411700 1 -- 172.21.15.57:0/335132502 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70815d590 con 0x7fb7080b3a50 2023-11-28T12:07:31.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.105+0000 7fb6fdffb700 1 -- 172.21.15.57:0/335132502 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70000aea0 con 0x7fb7080b3a50 2023-11-28T12:07:31.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.106+0000 7fb70f411700 1 -- 172.21.15.57:0/335132502 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70815de60 con 0x7fb7080b3a50 2023-11-28T12:07:31.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.107+0000 7fb70f411700 1 -- 172.21.15.57:0/335132502 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70815a890 con 0x7fb7080b3a50 2023-11-28T12:07:31.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.107+0000 7fb6fdffb700 1 -- 172.21.15.57:0/335132502 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb70002d070 con 0x7fb7080b3a50 2023-11-28T12:07:31.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.108+0000 7fb6fdffb700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb6ec0682f0 0x7fb6ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.108+0000 7fb6fdffb700 1 -- 172.21.15.57:0/335132502 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fb7000893d0 con 0x7fb7080b3a50 2023-11-28T12:07:31.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.108+0000 7fb70d1ad700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb6ec0682f0 0x7fb6ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.109+0000 7fb70d1ad700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb6ec0682f0 0x7fb6ec06a7a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb7080c8220 tx=0x7fb704008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:31.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.112+0000 7fb6fdffb700 1 -- 172.21.15.57:0/335132502 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb700055ac0 con 0x7fb7080b3a50 2023-11-28T12:07:31.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.313+0000 7fb70f411700 1 -- 172.21.15.57:0/335132502 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7080cd340 con 0x7fb7080b3a50 2023-11-28T12:07:31.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.313+0000 7fb6fdffb700 1 -- 172.21.15.57:0/335132502 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7000590e0 con 0x7fb7080b3a50 2023-11-28T12:07:31.315 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-11-28T12:07:31.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.316+0000 7fb70f411700 1 -- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb6ec0682f0 msgr2=0x7fb6ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.316+0000 7fb70f411700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb6ec0682f0 0x7fb6ec06a7a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb7080c8220 tx=0x7fb704008040 comp rx=0 tx=0).stop 2023-11-28T12:07:31.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.316+0000 7fb70f411700 1 -- 172.21.15.57:0/335132502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7080b3a50 msgr2=0x7fb7080cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.316+0000 7fb70f411700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7080b3a50 0x7fb7080cb790 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb70000d830 tx=0x7fb700004ad0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.316+0000 7fb70f411700 1 -- 172.21.15.57:0/335132502 shutdown_connections 2023-11-28T12:07:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.316+0000 7fb70f411700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb6ec0682f0 0x7fb6ec06a7a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.316+0000 7fb70f411700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7080b3a50 0x7fb7080cb790 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.316+0000 7fb70f411700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7080af0a0 0x7fb7080caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.316+0000 7fb70f411700 1 --2- 172.21.15.57:0/335132502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7080b1650 0x7fb7081f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.316+0000 7fb70f411700 1 -- 172.21.15.57:0/335132502 >> 172.21.15.57:0/335132502 conn(0x7fb70806c010 msgr2=0x7fb7080cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.316+0000 7fb70f411700 1 -- 172.21.15.57:0/335132502 shutdown_connections 2023-11-28T12:07:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.316+0000 7fb70f411700 1 -- 172.21.15.57:0/335132502 wait complete. 2023-11-28T12:07:31.331 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:07:31.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.465+0000 7f887fc0b700 1 Processor -- start 2023-11-28T12:07:31.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.465+0000 7f887fc0b700 1 -- start start 2023-11-28T12:07:31.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.466+0000 7f887fc0b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8878070d90 0x7f88780711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.466+0000 7f887fc0b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f887815e0b0 0x7f887815e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.466+0000 7f887fc0b700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8878151a50 con 0x7f887806a270 2023-11-28T12:07:31.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.466+0000 7f887fc0b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887815b440 con 0x7f8878070d90 2023-11-28T12:07:31.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.466+0000 7f887fc0b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8878071ac0 con 0x7f887815e0b0 2023-11-28T12:07:31.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.466+0000 7f887d1a6700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8878070d90 0x7f88780711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.466+0000 7f887d9a7700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f887806a270 0x7f887806a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49884/0 (socket says 172.21.15.57:49884) 2023-11-28T12:07:31.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.466+0000 7f887d1a6700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8878070d90 0x7f88780711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39048/0 (socket says 172.21.15.57:39048) 2023-11-28T12:07:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.466+0000 7f887d9a7700 1 -- 172.21.15.57:0/2999085730 learned_addr learned my addr 172.21.15.57:0/2999085730 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.466+0000 7f887e1a8700 1 --2- 172.21.15.57:0/2999085730 >> v2:172.21.15.181:3300/0 conn(0x7f887815e0b0 0x7f887815e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.467+0000 7f887d1a6700 1 -- 172.21.15.57:0/2999085730 >> v2:172.21.15.181:3300/0 conn(0x7f887815e0b0 msgr2=0x7f887815e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.467+0000 7f887d1a6700 1 --2- 172.21.15.57:0/2999085730 >> v2:172.21.15.181:3300/0 conn(0x7f887815e0b0 0x7f887815e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.467+0000 7f887d1a6700 1 -- 172.21.15.57:0/2999085730 >> v1:172.21.15.57:6789/0 conn(0x7f887806a270 legacy=0x7f887806a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.467+0000 7f887d1a6700 1 -- 172.21.15.57:0/2999085730 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887815adf0 con 0x7f8878070d90 2023-11-28T12:07:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.467+0000 7f887c9a5700 1 -- 172.21.15.57:0/2999085730 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816015255 0 0) 0x7f8878151a50 con 0x7f887806a270 2023-11-28T12:07:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.467+0000 7f887d1a6700 1 --2- 172.21.15.57:0/2999085730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8878070d90 0x7f88780711e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f887815dff0 tx=0x7f8868009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309c49de86faeafa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.467+0000 7f887c9a5700 1 -- 172.21.15.57:0/2999085730 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868011830 con 0x7f8878070d90 2023-11-28T12:07:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.467+0000 7f887c9a5700 1 -- 172.21.15.57:0/2999085730 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8868011990 con 0x7f8878070d90 2023-11-28T12:07:31.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.467+0000 7f887c9a5700 1 -- 172.21.15.57:0/2999085730 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868011d20 con 0x7f8878070d90 2023-11-28T12:07:31.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.467+0000 7f887fc0b700 1 -- 172.21.15.57:0/2999085730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8878070d90 msgr2=0x7f88780711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.467+0000 7f887fc0b700 1 --2- 172.21.15.57:0/2999085730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8878070d90 0x7f88780711e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f887815dff0 tx=0x7f8868009b20 comp rx=0 tx=0).stop 2023-11-28T12:07:31.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.468+0000 7f887fc0b700 1 -- 172.21.15.57:0/2999085730 shutdown_connections 2023-11-28T12:07:31.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.468+0000 7f887fc0b700 1 --2- 172.21.15.57:0/2999085730 >> v2:172.21.15.181:3300/0 conn(0x7f887815e0b0 0x7f887815e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.468+0000 7f887fc0b700 1 --2- 172.21.15.57:0/2999085730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8878070d90 0x7f88780711e0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.468+0000 7f887fc0b700 1 -- 172.21.15.57:0/2999085730 >> 172.21.15.57:0/2999085730 conn(0x7f887806c010 msgr2=0x7f88780cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:31.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.468+0000 7f887fc0b700 1 -- 172.21.15.57:0/2999085730 shutdown_connections 2023-11-28T12:07:31.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.468+0000 7f887fc0b700 1 -- 172.21.15.57:0/2999085730 wait complete. 2023-11-28T12:07:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.469+0000 7f887fc0b700 1 Processor -- start 2023-11-28T12:07:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.469+0000 7f887fc0b700 1 -- start start 2023-11-28T12:07:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887fc0b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f887806a270 0x7f88780c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887fc0b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8878070d90 0x7f88780c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887fc0b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f887815e0b0 0x7f8878159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887fc0b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887815ca90 con 0x7f887815e0b0 2023-11-28T12:07:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887fc0b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887815ba80 con 0x7f887806a270 2023-11-28T12:07:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887fc0b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887815c600 con 0x7f8878070d90 2023-11-28T12:07:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887e1a8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f887815e0b0 0x7f8878159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887e1a8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f887815e0b0 0x7f8878159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39054/0 (socket says 172.21.15.57:39054) 2023-11-28T12:07:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887e1a8700 1 -- 172.21.15.57:0/467058960 learned_addr learned my addr 172.21.15.57:0/467058960 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887d1a6700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8878070d90 0x7f88780c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887d9a7700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f887806a270 0x7f88780c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887e1a8700 1 -- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8878070d90 msgr2=0x7f88780c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887e1a8700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8878070d90 0x7f88780c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887e1a8700 1 -- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f887806a270 msgr2=0x7f88780c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887e1a8700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f887806a270 0x7f88780c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887e1a8700 1 -- 172.21.15.57:0/467058960 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88680094a0 con 0x7f887815e0b0 2023-11-28T12:07:31.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887d9a7700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f887806a270 0x7f88780c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.470+0000 7f887e1a8700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f887815e0b0 0x7f8878159590 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8870007e90 tx=0x7f887000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.471+0000 7f886e7fc700 1 -- 172.21.15.57:0/467058960 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887000ede0 con 0x7f887815e0b0 2023-11-28T12:07:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.471+0000 7f886e7fc700 1 -- 172.21.15.57:0/467058960 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8870014420 con 0x7f887815e0b0 2023-11-28T12:07:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.471+0000 7f887fc0b700 1 -- 172.21.15.57:0/467058960 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887815aac0 con 0x7f887815e0b0 2023-11-28T12:07:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.471+0000 7f886e7fc700 1 -- 172.21.15.57:0/467058960 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88700147d0 con 0x7f887815e0b0 2023-11-28T12:07:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.471+0000 7f887fc0b700 1 -- 172.21.15.57:0/467058960 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887815b0d0 con 0x7f887815e0b0 2023-11-28T12:07:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.472+0000 7f887fc0b700 1 -- 172.21.15.57:0/467058960 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8878071720 con 0x7f887815e0b0 2023-11-28T12:07:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.472+0000 7f886e7fc700 1 -- 172.21.15.57:0/467058960 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8870012050 con 0x7f887815e0b0 2023-11-28T12:07:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.473+0000 7f886e7fc700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f885c0682f0 0x7f885c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.473+0000 7f886e7fc700 1 -- 172.21.15.57:0/467058960 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f88700872f0 con 0x7f887815e0b0 2023-11-28T12:07:31.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.473+0000 7f887d9a7700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f885c0682f0 0x7f885c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.474+0000 7f887d9a7700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f885c0682f0 0x7f885c06a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8874000f60 tx=0x7f8874009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:31.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.477+0000 7f886e7fc700 1 -- 172.21.15.57:0/467058960 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8870053a60 con 0x7f887815e0b0 2023-11-28T12:07:31.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.729+0000 7f887fc0b700 1 -- 172.21.15.57:0/467058960 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88780743f0 con 0x7f887815e0b0 2023-11-28T12:07:31.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.729+0000 7f886e7fc700 1 -- 172.21.15.57:0/467058960 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4652 (secure 0 0 0) 0x7f8870057080 con 0x7f887815e0b0 2023-11-28T12:07:31.731 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:31.731 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2317025916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2317025916},{"type":"v1","addr":"172.21.15.181:6836","nonce":2317025916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3486085199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3486085199},{"type":"v1","addr":"172.21.15.181:6837","nonce":3486085199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/4272259116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":4272259116},{"type":"v1","addr":"172.21.15.57:6834","nonce":4272259116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:04:52.360416+0000","modified":"2023-11-28T12:05:08.579934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.57:6835/685888843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":685888843},{"type":"v1","addr":"172.21.15.57:6835","nonce":685888843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T12:07:31.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.732+0000 7f887fc0b700 1 -- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f885c0682f0 msgr2=0x7f885c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.732+0000 7f887fc0b700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f885c0682f0 0x7f885c06a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8874000f60 tx=0x7f8874009040 comp rx=0 tx=0).stop 2023-11-28T12:07:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.732+0000 7f887fc0b700 1 -- 172.21.15.57:0/467058960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f887815e0b0 msgr2=0x7f8878159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.732+0000 7f887fc0b700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f887815e0b0 0x7f8878159590 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8870007e90 tx=0x7f887000e3c0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.732+0000 7f887fc0b700 1 -- 172.21.15.57:0/467058960 shutdown_connections 2023-11-28T12:07:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.732+0000 7f887fc0b700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f885c0682f0 0x7f885c06a7a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.732+0000 7f887fc0b700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f887815e0b0 0x7f8878159590 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.732+0000 7f887fc0b700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f887806a270 0x7f88780c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.732+0000 7f887fc0b700 1 --2- 172.21.15.57:0/467058960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8878070d90 0x7f88780c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.732+0000 7f887fc0b700 1 -- 172.21.15.57:0/467058960 >> 172.21.15.57:0/467058960 conn(0x7f887806c010 msgr2=0x7f88780afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:31.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.732+0000 7f887fc0b700 1 -- 172.21.15.57:0/467058960 shutdown_connections 2023-11-28T12:07:31.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.732+0000 7f887fc0b700 1 -- 172.21.15.57:0/467058960 wait complete. 2023-11-28T12:07:31.735 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2023-11-28T12:07:31.746 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:07:31.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.881+0000 7fcfeda7d700 1 Processor -- start 2023-11-28T12:07:31.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.881+0000 7fcfeda7d700 1 -- start start 2023-11-28T12:07:31.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.882+0000 7fcfeda7d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcfe80ce040 0x7fcfe815d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.882+0000 7fcfeda7d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe813fc90 con 0x7fcfe80b1770 2023-11-28T12:07:31.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.882+0000 7fcfeda7d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe813f480 con 0x7fcfe8142db0 2023-11-28T12:07:31.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.882+0000 7fcfeda7d700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe813f6d0 con 0x7fcfe80ce040 2023-11-28T12:07:31.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.882+0000 7fcfe77fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcfe80ce040 0x7fcfe815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.882+0000 7fcfe7fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcfe80b1770 0x7fcfe80b1b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49894/0 (socket says 172.21.15.57:49894) 2023-11-28T12:07:31.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.882+0000 7fcfe7fff700 1 -- 172.21.15.57:0/3295795751 learned_addr learned my addr 172.21.15.57:0/3295795751 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:31.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.882+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3295795751 >> v1:172.21.15.181:6789/0 conn(0x7fcfe8142db0 legacy=0x7fcfe813ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:31.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.882+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3295795751 >> v1:172.21.15.57:6789/0 conn(0x7fcfe80b1770 legacy=0x7fcfe80b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.882+0000 7fcfe67fc700 1 -- 172.21.15.57:0/3295795751 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374800 0 0) 0x7fcfe813fc90 con 0x7fcfe80b1770 2023-11-28T12:07:31.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.882+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3295795751 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe813f9b0 con 0x7fcfe80ce040 2023-11-28T12:07:31.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.883+0000 7fcfe77fe700 1 --2- 172.21.15.57:0/3295795751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfe80ce040 0x7fcfe815d900 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009840 tx=0x7fcfd8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a4b99fb49beacd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:31.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.883+0000 7fcfe67fc700 1 -- 172.21.15.57:0/3295795751 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8011830 con 0x7fcfe80ce040 2023-11-28T12:07:31.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.883+0000 7fcfe67fc700 1 -- 172.21.15.57:0/3295795751 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd8011990 con 0x7fcfe80ce040 2023-11-28T12:07:31.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.883+0000 7fcfe67fc700 1 -- 172.21.15.57:0/3295795751 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8011d20 con 0x7fcfe80ce040 2023-11-28T12:07:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.883+0000 7fcfeda7d700 1 -- 172.21.15.57:0/3295795751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfe80ce040 msgr2=0x7fcfe815d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.883+0000 7fcfeda7d700 1 --2- 172.21.15.57:0/3295795751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfe80ce040 0x7fcfe815d900 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009840 tx=0x7fcfd8009b50 comp rx=0 tx=0).stop 2023-11-28T12:07:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.883+0000 7fcfeda7d700 1 -- 172.21.15.57:0/3295795751 shutdown_connections 2023-11-28T12:07:31.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.883+0000 7fcfeda7d700 1 --2- 172.21.15.57:0/3295795751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfe80ce040 0x7fcfe815d900 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.883+0000 7fcfeda7d700 1 -- 172.21.15.57:0/3295795751 >> 172.21.15.57:0/3295795751 conn(0x7fcfe806c010 msgr2=0x7fcfe80cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:31.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.883+0000 7fcfeda7d700 1 -- 172.21.15.57:0/3295795751 shutdown_connections 2023-11-28T12:07:31.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.883+0000 7fcfeda7d700 1 -- 172.21.15.57:0/3295795751 wait complete. 2023-11-28T12:07:31.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.885+0000 7fcfeda7d700 1 Processor -- start 2023-11-28T12:07:31.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.885+0000 7fcfeda7d700 1 -- start start 2023-11-28T12:07:31.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfeda7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfe80b1770 0x7fcfe80b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfeda7d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfe80ce040 0x7fcfe80cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfeda7d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfe8142db0 0x7fcfe80cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfeda7d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80b3ef0 con 0x7fcfe8142db0 2023-11-28T12:07:31.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfeda7d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80b5990 con 0x7fcfe80b1770 2023-11-28T12:07:31.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfeda7d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80c3390 con 0x7fcfe80ce040 2023-11-28T12:07:31.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfe7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfe8142db0 0x7fcfe80cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfe7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfe8142db0 0x7fcfe80cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39064/0 (socket says 172.21.15.57:39064) 2023-11-28T12:07:31.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfe7fff700 1 -- 172.21.15.57:0/2124498432 learned_addr learned my addr 172.21.15.57:0/2124498432 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:31.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfe77fe700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfe80b1770 0x7fcfe80b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfe6ffd700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfe80ce040 0x7fcfe80cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfe7fff700 1 -- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfe80ce040 msgr2=0x7fcfe80cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfe7fff700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfe80ce040 0x7fcfe80cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfe7fff700 1 -- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfe80b1770 msgr2=0x7fcfe80b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:31.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfe7fff700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfe80b1770 0x7fcfe80b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:31.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfe7fff700 1 -- 172.21.15.57:0/2124498432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80094a0 con 0x7fcfe8142db0 2023-11-28T12:07:31.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfe7fff700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfe8142db0 0x7fcfe80cccb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000cba0 tx=0x7fcfe000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:31.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.886+0000 7fcfdffff700 1 -- 172.21.15.57:0/2124498432 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe000aa70 con 0x7fcfe8142db0 2023-11-28T12:07:31.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.887+0000 7fcfdffff700 1 -- 172.21.15.57:0/2124498432 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe000abf0 con 0x7fcfe8142db0 2023-11-28T12:07:31.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.887+0000 7fcfdffff700 1 -- 172.21.15.57:0/2124498432 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00070d0 con 0x7fcfe8142db0 2023-11-28T12:07:31.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.887+0000 7fcfeda7d700 1 -- 172.21.15.57:0/2124498432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe815d3e0 con 0x7fcfe8142db0 2023-11-28T12:07:31.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.887+0000 7fcfeda7d700 1 -- 172.21.15.57:0/2124498432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe815d100 con 0x7fcfe8142db0 2023-11-28T12:07:31.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.888+0000 7fcfeda7d700 1 -- 172.21.15.57:0/2124498432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe80ce410 con 0x7fcfe8142db0 2023-11-28T12:07:31.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.888+0000 7fcfdffff700 1 -- 172.21.15.57:0/2124498432 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcfe0010070 con 0x7fcfe8142db0 2023-11-28T12:07:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.888+0000 7fcfdffff700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcfc80682f0 0x7fcfc806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.889+0000 7fcfdffff700 1 -- 172.21.15.57:0/2124498432 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fcfe00855c0 con 0x7fcfe8142db0 2023-11-28T12:07:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.889+0000 7fcfe77fe700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcfc80682f0 0x7fcfc806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:31.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.889+0000 7fcfe77fe700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcfc80682f0 0x7fcfc806a7a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80090b0 tx=0x7fcfd801c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:31.893+0000 7fcfdffff700 1 -- 172.21.15.57:0/2124498432 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcfe0051da0 con 0x7fcfe8142db0 2023-11-28T12:07:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.095+0000 7fcfeda7d700 1 -- 172.21.15.57:0/2124498432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfe80ced70 con 0x7fcfe8142db0 2023-11-28T12:07:32.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.096+0000 7fcfdffff700 1 -- 172.21.15.57:0/2124498432 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18056 (secure 0 0 0) 0x7fcfe00553c0 con 0x7fcfe8142db0 2023-11-28T12:07:32.097 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:32.098 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":65,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:05:34.795435+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:04:50.522537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T12:04:51.333855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.32","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:07:32.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.098+0000 7fcfeda7d700 1 -- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcfc80682f0 msgr2=0x7fcfc806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:32.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.098+0000 7fcfeda7d700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcfc80682f0 0x7fcfc806a7a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80090b0 tx=0x7fcfd801c040 comp rx=0 tx=0).stop 2023-11-28T12:07:32.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.098+0000 7fcfeda7d700 1 -- 172.21.15.57:0/2124498432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfe8142db0 msgr2=0x7fcfe80cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:32.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.098+0000 7fcfeda7d700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfe8142db0 0x7fcfe80cccb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000cba0 tx=0x7fcfe000ceb0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.098+0000 7fcfeda7d700 1 -- 172.21.15.57:0/2124498432 shutdown_connections 2023-11-28T12:07:32.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.098+0000 7fcfeda7d700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fcfc80682f0 0x7fcfc806a7a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.098+0000 7fcfeda7d700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcfe8142db0 0x7fcfe80cccb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.098+0000 7fcfeda7d700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfe80b1770 0x7fcfe80b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.098+0000 7fcfeda7d700 1 --2- 172.21.15.57:0/2124498432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfe80ce040 0x7fcfe80cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.098+0000 7fcfeda7d700 1 -- 172.21.15.57:0/2124498432 >> 172.21.15.57:0/2124498432 conn(0x7fcfe806c010 msgr2=0x7fcfe8071870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:32.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.098+0000 7fcfeda7d700 1 -- 172.21.15.57:0/2124498432 shutdown_connections 2023-11-28T12:07:32.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.098+0000 7fcfeda7d700 1 -- 172.21.15.57:0/2124498432 wait complete. 2023-11-28T12:07:32.114 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T12:07:32.115 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:07:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.249+0000 7fb75e6e8700 1 Processor -- start 2023-11-28T12:07:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.250+0000 7fb75e6e8700 1 -- start start 2023-11-28T12:07:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.250+0000 7fb75e6e8700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb75806a270 0x7fb75806a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.250+0000 7fb75e6e8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb758070d90 0x7fb7580711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.250+0000 7fb75e6e8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb75806da40 0x7fb75806dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.250+0000 7fb75e6e8700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb758151a50 con 0x7fb75806a270 2023-11-28T12:07:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.250+0000 7fb75e6e8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb75815b440 con 0x7fb758070d90 2023-11-28T12:07:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.250+0000 7fb75e6e8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75806ab80 con 0x7fb75806da40 2023-11-28T12:07:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb757fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb75806a270 0x7fb75806a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb757fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb75806a270 0x7fb75806a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39080/0 (socket says 172.21.15.57:39080) 2023-11-28T12:07:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb757fff700 1 -- 172.21.15.57:0/1833117622 learned_addr learned my addr 172.21.15.57:0/1833117622 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb75cc85700 1 --2- 172.21.15.57:0/1833117622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb75806da40 0x7fb75806dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb7577fe700 1 --2- 172.21.15.57:0/1833117622 >> v2:172.21.15.181:3300/0 conn(0x7fb758070d90 0x7fb7580711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb757fff700 1 -- 172.21.15.57:0/1833117622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb75806da40 msgr2=0x7fb75806dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb757fff700 1 --2- 172.21.15.57:0/1833117622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb75806da40 0x7fb75806dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb757fff700 1 -- 172.21.15.57:0/1833117622 >> v2:172.21.15.181:3300/0 conn(0x7fb758070d90 msgr2=0x7fb7580711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb757fff700 1 --2- 172.21.15.57:0/1833117622 >> v2:172.21.15.181:3300/0 conn(0x7fb758070d90 0x7fb7580711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb757fff700 1 -- 172.21.15.57:0/1833117622 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb758071ac0 con 0x7fb75806a270 2023-11-28T12:07:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb7577fe700 1 --2- 172.21.15.57:0/1833117622 >> v2:172.21.15.181:3300/0 conn(0x7fb758070d90 0x7fb7580711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb757fff700 1 --2- 172.21.15.57:0/1833117622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb75806a270 0x7fb75806a640 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb7580cd4f0 tx=0x7fb748009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1f74e585ec9aa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb756ffd700 1 -- 172.21.15.57:0/1833117622 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb748011830 con 0x7fb75806a270 2023-11-28T12:07:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb756ffd700 1 -- 172.21.15.57:0/1833117622 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb748011990 con 0x7fb75806a270 2023-11-28T12:07:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.251+0000 7fb756ffd700 1 -- 172.21.15.57:0/1833117622 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb748011d20 con 0x7fb75806a270 2023-11-28T12:07:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.252+0000 7fb75e6e8700 1 -- 172.21.15.57:0/1833117622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb75806a270 msgr2=0x7fb75806a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.252+0000 7fb75e6e8700 1 --2- 172.21.15.57:0/1833117622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb75806a270 0x7fb75806a640 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb7580cd4f0 tx=0x7fb748009bd0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.252+0000 7fb75e6e8700 1 -- 172.21.15.57:0/1833117622 shutdown_connections 2023-11-28T12:07:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.252+0000 7fb75e6e8700 1 --2- 172.21.15.57:0/1833117622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb75806da40 0x7fb75806dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.252+0000 7fb75e6e8700 1 --2- 172.21.15.57:0/1833117622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb75806a270 0x7fb75806a640 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.252+0000 7fb75e6e8700 1 --2- 172.21.15.57:0/1833117622 >> v2:172.21.15.181:3300/0 conn(0x7fb758070d90 0x7fb7580711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.252+0000 7fb75e6e8700 1 -- 172.21.15.57:0/1833117622 >> 172.21.15.57:0/1833117622 conn(0x7fb75806c010 msgr2=0x7fb7580cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.252+0000 7fb75e6e8700 1 -- 172.21.15.57:0/1833117622 shutdown_connections 2023-11-28T12:07:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.252+0000 7fb75e6e8700 1 -- 172.21.15.57:0/1833117622 wait complete. 2023-11-28T12:07:32.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.253+0000 7fb75e6e8700 1 Processor -- start 2023-11-28T12:07:32.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.253+0000 7fb75e6e8700 1 -- start start 2023-11-28T12:07:32.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.254+0000 7fb75e6e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb75806a270 0x7fb758149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.254+0000 7fb75e6e8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb75806da40 0x7fb7580c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.254+0000 7fb75e6e8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb758070d90 0x7fb7580c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.254+0000 7fb75e6e8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7580c6960 con 0x7fb75806da40 2023-11-28T12:07:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.254+0000 7fb75e6e8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7581496d0 con 0x7fb75806a270 2023-11-28T12:07:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.254+0000 7fb75e6e8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7580c60c0 con 0x7fb758070d90 2023-11-28T12:07:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.254+0000 7fb7577fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb75806da40 0x7fb7580c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.254+0000 7fb7577fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb75806da40 0x7fb7580c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39092/0 (socket says 172.21.15.57:39092) 2023-11-28T12:07:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.254+0000 7fb7577fe700 1 -- 172.21.15.57:0/3992324997 learned_addr learned my addr 172.21.15.57:0/3992324997 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.254+0000 7fb75cc85700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb758070d90 0x7fb7580c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb757fff700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb75806a270 0x7fb758149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb7577fe700 1 -- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb758070d90 msgr2=0x7fb7580c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb7577fe700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb758070d90 0x7fb7580c7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb7577fe700 1 -- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb75806a270 msgr2=0x7fb758149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb7577fe700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb75806a270 0x7fb758149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb7577fe700 1 -- 172.21.15.57:0/3992324997 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb748009550 con 0x7fb75806da40 2023-11-28T12:07:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb75cc85700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb758070d90 0x7fb7580c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb757fff700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb75806a270 0x7fb758149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb7577fe700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb75806da40 0x7fb7580c7740 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00e810 tx=0x7fb74c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb754ff9700 1 -- 172.21.15.57:0/3992324997 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c00c020 con 0x7fb75806da40 2023-11-28T12:07:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb754ff9700 1 -- 172.21.15.57:0/3992324997 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74c004250 con 0x7fb75806da40 2023-11-28T12:07:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb75e6e8700 1 -- 172.21.15.57:0/3992324997 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75815b0d0 con 0x7fb75806da40 2023-11-28T12:07:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb754ff9700 1 -- 172.21.15.57:0/3992324997 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0036a0 con 0x7fb75806da40 2023-11-28T12:07:32.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.255+0000 7fb75e6e8700 1 -- 172.21.15.57:0/3992324997 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75815adf0 con 0x7fb75806da40 2023-11-28T12:07:32.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.256+0000 7fb75e6e8700 1 -- 172.21.15.57:0/3992324997 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75815aac0 con 0x7fb75806da40 2023-11-28T12:07:32.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.257+0000 7fb754ff9700 1 -- 172.21.15.57:0/3992324997 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb74c022070 con 0x7fb75806da40 2023-11-28T12:07:32.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.257+0000 7fb754ff9700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7440682f0 0x7fb74406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:32.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.257+0000 7fb754ff9700 1 -- 172.21.15.57:0/3992324997 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fb74c085aa0 con 0x7fb75806da40 2023-11-28T12:07:32.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.257+0000 7fb757fff700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7440682f0 0x7fb74406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:32.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.258+0000 7fb757fff700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7440682f0 0x7fb74406a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb748009c10 tx=0x7fb748008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:32.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.262+0000 7fb754ff9700 1 -- 172.21.15.57:0/3992324997 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb74c052280 con 0x7fb75806da40 2023-11-28T12:07:32.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.509+0000 7fb75e6e8700 1 -- 172.21.15.57:0/3992324997 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7580743f0 con 0x7fb75806da40 2023-11-28T12:07:32.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.509+0000 7fb754ff9700 1 -- 172.21.15.57:0/3992324997 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4652 (secure 0 0 0) 0x7fb74c00f040 con 0x7fb75806da40 2023-11-28T12:07:32.511 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:32.511 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2317025916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2317025916},{"type":"v1","addr":"172.21.15.181:6836","nonce":2317025916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3486085199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3486085199},{"type":"v1","addr":"172.21.15.181:6837","nonce":3486085199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/4272259116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":4272259116},{"type":"v1","addr":"172.21.15.57:6834","nonce":4272259116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:04:52.360416+0000","modified":"2023-11-28T12:05:08.579934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.57:6835/685888843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":685888843},{"type":"v1","addr":"172.21.15.57:6835","nonce":685888843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T12:07:32.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.511+0000 7fb75e6e8700 1 -- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7440682f0 msgr2=0x7fb74406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:32.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.511+0000 7fb75e6e8700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7440682f0 0x7fb74406a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb748009c10 tx=0x7fb748008030 comp rx=0 tx=0).stop 2023-11-28T12:07:32.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.511+0000 7fb75e6e8700 1 -- 172.21.15.57:0/3992324997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb75806da40 msgr2=0x7fb7580c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:32.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.511+0000 7fb75e6e8700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb75806da40 0x7fb7580c7740 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00e810 tx=0x7fb74c00ebd0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.511+0000 7fb75e6e8700 1 -- 172.21.15.57:0/3992324997 shutdown_connections 2023-11-28T12:07:32.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.511+0000 7fb75e6e8700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb7440682f0 0x7fb74406a7a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.511+0000 7fb75e6e8700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb758070d90 0x7fb7580c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.511+0000 7fb75e6e8700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb75806a270 0x7fb758149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.511+0000 7fb75e6e8700 1 --2- 172.21.15.57:0/3992324997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb75806da40 0x7fb7580c7740 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.511+0000 7fb75e6e8700 1 -- 172.21.15.57:0/3992324997 >> 172.21.15.57:0/3992324997 conn(0x7fb75806c010 msgr2=0x7fb75815d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:32.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.511+0000 7fb75e6e8700 1 -- 172.21.15.57:0/3992324997 shutdown_connections 2023-11-28T12:07:32.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.511+0000 7fb75e6e8700 1 -- 172.21.15.57:0/3992324997 wait complete. 2023-11-28T12:07:32.514 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2023-11-28T12:07:32.526 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:07:32.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.661+0000 7f33e9c91700 1 Processor -- start 2023-11-28T12:07:32.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.661+0000 7f33e9c91700 1 -- start start 2023-11-28T12:07:32.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.662+0000 7f33e9c91700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40ce040 0x7f33e415d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:32.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.662+0000 7f33e9c91700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f33e40b1770 0x7f33e40b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:32.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.662+0000 7f33e9c91700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e413fc90 con 0x7f33e4142db0 2023-11-28T12:07:32.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.662+0000 7f33e9c91700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e40b2130 con 0x7f33e40b1770 2023-11-28T12:07:32.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.662+0000 7f33e9c91700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e413f480 con 0x7f33e40ce040 2023-11-28T12:07:32.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.662+0000 7f33e3fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f33e40b1770 0x7f33e40b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:32.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.662+0000 7f33e3fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f33e40b1770 0x7f33e40b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39102/0 (socket says 172.21.15.57:39102) 2023-11-28T12:07:32.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.662+0000 7f33e3fff700 1 -- 172.21.15.57:0/1176598137 learned_addr learned my addr 172.21.15.57:0/1176598137 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:32.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.662+0000 7f33e37fe700 1 --2- 172.21.15.57:0/1176598137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40ce040 0x7f33e415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:32.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.663+0000 7f33e3fff700 1 -- 172.21.15.57:0/1176598137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40ce040 msgr2=0x7f33e415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:32.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.663+0000 7f33e3fff700 1 --2- 172.21.15.57:0/1176598137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40ce040 0x7f33e415d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.663+0000 7f33e3fff700 1 -- 172.21.15.57:0/1176598137 >> v1:172.21.15.181:6789/0 conn(0x7f33e4142db0 legacy=0x7f33e413ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:32.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.663+0000 7f33e3fff700 1 -- 172.21.15.57:0/1176598137 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e413f6d0 con 0x7f33e40b1770 2023-11-28T12:07:32.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.663+0000 7f33e37fe700 1 --2- 172.21.15.57:0/1176598137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40ce040 0x7f33e415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:32.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.663+0000 7f33e3fff700 1 --2- 172.21.15.57:0/1176598137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33e40b1770 0x7f33e40b1bf0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009e50 tx=0x7f33dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44329acf9c431e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:32.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.663+0000 7f33e27fc700 1 -- 172.21.15.57:0/1176598137 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc00e070 con 0x7f33e40b1770 2023-11-28T12:07:32.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.663+0000 7f33e27fc700 1 -- 172.21.15.57:0/1176598137 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc003a20 con 0x7f33e40b1770 2023-11-28T12:07:32.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.663+0000 7f33e27fc700 1 -- 172.21.15.57:0/1176598137 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc003dd0 con 0x7f33e40b1770 2023-11-28T12:07:32.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.663+0000 7f33e9c91700 1 -- 172.21.15.57:0/1176598137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33e40b1770 msgr2=0x7f33e40b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:32.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.663+0000 7f33e9c91700 1 --2- 172.21.15.57:0/1176598137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33e40b1770 0x7f33e40b1bf0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009e50 tx=0x7f33dc00b330 comp rx=0 tx=0).stop 2023-11-28T12:07:32.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.664+0000 7f33e9c91700 1 -- 172.21.15.57:0/1176598137 shutdown_connections 2023-11-28T12:07:32.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.664+0000 7f33e9c91700 1 --2- 172.21.15.57:0/1176598137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33e40b1770 0x7f33e40b1bf0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.664+0000 7f33e9c91700 1 --2- 172.21.15.57:0/1176598137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40ce040 0x7f33e415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.664+0000 7f33e9c91700 1 -- 172.21.15.57:0/1176598137 >> 172.21.15.57:0/1176598137 conn(0x7f33e406c010 msgr2=0x7f33e40cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:32.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.664+0000 7f33e9c91700 1 -- 172.21.15.57:0/1176598137 shutdown_connections 2023-11-28T12:07:32.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.664+0000 7f33e9c91700 1 -- 172.21.15.57:0/1176598137 wait complete. 2023-11-28T12:07:32.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.665+0000 7f33e9c91700 1 Processor -- start 2023-11-28T12:07:32.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.665+0000 7f33e9c91700 1 -- start start 2023-11-28T12:07:32.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e9c91700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40b1770 0x7f33e40c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:32.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e9c91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e40ce040 0x7f33e41faf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:32.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e9c91700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33e4142db0 0x7f33e4140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:32.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e9c91700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40c62f0 con 0x7f33e4142db0 2023-11-28T12:07:32.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e9c91700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40c6010 con 0x7f33e40ce040 2023-11-28T12:07:32.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e9c91700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40b2cc0 con 0x7f33e40b1770 2023-11-28T12:07:32.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33e4142db0 0x7f33e4140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:32.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33e4142db0 0x7f33e4140a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39112/0 (socket says 172.21.15.57:39112) 2023-11-28T12:07:32.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e3fff700 1 -- 172.21.15.57:0/242072077 learned_addr learned my addr 172.21.15.57:0/242072077 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:32.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e37fe700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40b1770 0x7f33e40c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:32.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e2ffd700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e40ce040 0x7f33e41faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:32.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e3fff700 1 -- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40b1770 msgr2=0x7f33e40c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:32.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e3fff700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40b1770 0x7f33e40c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e3fff700 1 -- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e40ce040 msgr2=0x7f33e41faf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:32.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e3fff700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e40ce040 0x7f33e41faf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e3fff700 1 -- 172.21.15.57:0/242072077 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc009b30 con 0x7f33e4142db0 2023-11-28T12:07:32.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.666+0000 7f33e2ffd700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e40ce040 0x7f33e41faf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:32.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.667+0000 7f33e3fff700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33e4142db0 0x7f33e4140a50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f33e4070850 tx=0x7f33dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:32.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.667+0000 7f33dbfff700 1 -- 172.21.15.57:0/242072077 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc004440 con 0x7f33e4142db0 2023-11-28T12:07:32.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.667+0000 7f33dbfff700 1 -- 172.21.15.57:0/242072077 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc0045a0 con 0x7f33e4142db0 2023-11-28T12:07:32.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.667+0000 7f33e9c91700 1 -- 172.21.15.57:0/242072077 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e415d100 con 0x7f33e4142db0 2023-11-28T12:07:32.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.667+0000 7f33dbfff700 1 -- 172.21.15.57:0/242072077 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc004950 con 0x7f33e4142db0 2023-11-28T12:07:32.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.667+0000 7f33e9c91700 1 -- 172.21.15.57:0/242072077 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e413f9b0 con 0x7f33e4142db0 2023-11-28T12:07:32.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.668+0000 7f33e9c91700 1 -- 172.21.15.57:0/242072077 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e415d3e0 con 0x7f33e4142db0 2023-11-28T12:07:32.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.669+0000 7f33dbfff700 1 -- 172.21.15.57:0/242072077 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f33dc02c070 con 0x7f33e4142db0 2023-11-28T12:07:32.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.670+0000 7f33dbfff700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f33c40682f0 0x7f33c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:32.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.670+0000 7f33dbfff700 1 -- 172.21.15.57:0/242072077 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f33dc087b00 con 0x7f33e4142db0 2023-11-28T12:07:32.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.670+0000 7f33e37fe700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f33c40682f0 0x7f33c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:32.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.670+0000 7f33e37fe700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f33c40682f0 0x7f33c406a7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f33cc006fd0 tx=0x7f33cc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:32.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.673+0000 7f33dbfff700 1 -- 172.21.15.57:0/242072077 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f33dc0542e0 con 0x7f33e4142db0 2023-11-28T12:07:32.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.917+0000 7f33e9c91700 1 -- 172.21.15.57:0/242072077 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33e40b2b20 con 0x7f33e4142db0 2023-11-28T12:07:32.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.917+0000 7f33dbfff700 1 -- 172.21.15.57:0/242072077 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4652 (secure 0 0 0) 0x7f33e40b2b20 con 0x7f33e4142db0 2023-11-28T12:07:32.919 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:32.919 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2317025916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2317025916},{"type":"v1","addr":"172.21.15.181:6836","nonce":2317025916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3486085199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3486085199},{"type":"v1","addr":"172.21.15.181:6837","nonce":3486085199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/4272259116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":4272259116},{"type":"v1","addr":"172.21.15.57:6834","nonce":4272259116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T12:04:52.360416+0000","modified":"2023-11-28T12:05:08.579934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.57:6835/685888843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":685888843},{"type":"v1","addr":"172.21.15.57:6835","nonce":685888843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T12:07:32.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.920+0000 7f33e9c91700 1 -- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f33c40682f0 msgr2=0x7f33c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:32.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.920+0000 7f33e9c91700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f33c40682f0 0x7f33c406a7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f33cc006fd0 tx=0x7f33cc008040 comp rx=0 tx=0).stop 2023-11-28T12:07:32.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.920+0000 7f33e9c91700 1 -- 172.21.15.57:0/242072077 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33e4142db0 msgr2=0x7f33e4140a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:32.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.920+0000 7f33e9c91700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33e4142db0 0x7f33e4140a50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f33e4070850 tx=0x7f33dc004ad0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.920+0000 7f33e9c91700 1 -- 172.21.15.57:0/242072077 shutdown_connections 2023-11-28T12:07:32.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.920+0000 7f33e9c91700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f33c40682f0 0x7f33c406a7a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.920+0000 7f33e9c91700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33e4142db0 0x7f33e4140a50 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.920+0000 7f33e9c91700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e40b1770 0x7f33e40c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.920+0000 7f33e9c91700 1 --2- 172.21.15.57:0/242072077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e40ce040 0x7f33e41faf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:32.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.920+0000 7f33e9c91700 1 -- 172.21.15.57:0/242072077 >> 172.21.15.57:0/242072077 conn(0x7f33e406c010 msgr2=0x7f33e413ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:32.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.920+0000 7f33e9c91700 1 -- 172.21.15.57:0/242072077 shutdown_connections 2023-11-28T12:07:32.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:32.920+0000 7f33e9c91700 1 -- 172.21.15.57:0/242072077 wait complete. 2023-11-28T12:07:32.922 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2023-11-28T12:07:32.935 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:07:33.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.069+0000 7f7ad940d700 1 Processor -- start 2023-11-28T12:07:33.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.069+0000 7f7ad940d700 1 -- start start 2023-11-28T12:07:33.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.070+0000 7f7ad940d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7ad40cd5a0 0x7f7ad40cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:33.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.070+0000 7f7ad940d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad40b22a0 0x7f7ad40cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:33.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.070+0000 7f7ad940d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad40d1ee0 con 0x7f7ad406ab10 2023-11-28T12:07:33.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.070+0000 7f7ad940d700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad40cdb10 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.070+0000 7f7ad940d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40d1890 con 0x7f7ad40b22a0 2023-11-28T12:07:33.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.070+0000 7f7ad27fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7ad40cd5a0 0x7f7ad40cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:33.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.070+0000 7f7ad2ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f7ad406ab10 0x7f7ad406aee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49908/0 (socket says 172.21.15.57:49908) 2023-11-28T12:07:33.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.070+0000 7f7ad2ffd700 1 -- 172.21.15.57:0/4218669784 learned_addr learned my addr 172.21.15.57:0/4218669784 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:33.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.070+0000 7f7ad37fe700 1 --2- 172.21.15.57:0/4218669784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad40b22a0 0x7f7ad40cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:33.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.071+0000 7f7ad27fc700 1 -- 172.21.15.57:0/4218669784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad40b22a0 msgr2=0x7f7ad40cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:33.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.071+0000 7f7ad27fc700 1 --2- 172.21.15.57:0/4218669784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad40b22a0 0x7f7ad40cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:33.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.071+0000 7f7ad27fc700 1 -- 172.21.15.57:0/4218669784 >> v1:172.21.15.57:6789/0 conn(0x7f7ad406ab10 legacy=0x7f7ad406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:33.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.071+0000 7f7ad27fc700 1 -- 172.21.15.57:0/4218669784 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad40d1b70 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.071+0000 7f7ad1ffb700 1 -- 172.21.15.57:0/4218669784 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899776579 0 0) 0x7f7ad40d1ee0 con 0x7f7ad406ab10 2023-11-28T12:07:33.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.071+0000 7f7ad37fe700 1 --2- 172.21.15.57:0/4218669784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad40b22a0 0x7f7ad40cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:33.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.071+0000 7f7ad27fc700 1 --2- 172.21.15.57:0/4218669784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ad40cd5a0 0x7f7ad40cf440 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009840 tx=0x7f7ac8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685feffcc38b47cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:33.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.071+0000 7f7ad1ffb700 1 -- 172.21.15.57:0/4218669784 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8011830 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.071+0000 7f7ad1ffb700 1 -- 172.21.15.57:0/4218669784 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac8011990 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.071+0000 7f7ad1ffb700 1 -- 172.21.15.57:0/4218669784 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8011d20 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.072+0000 7f7ad940d700 1 -- 172.21.15.57:0/4218669784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ad40cd5a0 msgr2=0x7f7ad40cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:33.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.072+0000 7f7ad940d700 1 --2- 172.21.15.57:0/4218669784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ad40cd5a0 0x7f7ad40cf440 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009840 tx=0x7f7ac8009b50 comp rx=0 tx=0).stop 2023-11-28T12:07:33.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.072+0000 7f7ad940d700 1 -- 172.21.15.57:0/4218669784 shutdown_connections 2023-11-28T12:07:33.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.072+0000 7f7ad940d700 1 --2- 172.21.15.57:0/4218669784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad40b22a0 0x7f7ad40cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:33.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.072+0000 7f7ad940d700 1 --2- 172.21.15.57:0/4218669784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ad40cd5a0 0x7f7ad40cf440 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:33.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.072+0000 7f7ad940d700 1 -- 172.21.15.57:0/4218669784 >> 172.21.15.57:0/4218669784 conn(0x7f7ad406c520 msgr2=0x7f7ad406c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:33.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.072+0000 7f7ad940d700 1 -- 172.21.15.57:0/4218669784 shutdown_connections 2023-11-28T12:07:33.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.072+0000 7f7ad940d700 1 -- 172.21.15.57:0/4218669784 wait complete. 2023-11-28T12:07:33.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.073+0000 7f7ad940d700 1 Processor -- start 2023-11-28T12:07:33.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.074+0000 7f7ad940d700 1 -- start start 2023-11-28T12:07:33.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.074+0000 7f7ad940d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad406ab10 0x7f7ad41e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:33.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad940d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad40b22a0 0x7f7ad40cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:33.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad940d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ad40cd5a0 0x7f7ad41e75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad940d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad41e1b50 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad940d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad415a970 con 0x7f7ad40b22a0 2023-11-28T12:07:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad940d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40cada0 con 0x7f7ad406ab10 2023-11-28T12:07:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ad40cd5a0 0x7f7ad41e75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ad40cd5a0 0x7f7ad41e75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39140/0 (socket says 172.21.15.57:39140) 2023-11-28T12:07:33.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad37fe700 1 -- 172.21.15.57:0/2757553863 learned_addr learned my addr 172.21.15.57:0/2757553863 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:33.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad2ffd700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad406ab10 0x7f7ad41e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:33.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad27fc700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad40b22a0 0x7f7ad40cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:33.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad37fe700 1 -- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad406ab10 msgr2=0x7f7ad41e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:33.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad37fe700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad406ab10 0x7f7ad41e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:33.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad37fe700 1 -- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad40b22a0 msgr2=0x7f7ad40cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:33.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad37fe700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad40b22a0 0x7f7ad40cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:33.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad37fe700 1 -- 172.21.15.57:0/2757553863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80094a0 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad27fc700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad40b22a0 0x7f7ad40cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:33.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad37fe700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ad40cd5a0 0x7f7ad41e75d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009fa0 tx=0x7f7acc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:33.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7abb7fe700 1 -- 172.21.15.57:0/2757553863 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc00de60 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7abb7fe700 1 -- 172.21.15.57:0/2757553863 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc00f040 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad940d700 1 -- 172.21.15.57:0/2757553863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad40b1ec0 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7abb7fe700 1 -- 172.21.15.57:0/2757553863 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc003c80 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.075+0000 7f7ad940d700 1 -- 172.21.15.57:0/2757553863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad415cad0 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.077+0000 7f7ad940d700 1 -- 172.21.15.57:0/2757553863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad40b2050 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.077+0000 7f7abb7fe700 1 -- 172.21.15.57:0/2757553863 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7acc01c030 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.077+0000 7f7abb7fe700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7ac00682f0 0x7f7ac006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.078+0000 7f7abb7fe700 1 -- 172.21.15.57:0/2757553863 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f7acc085d00 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.078+0000 7f7ad2ffd700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7ac00682f0 0x7f7ac006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.078+0000 7f7ad2ffd700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7ac00682f0 0x7f7ac006a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4000f60 tx=0x7f7ac4007e70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:33.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.082+0000 7f7abb7fe700 1 -- 172.21.15.57:0/2757553863 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7acc089030 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.280+0000 7f7ad940d700 1 -- 172.21.15.57:0/2757553863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ad41f27c0 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.282+0000 7f7abb7fe700 1 -- 172.21.15.57:0/2757553863 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18056 (secure 0 0 0) 0x7f7acc052480 con 0x7f7ad40cd5a0 2023-11-28T12:07:33.284 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:33.284 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":65,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:05:34.795435+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T12:04:50.522537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T12:04:51.333855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.32","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:07:33.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.284+0000 7f7ad940d700 1 -- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7ac00682f0 msgr2=0x7f7ac006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:33.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.284+0000 7f7ad940d700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7ac00682f0 0x7f7ac006a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4000f60 tx=0x7f7ac4007e70 comp rx=0 tx=0).stop 2023-11-28T12:07:33.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.284+0000 7f7ad940d700 1 -- 172.21.15.57:0/2757553863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ad40cd5a0 msgr2=0x7f7ad41e75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:33.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.284+0000 7f7ad940d700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ad40cd5a0 0x7f7ad41e75d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009fa0 tx=0x7f7acc00d3b0 comp rx=0 tx=0).stop 2023-11-28T12:07:33.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.284+0000 7f7ad940d700 1 -- 172.21.15.57:0/2757553863 shutdown_connections 2023-11-28T12:07:33.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.284+0000 7f7ad940d700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7ac00682f0 0x7f7ac006a7a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:33.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.284+0000 7f7ad940d700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ad40cd5a0 0x7f7ad41e75d0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:33.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.284+0000 7f7ad940d700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad406ab10 0x7f7ad41e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:33.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.284+0000 7f7ad940d700 1 --2- 172.21.15.57:0/2757553863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad40b22a0 0x7f7ad40cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:33.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.284+0000 7f7ad940d700 1 -- 172.21.15.57:0/2757553863 >> 172.21.15.57:0/2757553863 conn(0x7f7ad406c520 msgr2=0x7f7ad40cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:33.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.284+0000 7f7ad940d700 1 -- 172.21.15.57:0/2757553863 shutdown_connections 2023-11-28T12:07:33.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.284+0000 7f7ad940d700 1 -- 172.21.15.57:0/2757553863 wait complete. 2023-11-28T12:07:33.299 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T12:07:33.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.434+0000 7f4f04f46700 1 Processor -- start 2023-11-28T12:07:33.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.434+0000 7f4f04f46700 1 -- start start 2023-11-28T12:07:33.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.435+0000 7f4f04f46700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4f0006d170 0x7f4f000b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:33.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.435+0000 7f4f04f46700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f00161370 con 0x7f4f0015fe50 2023-11-28T12:07:33.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.435+0000 7f4f04f46700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f000cf700 con 0x7f4f000b27d0 2023-11-28T12:07:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.435+0000 7f4f04f46700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f0006d5c0 con 0x7f4f0006d170 2023-11-28T12:07:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.435+0000 7f4efdd9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4f0006d170 0x7f4f000b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.435+0000 7f4efe59c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4f0015fe50 0x7f4f00160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49918/0 (socket says 172.21.15.57:49918) 2023-11-28T12:07:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.435+0000 7f4efe59c700 1 -- 172.21.15.57:0/1730360158 learned_addr learned my addr 172.21.15.57:0/1730360158 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.436+0000 7f4efd59a700 1 -- 172.21.15.57:0/1730360158 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969187607 0 0) 0x7f4f00161370 con 0x7f4f0015fe50 2023-11-28T12:07:33.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.436+0000 7f4efd59a700 1 -- 172.21.15.57:0/1730360158 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eec0035b0 con 0x7f4f0015fe50 2023-11-28T12:07:33.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.436+0000 7f4efdd9b700 1 -- 172.21.15.57:0/1730360158 >> v1:172.21.15.181:6789/0 conn(0x7f4f000b27d0 legacy=0x7f4f000cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:33.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.436+0000 7f4efdd9b700 1 -- 172.21.15.57:0/1730360158 >> v1:172.21.15.57:6789/0 conn(0x7f4f0015fe50 legacy=0x7f4f00160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:33.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.436+0000 7f4efdd9b700 1 -- 172.21.15.57:0/1730360158 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f000cedd0 con 0x7f4f0006d170 2023-11-28T12:07:33.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.436+0000 7f4efdd9b700 1 --2- 172.21.15.57:0/1730360158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f0006d170 0x7f4f000b3b80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009840 tx=0x7f4ee8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3d20bbfd6e249 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:33.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.436+0000 7f4efd59a700 1 -- 172.21.15.57:0/1730360158 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8011830 con 0x7f4f0006d170 2023-11-28T12:07:33.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.436+0000 7f4efd59a700 1 -- 172.21.15.57:0/1730360158 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee8011990 con 0x7f4f0006d170 2023-11-28T12:07:33.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.436+0000 7f4efd59a700 1 -- 172.21.15.57:0/1730360158 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8011d00 con 0x7f4f0006d170 2023-11-28T12:07:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.437+0000 7f4f04f46700 1 -- 172.21.15.57:0/1730360158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f0006d170 msgr2=0x7f4f000b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.437+0000 7f4f04f46700 1 --2- 172.21.15.57:0/1730360158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f0006d170 0x7f4f000b3b80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009840 tx=0x7f4ee8009b50 comp rx=0 tx=0).stop 2023-11-28T12:07:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.437+0000 7f4f04f46700 1 -- 172.21.15.57:0/1730360158 shutdown_connections 2023-11-28T12:07:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.437+0000 7f4f04f46700 1 --2- 172.21.15.57:0/1730360158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f0006d170 0x7f4f000b3b80 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.437+0000 7f4f04f46700 1 -- 172.21.15.57:0/1730360158 >> 172.21.15.57:0/1730360158 conn(0x7f4f0006ea80 msgr2=0x7f4f0006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.437+0000 7f4f04f46700 1 -- 172.21.15.57:0/1730360158 shutdown_connections 2023-11-28T12:07:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.437+0000 7f4f04f46700 1 -- 172.21.15.57:0/1730360158 wait complete. 2023-11-28T12:07:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.438+0000 7f4f04f46700 1 Processor -- start 2023-11-28T12:07:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.438+0000 7f4f04f46700 1 -- start start 2023-11-28T12:07:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4f04f46700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f0006d170 0x7f4f000ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4f04f46700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f000b27d0 0x7f4f0015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4f04f46700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f0015fe50 0x7f4f0015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:33.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4f04f46700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0015b830 con 0x7f4f0015fe50 2023-11-28T12:07:33.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4f04f46700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000c8c50 con 0x7f4f000b27d0 2023-11-28T12:07:33.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4f04f46700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000c8f30 con 0x7f4f0006d170 2023-11-28T12:07:33.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4efed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f0015fe50 0x7f4f0015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:33.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4efed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f0015fe50 0x7f4f0015b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39160/0 (socket says 172.21.15.57:39160) 2023-11-28T12:07:33.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4efed9d700 1 -- 172.21.15.57:0/3027525352 learned_addr learned my addr 172.21.15.57:0/3027525352 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:33.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4efe59c700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f0006d170 0x7f4f000ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4efdd9b700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f000b27d0 0x7f4f0015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4efed9d700 1 -- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f0006d170 msgr2=0x7f4f000ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4efed9d700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f0006d170 0x7f4f000ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4efed9d700 1 -- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f000b27d0 msgr2=0x7f4f0015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4efed9d700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f000b27d0 0x7f4f0015adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.439+0000 7f4efed9d700 1 -- 172.21.15.57:0/3027525352 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee80094a0 con 0x7f4f0015fe50 2023-11-28T12:07:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.440+0000 7f4efed9d700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f0015fe50 0x7f4f0015b2f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800cc50 tx=0x7f4ef800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.440+0000 7f4ef6ffd700 1 -- 172.21.15.57:0/3027525352 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8003c00 con 0x7f4f0015fe50 2023-11-28T12:07:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.440+0000 7f4ef6ffd700 1 -- 172.21.15.57:0/3027525352 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef8003d60 con 0x7f4f0015fe50 2023-11-28T12:07:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.440+0000 7f4f04f46700 1 -- 172.21.15.57:0/3027525352 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f000cf390 con 0x7f4f0015fe50 2023-11-28T12:07:33.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.440+0000 7f4ef6ffd700 1 -- 172.21.15.57:0/3027525352 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800aab0 con 0x7f4f0015fe50 2023-11-28T12:07:33.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.440+0000 7f4f04f46700 1 -- 172.21.15.57:0/3027525352 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f000cf0b0 con 0x7f4f0015fe50 2023-11-28T12:07:33.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.441+0000 7f4f04f46700 1 -- 172.21.15.57:0/3027525352 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f000b2c30 con 0x7f4f0015fe50 2023-11-28T12:07:33.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.441+0000 7f4ef6ffd700 1 -- 172.21.15.57:0/3027525352 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4ef8010050 con 0x7f4f0015fe50 2023-11-28T12:07:33.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.442+0000 7f4ef6ffd700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4ee00682f0 0x7f4ee006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:33.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.442+0000 7f4ef6ffd700 1 -- 172.21.15.57:0/3027525352 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f4ef8085b50 con 0x7f4f0015fe50 2023-11-28T12:07:33.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.442+0000 7f4efe59c700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4ee00682f0 0x7f4ee006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:33.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.443+0000 7f4efe59c700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4ee00682f0 0x7f4ee006a7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0007fd0 tx=0x7f4ef0007dc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:33.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.446+0000 7f4ef6ffd700 1 -- 172.21.15.57:0/3027525352 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4ef8052330 con 0x7f4f0015fe50 2023-11-28T12:07:33.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:33.692+0000 7f4f04f46700 1 -- 172.21.15.57:0/3027525352 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4f000743f0 con 0x7f4f0015fe50 2023-11-28T12:07:34.651 INFO:tasks.ceph.mon.a.smithi057.stderr:2023-11-28T12:07:34.650+0000 7fb777740700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T12:07:34.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.662+0000 7f4ef6ffd700 1 -- 172.21.15.57:0/3027525352 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4ef8055950 con 0x7f4f0015fe50 2023-11-28T12:07:34.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.665+0000 7f4f04f46700 1 -- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4ee00682f0 msgr2=0x7f4ee006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.665+0000 7f4f04f46700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4ee00682f0 0x7f4ee006a7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0007fd0 tx=0x7f4ef0007dc0 comp rx=0 tx=0).stop 2023-11-28T12:07:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.665+0000 7f4f04f46700 1 -- 172.21.15.57:0/3027525352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f0015fe50 msgr2=0x7f4f0015b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.665+0000 7f4f04f46700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f0015fe50 0x7f4f0015b2f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800cc50 tx=0x7f4ef800cf60 comp rx=0 tx=0).stop 2023-11-28T12:07:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.665+0000 7f4f04f46700 1 -- 172.21.15.57:0/3027525352 shutdown_connections 2023-11-28T12:07:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.665+0000 7f4f04f46700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4ee00682f0 0x7f4ee006a7a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.665+0000 7f4f04f46700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f0015fe50 0x7f4f0015b2f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.665+0000 7f4f04f46700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f0006d170 0x7f4f000ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.665+0000 7f4f04f46700 1 --2- 172.21.15.57:0/3027525352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f000b27d0 0x7f4f0015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.665+0000 7f4f04f46700 1 -- 172.21.15.57:0/3027525352 >> 172.21.15.57:0/3027525352 conn(0x7f4f0006ea80 msgr2=0x7f4f00160a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:34.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.665+0000 7f4f04f46700 1 -- 172.21.15.57:0/3027525352 shutdown_connections 2023-11-28T12:07:34.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.665+0000 7f4f04f46700 1 -- 172.21.15.57:0/3027525352 wait complete. 2023-11-28T12:07:34.668 INFO:teuthology.orchestra.run.smithi057.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T12:07:34.669 INFO:tasks.ceph.mds.a.smithi057.stderr: -8571> 2023-11-28T12:04:57.891+0000 7f92e9fc6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T12:07:34.671 INFO:tasks.ceph.mds.a.smithi057.stderr: -6957> 2023-11-28T12:05:06.549+0000 7f92e67bf700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6396=0-4194304@1} caps={6396=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6396 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5596a631fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T12:07:34.682 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T12:07:34.707 INFO:tasks.ceph.mds.a.smithi057.stdout:starting mds.a at 2023-11-28T12:07:34.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.823+0000 7f7b41172700 1 Processor -- start 2023-11-28T12:07:34.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.823+0000 7f7b41172700 1 -- start start 2023-11-28T12:07:34.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.824+0000 7f7b41172700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b3c0cf710 0x7f7b3c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:34.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.824+0000 7f7b41172700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7b3c071890 0x7f7b3c071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:34.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.824+0000 7f7b41172700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3c15cc10 con 0x7f7b3c06a270 2023-11-28T12:07:34.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.824+0000 7f7b41172700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b3c06c9b0 con 0x7f7b3c071890 2023-11-28T12:07:34.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.824+0000 7f7b41172700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c06cc90 con 0x7f7b3c0cf710 2023-11-28T12:07:34.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.824+0000 7f7b3a59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7b3c071890 0x7f7b3c071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:34.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.824+0000 7f7b3b59e700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f7b3c06a270 0x7f7b3c0b1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47268/0 (socket says 172.21.15.57:47268) 2023-11-28T12:07:34.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.824+0000 7f7b3b59e700 1 -- 172.21.15.57:0/2964668505 learned_addr learned my addr 172.21.15.57:0/2964668505 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:34.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.824+0000 7f7b3ad9d700 1 --2- 172.21.15.57:0/2964668505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b3c0cf710 0x7f7b3c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:34.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.825+0000 7f7b3a59c700 1 -- 172.21.15.57:0/2964668505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b3c0cf710 msgr2=0x7f7b3c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:34.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.825+0000 7f7b3a59c700 1 --2- 172.21.15.57:0/2964668505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b3c0cf710 0x7f7b3c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:34.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.825+0000 7f7b3a59c700 1 -- 172.21.15.57:0/2964668505 >> v1:172.21.15.57:6789/0 conn(0x7f7b3c06a270 legacy=0x7f7b3c0b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:34.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.825+0000 7f7b3a59c700 1 -- 172.21.15.57:0/2964668505 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3c06cf70 con 0x7f7b3c071890 2023-11-28T12:07:34.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.825+0000 7f7b39d9b700 1 -- 172.21.15.57:0/2964668505 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560581024 0 0) 0x7f7b3c15cc10 con 0x7f7b3c06a270 2023-11-28T12:07:34.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.825+0000 7f7b3a59c700 1 --2- 172.21.15.57:0/2964668505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c071890 0x7f7b3c071220 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009840 tx=0x7f7b30009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed26f2a37b6f9328 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:34.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.825+0000 7f7b39d9b700 1 -- 172.21.15.57:0/2964668505 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30011830 con 0x7f7b3c071890 2023-11-28T12:07:34.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.825+0000 7f7b39d9b700 1 -- 172.21.15.57:0/2964668505 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b30011990 con 0x7f7b3c071890 2023-11-28T12:07:34.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.825+0000 7f7b39d9b700 1 -- 172.21.15.57:0/2964668505 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30011d20 con 0x7f7b3c071890 2023-11-28T12:07:34.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.826+0000 7f7b41172700 1 -- 172.21.15.57:0/2964668505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c071890 msgr2=0x7f7b3c071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:34.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.826+0000 7f7b41172700 1 --2- 172.21.15.57:0/2964668505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c071890 0x7f7b3c071220 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009840 tx=0x7f7b30009b50 comp rx=0 tx=0).stop 2023-11-28T12:07:34.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.826+0000 7f7b41172700 1 -- 172.21.15.57:0/2964668505 shutdown_connections 2023-11-28T12:07:34.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.826+0000 7f7b41172700 1 --2- 172.21.15.57:0/2964668505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b3c0cf710 0x7f7b3c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:34.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.826+0000 7f7b41172700 1 --2- 172.21.15.57:0/2964668505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c071890 0x7f7b3c071220 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:34.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.826+0000 7f7b41172700 1 -- 172.21.15.57:0/2964668505 >> 172.21.15.57:0/2964668505 conn(0x7f7b3c06c010 msgr2=0x7f7b3c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:34.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.826+0000 7f7b41172700 1 -- 172.21.15.57:0/2964668505 shutdown_connections 2023-11-28T12:07:34.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.826+0000 7f7b41172700 1 -- 172.21.15.57:0/2964668505 wait complete. 2023-11-28T12:07:34.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.828+0000 7f7b41172700 1 Processor -- start 2023-11-28T12:07:34.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.828+0000 7f7b41172700 1 -- start start 2023-11-28T12:07:34.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b41172700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b3c06a270 0x7f7b3c155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:34.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b41172700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b3c071890 0x7f7b3c0b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b41172700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c0cf710 0x7f7b3c15a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b41172700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c155890 con 0x7f7b3c0cf710 2023-11-28T12:07:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b41172700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c0b4a30 con 0x7f7b3c071890 2023-11-28T12:07:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b41172700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c0b4d10 con 0x7f7b3c06a270 2023-11-28T12:07:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b3b59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c0cf710 0x7f7b3c15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b3b59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c0cf710 0x7f7b3c15a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59012/0 (socket says 172.21.15.57:59012) 2023-11-28T12:07:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b3b59e700 1 -- 172.21.15.57:0/2933332227 learned_addr learned my addr 172.21.15.57:0/2933332227 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b3a59c700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b3c071890 0x7f7b3c0b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b3ad9d700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b3c06a270 0x7f7b3c155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b3b59e700 1 -- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b3c06a270 msgr2=0x7f7b3c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b3b59e700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b3c06a270 0x7f7b3c155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b3b59e700 1 -- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b3c071890 msgr2=0x7f7b3c0b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:34.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b3b59e700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b3c071890 0x7f7b3c0b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:34.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.829+0000 7f7b3b59e700 1 -- 172.21.15.57:0/2933332227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b300094a0 con 0x7f7b3c0cf710 2023-11-28T12:07:34.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.830+0000 7f7b3b59e700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c0cf710 0x7f7b3c15a270 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7b34002230 tx=0x7f7b3400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:34.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.830+0000 7f7b237fe700 1 -- 172.21.15.57:0/2933332227 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3400ac20 con 0x7f7b3c0cf710 2023-11-28T12:07:34.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.830+0000 7f7b237fe700 1 -- 172.21.15.57:0/2933332227 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3400ada0 con 0x7f7b3c0cf710 2023-11-28T12:07:34.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.830+0000 7f7b41172700 1 -- 172.21.15.57:0/2933332227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3c06d530 con 0x7f7b3c0cf710 2023-11-28T12:07:34.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.830+0000 7f7b237fe700 1 -- 172.21.15.57:0/2933332227 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b34007310 con 0x7f7b3c0cf710 2023-11-28T12:07:34.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.830+0000 7f7b41172700 1 -- 172.21.15.57:0/2933332227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3c06d250 con 0x7f7b3c0cf710 2023-11-28T12:07:34.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.831+0000 7f7b41172700 1 -- 172.21.15.57:0/2933332227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3c06a6f0 con 0x7f7b3c0cf710 2023-11-28T12:07:34.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.831+0000 7f7b237fe700 1 -- 172.21.15.57:0/2933332227 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7b3401a030 con 0x7f7b3c0cf710 2023-11-28T12:07:34.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.832+0000 7f7b237fe700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b280682f0 0x7f7b2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:34.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.832+0000 7f7b237fe700 1 -- 172.21.15.57:0/2933332227 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f7b34085b20 con 0x7f7b3c0cf710 2023-11-28T12:07:34.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.832+0000 7f7b3ad9d700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b280682f0 0x7f7b2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:34.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.833+0000 7f7b3ad9d700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b280682f0 0x7f7b2806a7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009ad0 tx=0x7f7b2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:34.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:34.837+0000 7f7b237fe700 1 -- 172.21.15.57:0/2933332227 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7b340522b0 con 0x7f7b3c0cf710 2023-11-28T12:07:35.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.083+0000 7f7b41172700 1 -- 172.21.15.57:0/2933332227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7b3c0743f0 con 0x7f7b3c0cf710 2023-11-28T12:07:35.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.699+0000 7f7b237fe700 1 -- 172.21.15.57:0/2933332227 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7b340558d0 con 0x7f7b3c0cf710 2023-11-28T12:07:35.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.702+0000 7f7b41172700 1 -- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b280682f0 msgr2=0x7f7b2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:35.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.702+0000 7f7b41172700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b280682f0 0x7f7b2806a7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009ad0 tx=0x7f7b2c008040 comp rx=0 tx=0).stop 2023-11-28T12:07:35.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.702+0000 7f7b41172700 1 -- 172.21.15.57:0/2933332227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c0cf710 msgr2=0x7f7b3c15a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:35.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.702+0000 7f7b41172700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c0cf710 0x7f7b3c15a270 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7b34002230 tx=0x7f7b3400a060 comp rx=0 tx=0).stop 2023-11-28T12:07:35.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.702+0000 7f7b41172700 1 -- 172.21.15.57:0/2933332227 shutdown_connections 2023-11-28T12:07:35.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.702+0000 7f7b41172700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f7b280682f0 0x7f7b2806a7a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:35.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.702+0000 7f7b41172700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b3c0cf710 0x7f7b3c15a270 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:35.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.702+0000 7f7b41172700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b3c06a270 0x7f7b3c155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:35.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.702+0000 7f7b41172700 1 --2- 172.21.15.57:0/2933332227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b3c071890 0x7f7b3c0b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:35.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.702+0000 7f7b41172700 1 -- 172.21.15.57:0/2933332227 >> 172.21.15.57:0/2933332227 conn(0x7f7b3c06c010 msgr2=0x7f7b3c06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:35.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.702+0000 7f7b41172700 1 -- 172.21.15.57:0/2933332227 shutdown_connections 2023-11-28T12:07:35.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.702+0000 7f7b41172700 1 -- 172.21.15.57:0/2933332227 wait complete. 2023-11-28T12:07:35.719 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T12:07:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.907+0000 7fd74138e700 1 Processor -- start 2023-11-28T12:07:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.907+0000 7fd74138e700 1 -- start start 2023-11-28T12:07:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.907+0000 7fd74138e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd73c0d0cb0 0x7fd73c0d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.907+0000 7fd74138e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd73c0d15c0 con 0x7fd73c0ccef0 2023-11-28T12:07:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.907+0000 7fd74138e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd73c0cbe50 con 0x7fd73c15c8f0 2023-11-28T12:07:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.907+0000 7fd74138e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd73c0cc130 con 0x7fd73c0d0cb0 2023-11-28T12:07:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.908+0000 7fd73affd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd73c0d0cb0 0x7fd73c0d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.908+0000 7fd73b7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd73c0ccef0 0x7fd73c0ad980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47276/0 (socket says 172.21.15.57:47276) 2023-11-28T12:07:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.908+0000 7fd73b7fe700 1 -- 172.21.15.57:0/1712142087 learned_addr learned my addr 172.21.15.57:0/1712142087 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.908+0000 7fd73affd700 1 -- 172.21.15.57:0/1712142087 >> v1:172.21.15.181:6789/0 conn(0x7fd73c15c8f0 legacy=0x7fd73c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.908+0000 7fd73affd700 1 -- 172.21.15.57:0/1712142087 >> v1:172.21.15.57:6789/0 conn(0x7fd73c0ccef0 legacy=0x7fd73c0ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.908+0000 7fd73affd700 1 -- 172.21.15.57:0/1712142087 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c0cc410 con 0x7fd73c0d0cb0 2023-11-28T12:07:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.908+0000 7fd73affd700 1 --2- 172.21.15.57:0/1712142087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd73c0d0cb0 0x7fd73c0d1080 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009820 tx=0x7fd72c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec0467627a51e14 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.908+0000 7fd739ffb700 1 -- 172.21.15.57:0/1712142087 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c005e30 con 0x7fd73c0d0cb0 2023-11-28T12:07:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.908+0000 7fd739ffb700 1 -- 172.21.15.57:0/1712142087 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72c006030 con 0x7fd73c0d0cb0 2023-11-28T12:07:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.908+0000 7fd739ffb700 1 -- 172.21.15.57:0/1712142087 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c0063e0 con 0x7fd73c0d0cb0 2023-11-28T12:07:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.909+0000 7fd74138e700 1 -- 172.21.15.57:0/1712142087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd73c0d0cb0 msgr2=0x7fd73c0d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.909+0000 7fd74138e700 1 --2- 172.21.15.57:0/1712142087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd73c0d0cb0 0x7fd73c0d1080 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009820 tx=0x7fd72c009b30 comp rx=0 tx=0).stop 2023-11-28T12:07:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.909+0000 7fd74138e700 1 -- 172.21.15.57:0/1712142087 shutdown_connections 2023-11-28T12:07:35.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.909+0000 7fd74138e700 1 --2- 172.21.15.57:0/1712142087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd73c0d0cb0 0x7fd73c0d1080 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:35.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.909+0000 7fd74138e700 1 -- 172.21.15.57:0/1712142087 >> 172.21.15.57:0/1712142087 conn(0x7fd73c06ab90 msgr2=0x7fd73c099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:35.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.909+0000 7fd74138e700 1 -- 172.21.15.57:0/1712142087 shutdown_connections 2023-11-28T12:07:35.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.909+0000 7fd74138e700 1 -- 172.21.15.57:0/1712142087 wait complete. 2023-11-28T12:07:35.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.910+0000 7fd74138e700 1 Processor -- start 2023-11-28T12:07:35.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.911+0000 7fd74138e700 1 -- start start 2023-11-28T12:07:35.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.911+0000 7fd74138e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd73c0ccef0 0x7fd73c15ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:35.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.911+0000 7fd74138e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd73c0d0cb0 0x7fd73c150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:35.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.911+0000 7fd74138e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd73c15c8f0 0x7fd73c15e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:35.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.911+0000 7fd74138e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c15e720 con 0x7fd73c15c8f0 2023-11-28T12:07:35.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.911+0000 7fd74138e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c0b5a30 con 0x7fd73c0d0cb0 2023-11-28T12:07:35.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.911+0000 7fd74138e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c15ff70 con 0x7fd73c0ccef0 2023-11-28T12:07:35.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.911+0000 7fd73b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd73c15c8f0 0x7fd73c15e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:35.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.911+0000 7fd73b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd73c15c8f0 0x7fd73c15e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59030/0 (socket says 172.21.15.57:59030) 2023-11-28T12:07:35.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd73b7fe700 1 -- 172.21.15.57:0/1627104731 learned_addr learned my addr 172.21.15.57:0/1627104731 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:35.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd73affd700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd73c0ccef0 0x7fd73c15ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:35.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd73a7fc700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd73c0d0cb0 0x7fd73c150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:35.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd73b7fe700 1 -- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd73c0ccef0 msgr2=0x7fd73c15ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:35.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd73b7fe700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd73c0ccef0 0x7fd73c15ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:35.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd73b7fe700 1 -- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd73c0d0cb0 msgr2=0x7fd73c150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:35.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd73b7fe700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd73c0d0cb0 0x7fd73c150250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:35.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd73b7fe700 1 -- 172.21.15.57:0/1627104731 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c009430 con 0x7fd73c15c8f0 2023-11-28T12:07:35.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd73a7fc700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd73c0d0cb0 0x7fd73c150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:35.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd73b7fe700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd73c15c8f0 0x7fd73c15e1e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd73400cc20 tx=0x7fd7340021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:35.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd7237fe700 1 -- 172.21.15.57:0/1627104731 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73400ab00 con 0x7fd73c15c8f0 2023-11-28T12:07:35.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd7237fe700 1 -- 172.21.15.57:0/1627104731 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73400ac60 con 0x7fd73c15c8f0 2023-11-28T12:07:35.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd74138e700 1 -- 172.21.15.57:0/1627104731 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73c0cc9d0 con 0x7fd73c15c8f0 2023-11-28T12:07:35.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd7237fe700 1 -- 172.21.15.57:0/1627104731 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734012400 con 0x7fd73c15c8f0 2023-11-28T12:07:35.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.912+0000 7fd74138e700 1 -- 172.21.15.57:0/1627104731 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73c0cc6f0 con 0x7fd73c15c8f0 2023-11-28T12:07:35.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.913+0000 7fd74138e700 1 -- 172.21.15.57:0/1627104731 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73c06ba90 con 0x7fd73c15c8f0 2023-11-28T12:07:35.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.914+0000 7fd7237fe700 1 -- 172.21.15.57:0/1627104731 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd734019030 con 0x7fd73c15c8f0 2023-11-28T12:07:35.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.914+0000 7fd7237fe700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd7280682f0 0x7fd72806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:35.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.914+0000 7fd7237fe700 1 -- 172.21.15.57:0/1627104731 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6264+0+0 (secure 0 0 0) 0x7fd7340857c0 con 0x7fd73c15c8f0 2023-11-28T12:07:35.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.914+0000 7fd73affd700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd7280682f0 0x7fd72806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:35.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.915+0000 7fd73affd700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd7280682f0 0x7fd72806a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd72c013040 tx=0x7fd72c008070 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:35.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:35.919+0000 7fd7237fe700 1 -- 172.21.15.57:0/1627104731 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd73404b5e0 con 0x7fd73c15c8f0 2023-11-28T12:07:36.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.117+0000 7fd74138e700 1 -- 172.21.15.57:0/1627104731 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd73c0743f0 con 0x7fd73c15c8f0 2023-11-28T12:07:36.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.705+0000 7fd7237fe700 1 -- 172.21.15.57:0/1627104731 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd734052000 con 0x7fd73c15c8f0 2023-11-28T12:07:36.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.710+0000 7fd74138e700 1 -- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd7280682f0 msgr2=0x7fd72806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:36.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.710+0000 7fd74138e700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd7280682f0 0x7fd72806a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd72c013040 tx=0x7fd72c008070 comp rx=0 tx=0).stop 2023-11-28T12:07:36.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.710+0000 7fd74138e700 1 -- 172.21.15.57:0/1627104731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd73c15c8f0 msgr2=0x7fd73c15e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:36.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.710+0000 7fd74138e700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd73c15c8f0 0x7fd73c15e1e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd73400cc20 tx=0x7fd7340021c0 comp rx=0 tx=0).stop 2023-11-28T12:07:36.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.711+0000 7fd74138e700 1 -- 172.21.15.57:0/1627104731 shutdown_connections 2023-11-28T12:07:36.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.711+0000 7fd74138e700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd7280682f0 0x7fd72806a7a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:36.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.711+0000 7fd74138e700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd73c15c8f0 0x7fd73c15e1e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:36.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.711+0000 7fd74138e700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd73c0ccef0 0x7fd73c15ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:36.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.711+0000 7fd74138e700 1 --2- 172.21.15.57:0/1627104731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd73c0d0cb0 0x7fd73c150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:36.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.711+0000 7fd74138e700 1 -- 172.21.15.57:0/1627104731 >> 172.21.15.57:0/1627104731 conn(0x7fd73c06ab90 msgr2=0x7fd73c0cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:36.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.712+0000 7fd74138e700 1 -- 172.21.15.57:0/1627104731 shutdown_connections 2023-11-28T12:07:36.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.712+0000 7fd74138e700 1 -- 172.21.15.57:0/1627104731 wait complete. 2023-11-28T12:07:36.714 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' removed 2023-11-28T12:07:36.729 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T12:07:36.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.874+0000 7f06b2194700 1 Processor -- start 2023-11-28T12:07:36.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.874+0000 7f06b2194700 1 -- start start 2023-11-28T12:07:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.875+0000 7f06b2194700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f06ac0cf030 0x7f06ac1650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.875+0000 7f06b2194700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06ac0af050 con 0x7f06ac0af4e0 2023-11-28T12:07:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.875+0000 7f06b2194700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06ac0ae440 con 0x7f06ac15a4a0 2023-11-28T12:07:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.875+0000 7f06b2194700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ac15a870 con 0x7f06ac0cf030 2023-11-28T12:07:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.875+0000 7f06abfff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f06ac0cf030 0x7f06ac1650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:36.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.875+0000 7f06aaffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f06ac0af4e0 0x7f06ac0ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47290/0 (socket says 172.21.15.57:47290) 2023-11-28T12:07:36.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.875+0000 7f06aaffd700 1 -- 172.21.15.57:0/2295696227 learned_addr learned my addr 172.21.15.57:0/2295696227 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:36.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.875+0000 7f06abfff700 1 --2- 172.21.15.57:0/2295696227 >> v2:172.21.15.57:3300/0 conn(0x7f06ac0cf030 0x7f06ac1650c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59040/0 (socket says 172.21.15.57:59040) 2023-11-28T12:07:36.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.875+0000 7f06abfff700 1 -- 172.21.15.57:0/2295696227 >> v1:172.21.15.181:6789/0 conn(0x7f06ac15a4a0 legacy=0x7f06ac0ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:36.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.875+0000 7f06abfff700 1 -- 172.21.15.57:0/2295696227 >> v1:172.21.15.57:6789/0 conn(0x7f06ac0af4e0 legacy=0x7f06ac0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:36.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.875+0000 7f06aa7fc700 1 -- 172.21.15.57:0/2295696227 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246875536 0 0) 0x7f06ac0af050 con 0x7f06ac0af4e0 2023-11-28T12:07:36.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.875+0000 7f06abfff700 1 -- 172.21.15.57:0/2295696227 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac0ae720 con 0x7f06ac0cf030 2023-11-28T12:07:36.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.876+0000 7f06abfff700 1 --2- 172.21.15.57:0/2295696227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06ac0cf030 0x7f06ac1650c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009dc0 tx=0x7f06a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bbcf62196c417 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:36.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.876+0000 7f06aa7fc700 1 -- 172.21.15.57:0/2295696227 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a400e070 con 0x7f06ac0cf030 2023-11-28T12:07:36.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.876+0000 7f06aa7fc700 1 -- 172.21.15.57:0/2295696227 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a400bb50 con 0x7f06ac0cf030 2023-11-28T12:07:36.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.876+0000 7f06aa7fc700 1 -- 172.21.15.57:0/2295696227 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4003a20 con 0x7f06ac0cf030 2023-11-28T12:07:36.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.876+0000 7f06b2194700 1 -- 172.21.15.57:0/2295696227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06ac0cf030 msgr2=0x7f06ac1650c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:36.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.876+0000 7f06b2194700 1 --2- 172.21.15.57:0/2295696227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06ac0cf030 0x7f06ac1650c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009dc0 tx=0x7f06a400b330 comp rx=0 tx=0).stop 2023-11-28T12:07:36.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.876+0000 7f06b2194700 1 -- 172.21.15.57:0/2295696227 shutdown_connections 2023-11-28T12:07:36.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.876+0000 7f06b2194700 1 --2- 172.21.15.57:0/2295696227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06ac0cf030 0x7f06ac1650c0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:36.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.876+0000 7f06b2194700 1 -- 172.21.15.57:0/2295696227 >> 172.21.15.57:0/2295696227 conn(0x7f06ac06ab90 msgr2=0x7f06ac099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:36.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.876+0000 7f06b2194700 1 -- 172.21.15.57:0/2295696227 shutdown_connections 2023-11-28T12:07:36.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.877+0000 7f06b2194700 1 -- 172.21.15.57:0/2295696227 wait complete. 2023-11-28T12:07:36.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.878+0000 7f06b2194700 1 Processor -- start 2023-11-28T12:07:36.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.878+0000 7f06b2194700 1 -- start start 2023-11-28T12:07:36.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06b2194700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06ac0af4e0 0x7f06ac1587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:36.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06b2194700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06ac0cf030 0x7f06ac09b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:36.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06b2194700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06ac15a4a0 0x7f06ac09b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:36.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06b2194700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac158cf0 con 0x7f06ac0cf030 2023-11-28T12:07:36.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06b2194700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac09c8e0 con 0x7f06ac15a4a0 2023-11-28T12:07:36.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06b2194700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac09c010 con 0x7f06ac0af4e0 2023-11-28T12:07:36.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06aaffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06ac0cf030 0x7f06ac09b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:36.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06aaffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06ac0cf030 0x7f06ac09b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59054/0 (socket says 172.21.15.57:59054) 2023-11-28T12:07:36.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06aaffd700 1 -- 172.21.15.57:0/3716855363 learned_addr learned my addr 172.21.15.57:0/3716855363 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:36.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06abfff700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06ac15a4a0 0x7f06ac09b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:36.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06ab7fe700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06ac0af4e0 0x7f06ac1587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:36.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06aaffd700 1 -- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06ac0af4e0 msgr2=0x7f06ac1587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:36.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06aaffd700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06ac0af4e0 0x7f06ac1587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:36.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06aaffd700 1 -- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06ac15a4a0 msgr2=0x7f06ac09b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:36.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06aaffd700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06ac15a4a0 0x7f06ac09b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:36.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06aaffd700 1 -- 172.21.15.57:0/3716855363 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a4009a70 con 0x7f06ac0cf030 2023-11-28T12:07:36.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06abfff700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06ac15a4a0 0x7f06ac09b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:36.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.879+0000 7f06aaffd700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06ac0cf030 0x7f06ac09b4a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f069400c540 tx=0x7f069400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:36.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.880+0000 7f06a3fff700 1 -- 172.21.15.57:0/3716855363 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694004c80 con 0x7f06ac0cf030 2023-11-28T12:07:36.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.880+0000 7f06a3fff700 1 -- 172.21.15.57:0/3716855363 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694004de0 con 0x7f06ac0cf030 2023-11-28T12:07:36.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.880+0000 7f06b2194700 1 -- 172.21.15.57:0/3716855363 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac0aece0 con 0x7f06ac0cf030 2023-11-28T12:07:36.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.880+0000 7f06a3fff700 1 -- 172.21.15.57:0/3716855363 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694007310 con 0x7f06ac0cf030 2023-11-28T12:07:36.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.880+0000 7f06b2194700 1 -- 172.21.15.57:0/3716855363 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac0aea00 con 0x7f06ac0cf030 2023-11-28T12:07:36.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.881+0000 7f06b2194700 1 -- 172.21.15.57:0/3716855363 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ac0d0bd0 con 0x7f06ac0cf030 2023-11-28T12:07:36.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.881+0000 7f06a3fff700 1 -- 172.21.15.57:0/3716855363 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f069401a020 con 0x7f06ac0cf030 2023-11-28T12:07:36.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.882+0000 7f06a3fff700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f068c0682f0 0x7f068c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:36.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.882+0000 7f06a3fff700 1 -- 172.21.15.57:0/3716855363 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5797+0+0 (secure 0 0 0) 0x7f0694085770 con 0x7f06ac0cf030 2023-11-28T12:07:36.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.882+0000 7f06ab7fe700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f068c0682f0 0x7f068c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:36.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.882+0000 7f06ab7fe700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f068c0682f0 0x7f068c06a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f069c006fd0 tx=0x7f069c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:36.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:36.886+0000 7f06a3fff700 1 -- 172.21.15.57:0/3716855363 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f06940520d0 con 0x7f06ac0cf030 2023-11-28T12:07:37.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.085+0000 7f06b2194700 1 -- 172.21.15.57:0/3716855363 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f06ac0743f0 con 0x7f06ac0cf030 2023-11-28T12:07:37.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.712+0000 7f06a3fff700 1 -- 172.21.15.57:0/3716855363 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 169+0+0 (secure 0 0 0) 0x7f06940556f0 con 0x7f06ac0cf030 2023-11-28T12:07:37.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.716+0000 7f06b2194700 1 -- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f068c0682f0 msgr2=0x7f068c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:37.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.716+0000 7f06b2194700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f068c0682f0 0x7f068c06a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f069c006fd0 tx=0x7f069c008040 comp rx=0 tx=0).stop 2023-11-28T12:07:37.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.716+0000 7f06b2194700 1 -- 172.21.15.57:0/3716855363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06ac0cf030 msgr2=0x7f06ac09b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:37.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.716+0000 7f06b2194700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06ac0cf030 0x7f06ac09b4a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f069400c540 tx=0x7f069400c900 comp rx=0 tx=0).stop 2023-11-28T12:07:37.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.717+0000 7f06b2194700 1 -- 172.21.15.57:0/3716855363 shutdown_connections 2023-11-28T12:07:37.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.717+0000 7f06b2194700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f068c0682f0 0x7f068c06a7a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:37.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.717+0000 7f06b2194700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06ac15a4a0 0x7f06ac09b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:37.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.717+0000 7f06b2194700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06ac0af4e0 0x7f06ac1587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:37.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.717+0000 7f06b2194700 1 --2- 172.21.15.57:0/3716855363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06ac0cf030 0x7f06ac09b4a0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:37.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.717+0000 7f06b2194700 1 -- 172.21.15.57:0/3716855363 >> 172.21.15.57:0/3716855363 conn(0x7f06ac06ab90 msgr2=0x7f06ac0ad980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:37.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.717+0000 7f06b2194700 1 -- 172.21.15.57:0/3716855363 shutdown_connections 2023-11-28T12:07:37.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.717+0000 7f06b2194700 1 -- 172.21.15.57:0/3716855363 wait complete. 2023-11-28T12:07:37.720 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' removed 2023-11-28T12:07:37.734 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T12:07:37.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.874+0000 7fba04630700 1 Processor -- start 2023-11-28T12:07:37.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.874+0000 7fba04630700 1 -- start start 2023-11-28T12:07:37.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.875+0000 7fba04630700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9fc15a5f0 0x7fb9fc0d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:37.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.875+0000 7fba04630700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb9fc0b2ee0 0x7fb9fc0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:37.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.875+0000 7fba04630700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc15a9c0 con 0x7fb9fc0cc200 2023-11-28T12:07:37.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.875+0000 7fba04630700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9fc0c7b80 con 0x7fb9fc0b2ee0 2023-11-28T12:07:37.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.875+0000 7fba04630700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0cc990 con 0x7fb9fc15a5f0 2023-11-28T12:07:37.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.876+0000 7fba02bcd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb9fc0b2ee0 0x7fb9fc0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:37.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.876+0000 7fba02bcd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb9fc0b2ee0 0x7fb9fc0ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59066/0 (socket says 172.21.15.57:59066) 2023-11-28T12:07:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.876+0000 7fba02bcd700 1 -- 172.21.15.57:0/1392955393 learned_addr learned my addr 172.21.15.57:0/1392955393 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.876+0000 7fba023cc700 1 --2- 172.21.15.57:0/1392955393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9fc15a5f0 0x7fb9fc0d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.876+0000 7fba02bcd700 1 -- 172.21.15.57:0/1392955393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9fc15a5f0 msgr2=0x7fb9fc0d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.876+0000 7fba02bcd700 1 --2- 172.21.15.57:0/1392955393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9fc15a5f0 0x7fb9fc0d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.876+0000 7fba02bcd700 1 -- 172.21.15.57:0/1392955393 >> v1:172.21.15.181:6789/0 conn(0x7fb9fc0cc200 legacy=0x7fb9fc0b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.876+0000 7fba02bcd700 1 -- 172.21.15.57:0/1392955393 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc0ccc70 con 0x7fb9fc0b2ee0 2023-11-28T12:07:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.876+0000 7fba02bcd700 1 --2- 172.21.15.57:0/1392955393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9fc0b2ee0 0x7fb9fc0ced30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009e50 tx=0x7fb9f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fee30fd1e3bc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.876+0000 7fba013ca700 1 -- 172.21.15.57:0/1392955393 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f400e070 con 0x7fb9fc0b2ee0 2023-11-28T12:07:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.876+0000 7fba013ca700 1 -- 172.21.15.57:0/1392955393 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f400b910 con 0x7fb9fc0b2ee0 2023-11-28T12:07:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.876+0000 7fba013ca700 1 -- 172.21.15.57:0/1392955393 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f400bcc0 con 0x7fb9fc0b2ee0 2023-11-28T12:07:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.877+0000 7fba04630700 1 -- 172.21.15.57:0/1392955393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9fc0b2ee0 msgr2=0x7fb9fc0ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.877+0000 7fba04630700 1 --2- 172.21.15.57:0/1392955393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9fc0b2ee0 0x7fb9fc0ced30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009e50 tx=0x7fb9f400b330 comp rx=0 tx=0).stop 2023-11-28T12:07:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.877+0000 7fba04630700 1 -- 172.21.15.57:0/1392955393 shutdown_connections 2023-11-28T12:07:37.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.877+0000 7fba04630700 1 --2- 172.21.15.57:0/1392955393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9fc0b2ee0 0x7fb9fc0ced30 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:37.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.877+0000 7fba04630700 1 --2- 172.21.15.57:0/1392955393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9fc15a5f0 0x7fb9fc0d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:37.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.877+0000 7fba04630700 1 -- 172.21.15.57:0/1392955393 >> 172.21.15.57:0/1392955393 conn(0x7fb9fc06ea80 msgr2=0x7fb9fc06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:37.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.877+0000 7fba04630700 1 -- 172.21.15.57:0/1392955393 shutdown_connections 2023-11-28T12:07:37.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.877+0000 7fba04630700 1 -- 172.21.15.57:0/1392955393 wait complete. 2023-11-28T12:07:37.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.878+0000 7fba04630700 1 Processor -- start 2023-11-28T12:07:37.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.878+0000 7fba04630700 1 -- start start 2023-11-28T12:07:37.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.879+0000 7fba04630700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9fc0b2ee0 0x7fb9fc0c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:37.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.879+0000 7fba04630700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9fc0cc200 0x7fb9fc1f07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:37.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.879+0000 7fba04630700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9fc15a5f0 0x7fb9fc1f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:37.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.879+0000 7fba04630700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0c66b0 con 0x7fb9fc0cc200 2023-11-28T12:07:37.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.879+0000 7fba04630700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc15cae0 con 0x7fb9fc15a5f0 2023-11-28T12:07:37.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.879+0000 7fba04630700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0c5e10 con 0x7fb9fc0b2ee0 2023-11-28T12:07:37.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.879+0000 7fba01bcb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9fc0cc200 0x7fb9fc1f07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:37.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.879+0000 7fba01bcb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9fc0cc200 0x7fb9fc1f07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59074/0 (socket says 172.21.15.57:59074) 2023-11-28T12:07:37.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.879+0000 7fba01bcb700 1 -- 172.21.15.57:0/42656520 learned_addr learned my addr 172.21.15.57:0/42656520 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:37.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fba023cc700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9fc0b2ee0 0x7fb9fc0c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:37.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fba02bcd700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9fc15a5f0 0x7fb9fc1f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:37.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fba01bcb700 1 -- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9fc0b2ee0 msgr2=0x7fb9fc0c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:37.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fba01bcb700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9fc0b2ee0 0x7fb9fc0c55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:37.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fba01bcb700 1 -- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9fc15a5f0 msgr2=0x7fb9fc1f0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:37.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fba01bcb700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9fc15a5f0 0x7fb9fc1f0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:37.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fba01bcb700 1 -- 172.21.15.57:0/42656520 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f4009b30 con 0x7fb9fc0cc200 2023-11-28T12:07:37.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fba023cc700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9fc0b2ee0 0x7fb9fc0c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:37.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fba02bcd700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9fc15a5f0 0x7fb9fc1f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:37.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fba01bcb700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9fc0cc200 0x7fb9fc1f07c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c4e0 tx=0x7fb9ec00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:37.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fb9f2ffd700 1 -- 172.21.15.57:0/42656520 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0052d0 con 0x7fb9fc0cc200 2023-11-28T12:07:37.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fb9f2ffd700 1 -- 172.21.15.57:0/42656520 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec005430 con 0x7fb9fc0cc200 2023-11-28T12:07:37.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fba04630700 1 -- 172.21.15.57:0/42656520 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc0cd230 con 0x7fb9fc0cc200 2023-11-28T12:07:37.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fb9f2ffd700 1 -- 172.21.15.57:0/42656520 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0038c0 con 0x7fb9fc0cc200 2023-11-28T12:07:37.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.880+0000 7fba04630700 1 -- 172.21.15.57:0/42656520 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc0ccf50 con 0x7fb9fc0cc200 2023-11-28T12:07:37.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.881+0000 7fba04630700 1 -- 172.21.15.57:0/42656520 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9fc0cd510 con 0x7fb9fc0cc200 2023-11-28T12:07:37.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.882+0000 7fb9f2ffd700 1 -- 172.21.15.57:0/42656520 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb9ec019030 con 0x7fb9fc0cc200 2023-11-28T12:07:37.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.882+0000 7fb9f2ffd700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9e00682f0 0x7fb9e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:37.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.882+0000 7fb9f2ffd700 1 -- 172.21.15.57:0/42656520 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb9ec084d40 con 0x7fb9fc0cc200 2023-11-28T12:07:37.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.882+0000 7fba023cc700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9e00682f0 0x7fb9e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:37.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.883+0000 7fba023cc700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9e00682f0 0x7fb9e006a7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0c69c0 tx=0x7fb9f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:37.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:37.887+0000 7fb9f2ffd700 1 -- 172.21.15.57:0/42656520 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb9ec088030 con 0x7fb9fc0cc200 2023-11-28T12:07:38.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.083+0000 7fba04630700 1 -- 172.21.15.57:0/42656520 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb9fc0743f0 con 0x7fb9fc0cc200 2023-11-28T12:07:38.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.087+0000 7fb9f2ffd700 1 -- 172.21.15.57:0/42656520 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb9ec0518a0 con 0x7fb9fc0cc200 2023-11-28T12:07:38.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.090+0000 7fba04630700 1 -- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9e00682f0 msgr2=0x7fb9e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:38.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.090+0000 7fba04630700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9e00682f0 0x7fb9e006a7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0c69c0 tx=0x7fb9f8008040 comp rx=0 tx=0).stop 2023-11-28T12:07:38.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.090+0000 7fba04630700 1 -- 172.21.15.57:0/42656520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9fc0cc200 msgr2=0x7fb9fc1f07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:38.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.090+0000 7fba04630700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9fc0cc200 0x7fb9fc1f07c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c4e0 tx=0x7fb9ec00c8a0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.090+0000 7fba04630700 1 -- 172.21.15.57:0/42656520 shutdown_connections 2023-11-28T12:07:38.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.090+0000 7fba04630700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fb9e00682f0 0x7fb9e006a7a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.090+0000 7fba04630700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9fc15a5f0 0x7fb9fc1f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.090+0000 7fba04630700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9fc0b2ee0 0x7fb9fc0c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.090+0000 7fba04630700 1 --2- 172.21.15.57:0/42656520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9fc0cc200 0x7fb9fc1f07c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.090+0000 7fba04630700 1 -- 172.21.15.57:0/42656520 >> 172.21.15.57:0/42656520 conn(0x7fb9fc06ea80 msgr2=0x7fb9fc0b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:38.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.090+0000 7fba04630700 1 -- 172.21.15.57:0/42656520 shutdown_connections 2023-11-28T12:07:38.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.090+0000 7fba04630700 1 -- 172.21.15.57:0/42656520 wait complete. 2023-11-28T12:07:38.105 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T12:07:38.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.239+0000 7f1878d1e700 1 Processor -- start 2023-11-28T12:07:38.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.239+0000 7f1878d1e700 1 -- start start 2023-11-28T12:07:38.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.240+0000 7f1878d1e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f187406a270 0x7f187406a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:38.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.240+0000 7f1878d1e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187415cc80 0x7f187406de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:38.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.240+0000 7f1878d1e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187406ab80 con 0x7f187406e3a0 2023-11-28T12:07:38.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.240+0000 7f1878d1e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18740b2490 con 0x7f187406a270 2023-11-28T12:07:38.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.240+0000 7f1878d1e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18740b1810 con 0x7f187415cc80 2023-11-28T12:07:38.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.240+0000 7f187259c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f187406a270 0x7f187406a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:38.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.240+0000 7f1872d9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f187406e3a0 0x7f187406ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47292/0 (socket says 172.21.15.57:47292) 2023-11-28T12:07:38.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.240+0000 7f187259c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f187406a270 0x7f187406a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59090/0 (socket says 172.21.15.57:59090) 2023-11-28T12:07:38.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.240+0000 7f1872d9d700 1 -- 172.21.15.57:0/2258337673 learned_addr learned my addr 172.21.15.57:0/2258337673 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:38.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.240+0000 7f1871d9b700 1 --2- 172.21.15.57:0/2258337673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187415cc80 0x7f187406de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:38.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.241+0000 7f187259c700 1 -- 172.21.15.57:0/2258337673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187415cc80 msgr2=0x7f187406de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:38.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.241+0000 7f187259c700 1 --2- 172.21.15.57:0/2258337673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187415cc80 0x7f187406de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.241+0000 7f187259c700 1 -- 172.21.15.57:0/2258337673 >> v1:172.21.15.57:6789/0 conn(0x7f187406e3a0 legacy=0x7f187406ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:38.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.241+0000 7f187259c700 1 -- 172.21.15.57:0/2258337673 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18740b1af0 con 0x7f187406a270 2023-11-28T12:07:38.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.241+0000 7f187159a700 1 -- 172.21.15.57:0/2258337673 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684540945 0 0) 0x7f187406ab80 con 0x7f187406e3a0 2023-11-28T12:07:38.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.241+0000 7f187259c700 1 --2- 172.21.15.57:0/2258337673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f187406a270 0x7f187406a640 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f1864009840 tx=0x7f1864009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d4813a178fcae6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:38.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.241+0000 7f187159a700 1 -- 172.21.15.57:0/2258337673 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864011830 con 0x7f187406a270 2023-11-28T12:07:38.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.241+0000 7f187159a700 1 -- 172.21.15.57:0/2258337673 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1864011990 con 0x7f187406a270 2023-11-28T12:07:38.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.241+0000 7f187159a700 1 -- 172.21.15.57:0/2258337673 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864011d20 con 0x7f187406a270 2023-11-28T12:07:38.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.242+0000 7f1878d1e700 1 -- 172.21.15.57:0/2258337673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f187406a270 msgr2=0x7f187406a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:38.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.242+0000 7f1878d1e700 1 --2- 172.21.15.57:0/2258337673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f187406a270 0x7f187406a640 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f1864009840 tx=0x7f1864009b50 comp rx=0 tx=0).stop 2023-11-28T12:07:38.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.242+0000 7f1878d1e700 1 -- 172.21.15.57:0/2258337673 shutdown_connections 2023-11-28T12:07:38.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.242+0000 7f1878d1e700 1 --2- 172.21.15.57:0/2258337673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f187406a270 0x7f187406a640 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.242+0000 7f1878d1e700 1 --2- 172.21.15.57:0/2258337673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187415cc80 0x7f187406de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.242+0000 7f1878d1e700 1 -- 172.21.15.57:0/2258337673 >> 172.21.15.57:0/2258337673 conn(0x7f187406c010 msgr2=0x7f18740cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:38.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.242+0000 7f1878d1e700 1 -- 172.21.15.57:0/2258337673 shutdown_connections 2023-11-28T12:07:38.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.242+0000 7f1878d1e700 1 -- 172.21.15.57:0/2258337673 wait complete. 2023-11-28T12:07:38.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.243+0000 7f1878d1e700 1 Processor -- start 2023-11-28T12:07:38.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.243+0000 7f1878d1e700 1 -- start start 2023-11-28T12:07:38.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1878d1e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187406a270 0x7f18740cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:38.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1878d1e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f187406e3a0 0x7f18740c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:38.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1878d1e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f187415cc80 0x7f18740c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:38.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1878d1e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18740b3530 con 0x7f187406e3a0 2023-11-28T12:07:38.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1878d1e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18740b3810 con 0x7f187415cc80 2023-11-28T12:07:38.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1878d1e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18740b3af0 con 0x7f187406a270 2023-11-28T12:07:38.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1871d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f187406e3a0 0x7f18740c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:38.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1871d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f187406e3a0 0x7f18740c5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59106/0 (socket says 172.21.15.57:59106) 2023-11-28T12:07:38.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1871d9b700 1 -- 172.21.15.57:0/2072499377 learned_addr learned my addr 172.21.15.57:0/2072499377 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:38.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1872d9d700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f187415cc80 0x7f18740c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:38.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f187259c700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187406a270 0x7f18740cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:38.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1871d9b700 1 -- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187406a270 msgr2=0x7f18740cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:38.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1871d9b700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187406a270 0x7f18740cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1871d9b700 1 -- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f187415cc80 msgr2=0x7f18740c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:38.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1871d9b700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f187415cc80 0x7f18740c5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.244+0000 7f1871d9b700 1 -- 172.21.15.57:0/2072499377 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18640094a0 con 0x7f187406e3a0 2023-11-28T12:07:38.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.245+0000 7f1871d9b700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f187406e3a0 0x7f18740c5670 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f185c00e850 tx=0x7f185c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:38.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.245+0000 7f186affd700 1 -- 172.21.15.57:0/2072499377 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c0095b0 con 0x7f187406e3a0 2023-11-28T12:07:38.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.245+0000 7f186affd700 1 -- 172.21.15.57:0/2072499377 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c00ee50 con 0x7f187406e3a0 2023-11-28T12:07:38.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.245+0000 7f1878d1e700 1 -- 172.21.15.57:0/2072499377 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18740b20b0 con 0x7f187406e3a0 2023-11-28T12:07:38.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.245+0000 7f186affd700 1 -- 172.21.15.57:0/2072499377 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00ca70 con 0x7f187406e3a0 2023-11-28T12:07:38.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.245+0000 7f1878d1e700 1 -- 172.21.15.57:0/2072499377 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18740b1dd0 con 0x7f187406e3a0 2023-11-28T12:07:38.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.246+0000 7f1878d1e700 1 -- 172.21.15.57:0/2072499377 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187406e7e0 con 0x7f187406e3a0 2023-11-28T12:07:38.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.246+0000 7f186affd700 1 -- 172.21.15.57:0/2072499377 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f185c019030 con 0x7f187406e3a0 2023-11-28T12:07:38.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.247+0000 7f186affd700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f18540682f0 0x7f185406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:38.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.247+0000 7f186affd700 1 -- 172.21.15.57:0/2072499377 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f185c085820 con 0x7f187406e3a0 2023-11-28T12:07:38.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.247+0000 7f187259c700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f18540682f0 0x7f185406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:38.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.248+0000 7f187259c700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f18540682f0 0x7f185406a7a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f18640090b0 tx=0x7f1864008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:38.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.252+0000 7f186affd700 1 -- 172.21.15.57:0/2072499377 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f185c052380 con 0x7f187406e3a0 2023-11-28T12:07:38.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.535+0000 7f1878d1e700 1 -- 172.21.15.57:0/2072499377 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f18740743f0 con 0x7f187406e3a0 2023-11-28T12:07:38.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.715+0000 7f186affd700 1 -- 172.21.15.57:0/2072499377 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v280) v1 ==== 169+0+0 (secure 0 0 0) 0x7f185c0559a0 con 0x7f187406e3a0 2023-11-28T12:07:38.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.718+0000 7f1878d1e700 1 -- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f18540682f0 msgr2=0x7f185406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:38.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.718+0000 7f1878d1e700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f18540682f0 0x7f185406a7a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f18640090b0 tx=0x7f1864008030 comp rx=0 tx=0).stop 2023-11-28T12:07:38.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.718+0000 7f1878d1e700 1 -- 172.21.15.57:0/2072499377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f187406e3a0 msgr2=0x7f18740c5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:38.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.718+0000 7f1878d1e700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f187406e3a0 0x7f18740c5670 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f185c00e850 tx=0x7f185c00ec10 comp rx=0 tx=0).stop 2023-11-28T12:07:38.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.718+0000 7f1878d1e700 1 -- 172.21.15.57:0/2072499377 shutdown_connections 2023-11-28T12:07:38.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.718+0000 7f1878d1e700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f18540682f0 0x7f185406a7a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.718+0000 7f1878d1e700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f187415cc80 0x7f18740c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.718+0000 7f1878d1e700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187406a270 0x7f18740cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.718+0000 7f1878d1e700 1 --2- 172.21.15.57:0/2072499377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f187406e3a0 0x7f18740c5670 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:38.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.718+0000 7f1878d1e700 1 -- 172.21.15.57:0/2072499377 >> 172.21.15.57:0/2072499377 conn(0x7f187406c010 msgr2=0x7f18740afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:38.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.718+0000 7f1878d1e700 1 -- 172.21.15.57:0/2072499377 shutdown_connections 2023-11-28T12:07:38.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:38.718+0000 7f1878d1e700 1 -- 172.21.15.57:0/2072499377 wait complete. 2023-11-28T12:07:38.734 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ERROR 2023-11-28T12:07:38.735 INFO:tasks.cephfs_test_runner: 2023-11-28T12:07:38.735 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T12:07:38.735 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T12:07:38.735 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T12:07:38.735 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T12:07:38.735 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T12:07:38.735 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T12:07:38.735 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=100) 2023-11-28T12:07:38.735 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T12:07:38.736 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T12:07:38.736 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 100s and 0 retries 2023-11-28T12:07:38.736 INFO:tasks.cephfs_test_runner: 2023-11-28T12:07:38.736 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T12:07:38.736 INFO:tasks.cephfs_test_runner:Ran 3 tests in 323.080s 2023-11-28T12:07:38.736 INFO:tasks.cephfs_test_runner: 2023-11-28T12:07:38.736 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-28T12:07:38.736 INFO:tasks.cephfs_test_runner: 2023-11-28T12:07:38.736 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T12:07:38.736 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T12:07:38.736 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T12:07:38.736 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T12:07:38.737 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T12:07:38.737 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T12:07:38.737 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=100) 2023-11-28T12:07:38.737 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T12:07:38.737 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T12:07:38.737 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 100s and 0 retries 2023-11-28T12:07:38.737 INFO:tasks.cephfs_test_runner: 2023-11-28T12:07:38.737 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T12:07:38.846 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=2b616a9481d24e0da0c47448b7a9ab16 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T12:07:38.848 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-28T12:07:38.857 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-28T12:07:38.904 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-28T12:07:38.904 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:07:38.904 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:07:38.931 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16254224k,nr_inodes=4063556,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15749 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3254388k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T12:07:38.932 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-11-28T12:07:38.932 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:07:38.995 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16254224k,nr_inodes=4063556,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15749 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3254388k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T12:07:38.995 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:07:38.995 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:07:39.018 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251064k,nr_inodes=4062766,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14954 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253752k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T12:07:39.019 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-11-28T12:07:39.019 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T12:07:39.080 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16251064k,nr_inodes=4062766,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14954 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253752k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T12:07:39.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:07:39.081 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-11-28T12:07:39.104 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T12:07:39.105 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T12:07:39.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:07:39.105 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T12:07:39.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:07:39.195 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T12:07:39.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:07:39.249 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2023-11-28T12:07:39.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:07:39.316 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-11-28T12:07:39.338 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-28T12:07:39.338 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T12:07:39.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:07:39.339 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T12:07:39.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:07:39.423 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T12:07:39.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T12:07:39.486 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-11-28T12:07:39.568 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-28T12:07:39.578 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T12:07:39.578 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:07:39.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.723+0000 7f3d6f893700 1 Processor -- start 2023-11-28T12:07:39.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.724+0000 7f3d6f893700 1 -- start start 2023-11-28T12:07:39.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.724+0000 7f3d6f893700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3d68158cd0 0x7f3d680c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:39.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.724+0000 7f3d6f893700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d680cd1c0 0x7f3d680cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:39.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6f893700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d680ce0c0 0x7f3d6815c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:39.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6f893700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d681590a0 con 0x7f3d68158cd0 2023-11-28T12:07:39.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6f893700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d680718e0 con 0x7f3d680cd1c0 2023-11-28T12:07:39.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6f893700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d680ce500 con 0x7f3d680ce0c0 2023-11-28T12:07:39.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6d62f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3d68158cd0 0x7f3d680c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:39.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6d62f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3d68158cd0 0x7f3d680c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59112/0 (socket says 172.21.15.57:59112) 2023-11-28T12:07:39.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6d62f700 1 -- 172.21.15.57:0/2533010344 learned_addr learned my addr 172.21.15.57:0/2533010344 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:39.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6de30700 1 --2- 172.21.15.57:0/2533010344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d680ce0c0 0x7f3d6815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:39.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6ce2e700 1 --2- 172.21.15.57:0/2533010344 >> v2:172.21.15.181:3300/0 conn(0x7f3d680cd1c0 0x7f3d680cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:39.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6d62f700 1 -- 172.21.15.57:0/2533010344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d680ce0c0 msgr2=0x7f3d6815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:39.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6d62f700 1 --2- 172.21.15.57:0/2533010344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d680ce0c0 0x7f3d6815c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:39.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6d62f700 1 -- 172.21.15.57:0/2533010344 >> v2:172.21.15.181:3300/0 conn(0x7f3d680cd1c0 msgr2=0x7f3d680cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:39.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6d62f700 1 --2- 172.21.15.57:0/2533010344 >> v2:172.21.15.181:3300/0 conn(0x7f3d680cd1c0 0x7f3d680cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:39.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6d62f700 1 -- 172.21.15.57:0/2533010344 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6806be90 con 0x7f3d68158cd0 2023-11-28T12:07:39.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6de30700 1 --2- 172.21.15.57:0/2533010344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d680ce0c0 0x7f3d6815c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:39.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6ce2e700 1 --2- 172.21.15.57:0/2533010344 >> v2:172.21.15.181:3300/0 conn(0x7f3d680cd1c0 0x7f3d680cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:39.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d6d62f700 1 --2- 172.21.15.57:0/2533010344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d68158cd0 0x7f3d680c92c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3d680cc260 tx=0x7f3d64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff300fcc5bfe0e82 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:39.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d5bfff700 1 -- 172.21.15.57:0/2533010344 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400b870 con 0x7f3d68158cd0 2023-11-28T12:07:39.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d5bfff700 1 -- 172.21.15.57:0/2533010344 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6400b9d0 con 0x7f3d68158cd0 2023-11-28T12:07:39.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.725+0000 7f3d5bfff700 1 -- 172.21.15.57:0/2533010344 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400bd60 con 0x7f3d68158cd0 2023-11-28T12:07:39.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.726+0000 7f3d6f893700 1 -- 172.21.15.57:0/2533010344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d68158cd0 msgr2=0x7f3d680c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:39.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.726+0000 7f3d6f893700 1 --2- 172.21.15.57:0/2533010344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d68158cd0 0x7f3d680c92c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3d680cc260 tx=0x7f3d64009bd0 comp rx=0 tx=0).stop 2023-11-28T12:07:39.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.726+0000 7f3d6f893700 1 -- 172.21.15.57:0/2533010344 shutdown_connections 2023-11-28T12:07:39.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.726+0000 7f3d6f893700 1 --2- 172.21.15.57:0/2533010344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d680ce0c0 0x7f3d6815c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:39.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.726+0000 7f3d6f893700 1 --2- 172.21.15.57:0/2533010344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d68158cd0 0x7f3d680c92c0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:39.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.726+0000 7f3d6f893700 1 --2- 172.21.15.57:0/2533010344 >> v2:172.21.15.181:3300/0 conn(0x7f3d680cd1c0 0x7f3d680cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:39.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.726+0000 7f3d6f893700 1 -- 172.21.15.57:0/2533010344 >> 172.21.15.57:0/2533010344 conn(0x7f3d680cb660 msgr2=0x7f3d680cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:39.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.726+0000 7f3d6f893700 1 -- 172.21.15.57:0/2533010344 shutdown_connections 2023-11-28T12:07:39.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.726+0000 7f3d6f893700 1 -- 172.21.15.57:0/2533010344 wait complete. 2023-11-28T12:07:39.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.727+0000 7f3d6f893700 1 Processor -- start 2023-11-28T12:07:39.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.727+0000 7f3d6f893700 1 -- start start 2023-11-28T12:07:39.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.728+0000 7f3d6f893700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d680cd1c0 0x7f3d680c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:39.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.728+0000 7f3d6f893700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d680ce0c0 0x7f3d680c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:39.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.728+0000 7f3d6f893700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d68158cd0 0x7f3d681f0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:39.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.728+0000 7f3d6f893700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d680c2df0 con 0x7f3d68158cd0 2023-11-28T12:07:39.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.728+0000 7f3d6f893700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d680c30d0 con 0x7f3d680ce0c0 2023-11-28T12:07:39.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.728+0000 7f3d6f893700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d680c33b0 con 0x7f3d680cd1c0 2023-11-28T12:07:39.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.728+0000 7f3d6de30700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d68158cd0 0x7f3d681f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:39.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.728+0000 7f3d6de30700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d68158cd0 0x7f3d681f0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59116/0 (socket says 172.21.15.57:59116) 2023-11-28T12:07:39.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.728+0000 7f3d6de30700 1 -- 172.21.15.57:0/1219427160 learned_addr learned my addr 172.21.15.57:0/1219427160 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:39.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d6ce2e700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d680ce0c0 0x7f3d680c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:39.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d6d62f700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d680cd1c0 0x7f3d680c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:39.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d6de30700 1 -- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d680cd1c0 msgr2=0x7f3d680c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:39.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d6de30700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d680cd1c0 0x7f3d680c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:39.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d6de30700 1 -- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d680ce0c0 msgr2=0x7f3d680c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:39.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d6de30700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d680ce0c0 0x7f3d680c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:39.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d6de30700 1 -- 172.21.15.57:0/1219427160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d64009550 con 0x7f3d68158cd0 2023-11-28T12:07:39.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d6ce2e700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d680ce0c0 0x7f3d680c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:39.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d6d62f700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d680cd1c0 0x7f3d680c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:39.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d6de30700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d68158cd0 0x7f3d681f0570 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000cd50 tx=0x7f3d6000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:39.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d59ffb700 1 -- 172.21.15.57:0/1219427160 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000f480 con 0x7f3d68158cd0 2023-11-28T12:07:39.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d59ffb700 1 -- 172.21.15.57:0/1219427160 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6000f600 con 0x7f3d68158cd0 2023-11-28T12:07:39.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d6f893700 1 -- 172.21.15.57:0/1219427160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d680cb010 con 0x7f3d68158cd0 2023-11-28T12:07:39.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d59ffb700 1 -- 172.21.15.57:0/1219427160 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000f9b0 con 0x7f3d68158cd0 2023-11-28T12:07:39.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.729+0000 7f3d6f893700 1 -- 172.21.15.57:0/1219427160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d680cad30 con 0x7f3d68158cd0 2023-11-28T12:07:39.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.730+0000 7f3d6f893700 1 -- 172.21.15.57:0/1219427160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d680cb2f0 con 0x7f3d68158cd0 2023-11-28T12:07:39.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.731+0000 7f3d59ffb700 1 -- 172.21.15.57:0/1219427160 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3d60009030 con 0x7f3d68158cd0 2023-11-28T12:07:39.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.731+0000 7f3d59ffb700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d4c0682f0 0x7f3d4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:39.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.731+0000 7f3d59ffb700 1 -- 172.21.15.57:0/1219427160 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3d60086260 con 0x7f3d68158cd0 2023-11-28T12:07:39.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.731+0000 7f3d6d62f700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d4c0682f0 0x7f3d4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:39.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.732+0000 7f3d6d62f700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d4c0682f0 0x7f3d4c06a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009130 tx=0x7f3d64008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:39.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.735+0000 7f3d59ffb700 1 -- 172.21.15.57:0/1219427160 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3d60052dc0 con 0x7f3d68158cd0 2023-11-28T12:07:39.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.931+0000 7f3d6f893700 1 -- 172.21.15.57:0/1219427160 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3d680743f0 con 0x7f3d4c0682f0 2023-11-28T12:07:39.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.935+0000 7f3d59ffb700 1 -- 172.21.15.57:0/1219427160 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73587 (secure 0 0 0) 0x7f3d680743f0 con 0x7f3d4c0682f0 2023-11-28T12:07:39.936 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.936+0000 7f3d6f893700 1 -- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d4c0682f0 msgr2=0x7f3d4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.936+0000 7f3d6f893700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d4c0682f0 0x7f3d4c06a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009130 tx=0x7f3d64008030 comp rx=0 tx=0).stop 2023-11-28T12:07:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.936+0000 7f3d6f893700 1 -- 172.21.15.57:0/1219427160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d68158cd0 msgr2=0x7f3d681f0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.936+0000 7f3d6f893700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d68158cd0 0x7f3d681f0570 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000cd50 tx=0x7f3d6000a560 comp rx=0 tx=0).stop 2023-11-28T12:07:39.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.937+0000 7f3d6f893700 1 -- 172.21.15.57:0/1219427160 shutdown_connections 2023-11-28T12:07:39.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.937+0000 7f3d6f893700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f3d4c0682f0 0x7f3d4c06a7a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:39.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.937+0000 7f3d6f893700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d68158cd0 0x7f3d681f0570 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:39.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.937+0000 7f3d6f893700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d680cd1c0 0x7f3d680c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:39.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.937+0000 7f3d6f893700 1 --2- 172.21.15.57:0/1219427160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d680ce0c0 0x7f3d680c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:39.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.937+0000 7f3d6f893700 1 -- 172.21.15.57:0/1219427160 >> 172.21.15.57:0/1219427160 conn(0x7f3d680cb660 msgr2=0x7f3d680c8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:39.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.937+0000 7f3d6f893700 1 -- 172.21.15.57:0/1219427160 shutdown_connections 2023-11-28T12:07:39.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:39.937+0000 7f3d6f893700 1 -- 172.21.15.57:0/1219427160 wait complete. 2023-11-28T12:07:39.940 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:07:39.951 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2023-11-28T12:07:39.751638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":96408,"kb_used_data":7448,"kb_used_omap":0,"kb_used_meta":88896,"kb_avail":754878312,"statfs":{"total":773094113280,"available":772995391488,"internally_reserved":0,"allocated":7626752,"data_stored":4055234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91029504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12289,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-74,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.054812"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":59,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.699534+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:07:33.699534+0000","last_peered":"2023-11-28T12:07:33.699534+0000","last_clean":"2023-11-28T12:07:33.699534+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:07:33.699534+0000","last_undegraded":"2023-11-28T12:07:33.699534+0000","last_fullsized":"2023-11-28T12:07:33.699534+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.697778+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:07:33.697778+0000","last_peered":"2023-11-28T12:07:33.697778+0000","last_clean":"2023-11-28T12:07:33.697778+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:07:33.697778+0000","last_undegraded":"2023-11-28T12:07:33.697778+0000","last_fullsized":"2023-11-28T12:07:33.697778+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":57,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.770443+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:07:33.770443+0000","last_peered":"2023-11-28T12:07:33.770443+0000","last_clean":"2023-11-28T12:07:33.770443+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:07:33.770443+0000","last_undegraded":"2023-11-28T12:07:33.770443+0000","last_fullsized":"2023-11-28T12:07:33.770443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":202,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T12:05:41.751905+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:05:41.751905+0000","last_peered":"2023-11-28T12:05:41.751905+0000","last_clean":"2023-11-28T12:05:41.751905+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:05:41.751905+0000","last_undegraded":"2023-11-28T12:05:41.751905+0000","last_fullsized":"2023-11-28T12:05:41.751905+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T12:05:51.753233+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:05:51.753233+0000","last_peered":"2023-11-28T12:05:51.753233+0000","last_clean":"2023-11-28T12:05:51.753233+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:05:51.753233+0000","last_undegraded":"2023-11-28T12:05:51.753233+0000","last_fullsized":"2023-11-28T12:05:51.753233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T12:05:41.751958+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:05:41.751958+0000","last_peered":"2023-11-28T12:05:41.751958+0000","last_clean":"2023-11-28T12:05:41.751958+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:05:41.751958+0000","last_undegraded":"2023-11-28T12:05:41.751958+0000","last_fullsized":"2023-11-28T12:05:41.751958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T12:05:51.778799+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:05:51.778799+0000","last_peered":"2023-11-28T12:05:51.778799+0000","last_clean":"2023-11-28T12:05:51.778799+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:05:51.778799+0000","last_undegraded":"2023-11-28T12:05:51.778799+0000","last_fullsized":"2023-11-28T12:05:51.778799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.699894+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:07:33.699894+0000","last_peered":"2023-11-28T12:07:33.699894+0000","last_clean":"2023-11-28T12:07:33.699894+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:07:33.699894+0000","last_undegraded":"2023-11-28T12:07:33.699894+0000","last_fullsized":"2023-11-28T12:07:33.699894+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.700627+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:07:33.700627+0000","last_peered":"2023-11-28T12:07:33.700627+0000","last_clean":"2023-11-28T12:07:33.700627+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:07:33.700627+0000","last_undegraded":"2023-11-28T12:07:33.700627+0000","last_fullsized":"2023-11-28T12:07:33.700627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10980,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360860,"statfs":{"total":96636764160,"available":96625520640,"internally_reserved":0,"allocated":1404928,"data_stored":947368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.42099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71599999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.50700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77400000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.45000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.746,"5min":0.872,"15min":0.872},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.80700000000000005},"min":{"1min":0.48799999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.745,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63300000000000001,"15min":0.79200000000000004},"min":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72699999999999998,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.44800000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67200000000000004}]},{"osd":3,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.47699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.75900000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60999999999999999,"15min":0.66400000000000003},"min":{"1min":0.46200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67300000000000004,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.60099999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47399999999999998,"15min":0.499},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.56599999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47999999999999998,"15min":0.48799999999999999},"min":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68500000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.622}]},{"osd":5,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.497,"15min":0.51600000000000001},"min":{"1min":0.34599999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.59399999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52300000000000002,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65700000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.61199999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54900000000000004,"15min":0.58099999999999996},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.69599999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54600000000000004,"15min":0.57399999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.65000000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9492,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362348,"statfs":{"total":96636764160,"available":96627044352,"internally_reserved":0,"allocated":798720,"data_stored":356274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.498},"min":{"1min":0.39800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.746,"5min":1.446,"15min":1.446},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55200000000000005,"15min":0.51400000000000001},"min":{"1min":0.44600000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.80200000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46800000000000003}]},{"osd":1,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.50900000000000001},"min":{"1min":0.45300000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65800000000000003,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.52700000000000002},"min":{"1min":0.503,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":0.88,"15min":0.88},"last":0.71499999999999997}]},{"osd":2,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.51100000000000001},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72699999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":1.254,"15min":1.254},"last":0.48999999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58399999999999996,"15min":0.53300000000000003},"min":{"1min":0.40100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.67900000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.56399999999999995},"min":{"1min":0.48099999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72399999999999998,"5min":1.228,"15min":1.228},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46400000000000002,"15min":0.45000000000000001},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.58699999999999997,"5min":1.204,"15min":1.204},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.39300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66400000000000003,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68400000000000005,"5min":0.753,"15min":0.753},"last":0.433},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.42699999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68899999999999995,"5min":1.29,"15min":1.29},"last":0.627}]},{"osd":7,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52500000000000002,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69799999999999995,"5min":1.159,"15min":1.159},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65500000000000003,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14904,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":13504,"kb_avail":94356936,"statfs":{"total":96636764160,"available":96621502464,"internally_reserved":0,"allocated":1425408,"data_stored":958863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13828096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.371,"15min":0.39900000000000002},"min":{"1min":0.28399999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51000000000000001,"5min":1.2,"15min":1.2},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39800000000000002,"15min":0.40899999999999997},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60999999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.36299999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.379,"15min":0.373},"min":{"1min":0.26300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.44400000000000001},"min":{"1min":0.35399999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.62,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.314}]},{"osd":3,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46200000000000002,"15min":0.48199999999999998},"min":{"1min":0.38700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68200000000000005,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.44400000000000001,"15min":0.45400000000000001},"min":{"1min":0.373,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66300000000000003,"5min":1.03,"15min":1.03},"last":0.42499999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60999999999999999,"15min":0.60499999999999998},"min":{"1min":0.48299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.50600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.90800000000000003,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.68100000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.625,"15min":0.629},"min":{"1min":0.5,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83899999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.749},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.51800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95899999999999996,"5min":0.998,"15min":0.998},"last":0.57799999999999996}]},{"osd":6,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64400000000000002,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.55500000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85799999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71599999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64400000000000002,"15min":0.65900000000000003},"min":{"1min":0.54600000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.52600000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92300000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.63700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13776,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94358064,"statfs":{"total":96636764160,"available":96622657536,"internally_reserved":0,"allocated":794624,"data_stored":356240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42999999999999999,"15min":0.53900000000000003},"min":{"1min":0.27900000000000003,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54800000000000004,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42499999999999999,"15min":0.44900000000000001},"min":{"1min":0.27700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56699999999999995,"5min":1.667,"15min":1.667},"last":0.39500000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45800000000000002,"15min":0.55800000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59799999999999998,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.376},{"interface":"front","average":{"1min":0.439,"5min":0.433,"15min":0.438},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59299999999999997,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.44700000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.44400000000000001},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.63700000000000001,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.63100000000000001},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67700000000000005,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.42899999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89100000000000001,"5min":1.536,"15min":1.536},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67600000000000005,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92300000000000004,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91100000000000003,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.443,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84199999999999997,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.76900000000000002},"min":{"1min":0.54500000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65800000000000003,"15min":0.628},"min":{"1min":0.47999999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92900000000000005,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.51400000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96299999999999997,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.66800000000000004,"15min":0.65800000000000003},"min":{"1min":0.49399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90100000000000002,"5min":1.605,"15min":1.605},"last":0.59299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13712,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":12928,"kb_avail":94358128,"statfs":{"total":96636764160,"available":96622723072,"internally_reserved":0,"allocated":794624,"data_stored":356240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13238272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.377,"15min":0.375},"min":{"1min":0.27600000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.61799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.318},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61099999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.41999999999999998},"min":{"1min":0.33200000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.53200000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.28499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56899999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.38700000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42199999999999999,"15min":0.41299999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.372},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.44600000000000001},"min":{"1min":0.31900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.59199999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.51500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90500000000000003,"5min":1.004,"15min":1.004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.48299999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.89000000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71699999999999997}]},{"osd":5,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.52000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84199999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58399999999999996}]},{"osd":6,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.873,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85299999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64500000000000002}]},{"osd":7,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.496,"5min":0.435,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":1.004,"15min":1.004},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14032,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357808,"statfs":{"total":96636764160,"available":96622395392,"internally_reserved":0,"allocated":794624,"data_stored":356240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.246,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.59299999999999997,"5min":0.749,"15min":0.749},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.40300000000000002,"15min":0.434},"min":{"1min":0.29799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48599999999999999,"5min":1.238,"15min":1.238},"last":0.34699999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.40699999999999997,"15min":0.442},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.54800000000000004,"5min":1.53,"15min":1.53},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.45500000000000002,"15min":0.5},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56499999999999995,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46200000000000002,"15min":0.48699999999999999},"min":{"1min":0.33100000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60799999999999998,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.34899999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63400000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.624,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.621},{"interface":"front","average":{"1min":0.623,"5min":0.64200000000000002,"15min":0.65000000000000002},"min":{"1min":0.46600000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88800000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.73199999999999998}]},{"osd":5,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65900000000000003,"15min":0.68200000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92700000000000005,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.876,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.67000000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.45900000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66700000000000004,"15min":0.68100000000000005},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96999999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86499999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.67800000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.68500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9828,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362012,"statfs":{"total":96636764160,"available":96626700288,"internally_reserved":0,"allocated":815104,"data_stored":367747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.505},"min":{"1min":0.435,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68000000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72099999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.65000000000000002}]},{"osd":1,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.751,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55100000000000005,"15min":0.53000000000000003},"min":{"1min":0.35499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74399999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66400000000000003,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56799999999999995,"15min":0.54200000000000004},"min":{"1min":0.45000000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67900000000000005,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.61799999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56799999999999995,"15min":0.53600000000000003},"min":{"1min":0.46300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.752,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.47399999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.63}]},{"osd":5,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.46899999999999997,"15min":0.497},"min":{"1min":0.32900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48199999999999998,"15min":0.48999999999999999},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66000000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.50700000000000001}]},{"osd":6,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46800000000000003,"15min":0.48999999999999999},"min":{"1min":0.33200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.433},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52800000000000002,"15min":0.56399999999999995},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.48399999999999999}]},{"osd":7,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.51000000000000001,"15min":0.53800000000000003},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68400000000000005,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.54300000000000004,"15min":0.56499999999999995},"min":{"1min":0.34100000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67400000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9684,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362156,"statfs":{"total":96636764160,"available":96626847744,"internally_reserved":0,"allocated":798720,"data_stored":356262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.433,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68100000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.47499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72099999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61199999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69899999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.64300000000000002,"15min":0.86499999999999999},"min":{"1min":0.41499999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71599999999999997,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.56499999999999995}]},{"osd":2,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.45600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.47899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.52200000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.748,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.47399999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.75,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.65300000000000002}]},{"osd":4,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.32900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.371},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.499,"15min":0.52500000000000002},"min":{"1min":0.34999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65500000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.495,"15min":0.51400000000000001},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66400000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.624},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71399999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.50900000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.42299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68500000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.58499999999999996},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73499999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:07:39.954 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:07:40.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.088+0000 7fbe9c53d700 1 Processor -- start 2023-11-28T12:07:40.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.088+0000 7fbe9c53d700 1 -- start start 2023-11-28T12:07:40.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.089+0000 7fbe9c53d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe940ce040 0x7fbe9415d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe9c53d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9413fc90 con 0x7fbe94142db0 2023-11-28T12:07:40.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe9c53d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9413f480 con 0x7fbe940b1770 2023-11-28T12:07:40.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe9c53d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9413f6d0 con 0x7fbe940ce040 2023-11-28T12:07:40.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe99ad8700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fbe94142db0 0x7fbe9413ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47304/0 (socket says 172.21.15.57:47304) 2023-11-28T12:07:40.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe99ad8700 1 -- 172.21.15.57:0/2094355614 learned_addr learned my addr 172.21.15.57:0/2094355614 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:40.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe9a2d9700 1 --2- 172.21.15.57:0/2094355614 >> v2:172.21.15.181:3300/0 conn(0x7fbe940ce040 0x7fbe9415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe992d7700 1 -- 172.21.15.57:0/2094355614 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879903051 0 0) 0x7fbe9413fc90 con 0x7fbe94142db0 2023-11-28T12:07:40.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe992d7700 1 -- 172.21.15.57:0/2094355614 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe800035b0 con 0x7fbe94142db0 2023-11-28T12:07:40.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe992d7700 1 -- 172.21.15.57:0/2094355614 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109388177 0 0) 0x7fbe9413f480 con 0x7fbe940b1770 2023-11-28T12:07:40.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe992d7700 1 -- 172.21.15.57:0/2094355614 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe9413fc90 con 0x7fbe940b1770 2023-11-28T12:07:40.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe9a2d9700 1 -- 172.21.15.57:0/2094355614 >> v1:172.21.15.181:6789/0 conn(0x7fbe940b1770 legacy=0x7fbe940b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe9a2d9700 1 -- 172.21.15.57:0/2094355614 >> v1:172.21.15.57:6789/0 conn(0x7fbe94142db0 legacy=0x7fbe9413ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe9a2d9700 1 -- 172.21.15.57:0/2094355614 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9413f9b0 con 0x7fbe940ce040 2023-11-28T12:07:40.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.090+0000 7fbe9a2d9700 1 --2- 172.21.15.57:0/2094355614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940ce040 0x7fbe9415d900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009d90 tx=0x7fbe9000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=758b04369d58dc11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:40.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.091+0000 7fbe992d7700 1 -- 172.21.15.57:0/2094355614 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000e070 con 0x7fbe940ce040 2023-11-28T12:07:40.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.091+0000 7fbe992d7700 1 -- 172.21.15.57:0/2094355614 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9000bb20 con 0x7fbe940ce040 2023-11-28T12:07:40.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.091+0000 7fbe992d7700 1 -- 172.21.15.57:0/2094355614 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90012430 con 0x7fbe940ce040 2023-11-28T12:07:40.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.091+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2094355614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940ce040 msgr2=0x7fbe9415d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.091+0000 7fbe9c53d700 1 --2- 172.21.15.57:0/2094355614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940ce040 0x7fbe9415d900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009d90 tx=0x7fbe9000b330 comp rx=0 tx=0).stop 2023-11-28T12:07:40.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.091+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2094355614 shutdown_connections 2023-11-28T12:07:40.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.091+0000 7fbe9c53d700 1 --2- 172.21.15.57:0/2094355614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940ce040 0x7fbe9415d900 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.091+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2094355614 >> 172.21.15.57:0/2094355614 conn(0x7fbe9406c010 msgr2=0x7fbe940cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:40.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.091+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2094355614 shutdown_connections 2023-11-28T12:07:40.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.091+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2094355614 wait complete. 2023-11-28T12:07:40.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.093+0000 7fbe9c53d700 1 Processor -- start 2023-11-28T12:07:40.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.093+0000 7fbe9c53d700 1 -- start start 2023-11-28T12:07:40.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.093+0000 7fbe9c53d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940b1770 0x7fbe940c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.093+0000 7fbe9c53d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe940ce040 0x7fbe940c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.093+0000 7fbe9c53d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe94142db0 0x7fbe940b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.093+0000 7fbe9c53d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940c7c90 con 0x7fbe940ce040 2023-11-28T12:07:40.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.093+0000 7fbe9c53d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941554a0 con 0x7fbe940b1770 2023-11-28T12:07:40.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.093+0000 7fbe9c53d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe94155740 con 0x7fbe94142db0 2023-11-28T12:07:40.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe99ad8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe940ce040 0x7fbe940c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe99ad8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe940ce040 0x7fbe940c5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59132/0 (socket says 172.21.15.57:59132) 2023-11-28T12:07:40.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe99ad8700 1 -- 172.21.15.57:0/2519786037 learned_addr learned my addr 172.21.15.57:0/2519786037 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:40.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe9a2d9700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940b1770 0x7fbe940c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe9aada700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe94142db0 0x7fbe940b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe99ad8700 1 -- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe94142db0 msgr2=0x7fbe940b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe99ad8700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe94142db0 0x7fbe940b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe99ad8700 1 -- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940b1770 msgr2=0x7fbe940c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe99ad8700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940b1770 0x7fbe940c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe99ad8700 1 -- 172.21.15.57:0/2519786037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe90009a70 con 0x7fbe940ce040 2023-11-28T12:07:40.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe9aada700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe94142db0 0x7fbe940b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:40.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe99ad8700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe940ce040 0x7fbe940c5190 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400c700 tx=0x7fbe8400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:40.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe8affd700 1 -- 172.21.15.57:0/2519786037 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84005d70 con 0x7fbe940ce040 2023-11-28T12:07:40.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe8affd700 1 -- 172.21.15.57:0/2519786037 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe840070d0 con 0x7fbe940ce040 2023-11-28T12:07:40.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe8affd700 1 -- 172.21.15.57:0/2519786037 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84007480 con 0x7fbe940ce040 2023-11-28T12:07:40.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2519786037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9415d3e0 con 0x7fbe940ce040 2023-11-28T12:07:40.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.094+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2519786037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9415d100 con 0x7fbe940ce040 2023-11-28T12:07:40.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.095+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2519786037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe940ce410 con 0x7fbe940ce040 2023-11-28T12:07:40.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.096+0000 7fbe8affd700 1 -- 172.21.15.57:0/2519786037 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbe84019030 con 0x7fbe940ce040 2023-11-28T12:07:40.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.096+0000 7fbe8affd700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbe780682f0 0x7fbe7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.096+0000 7fbe8affd700 1 -- 172.21.15.57:0/2519786037 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbe84084ef0 con 0x7fbe940ce040 2023-11-28T12:07:40.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.096+0000 7fbe9a2d9700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbe780682f0 0x7fbe7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.097+0000 7fbe9a2d9700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbe780682f0 0x7fbe7806a7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbe90014040 tx=0x7fbe9001d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:40.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.101+0000 7fbe8affd700 1 -- 172.21.15.57:0/2519786037 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbe840519d0 con 0x7fbe940ce040 2023-11-28T12:07:40.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.293+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2519786037 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe940743f0 con 0x7fbe780682f0 2023-11-28T12:07:40.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.297+0000 7fbe8affd700 1 -- 172.21.15.57:0/2519786037 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73587 (secure 0 0 0) 0x7fbe940743f0 con 0x7fbe780682f0 2023-11-28T12:07:40.299 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:40.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.299+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbe780682f0 msgr2=0x7fbe7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.299+0000 7fbe9c53d700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbe780682f0 0x7fbe7806a7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbe90014040 tx=0x7fbe9001d040 comp rx=0 tx=0).stop 2023-11-28T12:07:40.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.299+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2519786037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe940ce040 msgr2=0x7fbe940c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.299+0000 7fbe9c53d700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe940ce040 0x7fbe940c5190 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400c700 tx=0x7fbe8400cac0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.299+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2519786037 shutdown_connections 2023-11-28T12:07:40.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.299+0000 7fbe9c53d700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fbe780682f0 0x7fbe7806a7a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.299+0000 7fbe9c53d700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe94142db0 0x7fbe940b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.299+0000 7fbe9c53d700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940b1770 0x7fbe940c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.299+0000 7fbe9c53d700 1 --2- 172.21.15.57:0/2519786037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbe940ce040 0x7fbe940c5190 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.299+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2519786037 >> 172.21.15.57:0/2519786037 conn(0x7fbe9406c010 msgr2=0x7fbe9406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:40.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.299+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2519786037 shutdown_connections 2023-11-28T12:07:40.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.299+0000 7fbe9c53d700 1 -- 172.21.15.57:0/2519786037 wait complete. 2023-11-28T12:07:40.302 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:07:40.314 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2023-11-28T12:07:39.751638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":96408,"kb_used_data":7448,"kb_used_omap":0,"kb_used_meta":88896,"kb_avail":754878312,"statfs":{"total":773094113280,"available":772995391488,"internally_reserved":0,"allocated":7626752,"data_stored":4055234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91029504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12289,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-74,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.054812"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":59,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.699534+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:07:33.699534+0000","last_peered":"2023-11-28T12:07:33.699534+0000","last_clean":"2023-11-28T12:07:33.699534+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:07:33.699534+0000","last_undegraded":"2023-11-28T12:07:33.699534+0000","last_fullsized":"2023-11-28T12:07:33.699534+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.697778+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:07:33.697778+0000","last_peered":"2023-11-28T12:07:33.697778+0000","last_clean":"2023-11-28T12:07:33.697778+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:07:33.697778+0000","last_undegraded":"2023-11-28T12:07:33.697778+0000","last_fullsized":"2023-11-28T12:07:33.697778+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":57,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.770443+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:07:33.770443+0000","last_peered":"2023-11-28T12:07:33.770443+0000","last_clean":"2023-11-28T12:07:33.770443+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:07:33.770443+0000","last_undegraded":"2023-11-28T12:07:33.770443+0000","last_fullsized":"2023-11-28T12:07:33.770443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":202,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T12:05:41.751905+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:05:41.751905+0000","last_peered":"2023-11-28T12:05:41.751905+0000","last_clean":"2023-11-28T12:05:41.751905+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:05:41.751905+0000","last_undegraded":"2023-11-28T12:05:41.751905+0000","last_fullsized":"2023-11-28T12:05:41.751905+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T12:05:51.753233+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:05:51.753233+0000","last_peered":"2023-11-28T12:05:51.753233+0000","last_clean":"2023-11-28T12:05:51.753233+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:05:51.753233+0000","last_undegraded":"2023-11-28T12:05:51.753233+0000","last_fullsized":"2023-11-28T12:05:51.753233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T12:05:41.751958+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:05:41.751958+0000","last_peered":"2023-11-28T12:05:41.751958+0000","last_clean":"2023-11-28T12:05:41.751958+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:05:41.751958+0000","last_undegraded":"2023-11-28T12:05:41.751958+0000","last_fullsized":"2023-11-28T12:05:41.751958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T12:05:51.778799+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:05:51.778799+0000","last_peered":"2023-11-28T12:05:51.778799+0000","last_clean":"2023-11-28T12:05:51.778799+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:05:51.778799+0000","last_undegraded":"2023-11-28T12:05:51.778799+0000","last_fullsized":"2023-11-28T12:05:51.778799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.699894+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:07:33.699894+0000","last_peered":"2023-11-28T12:07:33.699894+0000","last_clean":"2023-11-28T12:07:33.699894+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:07:33.699894+0000","last_undegraded":"2023-11-28T12:07:33.699894+0000","last_fullsized":"2023-11-28T12:07:33.699894+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.700627+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:07:33.700627+0000","last_peered":"2023-11-28T12:07:33.700627+0000","last_clean":"2023-11-28T12:07:33.700627+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:07:33.700627+0000","last_undegraded":"2023-11-28T12:07:33.700627+0000","last_fullsized":"2023-11-28T12:07:33.700627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10980,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360860,"statfs":{"total":96636764160,"available":96625520640,"internally_reserved":0,"allocated":1404928,"data_stored":947368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.42099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71599999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.50700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77400000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.45000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.746,"5min":0.872,"15min":0.872},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.80700000000000005},"min":{"1min":0.48799999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.745,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63300000000000001,"15min":0.79200000000000004},"min":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72699999999999998,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.44800000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67200000000000004}]},{"osd":3,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.47699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.75900000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60999999999999999,"15min":0.66400000000000003},"min":{"1min":0.46200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67300000000000004,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.60099999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47399999999999998,"15min":0.499},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.56599999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47999999999999998,"15min":0.48799999999999999},"min":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68500000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.622}]},{"osd":5,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.497,"15min":0.51600000000000001},"min":{"1min":0.34599999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.59399999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52300000000000002,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65700000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.61199999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54900000000000004,"15min":0.58099999999999996},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.69599999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54600000000000004,"15min":0.57399999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.65000000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9492,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362348,"statfs":{"total":96636764160,"available":96627044352,"internally_reserved":0,"allocated":798720,"data_stored":356274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.498},"min":{"1min":0.39800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.746,"5min":1.446,"15min":1.446},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55200000000000005,"15min":0.51400000000000001},"min":{"1min":0.44600000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.80200000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46800000000000003}]},{"osd":1,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.50900000000000001},"min":{"1min":0.45300000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65800000000000003,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.52700000000000002},"min":{"1min":0.503,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":0.88,"15min":0.88},"last":0.71499999999999997}]},{"osd":2,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.51100000000000001},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72699999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":1.254,"15min":1.254},"last":0.48999999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58399999999999996,"15min":0.53300000000000003},"min":{"1min":0.40100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.67900000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.56399999999999995},"min":{"1min":0.48099999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72399999999999998,"5min":1.228,"15min":1.228},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46400000000000002,"15min":0.45000000000000001},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.58699999999999997,"5min":1.204,"15min":1.204},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.39300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66400000000000003,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68400000000000005,"5min":0.753,"15min":0.753},"last":0.433},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.42699999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68899999999999995,"5min":1.29,"15min":1.29},"last":0.627}]},{"osd":7,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52500000000000002,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69799999999999995,"5min":1.159,"15min":1.159},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65500000000000003,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14904,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":13504,"kb_avail":94356936,"statfs":{"total":96636764160,"available":96621502464,"internally_reserved":0,"allocated":1425408,"data_stored":958863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13828096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.371,"15min":0.39900000000000002},"min":{"1min":0.28399999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51000000000000001,"5min":1.2,"15min":1.2},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39800000000000002,"15min":0.40899999999999997},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60999999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.36299999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.379,"15min":0.373},"min":{"1min":0.26300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.44400000000000001},"min":{"1min":0.35399999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.62,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.314}]},{"osd":3,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46200000000000002,"15min":0.48199999999999998},"min":{"1min":0.38700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68200000000000005,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.44400000000000001,"15min":0.45400000000000001},"min":{"1min":0.373,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66300000000000003,"5min":1.03,"15min":1.03},"last":0.42499999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60999999999999999,"15min":0.60499999999999998},"min":{"1min":0.48299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.50600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.90800000000000003,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.68100000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.625,"15min":0.629},"min":{"1min":0.5,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83899999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.749},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.51800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95899999999999996,"5min":0.998,"15min":0.998},"last":0.57799999999999996}]},{"osd":6,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64400000000000002,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.55500000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85799999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71599999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64400000000000002,"15min":0.65900000000000003},"min":{"1min":0.54600000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.52600000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92300000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.63700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13776,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94358064,"statfs":{"total":96636764160,"available":96622657536,"internally_reserved":0,"allocated":794624,"data_stored":356240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42999999999999999,"15min":0.53900000000000003},"min":{"1min":0.27900000000000003,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54800000000000004,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42499999999999999,"15min":0.44900000000000001},"min":{"1min":0.27700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56699999999999995,"5min":1.667,"15min":1.667},"last":0.39500000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45800000000000002,"15min":0.55800000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59799999999999998,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.376},{"interface":"front","average":{"1min":0.439,"5min":0.433,"15min":0.438},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59299999999999997,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.44700000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.44400000000000001},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.63700000000000001,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.63100000000000001},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67700000000000005,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.42899999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89100000000000001,"5min":1.536,"15min":1.536},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67600000000000005,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92300000000000004,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91100000000000003,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.443,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84199999999999997,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.76900000000000002},"min":{"1min":0.54500000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65800000000000003,"15min":0.628},"min":{"1min":0.47999999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92900000000000005,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.51400000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96299999999999997,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.66800000000000004,"15min":0.65800000000000003},"min":{"1min":0.49399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90100000000000002,"5min":1.605,"15min":1.605},"last":0.59299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13712,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":12928,"kb_avail":94358128,"statfs":{"total":96636764160,"available":96622723072,"internally_reserved":0,"allocated":794624,"data_stored":356240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13238272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.377,"15min":0.375},"min":{"1min":0.27600000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.61799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.318},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61099999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.41999999999999998},"min":{"1min":0.33200000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.53200000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.28499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56899999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.38700000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42199999999999999,"15min":0.41299999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.372},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.44600000000000001},"min":{"1min":0.31900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.59199999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.51500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90500000000000003,"5min":1.004,"15min":1.004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.48299999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.89000000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71699999999999997}]},{"osd":5,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.52000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84199999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58399999999999996}]},{"osd":6,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.873,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85299999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64500000000000002}]},{"osd":7,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.496,"5min":0.435,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":1.004,"15min":1.004},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14032,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357808,"statfs":{"total":96636764160,"available":96622395392,"internally_reserved":0,"allocated":794624,"data_stored":356240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.246,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.59299999999999997,"5min":0.749,"15min":0.749},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.40300000000000002,"15min":0.434},"min":{"1min":0.29799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48599999999999999,"5min":1.238,"15min":1.238},"last":0.34699999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.40699999999999997,"15min":0.442},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.54800000000000004,"5min":1.53,"15min":1.53},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.45500000000000002,"15min":0.5},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56499999999999995,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46200000000000002,"15min":0.48699999999999999},"min":{"1min":0.33100000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60799999999999998,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.34899999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63400000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.624,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.621},{"interface":"front","average":{"1min":0.623,"5min":0.64200000000000002,"15min":0.65000000000000002},"min":{"1min":0.46600000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88800000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.73199999999999998}]},{"osd":5,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65900000000000003,"15min":0.68200000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92700000000000005,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.876,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.67000000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.45900000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66700000000000004,"15min":0.68100000000000005},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96999999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86499999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.67800000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.68500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9828,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362012,"statfs":{"total":96636764160,"available":96626700288,"internally_reserved":0,"allocated":815104,"data_stored":367747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.505},"min":{"1min":0.435,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68000000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72099999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.65000000000000002}]},{"osd":1,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.751,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55100000000000005,"15min":0.53000000000000003},"min":{"1min":0.35499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74399999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66400000000000003,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56799999999999995,"15min":0.54200000000000004},"min":{"1min":0.45000000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67900000000000005,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.61799999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56799999999999995,"15min":0.53600000000000003},"min":{"1min":0.46300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.752,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.47399999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.63}]},{"osd":5,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.46899999999999997,"15min":0.497},"min":{"1min":0.32900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48199999999999998,"15min":0.48999999999999999},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66000000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.50700000000000001}]},{"osd":6,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46800000000000003,"15min":0.48999999999999999},"min":{"1min":0.33200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.433},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52800000000000002,"15min":0.56399999999999995},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.48399999999999999}]},{"osd":7,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.51000000000000001,"15min":0.53800000000000003},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68400000000000005,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.54300000000000004,"15min":0.56499999999999995},"min":{"1min":0.34100000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67400000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9684,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362156,"statfs":{"total":96636764160,"available":96626847744,"internally_reserved":0,"allocated":798720,"data_stored":356262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.433,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68100000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.47499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72099999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61199999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69899999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.64300000000000002,"15min":0.86499999999999999},"min":{"1min":0.41499999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71599999999999997,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.56499999999999995}]},{"osd":2,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.45600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.47899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.52200000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.748,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.47399999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.75,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.65300000000000002}]},{"osd":4,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.32900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.371},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.499,"15min":0.52500000000000002},"min":{"1min":0.34999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65500000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.495,"15min":0.51400000000000001},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66400000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.624},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71399999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.50900000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.42299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68500000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.58499999999999996},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73499999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:07:40.315 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T12:07:40.316 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:07:40.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.451+0000 7f95a135b700 1 Processor -- start 2023-11-28T12:07:40.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.451+0000 7f95a135b700 1 -- start start 2023-11-28T12:07:40.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.452+0000 7f95a135b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f959c06ab10 0x7f959c06aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.452+0000 7f95a135b700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959c0d1ee0 con 0x7f959c0d0450 2023-11-28T12:07:40.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.452+0000 7f95a135b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959c161120 con 0x7f959c0cd5a0 2023-11-28T12:07:40.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.452+0000 7f95a135b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959c06b420 con 0x7f959c06ab10 2023-11-28T12:07:40.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.452+0000 7f959affd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f959c06ab10 0x7f959c06aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.452+0000 7f959b7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f959c0d0450 0x7f959c160a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47316/0 (socket says 172.21.15.57:47316) 2023-11-28T12:07:40.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.452+0000 7f959b7fe700 1 -- 172.21.15.57:0/3073016099 learned_addr learned my addr 172.21.15.57:0/3073016099 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:40.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.453+0000 7f959affd700 1 -- 172.21.15.57:0/3073016099 >> v1:172.21.15.181:6789/0 conn(0x7f959c0cd5a0 legacy=0x7f959c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.453+0000 7f959affd700 1 -- 172.21.15.57:0/3073016099 >> v1:172.21.15.57:6789/0 conn(0x7f959c0d0450 legacy=0x7f959c160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.453+0000 7f959affd700 1 -- 172.21.15.57:0/3073016099 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c0cdb10 con 0x7f959c06ab10 2023-11-28T12:07:40.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.453+0000 7f9599ffb700 1 -- 172.21.15.57:0/3073016099 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668856899 0 0) 0x7f959c0d1ee0 con 0x7f959c0d0450 2023-11-28T12:07:40.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.453+0000 7f959affd700 1 --2- 172.21.15.57:0/3073016099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f959c06ab10 0x7f959c06aee0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f958c009840 tx=0x7f958c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aeb49bcaec06b7e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:40.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.453+0000 7f9599ffb700 1 -- 172.21.15.57:0/3073016099 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c011830 con 0x7f959c06ab10 2023-11-28T12:07:40.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.453+0000 7f9599ffb700 1 -- 172.21.15.57:0/3073016099 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c011990 con 0x7f959c06ab10 2023-11-28T12:07:40.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.453+0000 7f9599ffb700 1 -- 172.21.15.57:0/3073016099 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c011d20 con 0x7f959c06ab10 2023-11-28T12:07:40.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.454+0000 7f95a135b700 1 -- 172.21.15.57:0/3073016099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f959c06ab10 msgr2=0x7f959c06aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.454+0000 7f95a135b700 1 --2- 172.21.15.57:0/3073016099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f959c06ab10 0x7f959c06aee0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f958c009840 tx=0x7f958c009b50 comp rx=0 tx=0).stop 2023-11-28T12:07:40.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.454+0000 7f95a135b700 1 -- 172.21.15.57:0/3073016099 shutdown_connections 2023-11-28T12:07:40.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.454+0000 7f95a135b700 1 --2- 172.21.15.57:0/3073016099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f959c06ab10 0x7f959c06aee0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.454+0000 7f95a135b700 1 -- 172.21.15.57:0/3073016099 >> 172.21.15.57:0/3073016099 conn(0x7f959c06c520 msgr2=0x7f959c06c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:40.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.454+0000 7f95a135b700 1 -- 172.21.15.57:0/3073016099 shutdown_connections 2023-11-28T12:07:40.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.454+0000 7f95a135b700 1 -- 172.21.15.57:0/3073016099 wait complete. 2023-11-28T12:07:40.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.455+0000 7f95a135b700 1 Processor -- start 2023-11-28T12:07:40.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.455+0000 7f95a135b700 1 -- start start 2023-11-28T12:07:40.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f95a135b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f959c06ab10 0x7f959c1f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f95a135b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f959c0cd5a0 0x7f959c0ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f95a135b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f959c0d0450 0x7f959c0b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f95a135b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0cb5f0 con 0x7f959c0cd5a0 2023-11-28T12:07:40.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f95a135b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0caa70 con 0x7f959c06ab10 2023-11-28T12:07:40.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f95a135b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0cad50 con 0x7f959c0d0450 2023-11-28T12:07:40.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f959a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f959c0cd5a0 0x7f959c0ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f959a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f959c0cd5a0 0x7f959c0ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59146/0 (socket says 172.21.15.57:59146) 2023-11-28T12:07:40.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f959a7fc700 1 -- 172.21.15.57:0/2123967421 learned_addr learned my addr 172.21.15.57:0/2123967421 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:40.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f959b7fe700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f959c0d0450 0x7f959c0b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f959affd700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f959c06ab10 0x7f959c1f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f959a7fc700 1 -- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f959c0d0450 msgr2=0x7f959c0b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f959a7fc700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f959c0d0450 0x7f959c0b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f959a7fc700 1 -- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f959c06ab10 msgr2=0x7f959c1f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f959a7fc700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f959c06ab10 0x7f959c1f51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.456+0000 7f959a7fc700 1 -- 172.21.15.57:0/2123967421 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c0094a0 con 0x7f959c0cd5a0 2023-11-28T12:07:40.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.457+0000 7f959a7fc700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f959c0cd5a0 0x7f959c0ca020 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f958400c4b0 tx=0x7f958400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:40.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.457+0000 7f95937fe700 1 -- 172.21.15.57:0/2123967421 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95840052d0 con 0x7f959c0cd5a0 2023-11-28T12:07:40.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.457+0000 7f95937fe700 1 -- 172.21.15.57:0/2123967421 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9584005430 con 0x7f959c0cd5a0 2023-11-28T12:07:40.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.457+0000 7f95a135b700 1 -- 172.21.15.57:0/2123967421 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959c0d1b70 con 0x7f959c0cd5a0 2023-11-28T12:07:40.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.457+0000 7f95937fe700 1 -- 172.21.15.57:0/2123967421 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95840038c0 con 0x7f959c0cd5a0 2023-11-28T12:07:40.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.457+0000 7f95a135b700 1 -- 172.21.15.57:0/2123967421 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959c0d1890 con 0x7f959c0cd5a0 2023-11-28T12:07:40.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.458+0000 7f95a135b700 1 -- 172.21.15.57:0/2123967421 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959c15cad0 con 0x7f959c0cd5a0 2023-11-28T12:07:40.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.458+0000 7f95937fe700 1 -- 172.21.15.57:0/2123967421 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9584019030 con 0x7f959c0cd5a0 2023-11-28T12:07:40.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.459+0000 7f95937fe700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f957c0682f0 0x7f957c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.459+0000 7f95937fe700 1 -- 172.21.15.57:0/2123967421 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9584084cf0 con 0x7f959c0cd5a0 2023-11-28T12:07:40.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.459+0000 7f959affd700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f957c0682f0 0x7f957c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.459+0000 7f959affd700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f957c0682f0 0x7f957c06a7a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f958c000c00 tx=0x7f958c01c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:40.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.463+0000 7f95937fe700 1 -- 172.21.15.57:0/2123967421 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9584051850 con 0x7f959c0cd5a0 2023-11-28T12:07:40.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.659+0000 7f95a135b700 1 -- 172.21.15.57:0/2123967421 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f959c074490 con 0x7f957c0682f0 2023-11-28T12:07:40.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.662+0000 7f95937fe700 1 -- 172.21.15.57:0/2123967421 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73587 (secure 0 0 0) 0x7f959c074490 con 0x7f957c0682f0 2023-11-28T12:07:40.664 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:40.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.664+0000 7f95a135b700 1 -- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f957c0682f0 msgr2=0x7f957c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.664+0000 7f95a135b700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f957c0682f0 0x7f957c06a7a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f958c000c00 tx=0x7f958c01c040 comp rx=0 tx=0).stop 2023-11-28T12:07:40.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.664+0000 7f95a135b700 1 -- 172.21.15.57:0/2123967421 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f959c0cd5a0 msgr2=0x7f959c0ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.664+0000 7f95a135b700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f959c0cd5a0 0x7f959c0ca020 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f958400c4b0 tx=0x7f958400c870 comp rx=0 tx=0).stop 2023-11-28T12:07:40.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.664+0000 7f95a135b700 1 -- 172.21.15.57:0/2123967421 shutdown_connections 2023-11-28T12:07:40.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.664+0000 7f95a135b700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f957c0682f0 0x7f957c06a7a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.664+0000 7f95a135b700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f959c0d0450 0x7f959c0b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.664+0000 7f95a135b700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f959c06ab10 0x7f959c1f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.664+0000 7f95a135b700 1 --2- 172.21.15.57:0/2123967421 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f959c0cd5a0 0x7f959c0ca020 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.664+0000 7f95a135b700 1 -- 172.21.15.57:0/2123967421 >> 172.21.15.57:0/2123967421 conn(0x7f959c06c520 msgr2=0x7f959c0b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:40.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.665+0000 7f95a135b700 1 -- 172.21.15.57:0/2123967421 shutdown_connections 2023-11-28T12:07:40.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.665+0000 7f95a135b700 1 -- 172.21.15.57:0/2123967421 wait complete. 2023-11-28T12:07:40.668 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:07:40.679 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2023-11-28T12:07:39.751638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":96408,"kb_used_data":7448,"kb_used_omap":0,"kb_used_meta":88896,"kb_avail":754878312,"statfs":{"total":773094113280,"available":772995391488,"internally_reserved":0,"allocated":7626752,"data_stored":4055234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91029504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12289,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-74,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.054812"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":59,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.699534+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:07:33.699534+0000","last_peered":"2023-11-28T12:07:33.699534+0000","last_clean":"2023-11-28T12:07:33.699534+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:07:33.699534+0000","last_undegraded":"2023-11-28T12:07:33.699534+0000","last_fullsized":"2023-11-28T12:07:33.699534+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.697778+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:07:33.697778+0000","last_peered":"2023-11-28T12:07:33.697778+0000","last_clean":"2023-11-28T12:07:33.697778+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:07:33.697778+0000","last_undegraded":"2023-11-28T12:07:33.697778+0000","last_fullsized":"2023-11-28T12:07:33.697778+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":57,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.770443+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:07:33.770443+0000","last_peered":"2023-11-28T12:07:33.770443+0000","last_clean":"2023-11-28T12:07:33.770443+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:07:33.770443+0000","last_undegraded":"2023-11-28T12:07:33.770443+0000","last_fullsized":"2023-11-28T12:07:33.770443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":202,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T12:05:41.751905+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:05:41.751905+0000","last_peered":"2023-11-28T12:05:41.751905+0000","last_clean":"2023-11-28T12:05:41.751905+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:05:41.751905+0000","last_undegraded":"2023-11-28T12:05:41.751905+0000","last_fullsized":"2023-11-28T12:05:41.751905+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T12:05:51.753233+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:05:51.753233+0000","last_peered":"2023-11-28T12:05:51.753233+0000","last_clean":"2023-11-28T12:05:51.753233+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:05:51.753233+0000","last_undegraded":"2023-11-28T12:05:51.753233+0000","last_fullsized":"2023-11-28T12:05:51.753233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T12:05:41.751958+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:05:41.751958+0000","last_peered":"2023-11-28T12:05:41.751958+0000","last_clean":"2023-11-28T12:05:41.751958+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:05:41.751958+0000","last_undegraded":"2023-11-28T12:05:41.751958+0000","last_fullsized":"2023-11-28T12:05:41.751958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T12:05:51.778799+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:05:51.778799+0000","last_peered":"2023-11-28T12:05:51.778799+0000","last_clean":"2023-11-28T12:05:51.778799+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:05:51.778799+0000","last_undegraded":"2023-11-28T12:05:51.778799+0000","last_fullsized":"2023-11-28T12:05:51.778799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.699894+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:07:33.699894+0000","last_peered":"2023-11-28T12:07:33.699894+0000","last_clean":"2023-11-28T12:07:33.699894+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:07:33.699894+0000","last_undegraded":"2023-11-28T12:07:33.699894+0000","last_fullsized":"2023-11-28T12:07:33.699894+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T12:07:33.700627+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:07:33.700627+0000","last_peered":"2023-11-28T12:07:33.700627+0000","last_clean":"2023-11-28T12:07:33.700627+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:07:33.700627+0000","last_undegraded":"2023-11-28T12:07:33.700627+0000","last_fullsized":"2023-11-28T12:07:33.700627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10980,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360860,"statfs":{"total":96636764160,"available":96625520640,"internally_reserved":0,"allocated":1404928,"data_stored":947368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.42099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71599999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.50700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77400000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.45000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.746,"5min":0.872,"15min":0.872},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.80700000000000005},"min":{"1min":0.48799999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.745,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63300000000000001,"15min":0.79200000000000004},"min":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72699999999999998,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.44800000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67200000000000004}]},{"osd":3,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.47699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.75900000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60999999999999999,"15min":0.66400000000000003},"min":{"1min":0.46200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67300000000000004,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.60099999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47399999999999998,"15min":0.499},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.56599999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47999999999999998,"15min":0.48799999999999999},"min":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68500000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.622}]},{"osd":5,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.497,"15min":0.51600000000000001},"min":{"1min":0.34599999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.59399999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52300000000000002,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65700000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.61199999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54900000000000004,"15min":0.58099999999999996},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.69599999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54600000000000004,"15min":0.57399999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.65000000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9492,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362348,"statfs":{"total":96636764160,"available":96627044352,"internally_reserved":0,"allocated":798720,"data_stored":356274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.498},"min":{"1min":0.39800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.746,"5min":1.446,"15min":1.446},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55200000000000005,"15min":0.51400000000000001},"min":{"1min":0.44600000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.80200000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46800000000000003}]},{"osd":1,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.50900000000000001},"min":{"1min":0.45300000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65800000000000003,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.52700000000000002},"min":{"1min":0.503,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":0.88,"15min":0.88},"last":0.71499999999999997}]},{"osd":2,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.51100000000000001},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72699999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":1.254,"15min":1.254},"last":0.48999999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58399999999999996,"15min":0.53300000000000003},"min":{"1min":0.40100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.67900000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.56399999999999995},"min":{"1min":0.48099999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72399999999999998,"5min":1.228,"15min":1.228},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46400000000000002,"15min":0.45000000000000001},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.58699999999999997,"5min":1.204,"15min":1.204},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.39300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66400000000000003,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68400000000000005,"5min":0.753,"15min":0.753},"last":0.433},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.42699999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68899999999999995,"5min":1.29,"15min":1.29},"last":0.627}]},{"osd":7,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52500000000000002,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69799999999999995,"5min":1.159,"15min":1.159},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65500000000000003,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14904,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":13504,"kb_avail":94356936,"statfs":{"total":96636764160,"available":96621502464,"internally_reserved":0,"allocated":1425408,"data_stored":958863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13828096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.371,"15min":0.39900000000000002},"min":{"1min":0.28399999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51000000000000001,"5min":1.2,"15min":1.2},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39800000000000002,"15min":0.40899999999999997},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60999999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.36299999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.379,"15min":0.373},"min":{"1min":0.26300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.44400000000000001},"min":{"1min":0.35399999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.62,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.314}]},{"osd":3,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46200000000000002,"15min":0.48199999999999998},"min":{"1min":0.38700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68200000000000005,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.44400000000000001,"15min":0.45400000000000001},"min":{"1min":0.373,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66300000000000003,"5min":1.03,"15min":1.03},"last":0.42499999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60999999999999999,"15min":0.60499999999999998},"min":{"1min":0.48299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.50600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.90800000000000003,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.68100000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.625,"15min":0.629},"min":{"1min":0.5,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83899999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.749},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.51800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95899999999999996,"5min":0.998,"15min":0.998},"last":0.57799999999999996}]},{"osd":6,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64400000000000002,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.55500000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85799999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71599999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64400000000000002,"15min":0.65900000000000003},"min":{"1min":0.54600000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.52600000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92300000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.63700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13776,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94358064,"statfs":{"total":96636764160,"available":96622657536,"internally_reserved":0,"allocated":794624,"data_stored":356240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42999999999999999,"15min":0.53900000000000003},"min":{"1min":0.27900000000000003,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54800000000000004,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42499999999999999,"15min":0.44900000000000001},"min":{"1min":0.27700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56699999999999995,"5min":1.667,"15min":1.667},"last":0.39500000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45800000000000002,"15min":0.55800000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59799999999999998,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.376},{"interface":"front","average":{"1min":0.439,"5min":0.433,"15min":0.438},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59299999999999997,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.44700000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.44400000000000001},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.63700000000000001,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.63100000000000001},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67700000000000005,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.42899999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89100000000000001,"5min":1.536,"15min":1.536},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67600000000000005,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92300000000000004,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91100000000000003,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.443,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84199999999999997,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.76900000000000002},"min":{"1min":0.54500000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65800000000000003,"15min":0.628},"min":{"1min":0.47999999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92900000000000005,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.51400000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96299999999999997,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.66800000000000004,"15min":0.65800000000000003},"min":{"1min":0.49399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90100000000000002,"5min":1.605,"15min":1.605},"last":0.59299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13712,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":12928,"kb_avail":94358128,"statfs":{"total":96636764160,"available":96622723072,"internally_reserved":0,"allocated":794624,"data_stored":356240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13238272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.377,"15min":0.375},"min":{"1min":0.27600000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.61799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.318},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61099999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.41999999999999998},"min":{"1min":0.33200000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.53200000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.28499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56899999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.38700000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42199999999999999,"15min":0.41299999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.372},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.44600000000000001},"min":{"1min":0.31900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.59199999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.51500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90500000000000003,"5min":1.004,"15min":1.004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.48299999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.89000000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71699999999999997}]},{"osd":5,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.52000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84199999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58399999999999996}]},{"osd":6,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.873,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85299999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64500000000000002}]},{"osd":7,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.496,"5min":0.435,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":1.004,"15min":1.004},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14032,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357808,"statfs":{"total":96636764160,"available":96622395392,"internally_reserved":0,"allocated":794624,"data_stored":356240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.246,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.59299999999999997,"5min":0.749,"15min":0.749},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.40300000000000002,"15min":0.434},"min":{"1min":0.29799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48599999999999999,"5min":1.238,"15min":1.238},"last":0.34699999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.40699999999999997,"15min":0.442},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.54800000000000004,"5min":1.53,"15min":1.53},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.45500000000000002,"15min":0.5},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56499999999999995,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46200000000000002,"15min":0.48699999999999999},"min":{"1min":0.33100000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60799999999999998,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.34899999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63400000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.624,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.621},{"interface":"front","average":{"1min":0.623,"5min":0.64200000000000002,"15min":0.65000000000000002},"min":{"1min":0.46600000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88800000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.73199999999999998}]},{"osd":5,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65900000000000003,"15min":0.68200000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92700000000000005,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.876,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.67000000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.45900000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66700000000000004,"15min":0.68100000000000005},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96999999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Tue Nov 28 12:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86499999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.67800000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.68500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9828,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362012,"statfs":{"total":96636764160,"available":96626700288,"internally_reserved":0,"allocated":815104,"data_stored":367747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.505},"min":{"1min":0.435,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68000000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72099999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.65000000000000002}]},{"osd":1,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.751,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55100000000000005,"15min":0.53000000000000003},"min":{"1min":0.35499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74399999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66400000000000003,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56799999999999995,"15min":0.54200000000000004},"min":{"1min":0.45000000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67900000000000005,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.61799999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56799999999999995,"15min":0.53600000000000003},"min":{"1min":0.46300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.752,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.47399999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.63}]},{"osd":5,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.46899999999999997,"15min":0.497},"min":{"1min":0.32900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48199999999999998,"15min":0.48999999999999999},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66000000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.50700000000000001}]},{"osd":6,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46800000000000003,"15min":0.48999999999999999},"min":{"1min":0.33200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.433},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52800000000000002,"15min":0.56399999999999995},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.48399999999999999}]},{"osd":7,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.51000000000000001,"15min":0.53800000000000003},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68400000000000005,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.54300000000000004,"15min":0.56499999999999995},"min":{"1min":0.34100000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67400000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9684,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362156,"statfs":{"total":96636764160,"available":96626847744,"internally_reserved":0,"allocated":798720,"data_stored":356262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.433,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68100000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.47499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72099999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61199999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69899999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.64300000000000002,"15min":0.86499999999999999},"min":{"1min":0.41499999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71599999999999997,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.56499999999999995}]},{"osd":2,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.45600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.47899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.52200000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.748,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.47399999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.75,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.65300000000000002}]},{"osd":4,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.32900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.371},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.499,"15min":0.52500000000000002},"min":{"1min":0.34999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65500000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.495,"15min":0.51400000000000001},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66400000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.624},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71399999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.50900000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.42299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68500000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.58499999999999996},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73499999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:07:40.681 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:07:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.817+0000 7f20c60ac700 1 Processor -- start 2023-11-28T12:07:40.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.817+0000 7f20c60ac700 1 -- start start 2023-11-28T12:07:40.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20c60ac700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f20c006e400 0x7f20c006e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20c60ac700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f20c015e010 0x7f20c00ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20c60ac700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c015af30 con 0x7f20c00d15f0 2023-11-28T12:07:40.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20c60ac700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c015a9d0 con 0x7f20c006e400 2023-11-28T12:07:40.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20c60ac700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c01584e0 con 0x7f20c015e010 2023-11-28T12:07:40.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20bf7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f20c006e400 0x7f20c006e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20beffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f20c00d15f0 0x7f20c015b3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47330/0 (socket says 172.21.15.57:47330) 2023-11-28T12:07:40.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20beffd700 1 -- 172.21.15.57:0/2379562275 learned_addr learned my addr 172.21.15.57:0/2379562275 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:40.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20bffff700 1 --2- 172.21.15.57:0/2379562275 >> v2:172.21.15.181:3300/0 conn(0x7f20c015e010 0x7f20c00ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20bf7fe700 1 -- 172.21.15.57:0/2379562275 >> v2:172.21.15.181:3300/0 conn(0x7f20c015e010 msgr2=0x7f20c00ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:40.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20bf7fe700 1 --2- 172.21.15.57:0/2379562275 >> v2:172.21.15.181:3300/0 conn(0x7f20c015e010 0x7f20c00ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20bf7fe700 1 -- 172.21.15.57:0/2379562275 >> v1:172.21.15.57:6789/0 conn(0x7f20c00d15f0 legacy=0x7f20c015b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20bf7fe700 1 -- 172.21.15.57:0/2379562275 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c015ef40 con 0x7f20c006e400 2023-11-28T12:07:40.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20be7fc700 1 -- 172.21.15.57:0/2379562275 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739482724 0 0) 0x7f20c015af30 con 0x7f20c00d15f0 2023-11-28T12:07:40.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20bffff700 1 --2- 172.21.15.57:0/2379562275 >> v2:172.21.15.181:3300/0 conn(0x7f20c015e010 0x7f20c00ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20bf7fe700 1 --2- 172.21.15.57:0/2379562275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20c006e400 0x7f20c006e7d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f20c00b60f0 tx=0x7f20b4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9134b552f208437b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.818+0000 7f20be7fc700 1 -- 172.21.15.57:0/2379562275 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4011830 con 0x7f20c006e400 2023-11-28T12:07:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.819+0000 7f20be7fc700 1 -- 172.21.15.57:0/2379562275 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b4011990 con 0x7f20c006e400 2023-11-28T12:07:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.819+0000 7f20be7fc700 1 -- 172.21.15.57:0/2379562275 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4011d20 con 0x7f20c006e400 2023-11-28T12:07:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.819+0000 7f20c60ac700 1 -- 172.21.15.57:0/2379562275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20c006e400 msgr2=0x7f20c006e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.819+0000 7f20c60ac700 1 --2- 172.21.15.57:0/2379562275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20c006e400 0x7f20c006e7d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f20c00b60f0 tx=0x7f20b4009b20 comp rx=0 tx=0).stop 2023-11-28T12:07:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.819+0000 7f20c60ac700 1 -- 172.21.15.57:0/2379562275 shutdown_connections 2023-11-28T12:07:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.819+0000 7f20c60ac700 1 --2- 172.21.15.57:0/2379562275 >> v2:172.21.15.181:3300/0 conn(0x7f20c015e010 0x7f20c00ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.819+0000 7f20c60ac700 1 --2- 172.21.15.57:0/2379562275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20c006e400 0x7f20c006e7d0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.819+0000 7f20c60ac700 1 -- 172.21.15.57:0/2379562275 >> 172.21.15.57:0/2379562275 conn(0x7f20c006a270 msgr2=0x7f20c006a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:40.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.819+0000 7f20c60ac700 1 -- 172.21.15.57:0/2379562275 shutdown_connections 2023-11-28T12:07:40.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.819+0000 7f20c60ac700 1 -- 172.21.15.57:0/2379562275 wait complete. 2023-11-28T12:07:40.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.820+0000 7f20c60ac700 1 Processor -- start 2023-11-28T12:07:40.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.821+0000 7f20c60ac700 1 -- start start 2023-11-28T12:07:40.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.821+0000 7f20c60ac700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20c006e400 0x7f20c0152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.821+0000 7f20c60ac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c00d15f0 0x7f20c0151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.821+0000 7f20c60ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c015e010 0x7f20c00b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.821+0000 7f20c60ac700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c0152120 con 0x7f20c006e400 2023-11-28T12:07:40.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.821+0000 7f20c60ac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00cec50 con 0x7f20c015e010 2023-11-28T12:07:40.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.821+0000 7f20c60ac700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00c5c70 con 0x7f20c00d15f0 2023-11-28T12:07:40.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.821+0000 7f20bf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20c006e400 0x7f20c0152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20bf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20c006e400 0x7f20c0152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59162/0 (socket says 172.21.15.57:59162) 2023-11-28T12:07:40.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20bf7fe700 1 -- 172.21.15.57:0/467518570 learned_addr learned my addr 172.21.15.57:0/467518570 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:40.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20bffff700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c015e010 0x7f20c00b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20beffd700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c00d15f0 0x7f20c0151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20bf7fe700 1 -- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c00d15f0 msgr2=0x7f20c0151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20bf7fe700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c00d15f0 0x7f20c0151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20bf7fe700 1 -- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c015e010 msgr2=0x7f20c00b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:40.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20bf7fe700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c015e010 0x7f20c00b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:40.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20bf7fe700 1 -- 172.21.15.57:0/467518570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b40094a0 con 0x7f20c006e400 2023-11-28T12:07:40.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20bf7fe700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20c006e400 0x7f20c0152290 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f20c00b6eb0 tx=0x7f20b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:40.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20a7fff700 1 -- 172.21.15.57:0/467518570 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b40038a0 con 0x7f20c006e400 2023-11-28T12:07:40.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20a7fff700 1 -- 172.21.15.57:0/467518570 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b401f070 con 0x7f20c006e400 2023-11-28T12:07:40.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20c60ac700 1 -- 172.21.15.57:0/467518570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c015e980 con 0x7f20c006e400 2023-11-28T12:07:40.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20a7fff700 1 -- 172.21.15.57:0/467518570 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4003c40 con 0x7f20c006e400 2023-11-28T12:07:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.822+0000 7f20c60ac700 1 -- 172.21.15.57:0/467518570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c006ed10 con 0x7f20c006e400 2023-11-28T12:07:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.824+0000 7f20c60ac700 1 -- 172.21.15.57:0/467518570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c015ec60 con 0x7f20c006e400 2023-11-28T12:07:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.824+0000 7f20a7fff700 1 -- 172.21.15.57:0/467518570 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f20b400f050 con 0x7f20c006e400 2023-11-28T12:07:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.824+0000 7f20a7fff700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20ac0682f0 0x7f20ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.824+0000 7f20a7fff700 1 -- 172.21.15.57:0/467518570 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f20b4021070 con 0x7f20c006e400 2023-11-28T12:07:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.825+0000 7f20beffd700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20ac0682f0 0x7f20ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.825+0000 7f20beffd700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20ac0682f0 0x7f20ac06a7a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f20c00d1260 tx=0x7f20b0007c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:40.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:40.829+0000 7f20a7fff700 1 -- 172.21.15.57:0/467518570 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f20b4053f10 con 0x7f20c006e400 2023-11-28T12:07:41.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.027+0000 7f20c60ac700 1 -- 172.21.15.57:0/467518570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20c00743f0 con 0x7f20c006e400 2023-11-28T12:07:41.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.028+0000 7f20a7fff700 1 -- 172.21.15.57:0/467518570 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+14337 (secure 0 0 0) 0x7f20b4057530 con 0x7f20c006e400 2023-11-28T12:07:41.029 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:41.030 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":68,"fsid":"0309b83f-6570-4eee-96fa-279822cd7766","created":"2023-11-28T12:01:25.754420+0000","modified":"2023-11-28T12:07:37.703608+0000","last_up_change":"2023-11-28T12:01:36.202469+0000","last_in_change":"2023-11-28T12:01:28.297052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T12:01:37.189786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T12:01:37.712343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fe3ca0b-57ba-412d-ba00-5085b06b65ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":120754},{"type":"v1","addr":"172.21.15.57:6801","nonce":120754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":120754},{"type":"v1","addr":"172.21.15.57:6803","nonce":120754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":120754},{"type":"v1","addr":"172.21.15.57:6807","nonce":120754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":120754},{"type":"v1","addr":"172.21.15.57:6805","nonce":120754}]},"public_addr":"172.21.15.57:6801/120754","cluster_addr":"172.21.15.57:6803/120754","heartbeat_back_addr":"172.21.15.57:6807/120754","heartbeat_front_addr":"172.21.15.57:6805/120754","state":["exists","up"]},{"osd":1,"uuid":"f0ce5312-030d-4de4-9c05-e13a80b0a7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":120755},{"type":"v1","addr":"172.21.15.57:6809","nonce":120755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":120755},{"type":"v1","addr":"172.21.15.57:6811","nonce":120755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":120755},{"type":"v1","addr":"172.21.15.57:6815","nonce":120755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":120755},{"type":"v1","addr":"172.21.15.57:6813","nonce":120755}]},"public_addr":"172.21.15.57:6809/120755","cluster_addr":"172.21.15.57:6811/120755","heartbeat_back_addr":"172.21.15.57:6815/120755","heartbeat_front_addr":"172.21.15.57:6813/120755","state":["exists","up"]},{"osd":2,"uuid":"8756e996-2c62-40ae-a244-6d6a8d58d275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":120771},{"type":"v1","addr":"172.21.15.57:6818","nonce":120771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6819","nonce":120771},{"type":"v1","addr":"172.21.15.57:6821","nonce":120771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":120771},{"type":"v1","addr":"172.21.15.57:6828","nonce":120771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6823","nonce":120771},{"type":"v1","addr":"172.21.15.57:6824","nonce":120771}]},"public_addr":"172.21.15.57:6818/120771","cluster_addr":"172.21.15.57:6821/120771","heartbeat_back_addr":"172.21.15.57:6828/120771","heartbeat_front_addr":"172.21.15.57:6824/120771","state":["exists","up"]},{"osd":3,"uuid":"5cbd5261-bd50-44f1-8192-60d55157d358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6817","nonce":120770},{"type":"v1","addr":"172.21.15.57:6820","nonce":120770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":120770},{"type":"v1","addr":"172.21.15.57:6825","nonce":120770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":120770},{"type":"v1","addr":"172.21.15.57:6831","nonce":120770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6827","nonce":120770},{"type":"v1","addr":"172.21.15.57:6829","nonce":120770}]},"public_addr":"172.21.15.57:6820/120770","cluster_addr":"172.21.15.57:6825/120770","heartbeat_back_addr":"172.21.15.57:6831/120770","heartbeat_front_addr":"172.21.15.57:6829/120770","state":["exists","up"]},{"osd":4,"uuid":"7a633c56-49e1-4d74-926d-3ea86cab85ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":120958},{"type":"v1","addr":"172.21.15.181:6801","nonce":120958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":120958},{"type":"v1","addr":"172.21.15.181:6803","nonce":120958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":120958},{"type":"v1","addr":"172.21.15.181:6807","nonce":120958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":120958},{"type":"v1","addr":"172.21.15.181:6805","nonce":120958}]},"public_addr":"172.21.15.181:6801/120958","cluster_addr":"172.21.15.181:6803/120958","heartbeat_back_addr":"172.21.15.181:6807/120958","heartbeat_front_addr":"172.21.15.181:6805/120958","state":["exists","up"]},{"osd":5,"uuid":"25161636-254d-4a76-abac-21f688eca3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":120977},{"type":"v1","addr":"172.21.15.181:6825","nonce":120977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":120977},{"type":"v1","addr":"172.21.15.181:6827","nonce":120977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":120977},{"type":"v1","addr":"172.21.15.181:6831","nonce":120977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":120977},{"type":"v1","addr":"172.21.15.181:6829","nonce":120977}]},"public_addr":"172.21.15.181:6825/120977","cluster_addr":"172.21.15.181:6827/120977","heartbeat_back_addr":"172.21.15.181:6831/120977","heartbeat_front_addr":"172.21.15.181:6829/120977","state":["exists","up"]},{"osd":6,"uuid":"61c58456-c1bd-479b-902f-4db772993c6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":120984},{"type":"v1","addr":"172.21.15.181:6817","nonce":120984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":120984},{"type":"v1","addr":"172.21.15.181:6819","nonce":120984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":120984},{"type":"v1","addr":"172.21.15.181:6823","nonce":120984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":120984},{"type":"v1","addr":"172.21.15.181:6821","nonce":120984}]},"public_addr":"172.21.15.181:6817/120984","cluster_addr":"172.21.15.181:6819/120984","heartbeat_back_addr":"172.21.15.181:6823/120984","heartbeat_front_addr":"172.21.15.181:6821/120984","state":["exists","up"]},{"osd":7,"uuid":"e986e5f1-3003-4c66-9b57-d0d8f8121bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":120959},{"type":"v1","addr":"172.21.15.181:6809","nonce":120959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":120959},{"type":"v1","addr":"172.21.15.181:6811","nonce":120959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":120959},{"type":"v1","addr":"172.21.15.181:6815","nonce":120959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":120959},{"type":"v1","addr":"172.21.15.181:6813","nonce":120959}]},"public_addr":"172.21.15.181:6809/120959","cluster_addr":"172.21.15.181:6811/120959","heartbeat_back_addr":"172.21.15.181:6815/120959","heartbeat_front_addr":"172.21.15.181:6813/120959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.477468+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.554132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.493936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.533336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.482624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.437597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.444619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T12:01:33.490307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:6835/685888843":"2023-11-29T12:07:33.694004+0000","172.21.15.57:6833/685888843":"2023-11-29T12:07:33.694004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:07:41.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.030+0000 7f20c60ac700 1 -- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20ac0682f0 msgr2=0x7f20ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:41.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.030+0000 7f20c60ac700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20ac0682f0 0x7f20ac06a7a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f20c00d1260 tx=0x7f20b0007c00 comp rx=0 tx=0).stop 2023-11-28T12:07:41.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.030+0000 7f20c60ac700 1 -- 172.21.15.57:0/467518570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20c006e400 msgr2=0x7f20c0152290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:41.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.030+0000 7f20c60ac700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20c006e400 0x7f20c0152290 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f20c00b6eb0 tx=0x7f20b4004750 comp rx=0 tx=0).stop 2023-11-28T12:07:41.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.030+0000 7f20c60ac700 1 -- 172.21.15.57:0/467518570 shutdown_connections 2023-11-28T12:07:41.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.030+0000 7f20c60ac700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20ac0682f0 0x7f20ac06a7a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:41.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.030+0000 7f20c60ac700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20c015e010 0x7f20c00b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:41.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.030+0000 7f20c60ac700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20c006e400 0x7f20c0152290 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:41.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.030+0000 7f20c60ac700 1 --2- 172.21.15.57:0/467518570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20c00d15f0 0x7f20c0151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:41.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.030+0000 7f20c60ac700 1 -- 172.21.15.57:0/467518570 >> 172.21.15.57:0/467518570 conn(0x7f20c006a270 msgr2=0x7f20c00b55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:41.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.030+0000 7f20c60ac700 1 -- 172.21.15.57:0/467518570 shutdown_connections 2023-11-28T12:07:41.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:41.030+0000 7f20c60ac700 1 -- 172.21.15.57:0/467518570 wait complete. 2023-11-28T12:07:42.046 INFO:tasks.ceph:Scrubbing osd.0 2023-11-28T12:07:42.047 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-28T12:07:42.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.195+0000 7ff6b120d700 1 Processor -- start 2023-11-28T12:07:42.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.195+0000 7ff6b120d700 1 -- start start 2023-11-28T12:07:42.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6b120d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff6ac071320 0x7ff6ac06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6b120d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff6ac15dc40 0x7ff6ac160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6b120d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac0cdc10 con 0x7ff6ac15cce0 2023-11-28T12:07:42.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6b120d700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ac15d0b0 con 0x7ff6ac071320 2023-11-28T12:07:42.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6b120d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ac0cfa10 con 0x7ff6ac15dc40 2023-11-28T12:07:42.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6aa59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff6ac071320 0x7ff6ac06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6aad9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff6ac15cce0 0x7ff6ac070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47342/0 (socket says 172.21.15.57:47342) 2023-11-28T12:07:42.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6aad9d700 1 -- 172.21.15.57:0/197639232 learned_addr learned my addr 172.21.15.57:0/197639232 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:42.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6ab59e700 1 --2- 172.21.15.57:0/197639232 >> v2:172.21.15.181:3300/0 conn(0x7ff6ac15dc40 0x7ff6ac160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6aa59c700 1 -- 172.21.15.57:0/197639232 >> v2:172.21.15.181:3300/0 conn(0x7ff6ac15dc40 msgr2=0x7ff6ac160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:42.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6aa59c700 1 --2- 172.21.15.57:0/197639232 >> v2:172.21.15.181:3300/0 conn(0x7ff6ac15dc40 0x7ff6ac160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6aa59c700 1 -- 172.21.15.57:0/197639232 >> v1:172.21.15.57:6789/0 conn(0x7ff6ac15cce0 legacy=0x7ff6ac070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6aa59c700 1 -- 172.21.15.57:0/197639232 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac0cf730 con 0x7ff6ac071320 2023-11-28T12:07:42.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6a9d9b700 1 -- 172.21.15.57:0/197639232 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913256800 0 0) 0x7ff6ac0cdc10 con 0x7ff6ac15cce0 2023-11-28T12:07:42.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6ab59e700 1 --2- 172.21.15.57:0/197639232 >> v2:172.21.15.181:3300/0 conn(0x7ff6ac15dc40 0x7ff6ac160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:42.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.196+0000 7ff6aa59c700 1 --2- 172.21.15.57:0/197639232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6ac071320 0x7ff6ac06b9f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0ca210 tx=0x7ff69c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84380ca1e6ac9e59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.197+0000 7ff6a9d9b700 1 -- 172.21.15.57:0/197639232 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c011830 con 0x7ff6ac071320 2023-11-28T12:07:42.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.197+0000 7ff6a9d9b700 1 -- 172.21.15.57:0/197639232 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c011990 con 0x7ff6ac071320 2023-11-28T12:07:42.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.197+0000 7ff6a9d9b700 1 -- 172.21.15.57:0/197639232 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c011d20 con 0x7ff6ac071320 2023-11-28T12:07:42.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.197+0000 7ff6b120d700 1 -- 172.21.15.57:0/197639232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6ac071320 msgr2=0x7ff6ac06b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.197+0000 7ff6b120d700 1 --2- 172.21.15.57:0/197639232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6ac071320 0x7ff6ac06b9f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0ca210 tx=0x7ff69c009b20 comp rx=0 tx=0).stop 2023-11-28T12:07:42.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.197+0000 7ff6b120d700 1 -- 172.21.15.57:0/197639232 shutdown_connections 2023-11-28T12:07:42.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.197+0000 7ff6b120d700 1 --2- 172.21.15.57:0/197639232 >> v2:172.21.15.181:3300/0 conn(0x7ff6ac15dc40 0x7ff6ac160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.197+0000 7ff6b120d700 1 --2- 172.21.15.57:0/197639232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6ac071320 0x7ff6ac06b9f0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.197+0000 7ff6b120d700 1 -- 172.21.15.57:0/197639232 >> 172.21.15.57:0/197639232 conn(0x7ff6ac0cfc30 msgr2=0x7ff6ac0d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:42.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.197+0000 7ff6b120d700 1 -- 172.21.15.57:0/197639232 shutdown_connections 2023-11-28T12:07:42.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.197+0000 7ff6b120d700 1 -- 172.21.15.57:0/197639232 wait complete. 2023-11-28T12:07:42.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.199+0000 7ff6b120d700 1 Processor -- start 2023-11-28T12:07:42.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.199+0000 7ff6b120d700 1 -- start start 2023-11-28T12:07:42.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.199+0000 7ff6b120d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6ac071320 0x7ff6ac0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.199+0000 7ff6b120d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6ac15cce0 0x7ff6ac0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6b120d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6ac15dc40 0x7ff6ac1f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6b120d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0cc710 con 0x7ff6ac15cce0 2023-11-28T12:07:42.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6b120d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0cc120 con 0x7ff6ac15dc40 2023-11-28T12:07:42.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6b120d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac1f2e40 con 0x7ff6ac071320 2023-11-28T12:07:42.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6aa59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6ac15cce0 0x7ff6ac0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6aa59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6ac15cce0 0x7ff6ac0b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59192/0 (socket says 172.21.15.57:59192) 2023-11-28T12:07:42.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6aa59c700 1 -- 172.21.15.57:0/86129103 learned_addr learned my addr 172.21.15.57:0/86129103 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:42.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6ab59e700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6ac15dc40 0x7ff6ac1f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6aad9d700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6ac071320 0x7ff6ac0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6aa59c700 1 -- 172.21.15.57:0/86129103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6ac071320 msgr2=0x7ff6ac0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6aa59c700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6ac071320 0x7ff6ac0ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6aa59c700 1 -- 172.21.15.57:0/86129103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6ac15dc40 msgr2=0x7ff6ac1f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6aa59c700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6ac15dc40 0x7ff6ac1f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6aa59c700 1 -- 172.21.15.57:0/86129103 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c0094a0 con 0x7ff6ac15cce0 2023-11-28T12:07:42.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6aad9d700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6ac071320 0x7ff6ac0ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:42.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6aa59c700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6ac15cce0 0x7ff6ac0b4620 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff69c000c00 tx=0x7ff69c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6a37fe700 1 -- 172.21.15.57:0/86129103 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c0038a0 con 0x7ff6ac15cce0 2023-11-28T12:07:42.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6a37fe700 1 -- 172.21.15.57:0/86129103 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c01e070 con 0x7ff6ac15cce0 2023-11-28T12:07:42.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6b120d700 1 -- 172.21.15.57:0/86129103 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac15e0a0 con 0x7ff6ac15cce0 2023-11-28T12:07:42.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6a37fe700 1 -- 172.21.15.57:0/86129103 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c003c40 con 0x7ff6ac15cce0 2023-11-28T12:07:42.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.200+0000 7ff6b120d700 1 -- 172.21.15.57:0/86129103 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac0cd8a0 con 0x7ff6ac15cce0 2023-11-28T12:07:42.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.201+0000 7ff6b120d700 1 -- 172.21.15.57:0/86129103 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff6ac0cf3e0 con 0x7ff6ac15cce0 2023-11-28T12:07:42.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.202+0000 7ff6a37fe700 1 -- 172.21.15.57:0/86129103 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff69c00f050 con 0x7ff6ac15cce0 2023-11-28T12:07:42.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.202+0000 7ff6a37fe700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff68c0682f0 0x7ff68c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.202+0000 7ff6a37fe700 1 -- 172.21.15.57:0/86129103 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff69c086b90 con 0x7ff6ac15cce0 2023-11-28T12:07:42.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.202+0000 7ff6a37fe700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] conn(0x7ff68c06e760 0x7ff68c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.202+0000 7ff6a37fe700 1 -- 172.21.15.57:0/86129103 --> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff68c071240 con 0x7ff68c06e760 2023-11-28T12:07:42.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.202+0000 7ff6a37fe700 1 -- 172.21.15.57:0/86129103 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff69c08b2a0 con 0x7ff6ac15cce0 2023-11-28T12:07:42.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.203+0000 7ff6aad9d700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff68c0682f0 0x7ff68c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.203+0000 7ff6ab59e700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] conn(0x7ff68c06e760 0x7ff68c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.203+0000 7ff6ab59e700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] conn(0x7ff68c06e760 0x7ff68c070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.203+0000 7ff6aad9d700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff68c0682f0 0x7ff68c06a7a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff694000f60 tx=0x7ff694009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.204+0000 7ff6a37fe700 1 -- 172.21.15.57:0/86129103 <== osd.0 v2:172.21.15.57:6800/120754 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff68c071240 con 0x7ff68c06e760 2023-11-28T12:07:42.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.218+0000 7ff6b120d700 1 -- 172.21.15.57:0/86129103 --> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff6ac0cc850 con 0x7ff68c06e760 2023-11-28T12:07:42.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.219+0000 7ff6a37fe700 1 -- 172.21.15.57:0/86129103 <== osd.0 v2:172.21.15.57:6800/120754 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff6ac0cc850 con 0x7ff68c06e760 2023-11-28T12:07:42.225 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:07:42.225 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T12:07:42.225 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:07:42.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.219+0000 7ff6b120d700 1 -- 172.21.15.57:0/86129103 >> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] conn(0x7ff68c06e760 msgr2=0x7ff68c070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.219+0000 7ff6b120d700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.57:6800/120754,v1:172.21.15.57:6801/120754] conn(0x7ff68c06e760 0x7ff68c070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.219+0000 7ff6b120d700 1 -- 172.21.15.57:0/86129103 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff68c0682f0 msgr2=0x7ff68c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.219+0000 7ff6b120d700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff68c0682f0 0x7ff68c06a7a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff694000f60 tx=0x7ff694009040 comp rx=0 tx=0).stop 2023-11-28T12:07:42.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.219+0000 7ff6b120d700 1 -- 172.21.15.57:0/86129103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6ac15cce0 msgr2=0x7ff6ac0b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.219+0000 7ff6b120d700 1 --2- 172.21.15.57:0/86129103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6ac15cce0 0x7ff6ac0b4620 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff69c000c00 tx=0x7ff69c004750 comp rx=0 tx=0).stop 2023-11-28T12:07:42.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.219+0000 7ff6ab59e700 1 -- 172.21.15.57:0/86129103 reap_dead start 2023-11-28T12:07:42.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.219+0000 7ff6b120d700 1 -- 172.21.15.57:0/86129103 shutdown_connections 2023-11-28T12:07:42.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.219+0000 7ff6b120d700 1 -- 172.21.15.57:0/86129103 >> 172.21.15.57:0/86129103 conn(0x7ff6ac0cfc30 msgr2=0x7ff6ac0ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:42.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.219+0000 7ff6b120d700 1 -- 172.21.15.57:0/86129103 shutdown_connections 2023-11-28T12:07:42.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.219+0000 7ff6b120d700 1 -- 172.21.15.57:0/86129103 wait complete. 2023-11-28T12:07:42.234 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-28T12:07:42.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.369+0000 7fc223939700 1 Processor -- start 2023-11-28T12:07:42.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.369+0000 7fc223939700 1 -- start start 2023-11-28T12:07:42.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.370+0000 7fc223939700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc21c071890 0x7fc21c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.370+0000 7fc223939700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21c15cc10 con 0x7fc21c06a980 2023-11-28T12:07:42.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.370+0000 7fc223939700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21c06c9b0 con 0x7fc21c0cf710 2023-11-28T12:07:42.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.370+0000 7fc223939700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21c06cc90 con 0x7fc21c071890 2023-11-28T12:07:42.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.370+0000 7fc220ed4700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc21c06a980 0x7fc21c160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47356/0 (socket says 172.21.15.57:47356) 2023-11-28T12:07:42.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.370+0000 7fc220ed4700 1 -- 172.21.15.57:0/3985884955 learned_addr learned my addr 172.21.15.57:0/3985884955 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:42.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.370+0000 7fc221ed6700 1 --2- 172.21.15.57:0/3985884955 >> v2:172.21.15.181:3300/0 conn(0x7fc21c071890 0x7fc21c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.371+0000 7fc20ffff700 1 -- 172.21.15.57:0/3985884955 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198408065 0 0) 0x7fc21c15cc10 con 0x7fc21c06a980 2023-11-28T12:07:42.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.371+0000 7fc20ffff700 1 -- 172.21.15.57:0/3985884955 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2080035b0 con 0x7fc21c06a980 2023-11-28T12:07:42.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.371+0000 7fc20ffff700 1 -- 172.21.15.57:0/3985884955 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725925395 0 0) 0x7fc21c06c9b0 con 0x7fc21c0cf710 2023-11-28T12:07:42.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.371+0000 7fc20ffff700 1 -- 172.21.15.57:0/3985884955 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc21c15cc10 con 0x7fc21c0cf710 2023-11-28T12:07:42.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.371+0000 7fc221ed6700 1 -- 172.21.15.57:0/3985884955 >> v1:172.21.15.181:6789/0 conn(0x7fc21c0cf710 legacy=0x7fc21c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.371+0000 7fc221ed6700 1 -- 172.21.15.57:0/3985884955 >> v1:172.21.15.57:6789/0 conn(0x7fc21c06a980 legacy=0x7fc21c160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.371+0000 7fc221ed6700 1 -- 172.21.15.57:0/3985884955 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c06cf70 con 0x7fc21c071890 2023-11-28T12:07:42.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.371+0000 7fc221ed6700 1 --2- 172.21.15.57:0/3985884955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c071890 0x7fc21c0ce740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc214009d90 tx=0x7fc21400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe3ef85bb74eb586 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.371+0000 7fc20ffff700 1 -- 172.21.15.57:0/3985884955 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21400e070 con 0x7fc21c071890 2023-11-28T12:07:42.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.371+0000 7fc20ffff700 1 -- 172.21.15.57:0/3985884955 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21400bb20 con 0x7fc21c071890 2023-11-28T12:07:42.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.371+0000 7fc20ffff700 1 -- 172.21.15.57:0/3985884955 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21400be90 con 0x7fc21c071890 2023-11-28T12:07:42.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.372+0000 7fc223939700 1 -- 172.21.15.57:0/3985884955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c071890 msgr2=0x7fc21c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.372+0000 7fc223939700 1 --2- 172.21.15.57:0/3985884955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c071890 0x7fc21c0ce740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc214009d90 tx=0x7fc21400b330 comp rx=0 tx=0).stop 2023-11-28T12:07:42.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.372+0000 7fc223939700 1 -- 172.21.15.57:0/3985884955 shutdown_connections 2023-11-28T12:07:42.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.372+0000 7fc223939700 1 --2- 172.21.15.57:0/3985884955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c071890 0x7fc21c0ce740 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.372+0000 7fc223939700 1 -- 172.21.15.57:0/3985884955 >> 172.21.15.57:0/3985884955 conn(0x7fc21c06c010 msgr2=0x7fc21c0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:42.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.372+0000 7fc223939700 1 -- 172.21.15.57:0/3985884955 shutdown_connections 2023-11-28T12:07:42.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.372+0000 7fc223939700 1 -- 172.21.15.57:0/3985884955 wait complete. 2023-11-28T12:07:42.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.373+0000 7fc223939700 1 Processor -- start 2023-11-28T12:07:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.373+0000 7fc223939700 1 -- start start 2023-11-28T12:07:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.374+0000 7fc223939700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c06a980 0x7fc21c0c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.374+0000 7fc223939700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c071890 0x7fc21c0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.374+0000 7fc223939700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc21c0cf710 0x7fc21c150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.374+0000 7fc223939700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c151670 con 0x7fc21c0cf710 2023-11-28T12:07:42.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.374+0000 7fc223939700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c0c35c0 con 0x7fc21c06a980 2023-11-28T12:07:42.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.374+0000 7fc223939700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c151170 con 0x7fc21c071890 2023-11-28T12:07:42.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.374+0000 7fc221ed6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc21c0cf710 0x7fc21c150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.374+0000 7fc221ed6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc21c0cf710 0x7fc21c150c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59198/0 (socket says 172.21.15.57:59198) 2023-11-28T12:07:42.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.374+0000 7fc221ed6700 1 -- 172.21.15.57:0/3458024072 learned_addr learned my addr 172.21.15.57:0/3458024072 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:42.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc220ed4700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c071890 0x7fc21c0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc221ed6700 1 -- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c071890 msgr2=0x7fc21c0c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc2216d5700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c06a980 0x7fc21c0c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc221ed6700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c071890 0x7fc21c0c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc221ed6700 1 -- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c06a980 msgr2=0x7fc21c0c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc221ed6700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c06a980 0x7fc21c0c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc221ed6700 1 -- 172.21.15.57:0/3458024072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214009a70 con 0x7fc21c0cf710 2023-11-28T12:07:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc220ed4700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c071890 0x7fc21c0c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc2216d5700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c06a980 0x7fc21c0c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc221ed6700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc21c0cf710 0x7fc21c150c30 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc21400b680 tx=0x7fc214004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc20dffb700 1 -- 172.21.15.57:0/3458024072 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140098f0 con 0x7fc21c0cf710 2023-11-28T12:07:42.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc20dffb700 1 -- 172.21.15.57:0/3458024072 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc214020070 con 0x7fc21c0cf710 2023-11-28T12:07:42.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc223939700 1 -- 172.21.15.57:0/3458024072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21c06d530 con 0x7fc21c0cf710 2023-11-28T12:07:42.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc20dffb700 1 -- 172.21.15.57:0/3458024072 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140276c0 con 0x7fc21c0cf710 2023-11-28T12:07:42.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.375+0000 7fc223939700 1 -- 172.21.15.57:0/3458024072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c06d250 con 0x7fc21c0cf710 2023-11-28T12:07:42.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.376+0000 7fc223939700 1 -- 172.21.15.57:0/3458024072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21c0743f0 con 0x7fc21c0cf710 2023-11-28T12:07:42.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.377+0000 7fc20dffb700 1 -- 172.21.15.57:0/3458024072 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc21402e070 con 0x7fc21c0cf710 2023-11-28T12:07:42.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.377+0000 7fc20dffb700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc2000682f0 0x7fc20006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.377+0000 7fc20dffb700 1 -- 172.21.15.57:0/3458024072 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc214022070 con 0x7fc21c0cf710 2023-11-28T12:07:42.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.377+0000 7fc2216d5700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc2000682f0 0x7fc20006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.378+0000 7fc2216d5700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc2000682f0 0x7fc20006a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc21c15b6d0 tx=0x7fc218009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.382+0000 7fc20dffb700 1 -- 172.21.15.57:0/3458024072 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc214054200 con 0x7fc21c0cf710 2023-11-28T12:07:42.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.580+0000 7fc223939700 1 -- 172.21.15.57:0/3458024072 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc21c0711a0 con 0x7fc2000682f0 2023-11-28T12:07:42.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.581+0000 7fc20dffb700 1 -- 172.21.15.57:0/3458024072 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc21c0711a0 con 0x7fc2000682f0 2023-11-28T12:07:42.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.583+0000 7fc223939700 1 -- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc2000682f0 msgr2=0x7fc20006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.583+0000 7fc223939700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc2000682f0 0x7fc20006a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc21c15b6d0 tx=0x7fc218009040 comp rx=0 tx=0).stop 2023-11-28T12:07:42.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.583+0000 7fc223939700 1 -- 172.21.15.57:0/3458024072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc21c0cf710 msgr2=0x7fc21c150c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.583+0000 7fc223939700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc21c0cf710 0x7fc21c150c30 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc21400b680 tx=0x7fc214004ad0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.583+0000 7fc223939700 1 -- 172.21.15.57:0/3458024072 shutdown_connections 2023-11-28T12:07:42.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.583+0000 7fc223939700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fc2000682f0 0x7fc20006a7a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.583+0000 7fc223939700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc21c0cf710 0x7fc21c150c30 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.583+0000 7fc223939700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c06a980 0x7fc21c0c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.583+0000 7fc223939700 1 --2- 172.21.15.57:0/3458024072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c071890 0x7fc21c0c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.583+0000 7fc223939700 1 -- 172.21.15.57:0/3458024072 >> 172.21.15.57:0/3458024072 conn(0x7fc21c06c010 msgr2=0x7fc21c0af610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:42.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.583+0000 7fc223939700 1 -- 172.21.15.57:0/3458024072 shutdown_connections 2023-11-28T12:07:42.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.583+0000 7fc223939700 1 -- 172.21.15.57:0/3458024072 wait complete. 2023-11-28T12:07:42.588 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 0 to deep-scrub 2023-11-28T12:07:42.598 INFO:tasks.ceph:Scrubbing osd.1 2023-11-28T12:07:42.598 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-28T12:07:42.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.736+0000 7fd2bbe14700 1 Processor -- start 2023-11-28T12:07:42.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.736+0000 7fd2bbe14700 1 -- start start 2023-11-28T12:07:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.738+0000 7fd2bbe14700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd2b415b090 0x7fd2b415b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.738+0000 7fd2bbe14700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2b406c390 0x7fd2b406c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.738+0000 7fd2bbe14700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b4071500 con 0x7fd2b40c9bd0 2023-11-28T12:07:42.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.738+0000 7fd2bbe14700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b415b9a0 con 0x7fd2b415b090 2023-11-28T12:07:42.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.738+0000 7fd2bbe14700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b406cd50 con 0x7fd2b406c390 2023-11-28T12:07:42.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.738+0000 7fd2b93af700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd2b40c9bd0 0x7fd2b406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47366/0 (socket says 172.21.15.57:47366) 2023-11-28T12:07:42.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.738+0000 7fd2b93af700 1 -- 172.21.15.57:0/2218195646 learned_addr learned my addr 172.21.15.57:0/2218195646 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:42.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.738+0000 7fd2b9bb0700 1 --2- 172.21.15.57:0/2218195646 >> v2:172.21.15.181:3300/0 conn(0x7fd2b415b090 0x7fd2b415b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.738+0000 7fd2ba3b1700 1 --2- 172.21.15.57:0/2218195646 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2b406c390 0x7fd2b406c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.738+0000 7fd2b9bb0700 1 -- 172.21.15.57:0/2218195646 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2b406c390 msgr2=0x7fd2b406c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:42.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.738+0000 7fd2b9bb0700 1 --2- 172.21.15.57:0/2218195646 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2b406c390 0x7fd2b406c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.738+0000 7fd2b9bb0700 1 -- 172.21.15.57:0/2218195646 >> v1:172.21.15.57:6789/0 conn(0x7fd2b40c9bd0 legacy=0x7fd2b406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.738+0000 7fd2b9bb0700 1 -- 172.21.15.57:0/2218195646 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b406aaa0 con 0x7fd2b415b090 2023-11-28T12:07:42.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.739+0000 7fd2b9bb0700 1 --2- 172.21.15.57:0/2218195646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2b415b090 0x7fd2b415b460 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009840 tx=0x7fd2b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c452798ef410b77 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.740+0000 7fd2b8bae700 1 -- 172.21.15.57:0/2218195646 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0011830 con 0x7fd2b415b090 2023-11-28T12:07:42.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.740+0000 7fd2b8bae700 1 -- 172.21.15.57:0/2218195646 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b0011990 con 0x7fd2b415b090 2023-11-28T12:07:42.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.740+0000 7fd2b8bae700 1 -- 172.21.15.57:0/2218195646 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0011d20 con 0x7fd2b415b090 2023-11-28T12:07:42.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.740+0000 7fd2bbe14700 1 -- 172.21.15.57:0/2218195646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2b415b090 msgr2=0x7fd2b415b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.740+0000 7fd2bbe14700 1 --2- 172.21.15.57:0/2218195646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2b415b090 0x7fd2b415b460 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009840 tx=0x7fd2b0009b50 comp rx=0 tx=0).stop 2023-11-28T12:07:42.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.740+0000 7fd2bbe14700 1 -- 172.21.15.57:0/2218195646 shutdown_connections 2023-11-28T12:07:42.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.740+0000 7fd2bbe14700 1 --2- 172.21.15.57:0/2218195646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2b415b090 0x7fd2b415b460 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.740+0000 7fd2bbe14700 1 --2- 172.21.15.57:0/2218195646 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2b406c390 0x7fd2b406c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.740+0000 7fd2bbe14700 1 -- 172.21.15.57:0/2218195646 >> 172.21.15.57:0/2218195646 conn(0x7fd2b40cf370 msgr2=0x7fd2b40cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:42.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.741+0000 7fd2bbe14700 1 -- 172.21.15.57:0/2218195646 shutdown_connections 2023-11-28T12:07:42.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.741+0000 7fd2bbe14700 1 -- 172.21.15.57:0/2218195646 wait complete. 2023-11-28T12:07:42.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.742+0000 7fd2bbe14700 1 Processor -- start 2023-11-28T12:07:42.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.742+0000 7fd2bbe14700 1 -- start start 2023-11-28T12:07:42.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2bbe14700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2b406c390 0x7fd2b4153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2bbe14700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2b40c9bd0 0x7fd2b41538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2bbe14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2b415b090 0x7fd2b41f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2bbe14700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4157f20 con 0x7fd2b40c9bd0 2023-11-28T12:07:42.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2bbe14700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b41f2030 con 0x7fd2b415b090 2023-11-28T12:07:42.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2bbe14700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b40c9170 con 0x7fd2b406c390 2023-11-28T12:07:42.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2b93af700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2b40c9bd0 0x7fd2b41538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2b93af700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2b40c9bd0 0x7fd2b41538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59214/0 (socket says 172.21.15.57:59214) 2023-11-28T12:07:42.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2b93af700 1 -- 172.21.15.57:0/1021754101 learned_addr learned my addr 172.21.15.57:0/1021754101 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:42.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2ba3b1700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2b415b090 0x7fd2b41f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2b9bb0700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2b406c390 0x7fd2b4153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2b93af700 1 -- 172.21.15.57:0/1021754101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2b406c390 msgr2=0x7fd2b4153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2b93af700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2b406c390 0x7fd2b4153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2b93af700 1 -- 172.21.15.57:0/1021754101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2b415b090 msgr2=0x7fd2b41f2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2b93af700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2b415b090 0x7fd2b41f2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2b93af700 1 -- 172.21.15.57:0/1021754101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b00094a0 con 0x7fd2b40c9bd0 2023-11-28T12:07:42.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.743+0000 7fd2b93af700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2b40c9bd0 0x7fd2b41538e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800c430 tx=0x7fd2a800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.744+0000 7fd2a67fc700 1 -- 172.21.15.57:0/1021754101 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8005b70 con 0x7fd2b40c9bd0 2023-11-28T12:07:42.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.744+0000 7fd2a67fc700 1 -- 172.21.15.57:0/1021754101 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a8005cd0 con 0x7fd2b40c9bd0 2023-11-28T12:07:42.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.744+0000 7fd2bbe14700 1 -- 172.21.15.57:0/1021754101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b406b060 con 0x7fd2b40c9bd0 2023-11-28T12:07:42.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.744+0000 7fd2a67fc700 1 -- 172.21.15.57:0/1021754101 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a80072f0 con 0x7fd2b40c9bd0 2023-11-28T12:07:42.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.744+0000 7fd2bbe14700 1 -- 172.21.15.57:0/1021754101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b406ad80 con 0x7fd2b40c9bd0 2023-11-28T12:07:42.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.745+0000 7fd2bbe14700 1 -- 172.21.15.57:0/1021754101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd2b406b340 con 0x7fd2b40c9bd0 2023-11-28T12:07:42.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.745+0000 7fd2a67fc700 1 -- 172.21.15.57:0/1021754101 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd2a8019030 con 0x7fd2b40c9bd0 2023-11-28T12:07:42.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.746+0000 7fd2a67fc700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd2980682f0 0x7fd29806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.746+0000 7fd2a67fc700 1 -- 172.21.15.57:0/1021754101 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd2a8084b30 con 0x7fd2b40c9bd0 2023-11-28T12:07:42.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.746+0000 7fd2a67fc700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x7fd29806e760 0x7fd298070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.746+0000 7fd2a67fc700 1 -- 172.21.15.57:0/1021754101 --> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd298071240 con 0x7fd29806e760 2023-11-28T12:07:42.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.746+0000 7fd2b9bb0700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd2980682f0 0x7fd29806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.746+0000 7fd2a67fc700 1 -- 172.21.15.57:0/1021754101 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2a80892a0 con 0x7fd2b40c9bd0 2023-11-28T12:07:42.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.746+0000 7fd2ba3b1700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x7fd29806e760 0x7fd298070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.746+0000 7fd2ba3b1700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x7fd29806e760 0x7fd298070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.746+0000 7fd2b9bb0700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd2980682f0 0x7fd29806a7a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0000c00 tx=0x7fd2b001d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.747+0000 7fd2a67fc700 1 -- 172.21.15.57:0/1021754101 <== osd.1 v2:172.21.15.57:6808/120755 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd298071240 con 0x7fd29806e760 2023-11-28T12:07:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.762+0000 7fd2bbe14700 1 -- 172.21.15.57:0/1021754101 --> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd2b40743f0 con 0x7fd29806e760 2023-11-28T12:07:42.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.762+0000 7fd2a67fc700 1 -- 172.21.15.57:0/1021754101 <== osd.1 v2:172.21.15.57:6808/120755 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd2b40743f0 con 0x7fd29806e760 2023-11-28T12:07:42.764 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:07:42.764 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T12:07:42.764 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:07:42.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.763+0000 7fd2bbe14700 1 -- 172.21.15.57:0/1021754101 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x7fd29806e760 msgr2=0x7fd298070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.763+0000 7fd2bbe14700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.57:6808/120755,v1:172.21.15.57:6809/120755] conn(0x7fd29806e760 0x7fd298070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.763+0000 7fd2bbe14700 1 -- 172.21.15.57:0/1021754101 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd2980682f0 msgr2=0x7fd29806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.763+0000 7fd2bbe14700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fd2980682f0 0x7fd29806a7a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0000c00 tx=0x7fd2b001d040 comp rx=0 tx=0).stop 2023-11-28T12:07:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.763+0000 7fd2bbe14700 1 -- 172.21.15.57:0/1021754101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2b40c9bd0 msgr2=0x7fd2b41538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.763+0000 7fd2bbe14700 1 --2- 172.21.15.57:0/1021754101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2b40c9bd0 0x7fd2b41538e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800c430 tx=0x7fd2a800c740 comp rx=0 tx=0).stop 2023-11-28T12:07:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.763+0000 7fd2ba3b1700 1 -- 172.21.15.57:0/1021754101 reap_dead start 2023-11-28T12:07:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.763+0000 7fd2bbe14700 1 -- 172.21.15.57:0/1021754101 shutdown_connections 2023-11-28T12:07:42.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.763+0000 7fd2bbe14700 1 -- 172.21.15.57:0/1021754101 >> 172.21.15.57:0/1021754101 conn(0x7fd2b40cf370 msgr2=0x7fd2b40ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:42.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.763+0000 7fd2bbe14700 1 -- 172.21.15.57:0/1021754101 shutdown_connections 2023-11-28T12:07:42.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.763+0000 7fd2bbe14700 1 -- 172.21.15.57:0/1021754101 wait complete. 2023-11-28T12:07:42.777 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-28T12:07:42.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.913+0000 7f57fdbdf700 1 Processor -- start 2023-11-28T12:07:42.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.913+0000 7f57fdbdf700 1 -- start start 2023-11-28T12:07:42.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.914+0000 7f57fdbdf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f80b1650 0x7f57f80b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.914+0000 7f57fdbdf700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f8159ec0 con 0x7f57f80af0a0 2023-11-28T12:07:42.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.914+0000 7f57fdbdf700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f815aa60 con 0x7f57f81599b0 2023-11-28T12:07:42.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.914+0000 7f57fdbdf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80b1f60 con 0x7f57f80b1650 2023-11-28T12:07:42.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.914+0000 7f57f7fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f57f80af0a0 0x7f57f815a350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47368/0 (socket says 172.21.15.57:47368) 2023-11-28T12:07:42.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.914+0000 7f57f7fff700 1 -- 172.21.15.57:0/2882035062 learned_addr learned my addr 172.21.15.57:0/2882035062 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:42.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f77fe700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f80b1650 0x7f57f80b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014933291 0 0) 0x7f57f8159ec0 con 0x7f57f80af0a0 2023-11-28T12:07:42.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e40035b0 con 0x7f57f80af0a0 2023-11-28T12:07:42.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1773561603 0 0) 0x7f57e40035b0 con 0x7f57f80af0a0 2023-11-28T12:07:42.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f80b1650 msgr2=0x7f57f80b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:42.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f80b1650 0x7f57f80b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 >> v1:172.21.15.181:6789/0 conn(0x7f57f81599b0 legacy=0x7f57f815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f815d8a0 con 0x7f57f80af0a0 2023-11-28T12:07:42.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f57f00019f0 con 0x7f57f80af0a0 2023-11-28T12:07:42.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 >> v1:172.21.15.57:6789/0 conn(0x7f57f80af0a0 legacy=0x7f57f815a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57e4003a30 0x7f57e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f80b1650 0x7f57e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57e4004d90 0x7f57e4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80b1f60 con 0x7f57e4003a30 2023-11-28T12:07:42.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e400ba10 con 0x7f57e4004d90 2023-11-28T12:07:42.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e400ddb0 con 0x7f57f80b1650 2023-11-28T12:07:42.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f77fe700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57e4003a30 0x7f57e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f7fff700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57e4004d90 0x7f57e4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f6ffd700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f80b1650 0x7f57e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f77fe700 1 -- 172.21.15.57:0/2882035062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f80b1650 msgr2=0x7f57e4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:42.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f77fe700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f80b1650 0x7f57e4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.915+0000 7f57f77fe700 1 -- 172.21.15.57:0/2882035062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57e4004d90 msgr2=0x7f57e4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:42.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.916+0000 7f57f77fe700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57e4004d90 0x7f57e4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.916+0000 7f57f77fe700 1 -- 172.21.15.57:0/2882035062 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e40035b0 con 0x7f57e4003a30 2023-11-28T12:07:42.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.916+0000 7f57f7fff700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57e4004d90 0x7f57e4005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:42.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.916+0000 7f57f6ffd700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f80b1650 0x7f57e4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:42.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.916+0000 7f57f77fe700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57e4003a30 0x7f57e4003e00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009b60 tx=0x7f57e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835d231521e2355c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.916+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e800e6c0 con 0x7f57e4003a30 2023-11-28T12:07:42.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.916+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e800e820 con 0x7f57e4003a30 2023-11-28T12:07:42.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.916+0000 7f57f67fc700 1 -- 172.21.15.57:0/2882035062 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e800eb90 con 0x7f57e4003a30 2023-11-28T12:07:42.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.916+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2882035062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57e4003a30 msgr2=0x7f57e4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.916+0000 7f57fdbdf700 1 --2- 172.21.15.57:0/2882035062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57e4003a30 0x7f57e4003e00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009b60 tx=0x7f57e4005880 comp rx=0 tx=0).stop 2023-11-28T12:07:42.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.916+0000 7f57f7fff700 1 -- 172.21.15.57:0/2882035062 reap_dead start 2023-11-28T12:07:42.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.916+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2882035062 shutdown_connections 2023-11-28T12:07:42.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.916+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2882035062 >> 172.21.15.57:0/2882035062 conn(0x7f57f806c010 msgr2=0x7f57f80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:42.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.917+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2882035062 shutdown_connections 2023-11-28T12:07:42.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.917+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2882035062 wait complete. 2023-11-28T12:07:42.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.918+0000 7f57fdbdf700 1 Processor -- start 2023-11-28T12:07:42.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.918+0000 7f57fdbdf700 1 -- start start 2023-11-28T12:07:42.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57fdbdf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57e4003a30 0x7f57f814cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57fdbdf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57f80cb0b0 0x7f57f80cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57fdbdf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f815b9a0 0x7f57f815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57fdbdf700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80cba60 con 0x7f57f80cb0b0 2023-11-28T12:07:42.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57fdbdf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80b3b40 con 0x7f57f815b9a0 2023-11-28T12:07:42.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57fdbdf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80b2aa0 con 0x7f57e4003a30 2023-11-28T12:07:42.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57f6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57f80cb0b0 0x7f57f80cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57f6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57f80cb0b0 0x7f57f80cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59230/0 (socket says 172.21.15.57:59230) 2023-11-28T12:07:42.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57f6ffd700 1 -- 172.21.15.57:0/2095258758 learned_addr learned my addr 172.21.15.57:0/2095258758 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:42.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57f7fff700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f815b9a0 0x7f57f815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57f77fe700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57e4003a30 0x7f57f814cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57f6ffd700 1 -- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57e4003a30 msgr2=0x7f57f814cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57f6ffd700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57e4003a30 0x7f57f814cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57f6ffd700 1 -- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f815b9a0 msgr2=0x7f57f815be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57f6ffd700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f815b9a0 0x7f57f815be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.919+0000 7f57f6ffd700 1 -- 172.21.15.57:0/2095258758 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f815de60 con 0x7f57f80cb0b0 2023-11-28T12:07:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.920+0000 7f57f6ffd700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57f80cb0b0 0x7f57f80cb520 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f57e000e800 tx=0x7f57e000eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.920+0000 7f57effff700 1 -- 172.21.15.57:0/2095258758 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e00052e0 con 0x7f57f80cb0b0 2023-11-28T12:07:42.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.920+0000 7f57effff700 1 -- 172.21.15.57:0/2095258758 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e0005440 con 0x7f57f80cb0b0 2023-11-28T12:07:42.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.920+0000 7f57effff700 1 -- 172.21.15.57:0/2095258758 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e00038a0 con 0x7f57f80cb0b0 2023-11-28T12:07:42.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.920+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2095258758 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f815db80 con 0x7f57f80cb0b0 2023-11-28T12:07:42.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.920+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2095258758 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f80b1a20 con 0x7f57f80cb0b0 2023-11-28T12:07:42.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.921+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2095258758 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f80743f0 con 0x7f57f80cb0b0 2023-11-28T12:07:42.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.921+0000 7f57effff700 1 -- 172.21.15.57:0/2095258758 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f57e001b030 con 0x7f57f80cb0b0 2023-11-28T12:07:42.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.922+0000 7f57effff700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f57d80682f0 0x7f57d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:42.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.922+0000 7f57effff700 1 -- 172.21.15.57:0/2095258758 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f57e0084f50 con 0x7f57f80cb0b0 2023-11-28T12:07:42.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.922+0000 7f57f77fe700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f57d80682f0 0x7f57d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:42.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.923+0000 7f57f77fe700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f57d80682f0 0x7f57d806a7a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f57e80090b0 tx=0x7f57e801f040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:42.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:42.926+0000 7f57effff700 1 -- 172.21.15.57:0/2095258758 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f57e004d5e0 con 0x7f57f80cb0b0 2023-11-28T12:07:43.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.126+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2095258758 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f57f806e620 con 0x7f57d80682f0 2023-11-28T12:07:43.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.127+0000 7f57effff700 1 -- 172.21.15.57:0/2095258758 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f57f806e620 con 0x7f57d80682f0 2023-11-28T12:07:43.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.129+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f57d80682f0 msgr2=0x7f57d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.129+0000 7f57fdbdf700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f57d80682f0 0x7f57d806a7a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f57e80090b0 tx=0x7f57e801f040 comp rx=0 tx=0).stop 2023-11-28T12:07:43.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.129+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2095258758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57f80cb0b0 msgr2=0x7f57f80cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.129+0000 7f57fdbdf700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57f80cb0b0 0x7f57f80cb520 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f57e000e800 tx=0x7f57e000eb10 comp rx=0 tx=0).stop 2023-11-28T12:07:43.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.129+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2095258758 shutdown_connections 2023-11-28T12:07:43.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.129+0000 7f57fdbdf700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f57d80682f0 0x7f57d806a7a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.129+0000 7f57fdbdf700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f815b9a0 0x7f57f815be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.129+0000 7f57fdbdf700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57e4003a30 0x7f57f814cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.129+0000 7f57fdbdf700 1 --2- 172.21.15.57:0/2095258758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57f80cb0b0 0x7f57f80cb520 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.129+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2095258758 >> 172.21.15.57:0/2095258758 conn(0x7f57f806c010 msgr2=0x7f57f815d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:43.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.129+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2095258758 shutdown_connections 2023-11-28T12:07:43.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.129+0000 7f57fdbdf700 1 -- 172.21.15.57:0/2095258758 wait complete. 2023-11-28T12:07:43.133 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 1 to deep-scrub 2023-11-28T12:07:43.143 INFO:tasks.ceph:Scrubbing osd.2 2023-11-28T12:07:43.144 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-28T12:07:43.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.282+0000 7feca3d2c700 1 Processor -- start 2023-11-28T12:07:43.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.283+0000 7feca3d2c700 1 -- start start 2023-11-28T12:07:43.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.283+0000 7feca3d2c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec9c163f60 0x7fec9c0cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca3d2c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fec9c0c4430 0x7fec9c0b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca3d2c700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c0cbe60 con 0x7fec9c0ccf00 2023-11-28T12:07:43.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca3d2c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9c0cc140 con 0x7fec9c0c4430 2023-11-28T12:07:43.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca3d2c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0cc420 con 0x7fec9c163f60 2023-11-28T12:07:43.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca12c7700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fec9c0ccf00 0x7fec9c0c3d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47376/0 (socket says 172.21.15.57:47376) 2023-11-28T12:07:43.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca12c7700 1 -- 172.21.15.57:0/2917156262 learned_addr learned my addr 172.21.15.57:0/2917156262 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:43.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca1ac8700 1 --2- 172.21.15.57:0/2917156262 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec9c163f60 0x7fec9c0cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca22c9700 1 --2- 172.21.15.57:0/2917156262 >> v2:172.21.15.181:3300/0 conn(0x7fec9c0c4430 0x7fec9c0b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca0ac6700 1 -- 172.21.15.57:0/2917156262 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600533775 0 0) 0x7fec9c0cbe60 con 0x7fec9c0ccf00 2023-11-28T12:07:43.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca0ac6700 1 -- 172.21.15.57:0/2917156262 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec880035b0 con 0x7fec9c0ccf00 2023-11-28T12:07:43.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca22c9700 1 -- 172.21.15.57:0/2917156262 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec9c163f60 msgr2=0x7fec9c0cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:43.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca22c9700 1 --2- 172.21.15.57:0/2917156262 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec9c163f60 0x7fec9c0cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca22c9700 1 -- 172.21.15.57:0/2917156262 >> v1:172.21.15.57:6789/0 conn(0x7fec9c0ccf00 legacy=0x7fec9c0c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca22c9700 1 -- 172.21.15.57:0/2917156262 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c0cc700 con 0x7fec9c0c4430 2023-11-28T12:07:43.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca1ac8700 1 --2- 172.21.15.57:0/2917156262 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec9c163f60 0x7fec9c0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:07:43.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.284+0000 7feca22c9700 1 --2- 172.21.15.57:0/2917156262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec9c0c4430 0x7fec9c0b0560 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fec94009d90 tx=0x7fec9400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d4275c2f77b4038 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:43.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.285+0000 7feca0ac6700 1 -- 172.21.15.57:0/2917156262 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9400e070 con 0x7fec9c0c4430 2023-11-28T12:07:43.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.285+0000 7feca0ac6700 1 -- 172.21.15.57:0/2917156262 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9400bb20 con 0x7fec9c0c4430 2023-11-28T12:07:43.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.285+0000 7feca0ac6700 1 -- 172.21.15.57:0/2917156262 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94012430 con 0x7fec9c0c4430 2023-11-28T12:07:43.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.285+0000 7feca3d2c700 1 -- 172.21.15.57:0/2917156262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec9c0c4430 msgr2=0x7fec9c0b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.285+0000 7feca3d2c700 1 --2- 172.21.15.57:0/2917156262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec9c0c4430 0x7fec9c0b0560 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fec94009d90 tx=0x7fec9400b330 comp rx=0 tx=0).stop 2023-11-28T12:07:43.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.285+0000 7feca3d2c700 1 -- 172.21.15.57:0/2917156262 shutdown_connections 2023-11-28T12:07:43.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.285+0000 7feca3d2c700 1 --2- 172.21.15.57:0/2917156262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec9c0c4430 0x7fec9c0b0560 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.285+0000 7feca3d2c700 1 --2- 172.21.15.57:0/2917156262 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec9c163f60 0x7fec9c0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.285+0000 7feca3d2c700 1 -- 172.21.15.57:0/2917156262 >> 172.21.15.57:0/2917156262 conn(0x7fec9c06ab90 msgr2=0x7fec9c099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:43.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.285+0000 7feca3d2c700 1 -- 172.21.15.57:0/2917156262 shutdown_connections 2023-11-28T12:07:43.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.285+0000 7feca3d2c700 1 -- 172.21.15.57:0/2917156262 wait complete. 2023-11-28T12:07:43.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.287+0000 7feca3d2c700 1 Processor -- start 2023-11-28T12:07:43.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.287+0000 7feca3d2c700 1 -- start start 2023-11-28T12:07:43.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.287+0000 7feca3d2c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec9c0c4430 0x7fec9c0c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.287+0000 7feca3d2c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec9c0ccf00 0x7fec9c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.287+0000 7feca3d2c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec9c163f60 0x7fec9c15aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.287+0000 7feca3d2c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0b3ed0 con 0x7fec9c163f60 2023-11-28T12:07:43.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.287+0000 7feca3d2c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0b3b80 con 0x7fec9c0c4430 2023-11-28T12:07:43.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.287+0000 7feca3d2c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0c5180 con 0x7fec9c0ccf00 2023-11-28T12:07:43.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca22c9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec9c163f60 0x7fec9c15aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca22c9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec9c163f60 0x7fec9c15aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59234/0 (socket says 172.21.15.57:59234) 2023-11-28T12:07:43.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca22c9700 1 -- 172.21.15.57:0/1424413573 learned_addr learned my addr 172.21.15.57:0/1424413573 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:43.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca1ac8700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec9c0c4430 0x7fec9c0c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca12c7700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec9c0ccf00 0x7fec9c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca22c9700 1 -- 172.21.15.57:0/1424413573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec9c0ccf00 msgr2=0x7fec9c0b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca22c9700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec9c0ccf00 0x7fec9c0b6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca22c9700 1 -- 172.21.15.57:0/1424413573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec9c0c4430 msgr2=0x7fec9c0c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca22c9700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec9c0c4430 0x7fec9c0c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca22c9700 1 -- 172.21.15.57:0/1424413573 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec98009430 con 0x7fec9c163f60 2023-11-28T12:07:43.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca12c7700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec9c0ccf00 0x7fec9c0b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:43.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca22c9700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec9c163f60 0x7fec9c15aed0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0d1e50 tx=0x7fec94004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:43.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7fec8e7fc700 1 -- 172.21.15.57:0/1424413573 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec940129f0 con 0x7fec9c163f60 2023-11-28T12:07:43.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7fec8e7fc700 1 -- 172.21.15.57:0/1424413573 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec94020070 con 0x7fec9c163f60 2023-11-28T12:07:43.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca3d2c700 1 -- 172.21.15.57:0/1424413573 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec94009a70 con 0x7fec9c163f60 2023-11-28T12:07:43.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7fec8e7fc700 1 -- 172.21.15.57:0/1424413573 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94012db0 con 0x7fec9c163f60 2023-11-28T12:07:43.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.288+0000 7feca3d2c700 1 -- 172.21.15.57:0/1424413573 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c0cd840 con 0x7fec9c163f60 2023-11-28T12:07:43.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.289+0000 7feca3d2c700 1 -- 172.21.15.57:0/1424413573 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec9c0cc9e0 con 0x7fec9c163f60 2023-11-28T12:07:43.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.290+0000 7fec8e7fc700 1 -- 172.21.15.57:0/1424413573 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fec9402e070 con 0x7fec9c163f60 2023-11-28T12:07:43.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.290+0000 7fec8e7fc700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec800682f0 0x7fec8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.290+0000 7fec8e7fc700 1 -- 172.21.15.57:0/1424413573 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fec94022070 con 0x7fec9c163f60 2023-11-28T12:07:43.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.290+0000 7fec8e7fc700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] conn(0x7fec8006e760 0x7fec80070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.290+0000 7fec8e7fc700 1 -- 172.21.15.57:0/1424413573 --> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec80071240 con 0x7fec8006e760 2023-11-28T12:07:43.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.290+0000 7fec8e7fc700 1 -- 172.21.15.57:0/1424413573 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec94087f40 con 0x7fec9c163f60 2023-11-28T12:07:43.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.291+0000 7feca1ac8700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec800682f0 0x7fec8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.291+0000 7feca12c7700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] conn(0x7fec8006e760 0x7fec80070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.291+0000 7feca12c7700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] conn(0x7fec8006e760 0x7fec80070b70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:43.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.291+0000 7feca1ac8700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec800682f0 0x7fec8006a7a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0b1e80 tx=0x7fec980091c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:43.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.292+0000 7fec8e7fc700 1 -- 172.21.15.57:0/1424413573 <== osd.2 v2:172.21.15.57:6816/120771 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fec80071240 con 0x7fec8006e760 2023-11-28T12:07:43.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.307+0000 7feca3d2c700 1 -- 172.21.15.57:0/1424413573 --> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fec9c164330 con 0x7fec8006e760 2023-11-28T12:07:43.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.307+0000 7fec8e7fc700 1 -- 172.21.15.57:0/1424413573 <== osd.2 v2:172.21.15.57:6816/120771 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fec9c164330 con 0x7fec8006e760 2023-11-28T12:07:43.309 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:07:43.309 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T12:07:43.309 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:07:43.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.307+0000 7feca3d2c700 1 -- 172.21.15.57:0/1424413573 >> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] conn(0x7fec8006e760 msgr2=0x7fec80070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.307+0000 7feca3d2c700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.57:6816/120771,v1:172.21.15.57:6818/120771] conn(0x7fec8006e760 0x7fec80070b70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.307+0000 7feca3d2c700 1 -- 172.21.15.57:0/1424413573 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec800682f0 msgr2=0x7fec8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.307+0000 7feca3d2c700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec800682f0 0x7fec8006a7a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0b1e80 tx=0x7fec980091c0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.307+0000 7feca3d2c700 1 -- 172.21.15.57:0/1424413573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec9c163f60 msgr2=0x7fec9c15aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.307+0000 7feca3d2c700 1 --2- 172.21.15.57:0/1424413573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec9c163f60 0x7fec9c15aed0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0d1e50 tx=0x7fec94004ad0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.307+0000 7feca22c9700 1 -- 172.21.15.57:0/1424413573 reap_dead start 2023-11-28T12:07:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.307+0000 7feca3d2c700 1 -- 172.21.15.57:0/1424413573 shutdown_connections 2023-11-28T12:07:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.307+0000 7feca3d2c700 1 -- 172.21.15.57:0/1424413573 >> 172.21.15.57:0/1424413573 conn(0x7fec9c06ab90 msgr2=0x7fec9c162170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:43.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.307+0000 7feca3d2c700 1 -- 172.21.15.57:0/1424413573 shutdown_connections 2023-11-28T12:07:43.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.307+0000 7feca3d2c700 1 -- 172.21.15.57:0/1424413573 wait complete. 2023-11-28T12:07:43.322 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-28T12:07:43.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.457+0000 7f2027b33700 1 Processor -- start 2023-11-28T12:07:43.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.457+0000 7f2027b33700 1 -- start start 2023-11-28T12:07:43.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f2027b33700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2020160e20 0x7f2020070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f2027b33700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f20200ae660 0x7f202006c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f2027b33700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20200ce740 con 0x7f2020160a50 2023-11-28T12:07:43.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f2027b33700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2020071870 con 0x7f20200ae660 2023-11-28T12:07:43.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f2027b33700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20200af450 con 0x7f2020160e20 2023-11-28T12:07:43.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f20260d0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f20200ae660 0x7f202006c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f20260d0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f20200ae660 0x7f202006c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59236/0 (socket says 172.21.15.57:59236) 2023-11-28T12:07:43.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f20260d0700 1 -- 172.21.15.57:0/469133561 learned_addr learned my addr 172.21.15.57:0/469133561 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:43.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f20250ce700 1 --2- 172.21.15.57:0/469133561 >> v2:172.21.15.181:3300/0 conn(0x7f2020160e20 0x7f2020070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f20260d0700 1 -- 172.21.15.57:0/469133561 >> v2:172.21.15.181:3300/0 conn(0x7f2020160e20 msgr2=0x7f2020070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:43.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f20260d0700 1 --2- 172.21.15.57:0/469133561 >> v2:172.21.15.181:3300/0 conn(0x7f2020160e20 0x7f2020070bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f20260d0700 1 -- 172.21.15.57:0/469133561 >> v1:172.21.15.181:6789/0 conn(0x7f2020160a50 legacy=0x7f20200cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:43.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f20260d0700 1 -- 172.21.15.57:0/469133561 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20200712b0 con 0x7f20200ae660 2023-11-28T12:07:43.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f20250ce700 1 --2- 172.21.15.57:0/469133561 >> v2:172.21.15.181:3300/0 conn(0x7f2020160e20 0x7f2020070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:43.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f20260d0700 1 --2- 172.21.15.57:0/469133561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20200ae660 0x7f202006c880 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2020071c50 tx=0x7f201800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f972acf7cd254a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:43.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.458+0000 7f20248cd700 1 -- 172.21.15.57:0/469133561 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201800e070 con 0x7f20200ae660 2023-11-28T12:07:43.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.459+0000 7f20248cd700 1 -- 172.21.15.57:0/469133561 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2018003a20 con 0x7f20200ae660 2023-11-28T12:07:43.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.459+0000 7f20248cd700 1 -- 172.21.15.57:0/469133561 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018003dd0 con 0x7f20200ae660 2023-11-28T12:07:43.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.459+0000 7f2027b33700 1 -- 172.21.15.57:0/469133561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20200ae660 msgr2=0x7f202006c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.459+0000 7f2027b33700 1 --2- 172.21.15.57:0/469133561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20200ae660 0x7f202006c880 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2020071c50 tx=0x7f201800b330 comp rx=0 tx=0).stop 2023-11-28T12:07:43.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.459+0000 7f2027b33700 1 -- 172.21.15.57:0/469133561 shutdown_connections 2023-11-28T12:07:43.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.459+0000 7f2027b33700 1 --2- 172.21.15.57:0/469133561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20200ae660 0x7f202006c880 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.459+0000 7f2027b33700 1 --2- 172.21.15.57:0/469133561 >> v2:172.21.15.181:3300/0 conn(0x7f2020160e20 0x7f2020070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.459+0000 7f2027b33700 1 -- 172.21.15.57:0/469133561 >> 172.21.15.57:0/469133561 conn(0x7f202006c010 msgr2=0x7f20200cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:43.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.459+0000 7f2027b33700 1 -- 172.21.15.57:0/469133561 shutdown_connections 2023-11-28T12:07:43.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.459+0000 7f2027b33700 1 -- 172.21.15.57:0/469133561 wait complete. 2023-11-28T12:07:43.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.460+0000 7f2027b33700 1 Processor -- start 2023-11-28T12:07:43.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.461+0000 7f2027b33700 1 -- start start 2023-11-28T12:07:43.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.461+0000 7f2027b33700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20200ae660 0x7f20200ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.461+0000 7f2027b33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2020160a50 0x7f202006d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.461+0000 7f2027b33700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2020160e20 0x7f202006dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.461+0000 7f2027b33700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20200c3b70 con 0x7f20200ae660 2023-11-28T12:07:43.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.461+0000 7f2027b33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20200c3e50 con 0x7f2020160a50 2023-11-28T12:07:43.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.461+0000 7f2027b33700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20200c4130 con 0x7f2020160e20 2023-11-28T12:07:43.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.461+0000 7f20258cf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20200ae660 0x7f20200ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.461+0000 7f20258cf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20200ae660 0x7f20200ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59242/0 (socket says 172.21.15.57:59242) 2023-11-28T12:07:43.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.461+0000 7f20258cf700 1 -- 172.21.15.57:0/1479700176 learned_addr learned my addr 172.21.15.57:0/1479700176 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:43.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f20250ce700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2020160a50 0x7f202006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f20258cf700 1 -- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2020160e20 msgr2=0x7f202006dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f20260d0700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2020160e20 0x7f202006dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f20258cf700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2020160e20 0x7f202006dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f20258cf700 1 -- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2020160a50 msgr2=0x7f202006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f20258cf700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2020160a50 0x7f202006d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f20258cf700 1 -- 172.21.15.57:0/1479700176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2018009b30 con 0x7f20200ae660 2023-11-28T12:07:43.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f20250ce700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2020160a50 0x7f202006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:43.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f20260d0700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2020160e20 0x7f202006dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:43.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f20258cf700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20200ae660 0x7f20200ccd80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f201c00c4d0 tx=0x7f201c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:43.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f20167fc700 1 -- 172.21.15.57:0/1479700176 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c011790 con 0x7f20200ae660 2023-11-28T12:07:43.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f20167fc700 1 -- 172.21.15.57:0/1479700176 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c0118f0 con 0x7f20200ae660 2023-11-28T12:07:43.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f2027b33700 1 -- 172.21.15.57:0/1479700176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20200ce970 con 0x7f20200ae660 2023-11-28T12:07:43.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f20167fc700 1 -- 172.21.15.57:0/1479700176 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c011ca0 con 0x7f20200ae660 2023-11-28T12:07:43.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.462+0000 7f2027b33700 1 -- 172.21.15.57:0/1479700176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2020071590 con 0x7f20200ae660 2023-11-28T12:07:43.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.463+0000 7f2027b33700 1 -- 172.21.15.57:0/1479700176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20200cec50 con 0x7f20200ae660 2023-11-28T12:07:43.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.464+0000 7f20167fc700 1 -- 172.21.15.57:0/1479700176 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f201c010050 con 0x7f20200ae660 2023-11-28T12:07:43.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.464+0000 7f20167fc700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20040682f0 0x7f200406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.464+0000 7f20167fc700 1 -- 172.21.15.57:0/1479700176 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f201c085710 con 0x7f20200ae660 2023-11-28T12:07:43.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.464+0000 7f20250ce700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20040682f0 0x7f200406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.465+0000 7f20250ce700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20040682f0 0x7f200406a7a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f20200c54d0 tx=0x7f2010008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:43.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.469+0000 7f20167fc700 1 -- 172.21.15.57:0/1479700176 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f201c052270 con 0x7f20200ae660 2023-11-28T12:07:43.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.670+0000 7f2027b33700 1 -- 172.21.15.57:0/1479700176 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f20200743f0 con 0x7f20040682f0 2023-11-28T12:07:43.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.670+0000 7f20167fc700 1 -- 172.21.15.57:0/1479700176 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f20200743f0 con 0x7f20040682f0 2023-11-28T12:07:43.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.672+0000 7f2027b33700 1 -- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20040682f0 msgr2=0x7f200406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.672+0000 7f2027b33700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20040682f0 0x7f200406a7a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f20200c54d0 tx=0x7f2010008040 comp rx=0 tx=0).stop 2023-11-28T12:07:43.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.672+0000 7f2027b33700 1 -- 172.21.15.57:0/1479700176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20200ae660 msgr2=0x7f20200ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.672+0000 7f2027b33700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20200ae660 0x7f20200ccd80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f201c00c4d0 tx=0x7f201c00c890 comp rx=0 tx=0).stop 2023-11-28T12:07:43.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.673+0000 7f2027b33700 1 -- 172.21.15.57:0/1479700176 shutdown_connections 2023-11-28T12:07:43.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.673+0000 7f2027b33700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f20040682f0 0x7f200406a7a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.673+0000 7f2027b33700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2020160e20 0x7f202006dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.673+0000 7f2027b33700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20200ae660 0x7f20200ccd80 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.673+0000 7f2027b33700 1 --2- 172.21.15.57:0/1479700176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2020160a50 0x7f202006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.673+0000 7f2027b33700 1 -- 172.21.15.57:0/1479700176 >> 172.21.15.57:0/1479700176 conn(0x7f202006c010 msgr2=0x7f202006c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:43.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.673+0000 7f2027b33700 1 -- 172.21.15.57:0/1479700176 shutdown_connections 2023-11-28T12:07:43.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.673+0000 7f2027b33700 1 -- 172.21.15.57:0/1479700176 wait complete. 2023-11-28T12:07:43.676 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 2 to deep-scrub 2023-11-28T12:07:43.687 INFO:tasks.ceph:Scrubbing osd.3 2023-11-28T12:07:43.687 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-28T12:07:43.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.827+0000 7f2c672a7700 1 Processor -- start 2023-11-28T12:07:43.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.827+0000 7f2c672a7700 1 -- start start 2023-11-28T12:07:43.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c672a7700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2c6006ba90 0x7f2c6015a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c672a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c600ceea0 0x7f2c6015a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c672a7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c600cea10 con 0x7f2c6015e070 2023-11-28T12:07:43.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c672a7700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6015ae80 con 0x7f2c6006ba90 2023-11-28T12:07:43.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c672a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6015abd0 con 0x7f2c600ceea0 2023-11-28T12:07:43.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c64842700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2c6006ba90 0x7f2c6015a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c64842700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2c6006ba90 0x7f2c6015a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59248/0 (socket says 172.21.15.57:59248) 2023-11-28T12:07:43.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c64842700 1 -- 172.21.15.57:0/125572586 learned_addr learned my addr 172.21.15.57:0/125572586 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:43.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c65844700 1 --2- 172.21.15.57:0/125572586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c600ceea0 0x7f2c6015a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c64842700 1 -- 172.21.15.57:0/125572586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c600ceea0 msgr2=0x7f2c6015a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:43.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c64842700 1 --2- 172.21.15.57:0/125572586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c600ceea0 0x7f2c6015a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c64842700 1 -- 172.21.15.57:0/125572586 >> v1:172.21.15.181:6789/0 conn(0x7f2c6015e070 legacy=0x7f2c6006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:43.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c64842700 1 -- 172.21.15.57:0/125572586 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c600cbd10 con 0x7f2c6006ba90 2023-11-28T12:07:43.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.828+0000 7f2c65844700 1 --2- 172.21.15.57:0/125572586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c600ceea0 0x7f2c6015a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:43.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.829+0000 7f2c64842700 1 --2- 172.21.15.57:0/125572586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c6006ba90 0x7f2c6015a150 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2c500098f0 tx=0x7f2c50009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1438230e922b677 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:43.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.829+0000 7f2c57fff700 1 -- 172.21.15.57:0/125572586 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50004230 con 0x7f2c6006ba90 2023-11-28T12:07:43.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.829+0000 7f2c57fff700 1 -- 172.21.15.57:0/125572586 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c50004390 con 0x7f2c6006ba90 2023-11-28T12:07:43.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.829+0000 7f2c57fff700 1 -- 172.21.15.57:0/125572586 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c500036a0 con 0x7f2c6006ba90 2023-11-28T12:07:43.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.829+0000 7f2c672a7700 1 -- 172.21.15.57:0/125572586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c6006ba90 msgr2=0x7f2c6015a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.829+0000 7f2c672a7700 1 --2- 172.21.15.57:0/125572586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c6006ba90 0x7f2c6015a150 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2c500098f0 tx=0x7f2c50009c00 comp rx=0 tx=0).stop 2023-11-28T12:07:43.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.829+0000 7f2c672a7700 1 -- 172.21.15.57:0/125572586 shutdown_connections 2023-11-28T12:07:43.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.829+0000 7f2c672a7700 1 --2- 172.21.15.57:0/125572586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c600ceea0 0x7f2c6015a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.829+0000 7f2c672a7700 1 --2- 172.21.15.57:0/125572586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c6006ba90 0x7f2c6015a150 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.829+0000 7f2c672a7700 1 -- 172.21.15.57:0/125572586 >> 172.21.15.57:0/125572586 conn(0x7f2c600cfc30 msgr2=0x7f2c600d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:43.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.829+0000 7f2c672a7700 1 -- 172.21.15.57:0/125572586 shutdown_connections 2023-11-28T12:07:43.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.829+0000 7f2c672a7700 1 -- 172.21.15.57:0/125572586 wait complete. 2023-11-28T12:07:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.831+0000 7f2c672a7700 1 Processor -- start 2023-11-28T12:07:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.831+0000 7f2c672a7700 1 -- start start 2023-11-28T12:07:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.831+0000 7f2c672a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c6006ba90 0x7f2c60071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.831+0000 7f2c672a7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c600ceea0 0x7f2c600715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.831+0000 7f2c672a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c6015e070 0x7f2c6006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.831+0000 7f2c672a7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6006eb40 con 0x7f2c600ceea0 2023-11-28T12:07:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c672a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6006ee20 con 0x7f2c6006ba90 2023-11-28T12:07:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c672a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6013e1e0 con 0x7f2c6015e070 2023-11-28T12:07:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c64842700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c600ceea0 0x7f2c600715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c64842700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c600ceea0 0x7f2c600715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59250/0 (socket says 172.21.15.57:59250) 2023-11-28T12:07:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c64842700 1 -- 172.21.15.57:0/1653050423 learned_addr learned my addr 172.21.15.57:0/1653050423 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c65844700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c6015e070 0x7f2c6006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c65043700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c6006ba90 0x7f2c60071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c64842700 1 -- 172.21.15.57:0/1653050423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c6015e070 msgr2=0x7f2c6006ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c64842700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c6015e070 0x7f2c6006ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c64842700 1 -- 172.21.15.57:0/1653050423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c6006ba90 msgr2=0x7f2c60071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c64842700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c6006ba90 0x7f2c60071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c64842700 1 -- 172.21.15.57:0/1653050423 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c50009550 con 0x7f2c600ceea0 2023-11-28T12:07:43.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c65844700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c6015e070 0x7f2c6006ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:43.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c64842700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c600ceea0 0x7f2c600715d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009130 tx=0x7f2c50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:43.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c55ffb700 1 -- 172.21.15.57:0/1653050423 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50003c50 con 0x7f2c600ceea0 2023-11-28T12:07:43.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c55ffb700 1 -- 172.21.15.57:0/1653050423 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c50020070 con 0x7f2c600ceea0 2023-11-28T12:07:43.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c672a7700 1 -- 172.21.15.57:0/1653050423 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c600ce3c0 con 0x7f2c600ceea0 2023-11-28T12:07:43.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.832+0000 7f2c55ffb700 1 -- 172.21.15.57:0/1653050423 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5000b660 con 0x7f2c600ceea0 2023-11-28T12:07:43.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.833+0000 7f2c672a7700 1 -- 172.21.15.57:0/1653050423 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c600ce0e0 con 0x7f2c600ceea0 2023-11-28T12:07:43.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.833+0000 7f2c672a7700 1 -- 172.21.15.57:0/1653050423 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c6013e060 con 0x7f2c600ceea0 2023-11-28T12:07:43.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.834+0000 7f2c55ffb700 1 -- 172.21.15.57:0/1653050423 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2c5002b020 con 0x7f2c600ceea0 2023-11-28T12:07:43.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.835+0000 7f2c55ffb700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2c440682f0 0x7f2c4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.835+0000 7f2c55ffb700 1 -- 172.21.15.57:0/1653050423 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2c50089520 con 0x7f2c600ceea0 2023-11-28T12:07:43.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.835+0000 7f2c65043700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2c440682f0 0x7f2c4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.835+0000 7f2c55ffb700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] conn(0x7f2c4406e760 0x7f2c44070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:43.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.835+0000 7f2c55ffb700 1 -- 172.21.15.57:0/1653050423 --> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c44071240 con 0x7f2c4406e760 2023-11-28T12:07:43.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.835+0000 7f2c55ffb700 1 -- 172.21.15.57:0/1653050423 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c50006ea0 con 0x7f2c600ceea0 2023-11-28T12:07:43.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.835+0000 7f2c65844700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] conn(0x7f2c4406e760 0x7f2c44070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:43.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.835+0000 7f2c65844700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] conn(0x7f2c4406e760 0x7f2c44070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:43.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.835+0000 7f2c65043700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2c440682f0 0x7f2c4406a7a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c007fd0 tx=0x7f2c5c007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:43.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.836+0000 7f2c55ffb700 1 -- 172.21.15.57:0/1653050423 <== osd.3 v2:172.21.15.57:6817/120770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2c44071240 con 0x7f2c4406e760 2023-11-28T12:07:43.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.851+0000 7f2c672a7700 1 -- 172.21.15.57:0/1653050423 --> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2c600ce6a0 con 0x7f2c4406e760 2023-11-28T12:07:43.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.852+0000 7f2c55ffb700 1 -- 172.21.15.57:0/1653050423 <== osd.3 v2:172.21.15.57:6817/120770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2c600ce6a0 con 0x7f2c4406e760 2023-11-28T12:07:43.854 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:07:43.854 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T12:07:43.854 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:07:43.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.852+0000 7f2c672a7700 1 -- 172.21.15.57:0/1653050423 >> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] conn(0x7f2c4406e760 msgr2=0x7f2c44070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.852+0000 7f2c672a7700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.57:6817/120770,v1:172.21.15.57:6820/120770] conn(0x7f2c4406e760 0x7f2c44070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.852+0000 7f2c672a7700 1 -- 172.21.15.57:0/1653050423 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2c440682f0 msgr2=0x7f2c4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.852+0000 7f2c672a7700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f2c440682f0 0x7f2c4406a7a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c007fd0 tx=0x7f2c5c007ec0 comp rx=0 tx=0).stop 2023-11-28T12:07:43.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.852+0000 7f2c672a7700 1 -- 172.21.15.57:0/1653050423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c600ceea0 msgr2=0x7f2c600715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:43.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.852+0000 7f2c672a7700 1 --2- 172.21.15.57:0/1653050423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c600ceea0 0x7f2c600715d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009130 tx=0x7f2c50004750 comp rx=0 tx=0).stop 2023-11-28T12:07:43.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.852+0000 7f2c65844700 1 -- 172.21.15.57:0/1653050423 reap_dead start 2023-11-28T12:07:43.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.852+0000 7f2c672a7700 1 -- 172.21.15.57:0/1653050423 shutdown_connections 2023-11-28T12:07:43.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.852+0000 7f2c672a7700 1 -- 172.21.15.57:0/1653050423 >> 172.21.15.57:0/1653050423 conn(0x7f2c600cfc30 msgr2=0x7f2c601582c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:43.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.852+0000 7f2c672a7700 1 -- 172.21.15.57:0/1653050423 shutdown_connections 2023-11-28T12:07:43.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:43.852+0000 7f2c672a7700 1 -- 172.21.15.57:0/1653050423 wait complete. 2023-11-28T12:07:43.866 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-28T12:07:44.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.002+0000 7f63a5e83700 1 Processor -- start 2023-11-28T12:07:44.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.002+0000 7f63a5e83700 1 -- start start 2023-11-28T12:07:44.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f63a5e83700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a015a5f0 0x7f63a00d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f63a5e83700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a015a9c0 con 0x7f63a00cc200 2023-11-28T12:07:44.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f63a5e83700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a00c7b80 con 0x7f63a00b2ee0 2023-11-28T12:07:44.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f63a5e83700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00cc990 con 0x7f63a015a5f0 2023-11-28T12:07:44.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639effd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f63a00cc200 0x7f63a00b27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47384/0 (socket says 172.21.15.57:47384) 2023-11-28T12:07:44.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639effd700 1 -- 172.21.15.57:0/293553277 learned_addr learned my addr 172.21.15.57:0/293553277 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:44.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639f7fe700 1 --2- 172.21.15.57:0/293553277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a015a5f0 0x7f63a00d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217869233 0 0) 0x7f63a015a9c0 con 0x7f63a00cc200 2023-11-28T12:07:44.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f638c0035b0 con 0x7f63a00cc200 2023-11-28T12:07:44.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249742317 0 0) 0x7f63a00c7b80 con 0x7f63a00b2ee0 2023-11-28T12:07:44.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63a015a9c0 con 0x7f63a00b2ee0 2023-11-28T12:07:44.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1207961849 0 0) 0x7f638c0035b0 con 0x7f63a00cc200 2023-11-28T12:07:44.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a015a5f0 msgr2=0x7f63a00d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:44.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639e7fc700 1 --2- 172.21.15.57:0/293553277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a015a5f0 0x7f63a00d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 >> v1:172.21.15.181:6789/0 conn(0x7f63a00b2ee0 legacy=0x7f63a00ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a00ccc70 con 0x7f63a00cc200 2023-11-28T12:07:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f6390001670 con 0x7f63a00cc200 2023-11-28T12:07:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 >> v1:172.21.15.57:6789/0 conn(0x7f63a00cc200 legacy=0x7f63a00b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639e7fc700 1 --2- 172.21.15.57:0/293553277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f638c003bc0 0x7f638c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.003+0000 7f639e7fc700 1 --2- 172.21.15.57:0/293553277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a015a5f0 0x7f638c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639e7fc700 1 --2- 172.21.15.57:0/293553277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f638c004d90 0x7f638c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00cc990 con 0x7f638c003bc0 2023-11-28T12:07:44.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c00bac0 con 0x7f638c004d90 2023-11-28T12:07:44.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c00bc30 con 0x7f63a015a5f0 2023-11-28T12:07:44.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639f7fe700 1 --2- 172.21.15.57:0/293553277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f638c003bc0 0x7f638c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639effd700 1 --2- 172.21.15.57:0/293553277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a015a5f0 0x7f638c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639ffff700 1 --2- 172.21.15.57:0/293553277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f638c004d90 0x7f638c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639f7fe700 1 -- 172.21.15.57:0/293553277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a015a5f0 msgr2=0x7f638c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:44.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639f7fe700 1 --2- 172.21.15.57:0/293553277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a015a5f0 0x7f638c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639f7fe700 1 -- 172.21.15.57:0/293553277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f638c004d90 msgr2=0x7f638c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:44.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639f7fe700 1 --2- 172.21.15.57:0/293553277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f638c004d90 0x7f638c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639f7fe700 1 -- 172.21.15.57:0/293553277 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c0035b0 con 0x7f638c003bc0 2023-11-28T12:07:44.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639f7fe700 1 --2- 172.21.15.57:0/293553277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f638c003bc0 0x7f638c003fb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6388009b60 tx=0x7f638c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcae05bc173fecb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638800e660 con 0x7f638c003bc0 2023-11-28T12:07:44.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638800e7c0 con 0x7f638c003bc0 2023-11-28T12:07:44.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.004+0000 7f639e7fc700 1 -- 172.21.15.57:0/293553277 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638800eb30 con 0x7f638c003bc0 2023-11-28T12:07:44.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.005+0000 7f63a5e83700 1 -- 172.21.15.57:0/293553277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f638c003bc0 msgr2=0x7f638c003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.005+0000 7f63a5e83700 1 --2- 172.21.15.57:0/293553277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f638c003bc0 0x7f638c003fb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6388009b60 tx=0x7f638c005880 comp rx=0 tx=0).stop 2023-11-28T12:07:44.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.005+0000 7f639ffff700 1 -- 172.21.15.57:0/293553277 reap_dead start 2023-11-28T12:07:44.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.005+0000 7f63a5e83700 1 -- 172.21.15.57:0/293553277 shutdown_connections 2023-11-28T12:07:44.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.005+0000 7f63a5e83700 1 -- 172.21.15.57:0/293553277 >> 172.21.15.57:0/293553277 conn(0x7f63a006ea80 msgr2=0x7f63a006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:44.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.005+0000 7f63a5e83700 1 -- 172.21.15.57:0/293553277 shutdown_connections 2023-11-28T12:07:44.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.005+0000 7f63a5e83700 1 -- 172.21.15.57:0/293553277 wait complete. 2023-11-28T12:07:44.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.006+0000 7f63a5e83700 1 Processor -- start 2023-11-28T12:07:44.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.007+0000 7f63a5e83700 1 -- start start 2023-11-28T12:07:44.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.007+0000 7f63a5e83700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f638c003bc0 0x7f63a01e3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.007+0000 7f63a5e83700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a01e5c20 0x7f63a01e6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.007+0000 7f63a5e83700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a00c58b0 0x7f63a00c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.007+0000 7f63a5e83700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a01e4490 con 0x7f638c003bc0 2023-11-28T12:07:44.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.007+0000 7f63a5e83700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a0151060 con 0x7f63a00c58b0 2023-11-28T12:07:44.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.007+0000 7f63a5e83700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a01e65d0 con 0x7f63a01e5c20 2023-11-28T12:07:44.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.007+0000 7f639f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f638c003bc0 0x7f63a01e3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.007+0000 7f639f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f638c003bc0 0x7f63a01e3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59268/0 (socket says 172.21.15.57:59268) 2023-11-28T12:07:44.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.007+0000 7f639f7fe700 1 -- 172.21.15.57:0/199516051 learned_addr learned my addr 172.21.15.57:0/199516051 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:44.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f639effd700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a01e5c20 0x7f63a01e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f639ffff700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a00c58b0 0x7f63a00c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f639f7fe700 1 -- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a01e5c20 msgr2=0x7f63a01e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f639f7fe700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a01e5c20 0x7f63a01e6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f639f7fe700 1 -- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a00c58b0 msgr2=0x7f63a00c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f639f7fe700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a00c58b0 0x7f63a00c5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f639f7fe700 1 -- 172.21.15.57:0/199516051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a00cd230 con 0x7f638c003bc0 2023-11-28T12:07:44.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f639ffff700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a00c58b0 0x7f63a00c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:44.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f639f7fe700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f638c003bc0 0x7f63a01e3f50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6388004270 tx=0x7f63880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f6397fff700 1 -- 172.21.15.57:0/199516051 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388021070 con 0x7f638c003bc0 2023-11-28T12:07:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f6397fff700 1 -- 172.21.15.57:0/199516051 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63880036a0 con 0x7f638c003bc0 2023-11-28T12:07:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f6397fff700 1 -- 172.21.15.57:0/199516051 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638800e660 con 0x7f638c003bc0 2023-11-28T12:07:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f63a5e83700 1 -- 172.21.15.57:0/199516051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a00ccf50 con 0x7f638c003bc0 2023-11-28T12:07:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.008+0000 7f63a5e83700 1 -- 172.21.15.57:0/199516051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a00cd510 con 0x7f638c003bc0 2023-11-28T12:07:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.009+0000 7f63a5e83700 1 -- 172.21.15.57:0/199516051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a00743f0 con 0x7f638c003bc0 2023-11-28T12:07:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.010+0000 7f6397fff700 1 -- 172.21.15.57:0/199516051 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f638802e030 con 0x7f638c003bc0 2023-11-28T12:07:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.010+0000 7f6397fff700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63800682f0 0x7f638006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.011+0000 7f6397fff700 1 -- 172.21.15.57:0/199516051 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f63880871f0 con 0x7f638c003bc0 2023-11-28T12:07:44.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.011+0000 7f639effd700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63800682f0 0x7f638006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.011+0000 7f639effd700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63800682f0 0x7f638006a7a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6390000fd0 tx=0x7f6390009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.014+0000 7f6397fff700 1 -- 172.21.15.57:0/199516051 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6388053e10 con 0x7f638c003bc0 2023-11-28T12:07:44.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.214+0000 7f63a5e83700 1 -- 172.21.15.57:0/199516051 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f63a006ab80 con 0x7f63800682f0 2023-11-28T12:07:44.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.215+0000 7f6397fff700 1 -- 172.21.15.57:0/199516051 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f63a006ab80 con 0x7f63800682f0 2023-11-28T12:07:44.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.217+0000 7f63a5e83700 1 -- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63800682f0 msgr2=0x7f638006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.217+0000 7f63a5e83700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63800682f0 0x7f638006a7a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6390000fd0 tx=0x7f6390009040 comp rx=0 tx=0).stop 2023-11-28T12:07:44.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.217+0000 7f63a5e83700 1 -- 172.21.15.57:0/199516051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f638c003bc0 msgr2=0x7f63a01e3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.217+0000 7f63a5e83700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f638c003bc0 0x7f63a01e3f50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6388004270 tx=0x7f63880047c0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.217+0000 7f63a5e83700 1 -- 172.21.15.57:0/199516051 shutdown_connections 2023-11-28T12:07:44.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.217+0000 7f63a5e83700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f63800682f0 0x7f638006a7a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.217+0000 7f63a5e83700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63a00c58b0 0x7f63a00c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.217+0000 7f63a5e83700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f638c003bc0 0x7f63a01e3f50 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.217+0000 7f63a5e83700 1 --2- 172.21.15.57:0/199516051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63a01e5c20 0x7f63a01e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.217+0000 7f63a5e83700 1 -- 172.21.15.57:0/199516051 >> 172.21.15.57:0/199516051 conn(0x7f63a006ea80 msgr2=0x7f63a00cdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:44.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.217+0000 7f63a5e83700 1 -- 172.21.15.57:0/199516051 shutdown_connections 2023-11-28T12:07:44.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.217+0000 7f63a5e83700 1 -- 172.21.15.57:0/199516051 wait complete. 2023-11-28T12:07:44.221 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 3 to deep-scrub 2023-11-28T12:07:44.232 INFO:tasks.ceph:Scrubbing osd.4 2023-11-28T12:07:44.232 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-28T12:07:44.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.370+0000 7fec706db700 1 Processor -- start 2023-11-28T12:07:44.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.370+0000 7fec706db700 1 -- start start 2023-11-28T12:07:44.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.371+0000 7fec706db700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fec6806ba90 0x7fec6815a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.371+0000 7fec706db700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec680cea10 con 0x7fec6815e070 2023-11-28T12:07:44.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.371+0000 7fec706db700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec680cbd10 con 0x7fec680ceea0 2023-11-28T12:07:44.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.371+0000 7fec706db700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6815ae80 con 0x7fec6806ba90 2023-11-28T12:07:44.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.371+0000 7fec6dc76700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fec6806ba90 0x7fec6815a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.371+0000 7fec6e477700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fec6815e070 0x7fec6806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47398/0 (socket says 172.21.15.57:47398) 2023-11-28T12:07:44.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.371+0000 7fec6e477700 1 -- 172.21.15.57:0/293423404 learned_addr learned my addr 172.21.15.57:0/293423404 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:44.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.372+0000 7fec6dc76700 1 -- 172.21.15.57:0/293423404 >> v1:172.21.15.181:6789/0 conn(0x7fec680ceea0 legacy=0x7fec6815a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.372+0000 7fec6dc76700 1 -- 172.21.15.57:0/293423404 >> v1:172.21.15.57:6789/0 conn(0x7fec6815e070 legacy=0x7fec6806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.372+0000 7fec6d475700 1 -- 172.21.15.57:0/293423404 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443874876 0 0) 0x7fec680cea10 con 0x7fec6815e070 2023-11-28T12:07:44.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.372+0000 7fec6dc76700 1 -- 172.21.15.57:0/293423404 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec680ce0e0 con 0x7fec6806ba90 2023-11-28T12:07:44.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.372+0000 7fec6dc76700 1 --2- 172.21.15.57:0/293423404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec6806ba90 0x7fec6815a150 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fec64009840 tx=0x7fec64009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37d3fa479c95ef2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.372+0000 7fec6d475700 1 -- 172.21.15.57:0/293423404 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64011830 con 0x7fec6806ba90 2023-11-28T12:07:44.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.372+0000 7fec6d475700 1 -- 172.21.15.57:0/293423404 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec64011990 con 0x7fec6806ba90 2023-11-28T12:07:44.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.372+0000 7fec6d475700 1 -- 172.21.15.57:0/293423404 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64011d20 con 0x7fec6806ba90 2023-11-28T12:07:44.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.373+0000 7fec706db700 1 -- 172.21.15.57:0/293423404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec6806ba90 msgr2=0x7fec6815a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.373+0000 7fec706db700 1 --2- 172.21.15.57:0/293423404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec6806ba90 0x7fec6815a150 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fec64009840 tx=0x7fec64009b50 comp rx=0 tx=0).stop 2023-11-28T12:07:44.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.373+0000 7fec706db700 1 -- 172.21.15.57:0/293423404 shutdown_connections 2023-11-28T12:07:44.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.373+0000 7fec706db700 1 --2- 172.21.15.57:0/293423404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec6806ba90 0x7fec6815a150 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.373+0000 7fec706db700 1 -- 172.21.15.57:0/293423404 >> 172.21.15.57:0/293423404 conn(0x7fec680cfc30 msgr2=0x7fec680d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:44.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.373+0000 7fec706db700 1 -- 172.21.15.57:0/293423404 shutdown_connections 2023-11-28T12:07:44.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.373+0000 7fec706db700 1 -- 172.21.15.57:0/293423404 wait complete. 2023-11-28T12:07:44.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.374+0000 7fec706db700 1 Processor -- start 2023-11-28T12:07:44.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.374+0000 7fec706db700 1 -- start start 2023-11-28T12:07:44.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec706db700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec6806ba90 0x7fec680b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec706db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec680ceea0 0x7fec6814e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec706db700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec6815e070 0x7fec681f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec706db700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec680b5bc0 con 0x7fec6815e070 2023-11-28T12:07:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec706db700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6813d170 con 0x7fec680ceea0 2023-11-28T12:07:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec706db700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec680b51f0 con 0x7fec6806ba90 2023-11-28T12:07:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec6ec78700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec6815e070 0x7fec681f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec6ec78700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec6815e070 0x7fec681f54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59284/0 (socket says 172.21.15.57:59284) 2023-11-28T12:07:44.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec6ec78700 1 -- 172.21.15.57:0/2593672255 learned_addr learned my addr 172.21.15.57:0/2593672255 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:44.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec6e477700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec6806ba90 0x7fec680b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec6dc76700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec680ceea0 0x7fec6814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec6ec78700 1 -- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec6806ba90 msgr2=0x7fec680b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec6ec78700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec6806ba90 0x7fec680b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec6ec78700 1 -- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec680ceea0 msgr2=0x7fec6814e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec6ec78700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec680ceea0 0x7fec6814e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec6ec78700 1 -- 172.21.15.57:0/2593672255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec640094a0 con 0x7fec6815e070 2023-11-28T12:07:44.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.375+0000 7fec6e477700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec6806ba90 0x7fec680b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:44.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.376+0000 7fec6ec78700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec6815e070 0x7fec681f54a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fec6000cc80 tx=0x7fec6000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.376+0000 7fec5effd700 1 -- 172.21.15.57:0/2593672255 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6000ad40 con 0x7fec6815e070 2023-11-28T12:07:44.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.376+0000 7fec5effd700 1 -- 172.21.15.57:0/2593672255 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6000ce20 con 0x7fec6815e070 2023-11-28T12:07:44.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.376+0000 7fec706db700 1 -- 172.21.15.57:0/2593672255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec680ce6a0 con 0x7fec6815e070 2023-11-28T12:07:44.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.376+0000 7fec5effd700 1 -- 172.21.15.57:0/2593672255 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6000aea0 con 0x7fec6815e070 2023-11-28T12:07:44.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.376+0000 7fec706db700 1 -- 172.21.15.57:0/2593672255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec680ce3c0 con 0x7fec6815e070 2023-11-28T12:07:44.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.377+0000 7fec706db700 1 -- 172.21.15.57:0/2593672255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec68150080 con 0x7fec6815e070 2023-11-28T12:07:44.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.377+0000 7fec5effd700 1 -- 172.21.15.57:0/2593672255 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fec60010050 con 0x7fec6815e070 2023-11-28T12:07:44.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.378+0000 7fec5effd700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec4c0682f0 0x7fec4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.378+0000 7fec5effd700 1 -- 172.21.15.57:0/2593672255 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fec60086410 con 0x7fec6815e070 2023-11-28T12:07:44.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.378+0000 7fec6e477700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec4c0682f0 0x7fec4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.378+0000 7fec5effd700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] conn(0x7fec4c06e760 0x7fec4c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.378+0000 7fec5effd700 1 -- 172.21.15.57:0/2593672255 --> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec4c071240 con 0x7fec4c06e760 2023-11-28T12:07:44.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.378+0000 7fec5effd700 1 -- 172.21.15.57:0/2593672255 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec60086780 con 0x7fec6815e070 2023-11-28T12:07:44.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.378+0000 7fec6dc76700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] conn(0x7fec4c06e760 0x7fec4c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.379+0000 7fec6e477700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec4c0682f0 0x7fec4c06a7a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fec58000f60 tx=0x7fec58009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.379+0000 7fec6dc76700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] conn(0x7fec4c06e760 0x7fec4c070b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.380+0000 7fec5effd700 1 -- 172.21.15.57:0/2593672255 <== osd.4 v2:172.21.15.181:6800/120958 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fec4c071240 con 0x7fec4c06e760 2023-11-28T12:07:44.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.394+0000 7fec706db700 1 -- 172.21.15.57:0/2593672255 --> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fec680743f0 con 0x7fec4c06e760 2023-11-28T12:07:44.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.395+0000 7fec5effd700 1 -- 172.21.15.57:0/2593672255 <== osd.4 v2:172.21.15.181:6800/120958 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fec680743f0 con 0x7fec4c06e760 2023-11-28T12:07:44.397 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:07:44.397 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T12:07:44.397 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:07:44.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.395+0000 7fec706db700 1 -- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] conn(0x7fec4c06e760 msgr2=0x7fec4c070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.395+0000 7fec706db700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:6800/120958,v1:172.21.15.181:6801/120958] conn(0x7fec4c06e760 0x7fec4c070b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.395+0000 7fec706db700 1 -- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec4c0682f0 msgr2=0x7fec4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.395+0000 7fec706db700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fec4c0682f0 0x7fec4c06a7a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fec58000f60 tx=0x7fec58009040 comp rx=0 tx=0).stop 2023-11-28T12:07:44.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.395+0000 7fec706db700 1 -- 172.21.15.57:0/2593672255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec6815e070 msgr2=0x7fec681f54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.395+0000 7fec706db700 1 --2- 172.21.15.57:0/2593672255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec6815e070 0x7fec681f54a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fec6000cc80 tx=0x7fec6000a340 comp rx=0 tx=0).stop 2023-11-28T12:07:44.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.395+0000 7fec6ec78700 1 -- 172.21.15.57:0/2593672255 reap_dead start 2023-11-28T12:07:44.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.395+0000 7fec706db700 1 -- 172.21.15.57:0/2593672255 shutdown_connections 2023-11-28T12:07:44.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.395+0000 7fec706db700 1 -- 172.21.15.57:0/2593672255 >> 172.21.15.57:0/2593672255 conn(0x7fec680cfc30 msgr2=0x7fec6815a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:44.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.395+0000 7fec706db700 1 -- 172.21.15.57:0/2593672255 shutdown_connections 2023-11-28T12:07:44.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.395+0000 7fec706db700 1 -- 172.21.15.57:0/2593672255 wait complete. 2023-11-28T12:07:44.410 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-28T12:07:44.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.545+0000 7f0c2d586700 1 Processor -- start 2023-11-28T12:07:44.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.545+0000 7f0c2d586700 1 -- start start 2023-11-28T12:07:44.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.546+0000 7f0c2d586700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c2815c8c0 0x7f0c28160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.546+0000 7f0c2d586700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c280d2d30 con 0x7f0c280b5dc0 2023-11-28T12:07:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.546+0000 7f0c2d586700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c280d0b40 con 0x7f0c2806b220 2023-11-28T12:07:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.546+0000 7f0c2d586700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281611f0 con 0x7f0c2815c8c0 2023-11-28T12:07:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.546+0000 7f0c267fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f0c280b5dc0 0x7f0c2806ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:58902/0 (socket says 172.21.15.57:58902) 2023-11-28T12:07:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.546+0000 7f0c267fc700 1 -- 172.21.15.57:0/702739932 learned_addr learned my addr 172.21.15.57:0/702739932 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c26ffd700 1 --2- 172.21.15.57:0/702739932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c2815c8c0 0x7f0c28160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787683059 0 0) 0x7f0c10001110 con 0x7f0c280b5dc0 2023-11-28T12:07:44.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c140035b0 con 0x7f0c280b5dc0 2023-11-28T12:07:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499146555 0 0) 0x7f0c280d0b40 con 0x7f0c2806b220 2023-11-28T12:07:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c10001110 con 0x7f0c2806b220 2023-11-28T12:07:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1169282213 0 0) 0x7f0c140035b0 con 0x7f0c280b5dc0 2023-11-28T12:07:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c2815c8c0 msgr2=0x7f0c28160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 --2- 172.21.15.57:0/702739932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c2815c8c0 0x7f0c28160cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 >> v1:172.21.15.181:6789/0 conn(0x7f0c2806b220 legacy=0x7f0c280d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c280d2a50 con 0x7f0c280b5dc0 2023-11-28T12:07:44.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7f0c10001780 con 0x7f0c280b5dc0 2023-11-28T12:07:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 >> v1:172.21.15.57:6789/0 conn(0x7f0c280b5dc0 legacy=0x7f0c2806ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 --2- 172.21.15.57:0/702739932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c2815c8c0 0x7f0c14003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 --2- 172.21.15.57:0/702739932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c14004170 0x7f0c14004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 --2- 172.21.15.57:0/702739932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c14004d90 0x7f0c14005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281611f0 con 0x7f0c14004d90 2023-11-28T12:07:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1400bb20 con 0x7f0c14004170 2023-11-28T12:07:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1400bce0 con 0x7f0c2815c8c0 2023-11-28T12:07:44.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c277fe700 1 --2- 172.21.15.57:0/702739932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c14004d90 0x7f0c14005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c26ffd700 1 --2- 172.21.15.57:0/702739932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c2815c8c0 0x7f0c14003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.547+0000 7f0c267fc700 1 --2- 172.21.15.57:0/702739932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c14004170 0x7f0c14004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.548+0000 7f0c277fe700 1 -- 172.21.15.57:0/702739932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c2815c8c0 msgr2=0x7f0c14003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:44.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.548+0000 7f0c277fe700 1 --2- 172.21.15.57:0/702739932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c2815c8c0 0x7f0c14003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.548+0000 7f0c277fe700 1 -- 172.21.15.57:0/702739932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c14004170 msgr2=0x7f0c14004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:44.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.548+0000 7f0c277fe700 1 --2- 172.21.15.57:0/702739932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c14004170 0x7f0c14004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.548+0000 7f0c277fe700 1 -- 172.21.15.57:0/702739932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c140035b0 con 0x7f0c14004d90 2023-11-28T12:07:44.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.548+0000 7f0c277fe700 1 --2- 172.21.15.57:0/702739932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c14004d90 0x7f0c14005200 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f0c280d1f30 tx=0x7f0c14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42026aba51515212 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.548+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20001f30 con 0x7f0c14004d90 2023-11-28T12:07:44.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.548+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2000ce30 con 0x7f0c14004d90 2023-11-28T12:07:44.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.548+0000 7f0c25ffb700 1 -- 172.21.15.57:0/702739932 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20003c40 con 0x7f0c14004d90 2023-11-28T12:07:44.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.549+0000 7f0c2d586700 1 -- 172.21.15.57:0/702739932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c14004d90 msgr2=0x7f0c14005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.549+0000 7f0c2d586700 1 --2- 172.21.15.57:0/702739932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c14004d90 0x7f0c14005200 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f0c280d1f30 tx=0x7f0c14005880 comp rx=0 tx=0).stop 2023-11-28T12:07:44.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.549+0000 7f0c277fe700 1 -- 172.21.15.57:0/702739932 reap_dead start 2023-11-28T12:07:44.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.549+0000 7f0c2d586700 1 -- 172.21.15.57:0/702739932 shutdown_connections 2023-11-28T12:07:44.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.549+0000 7f0c2d586700 1 -- 172.21.15.57:0/702739932 >> 172.21.15.57:0/702739932 conn(0x7f0c2806c520 msgr2=0x7f0c2806c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:44.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.549+0000 7f0c2d586700 1 -- 172.21.15.57:0/702739932 shutdown_connections 2023-11-28T12:07:44.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.549+0000 7f0c2d586700 1 -- 172.21.15.57:0/702739932 wait complete. 2023-11-28T12:07:44.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.550+0000 7f0c2d586700 1 Processor -- start 2023-11-28T12:07:44.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.550+0000 7f0c2d586700 1 -- start start 2023-11-28T12:07:44.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.551+0000 7f0c2d586700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c281563b0 0x7f0c280c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.551+0000 7f0c2d586700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c280c9090 0x7f0c280c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.551+0000 7f0c2d586700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c281f56e0 0x7f0c281f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.551+0000 7f0c2d586700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280c7410 con 0x7f0c281f56e0 2023-11-28T12:07:44.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.551+0000 7f0c2d586700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280cb5a0 con 0x7f0c281563b0 2023-11-28T12:07:44.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.551+0000 7f0c2d586700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280c7c20 con 0x7f0c280c9090 2023-11-28T12:07:44.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.551+0000 7f0c277fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c281f56e0 0x7f0c281f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.551+0000 7f0c277fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c281f56e0 0x7f0c281f5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57088/0 (socket says 172.21.15.57:57088) 2023-11-28T12:07:44.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.551+0000 7f0c277fe700 1 -- 172.21.15.57:0/2538588792 learned_addr learned my addr 172.21.15.57:0/2538588792 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:44.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.551+0000 7f0c267fc700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c280c9090 0x7f0c280c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.551+0000 7f0c26ffd700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c281563b0 0x7f0c280c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.551+0000 7f0c277fe700 1 -- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c280c9090 msgr2=0x7f0c280c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.552+0000 7f0c277fe700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c280c9090 0x7f0c280c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.552+0000 7f0c277fe700 1 -- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c281563b0 msgr2=0x7f0c280c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.552+0000 7f0c277fe700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c281563b0 0x7f0c280c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.552+0000 7f0c277fe700 1 -- 172.21.15.57:0/2538588792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c280cda40 con 0x7f0c281f56e0 2023-11-28T12:07:44.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.552+0000 7f0c267fc700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c280c9090 0x7f0c280c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:44.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.552+0000 7f0c26ffd700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c281563b0 0x7f0c280c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:44.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.552+0000 7f0c277fe700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c281f56e0 0x7f0c281f5b50 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005040 tx=0x7f0c20001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.552+0000 7f0c1f7fe700 1 -- 172.21.15.57:0/2538588792 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c200050d0 con 0x7f0c281f56e0 2023-11-28T12:07:44.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.552+0000 7f0c1f7fe700 1 -- 172.21.15.57:0/2538588792 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c20005230 con 0x7f0c281f56e0 2023-11-28T12:07:44.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.552+0000 7f0c1f7fe700 1 -- 172.21.15.57:0/2538588792 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c200055c0 con 0x7f0c281f56e0 2023-11-28T12:07:44.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.552+0000 7f0c2d586700 1 -- 172.21.15.57:0/2538588792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c280cd760 con 0x7f0c281f56e0 2023-11-28T12:07:44.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.552+0000 7f0c2d586700 1 -- 172.21.15.57:0/2538588792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c280d2700 con 0x7f0c281f56e0 2023-11-28T12:07:44.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.553+0000 7f0c2d586700 1 -- 172.21.15.57:0/2538588792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c28067d70 con 0x7f0c281f56e0 2023-11-28T12:07:44.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.553+0000 7f0c1f7fe700 1 -- 172.21.15.57:0/2538588792 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0c20010050 con 0x7f0c281f56e0 2023-11-28T12:07:44.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.554+0000 7f0c1f7fe700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0c080682f0 0x7f0c0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.554+0000 7f0c1f7fe700 1 -- 172.21.15.57:0/2538588792 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0c200869a0 con 0x7f0c281f56e0 2023-11-28T12:07:44.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.554+0000 7f0c26ffd700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0c080682f0 0x7f0c0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.555+0000 7f0c26ffd700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0c080682f0 0x7f0c0806a7a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0c2806cbd0 tx=0x7f0c18008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.558+0000 7f0c1f7fe700 1 -- 172.21.15.57:0/2538588792 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0c20053500 con 0x7f0c281f56e0 2023-11-28T12:07:44.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.760+0000 7f0c2d586700 1 -- 172.21.15.57:0/2538588792 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c280743f0 con 0x7f0c080682f0 2023-11-28T12:07:44.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.760+0000 7f0c1f7fe700 1 -- 172.21.15.57:0/2538588792 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0c280743f0 con 0x7f0c080682f0 2023-11-28T12:07:44.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.762+0000 7f0c2d586700 1 -- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0c080682f0 msgr2=0x7f0c0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.762+0000 7f0c2d586700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0c080682f0 0x7f0c0806a7a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0c2806cbd0 tx=0x7f0c18008040 comp rx=0 tx=0).stop 2023-11-28T12:07:44.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.763+0000 7f0c2d586700 1 -- 172.21.15.57:0/2538588792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c281f56e0 msgr2=0x7f0c281f5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.763+0000 7f0c2d586700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c281f56e0 0x7f0c281f5b50 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005040 tx=0x7f0c20001650 comp rx=0 tx=0).stop 2023-11-28T12:07:44.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.763+0000 7f0c2d586700 1 -- 172.21.15.57:0/2538588792 shutdown_connections 2023-11-28T12:07:44.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.763+0000 7f0c2d586700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f0c080682f0 0x7f0c0806a7a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.763+0000 7f0c2d586700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c281f56e0 0x7f0c281f5b50 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.763+0000 7f0c2d586700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c281563b0 0x7f0c280c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.763+0000 7f0c2d586700 1 --2- 172.21.15.57:0/2538588792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c280c9090 0x7f0c280c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.763+0000 7f0c2d586700 1 -- 172.21.15.57:0/2538588792 >> 172.21.15.57:0/2538588792 conn(0x7f0c2806c520 msgr2=0x7f0c2815ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:44.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.763+0000 7f0c2d586700 1 -- 172.21.15.57:0/2538588792 shutdown_connections 2023-11-28T12:07:44.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.763+0000 7f0c2d586700 1 -- 172.21.15.57:0/2538588792 wait complete. 2023-11-28T12:07:44.771 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 4 to deep-scrub 2023-11-28T12:07:44.777 INFO:tasks.ceph:Scrubbing osd.5 2023-11-28T12:07:44.778 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-28T12:07:44.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.917+0000 7f502963f700 1 Processor -- start 2023-11-28T12:07:44.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.917+0000 7f502963f700 1 -- start start 2023-11-28T12:07:44.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f502963f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f502415cce0 0x7f50240b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f502963f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50240b22e0 0x7f50240d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f502963f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50240b1c90 0x7f50240d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f502963f700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50240b32b0 con 0x7f502415cce0 2023-11-28T12:07:44.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f502963f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50240b26e0 con 0x7f50240b1c90 2023-11-28T12:07:44.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f502963f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50240b2980 con 0x7f50240b22e0 2023-11-28T12:07:44.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f5022ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f502415cce0 0x7f50240b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f5022ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f502415cce0 0x7f50240b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57094/0 (socket says 172.21.15.57:57094) 2023-11-28T12:07:44.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f5022ffd700 1 -- 172.21.15.57:0/770153021 learned_addr learned my addr 172.21.15.57:0/770153021 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:44.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f50237fe700 1 --2- 172.21.15.57:0/770153021 >> v2:172.21.15.181:3300/0 conn(0x7f50240b1c90 0x7f50240d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f50227fc700 1 --2- 172.21.15.57:0/770153021 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50240b22e0 0x7f50240d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f5022ffd700 1 -- 172.21.15.57:0/770153021 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50240b22e0 msgr2=0x7f50240d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f5022ffd700 1 --2- 172.21.15.57:0/770153021 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50240b22e0 0x7f50240d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f5022ffd700 1 -- 172.21.15.57:0/770153021 >> v2:172.21.15.181:3300/0 conn(0x7f50240b1c90 msgr2=0x7f50240d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f5022ffd700 1 --2- 172.21.15.57:0/770153021 >> v2:172.21.15.181:3300/0 conn(0x7f50240b1c90 0x7f50240d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.918+0000 7f5022ffd700 1 -- 172.21.15.57:0/770153021 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50240b2c60 con 0x7f502415cce0 2023-11-28T12:07:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.919+0000 7f50227fc700 1 --2- 172.21.15.57:0/770153021 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50240b22e0 0x7f50240d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.919+0000 7f50237fe700 1 --2- 172.21.15.57:0/770153021 >> v2:172.21.15.181:3300/0 conn(0x7f50240b1c90 0x7f50240d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.919+0000 7f5022ffd700 1 --2- 172.21.15.57:0/770153021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f502415cce0 0x7f50240b1750 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f50240cd740 tx=0x7f5014009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f350868e8a2ed7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.919+0000 7f5021ffb700 1 -- 172.21.15.57:0/770153021 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501400b870 con 0x7f502415cce0 2023-11-28T12:07:44.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.919+0000 7f5021ffb700 1 -- 172.21.15.57:0/770153021 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501400b9d0 con 0x7f502415cce0 2023-11-28T12:07:44.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.919+0000 7f5021ffb700 1 -- 172.21.15.57:0/770153021 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501400bd60 con 0x7f502415cce0 2023-11-28T12:07:44.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.919+0000 7f502963f700 1 -- 172.21.15.57:0/770153021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f502415cce0 msgr2=0x7f50240b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.919+0000 7f502963f700 1 --2- 172.21.15.57:0/770153021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f502415cce0 0x7f50240b1750 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f50240cd740 tx=0x7f5014009bd0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.920+0000 7f502963f700 1 -- 172.21.15.57:0/770153021 shutdown_connections 2023-11-28T12:07:44.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.920+0000 7f502963f700 1 --2- 172.21.15.57:0/770153021 >> v2:172.21.15.181:3300/0 conn(0x7f50240b1c90 0x7f50240d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.920+0000 7f502963f700 1 --2- 172.21.15.57:0/770153021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f502415cce0 0x7f50240b1750 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.920+0000 7f502963f700 1 --2- 172.21.15.57:0/770153021 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50240b22e0 0x7f50240d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.920+0000 7f502963f700 1 -- 172.21.15.57:0/770153021 >> 172.21.15.57:0/770153021 conn(0x7f502409c4d0 msgr2=0x7f50240d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:44.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.920+0000 7f502963f700 1 -- 172.21.15.57:0/770153021 shutdown_connections 2023-11-28T12:07:44.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.920+0000 7f502963f700 1 -- 172.21.15.57:0/770153021 wait complete. 2023-11-28T12:07:44.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.921+0000 7f502963f700 1 Processor -- start 2023-11-28T12:07:44.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.921+0000 7f502963f700 1 -- start start 2023-11-28T12:07:44.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f502963f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50240b1c90 0x7f5024158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f502963f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f502415cce0 0x7f5024159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f502963f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50241598a0 0x7f50240cf940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f502963f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502409f4b0 con 0x7f50240b1c90 2023-11-28T12:07:44.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f502963f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502409d340 con 0x7f502415cce0 2023-11-28T12:07:44.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f502963f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50240b4a00 con 0x7f50241598a0 2023-11-28T12:07:44.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f5022ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50240b1c90 0x7f5024158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f5022ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50240b1c90 0x7f5024158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57096/0 (socket says 172.21.15.57:57096) 2023-11-28T12:07:44.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f5022ffd700 1 -- 172.21.15.57:0/4281456796 learned_addr learned my addr 172.21.15.57:0/4281456796 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:44.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f50237fe700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50241598a0 0x7f50240cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f50227fc700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f502415cce0 0x7f5024159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f5022ffd700 1 -- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50241598a0 msgr2=0x7f50240cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f5022ffd700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50241598a0 0x7f50240cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f5022ffd700 1 -- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f502415cce0 msgr2=0x7f5024159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f5022ffd700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f502415cce0 0x7f5024159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f5022ffd700 1 -- 172.21.15.57:0/4281456796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5014009550 con 0x7f50240b1c90 2023-11-28T12:07:44.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.922+0000 7f5022ffd700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50240b1c90 0x7f5024158ca0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5014014040 tx=0x7f5014004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.923+0000 7f501b7fe700 1 -- 172.21.15.57:0/4281456796 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5014004250 con 0x7f50240b1c90 2023-11-28T12:07:44.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.923+0000 7f501b7fe700 1 -- 172.21.15.57:0/4281456796 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501401e070 con 0x7f50240b1c90 2023-11-28T12:07:44.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.923+0000 7f502963f700 1 -- 172.21.15.57:0/4281456796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50240afd30 con 0x7f50240b1c90 2023-11-28T12:07:44.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.923+0000 7f501b7fe700 1 -- 172.21.15.57:0/4281456796 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50140036a0 con 0x7f50240b1c90 2023-11-28T12:07:44.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.923+0000 7f502963f700 1 -- 172.21.15.57:0/4281456796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50240b2f40 con 0x7f50240b1c90 2023-11-28T12:07:44.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.924+0000 7f502963f700 1 -- 172.21.15.57:0/4281456796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f50240afec0 con 0x7f50240b1c90 2023-11-28T12:07:44.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.924+0000 7f501b7fe700 1 -- 172.21.15.57:0/4281456796 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5014029020 con 0x7f50240b1c90 2023-11-28T12:07:44.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.924+0000 7f501b7fe700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50040682f0 0x7f500406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.925+0000 7f501b7fe700 1 -- 172.21.15.57:0/4281456796 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5014086b20 con 0x7f50240b1c90 2023-11-28T12:07:44.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.925+0000 7f501b7fe700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x7f500406e760 0x7f5004070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:44.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.925+0000 7f501b7fe700 1 -- 172.21.15.57:0/4281456796 --> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5004071240 con 0x7f500406e760 2023-11-28T12:07:44.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.925+0000 7f501b7fe700 1 -- 172.21.15.57:0/4281456796 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f501408b2a0 con 0x7f50240b1c90 2023-11-28T12:07:44.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.925+0000 7f50227fc700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50040682f0 0x7f500406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.925+0000 7f50237fe700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x7f500406e760 0x7f5004070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:44.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.925+0000 7f50227fc700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50040682f0 0x7f500406a7a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5024159ec0 tx=0x7f500c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.925+0000 7f50237fe700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x7f500406e760 0x7f5004070b70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:44.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.926+0000 7f501b7fe700 1 -- 172.21.15.57:0/4281456796 <== osd.5 v2:172.21.15.181:6824/120977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5004071240 con 0x7f500406e760 2023-11-28T12:07:44.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.941+0000 7f502963f700 1 -- 172.21.15.57:0/4281456796 --> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f50240743f0 con 0x7f500406e760 2023-11-28T12:07:44.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.941+0000 7f501b7fe700 1 -- 172.21.15.57:0/4281456796 <== osd.5 v2:172.21.15.181:6824/120977 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f50240743f0 con 0x7f500406e760 2023-11-28T12:07:44.943 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:07:44.943 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T12:07:44.943 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:07:44.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.942+0000 7f502963f700 1 -- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x7f500406e760 msgr2=0x7f5004070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.942+0000 7f502963f700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:6824/120977,v1:172.21.15.181:6825/120977] conn(0x7f500406e760 0x7f5004070b70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:44.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.942+0000 7f502963f700 1 -- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50040682f0 msgr2=0x7f500406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.942+0000 7f502963f700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f50040682f0 0x7f500406a7a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5024159ec0 tx=0x7f500c008040 comp rx=0 tx=0).stop 2023-11-28T12:07:44.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.942+0000 7f502963f700 1 -- 172.21.15.57:0/4281456796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50240b1c90 msgr2=0x7f5024158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:44.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.942+0000 7f502963f700 1 --2- 172.21.15.57:0/4281456796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50240b1c90 0x7f5024158ca0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5014014040 tx=0x7f5014004750 comp rx=0 tx=0).stop 2023-11-28T12:07:44.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.942+0000 7f50237fe700 1 -- 172.21.15.57:0/4281456796 reap_dead start 2023-11-28T12:07:44.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.942+0000 7f502963f700 1 -- 172.21.15.57:0/4281456796 shutdown_connections 2023-11-28T12:07:44.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.942+0000 7f502963f700 1 -- 172.21.15.57:0/4281456796 >> 172.21.15.57:0/4281456796 conn(0x7f502409c4d0 msgr2=0x7f50240b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:44.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.942+0000 7f502963f700 1 -- 172.21.15.57:0/4281456796 shutdown_connections 2023-11-28T12:07:44.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:44.942+0000 7f502963f700 1 -- 172.21.15.57:0/4281456796 wait complete. 2023-11-28T12:07:44.956 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-28T12:07:45.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.091+0000 7fa96a6f9700 1 Processor -- start 2023-11-28T12:07:45.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.091+0000 7fa96a6f9700 1 -- start start 2023-11-28T12:07:45.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.092+0000 7fa96a6f9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9641599b0 0x7fa96415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.092+0000 7fa96a6f9700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa964159ec0 con 0x7fa9640b1650 2023-11-28T12:07:45.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.092+0000 7fa96a6f9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96415aa60 con 0x7fa9640af0a0 2023-11-28T12:07:45.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.092+0000 7fa96a6f9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96415d8a0 con 0x7fa9641599b0 2023-11-28T12:07:45.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.092+0000 7fa963fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa9640b1650 0x7fa9640b1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:58916/0 (socket says 172.21.15.57:58916) 2023-11-28T12:07:45.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.092+0000 7fa963fff700 1 -- 172.21.15.57:0/2557073022 learned_addr learned my addr 172.21.15.57:0/2557073022 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:45.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa9637fe700 1 --2- 172.21.15.57:0/2557073022 >> v2:172.21.15.181:3300/0 conn(0x7fa9641599b0 0x7fa96415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139018783 0 0) 0x7fa964159ec0 con 0x7fa9640b1650 2023-11-28T12:07:45.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa94c0035b0 con 0x7fa9640b1650 2023-11-28T12:07:45.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 887859019 0 0) 0x7fa94c0035b0 con 0x7fa9640b1650 2023-11-28T12:07:45.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 >> v2:172.21.15.181:3300/0 conn(0x7fa9641599b0 msgr2=0x7fa96415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:45.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 --2- 172.21.15.57:0/2557073022 >> v2:172.21.15.181:3300/0 conn(0x7fa9641599b0 0x7fa96415d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 >> v1:172.21.15.181:6789/0 conn(0x7fa9640af0a0 legacy=0x7fa96415a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96415db80 con 0x7fa9640b1650 2023-11-28T12:07:45.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3789048806 0 0) 0x7fa954001670 con 0x7fa9640b1650 2023-11-28T12:07:45.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 >> v1:172.21.15.57:6789/0 conn(0x7fa9640b1650 legacy=0x7fa9640b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 --2- 172.21.15.57:0/2557073022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa94c003a00 0x7fa94c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 --2- 172.21.15.57:0/2557073022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94c004850 0x7fa94c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 --2- 172.21.15.57:0/2557073022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa94c005200 0x7fa94c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96415d8a0 con 0x7fa94c003a00 2023-11-28T12:07:45.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c00c080 con 0x7fa94c004850 2023-11-28T12:07:45.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c00c240 con 0x7fa94c005200 2023-11-28T12:07:45.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa963fff700 1 --2- 172.21.15.57:0/2557073022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa94c003a00 0x7fa94c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa9637fe700 1 --2- 172.21.15.57:0/2557073022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94c004850 0x7fa94c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.093+0000 7fa968c96700 1 --2- 172.21.15.57:0/2557073022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa94c005200 0x7fa94c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.094+0000 7fa963fff700 1 -- 172.21.15.57:0/2557073022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa94c005200 msgr2=0x7fa94c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:45.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.094+0000 7fa963fff700 1 --2- 172.21.15.57:0/2557073022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa94c005200 0x7fa94c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.094+0000 7fa963fff700 1 -- 172.21.15.57:0/2557073022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94c004850 msgr2=0x7fa94c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:45.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.094+0000 7fa963fff700 1 --2- 172.21.15.57:0/2557073022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94c004850 0x7fa94c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.094+0000 7fa963fff700 1 -- 172.21.15.57:0/2557073022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c0035b0 con 0x7fa94c003a00 2023-11-28T12:07:45.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.094+0000 7fa968c96700 1 -- 172.21.15.57:0/2557073022 reap_dead start 2023-11-28T12:07:45.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.094+0000 7fa963fff700 1 --2- 172.21.15.57:0/2557073022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa94c003a00 0x7fa94c003e10 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa95400c240 tx=0x7fa94c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b757209214dce974 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:45.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.094+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954007c10 con 0x7fa94c003a00 2023-11-28T12:07:45.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.094+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95400cde0 con 0x7fa94c003a00 2023-11-28T12:07:45.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.094+0000 7fa962ffd700 1 -- 172.21.15.57:0/2557073022 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954005ba0 con 0x7fa94c003a00 2023-11-28T12:07:45.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.095+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2557073022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa94c003a00 msgr2=0x7fa94c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.095+0000 7fa96a6f9700 1 --2- 172.21.15.57:0/2557073022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa94c003a00 0x7fa94c003e10 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa95400c240 tx=0x7fa94c005880 comp rx=0 tx=0).stop 2023-11-28T12:07:45.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.095+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2557073022 shutdown_connections 2023-11-28T12:07:45.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.095+0000 7fa96a6f9700 1 --2- 172.21.15.57:0/2557073022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa94c003a00 0x7fa94c003e10 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.095+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2557073022 >> 172.21.15.57:0/2557073022 conn(0x7fa96406c010 msgr2=0x7fa9640cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:45.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.095+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2557073022 shutdown_connections 2023-11-28T12:07:45.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.095+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2557073022 wait complete. 2023-11-28T12:07:45.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.096+0000 7fa96a6f9700 1 Processor -- start 2023-11-28T12:07:45.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.096+0000 7fa96a6f9700 1 -- start start 2023-11-28T12:07:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa96a6f9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa94c003a00 0x7fa9640c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa96a6f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9640b5200 0x7fa96406a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa96a6f9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9640b5670 0x7fa96406a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa96a6f9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9640b4d70 con 0x7fa9640b5670 2023-11-28T12:07:45.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa96a6f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9640b4160 con 0x7fa9640b5200 2023-11-28T12:07:45.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa96a6f9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9640b4440 con 0x7fa94c003a00 2023-11-28T12:07:45.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa968c96700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9640b5670 0x7fa96406a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa968c96700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9640b5670 0x7fa96406a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57124/0 (socket says 172.21.15.57:57124) 2023-11-28T12:07:45.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa968c96700 1 -- 172.21.15.57:0/2282251881 learned_addr learned my addr 172.21.15.57:0/2282251881 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:45.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa963fff700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa94c003a00 0x7fa9640c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa9637fe700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9640b5200 0x7fa96406a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa968c96700 1 -- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa94c003a00 msgr2=0x7fa9640c28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa968c96700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa94c003a00 0x7fa9640c28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa968c96700 1 -- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9640b5200 msgr2=0x7fa96406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa968c96700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9640b5200 0x7fa96406a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa968c96700 1 -- 172.21.15.57:0/2282251881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96415a350 con 0x7fa9640b5670 2023-11-28T12:07:45.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.097+0000 7fa9637fe700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9640b5200 0x7fa96406a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:45.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.098+0000 7fa968c96700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9640b5670 0x7fa96406a7b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa9640b2620 tx=0x7fa95c00d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:45.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.098+0000 7fa960ff9700 1 -- 172.21.15.57:0/2282251881 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c003c10 con 0x7fa9640b5670 2023-11-28T12:07:45.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.098+0000 7fa960ff9700 1 -- 172.21.15.57:0/2282251881 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95c003d90 con 0x7fa9640b5670 2023-11-28T12:07:45.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.098+0000 7fa960ff9700 1 -- 172.21.15.57:0/2282251881 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c00da10 con 0x7fa9640b5670 2023-11-28T12:07:45.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.098+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2282251881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96415de60 con 0x7fa9640b5670 2023-11-28T12:07:45.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.098+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2282251881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96415a4e0 con 0x7fa9640b5670 2023-11-28T12:07:45.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.099+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2282251881 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9640743f0 con 0x7fa9640b5670 2023-11-28T12:07:45.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.100+0000 7fa960ff9700 1 -- 172.21.15.57:0/2282251881 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa95c00db70 con 0x7fa9640b5670 2023-11-28T12:07:45.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.100+0000 7fa960ff9700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa9500682f0 0x7fa95006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.100+0000 7fa960ff9700 1 -- 172.21.15.57:0/2282251881 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa95c0866d0 con 0x7fa9640b5670 2023-11-28T12:07:45.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.100+0000 7fa963fff700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa9500682f0 0x7fa95006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.101+0000 7fa963fff700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa9500682f0 0x7fa95006a7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa954000c00 tx=0x7fa954007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:45.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.104+0000 7fa960ff9700 1 -- 172.21.15.57:0/2282251881 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa95c0532f0 con 0x7fa9640b5670 2023-11-28T12:07:45.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.304+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2282251881 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9640b4eb0 con 0x7fa9500682f0 2023-11-28T12:07:45.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.304+0000 7fa960ff9700 1 -- 172.21.15.57:0/2282251881 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa9640b4eb0 con 0x7fa9500682f0 2023-11-28T12:07:45.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.306+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa9500682f0 msgr2=0x7fa95006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.306+0000 7fa96a6f9700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa9500682f0 0x7fa95006a7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa954000c00 tx=0x7fa954007000 comp rx=0 tx=0).stop 2023-11-28T12:07:45.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.306+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2282251881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9640b5670 msgr2=0x7fa96406a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.306+0000 7fa96a6f9700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9640b5670 0x7fa96406a7b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa9640b2620 tx=0x7fa95c00d150 comp rx=0 tx=0).stop 2023-11-28T12:07:45.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.306+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2282251881 shutdown_connections 2023-11-28T12:07:45.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.306+0000 7fa96a6f9700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa9500682f0 0x7fa95006a7a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.306+0000 7fa96a6f9700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9640b5670 0x7fa96406a7b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.306+0000 7fa96a6f9700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa94c003a00 0x7fa9640c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.306+0000 7fa96a6f9700 1 --2- 172.21.15.57:0/2282251881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9640b5200 0x7fa96406a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.306+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2282251881 >> 172.21.15.57:0/2282251881 conn(0x7fa96406c010 msgr2=0x7fa9640b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:45.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.306+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2282251881 shutdown_connections 2023-11-28T12:07:45.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.306+0000 7fa96a6f9700 1 -- 172.21.15.57:0/2282251881 wait complete. 2023-11-28T12:07:45.312 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 5 to deep-scrub 2023-11-28T12:07:45.322 INFO:tasks.ceph:Scrubbing osd.6 2023-11-28T12:07:45.323 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-28T12:07:45.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.461+0000 7ff9f7b25700 1 Processor -- start 2023-11-28T12:07:45.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.462+0000 7ff9f7b25700 1 -- start start 2023-11-28T12:07:45.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.462+0000 7ff9f7b25700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9f015a4a0 0x7ff9f00ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.462+0000 7ff9f7b25700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff9f00cf030 0x7ff9f00b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.462+0000 7ff9f7b25700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f00af050 con 0x7ff9f00af4e0 2023-11-28T12:07:45.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.462+0000 7ff9f7b25700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f015a870 con 0x7ff9f00cf030 2023-11-28T12:07:45.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.462+0000 7ff9f7b25700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00ae440 con 0x7ff9f015a4a0 2023-11-28T12:07:45.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.463+0000 7ff9f60c2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff9f00cf030 0x7ff9f00b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.463+0000 7ff9f50c0700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff9f00af4e0 0x7ff9f00ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:58922/0 (socket says 172.21.15.57:58922) 2023-11-28T12:07:45.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.463+0000 7ff9f50c0700 1 -- 172.21.15.57:0/3305041852 learned_addr learned my addr 172.21.15.57:0/3305041852 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:45.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.463+0000 7ff9f58c1700 1 --2- 172.21.15.57:0/3305041852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9f015a4a0 0x7ff9f00ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.463+0000 7ff9f60c2700 1 -- 172.21.15.57:0/3305041852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9f015a4a0 msgr2=0x7ff9f00ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:45.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.463+0000 7ff9f60c2700 1 --2- 172.21.15.57:0/3305041852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9f015a4a0 0x7ff9f00ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.463+0000 7ff9f60c2700 1 -- 172.21.15.57:0/3305041852 >> v1:172.21.15.57:6789/0 conn(0x7ff9f00af4e0 legacy=0x7ff9f00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.463+0000 7ff9f48bf700 1 -- 172.21.15.57:0/3305041852 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980629568 0 0) 0x7ff9f00af050 con 0x7ff9f00af4e0 2023-11-28T12:07:45.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.463+0000 7ff9f60c2700 1 -- 172.21.15.57:0/3305041852 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f00ae720 con 0x7ff9f00cf030 2023-11-28T12:07:45.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.463+0000 7ff9f60c2700 1 --2- 172.21.15.57:0/3305041852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9f00cf030 0x7ff9f00b0650 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009dc0 tx=0x7ff9e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c3fa7a07d632aa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:45.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.463+0000 7ff9f48bf700 1 -- 172.21.15.57:0/3305041852 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e800e070 con 0x7ff9f00cf030 2023-11-28T12:07:45.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.463+0000 7ff9f48bf700 1 -- 172.21.15.57:0/3305041852 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e800bb90 con 0x7ff9f00cf030 2023-11-28T12:07:45.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.463+0000 7ff9f48bf700 1 -- 172.21.15.57:0/3305041852 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8012430 con 0x7ff9f00cf030 2023-11-28T12:07:45.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.464+0000 7ff9f7b25700 1 -- 172.21.15.57:0/3305041852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9f00cf030 msgr2=0x7ff9f00b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.464+0000 7ff9f7b25700 1 --2- 172.21.15.57:0/3305041852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9f00cf030 0x7ff9f00b0650 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009dc0 tx=0x7ff9e800b330 comp rx=0 tx=0).stop 2023-11-28T12:07:45.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.464+0000 7ff9f7b25700 1 -- 172.21.15.57:0/3305041852 shutdown_connections 2023-11-28T12:07:45.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.464+0000 7ff9f7b25700 1 --2- 172.21.15.57:0/3305041852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9f00cf030 0x7ff9f00b0650 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.464+0000 7ff9f7b25700 1 --2- 172.21.15.57:0/3305041852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9f015a4a0 0x7ff9f00ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.464+0000 7ff9f7b25700 1 -- 172.21.15.57:0/3305041852 >> 172.21.15.57:0/3305041852 conn(0x7ff9f006ab90 msgr2=0x7ff9f0099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:45.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.464+0000 7ff9f7b25700 1 -- 172.21.15.57:0/3305041852 shutdown_connections 2023-11-28T12:07:45.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.464+0000 7ff9f7b25700 1 -- 172.21.15.57:0/3305041852 wait complete. 2023-11-28T12:07:45.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.465+0000 7ff9f7b25700 1 Processor -- start 2023-11-28T12:07:45.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.465+0000 7ff9f7b25700 1 -- start start 2023-11-28T12:07:45.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.466+0000 7ff9f7b25700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9f00af4e0 0x7ff9f009e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.466+0000 7ff9f7b25700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9f00cf030 0x7ff9f0148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.466+0000 7ff9f7b25700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9f015a4a0 0x7ff9f0157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.466+0000 7ff9f7b25700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f0156600 con 0x7ff9f00af4e0 2023-11-28T12:07:45.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.466+0000 7ff9f7b25700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f009d420 con 0x7ff9f00cf030 2023-11-28T12:07:45.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.466+0000 7ff9f7b25700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f009d700 con 0x7ff9f015a4a0 2023-11-28T12:07:45.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.466+0000 7ff9f58c1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9f00af4e0 0x7ff9f009e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.466+0000 7ff9f58c1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9f00af4e0 0x7ff9f009e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57132/0 (socket says 172.21.15.57:57132) 2023-11-28T12:07:45.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.466+0000 7ff9f58c1700 1 -- 172.21.15.57:0/4112210837 learned_addr learned my addr 172.21.15.57:0/4112210837 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:45.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.467+0000 7ff9f60c2700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9f015a4a0 0x7ff9f0157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.467+0000 7ff9f58c1700 1 -- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9f015a4a0 msgr2=0x7ff9f0157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.467+0000 7ff9f50c0700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9f00cf030 0x7ff9f0148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.467+0000 7ff9f58c1700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9f015a4a0 0x7ff9f0157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.467+0000 7ff9f58c1700 1 -- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9f00cf030 msgr2=0x7ff9f0148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.467+0000 7ff9f58c1700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9f00cf030 0x7ff9f0148fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.467+0000 7ff9f58c1700 1 -- 172.21.15.57:0/4112210837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e8009a70 con 0x7ff9f00af4e0 2023-11-28T12:07:45.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.467+0000 7ff9f58c1700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9f00af4e0 0x7ff9f009e4c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00d7d0 tx=0x7ff9ec00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:45.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.467+0000 7ff9e27fc700 1 -- 172.21.15.57:0/4112210837 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec0125f0 con 0x7ff9f00af4e0 2023-11-28T12:07:45.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.467+0000 7ff9e27fc700 1 -- 172.21.15.57:0/4112210837 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec012750 con 0x7ff9f00af4e0 2023-11-28T12:07:45.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.467+0000 7ff9f7b25700 1 -- 172.21.15.57:0/4112210837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f00aece0 con 0x7ff9f00af4e0 2023-11-28T12:07:45.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.467+0000 7ff9e27fc700 1 -- 172.21.15.57:0/4112210837 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec012b00 con 0x7ff9f00af4e0 2023-11-28T12:07:45.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.467+0000 7ff9f7b25700 1 -- 172.21.15.57:0/4112210837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f00aea00 con 0x7ff9f00af4e0 2023-11-28T12:07:45.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.468+0000 7ff9f7b25700 1 -- 172.21.15.57:0/4112210837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff9f00d0bd0 con 0x7ff9f00af4e0 2023-11-28T12:07:45.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.469+0000 7ff9e27fc700 1 -- 172.21.15.57:0/4112210837 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff9ec012c60 con 0x7ff9f00af4e0 2023-11-28T12:07:45.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.469+0000 7ff9e27fc700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff9d4068320 0x7ff9d406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.469+0000 7ff9e27fc700 1 -- 172.21.15.57:0/4112210837 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff9ec0869c0 con 0x7ff9f00af4e0 2023-11-28T12:07:45.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.469+0000 7ff9e27fc700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x7ff9d406e790 0x7ff9d4070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.469+0000 7ff9e27fc700 1 -- 172.21.15.57:0/4112210837 --> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9d4071270 con 0x7ff9d406e790 2023-11-28T12:07:45.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.469+0000 7ff9e27fc700 1 -- 172.21.15.57:0/4112210837 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9ec086d30 con 0x7ff9f00af4e0 2023-11-28T12:07:45.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.469+0000 7ff9f50c0700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff9d4068320 0x7ff9d406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.470+0000 7ff9f60c2700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x7ff9d406e790 0x7ff9d4070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.470+0000 7ff9f50c0700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff9d4068320 0x7ff9d406a7d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0158790 tx=0x7ff9e4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:45.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.470+0000 7ff9f60c2700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x7ff9d406e790 0x7ff9d4070ba0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:45.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.471+0000 7ff9e27fc700 1 -- 172.21.15.57:0/4112210837 <== osd.6 v2:172.21.15.181:6816/120984 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff9d4071270 con 0x7ff9d406e790 2023-11-28T12:07:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.486+0000 7ff9f7b25700 1 -- 172.21.15.57:0/4112210837 --> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff9f00743f0 con 0x7ff9d406e790 2023-11-28T12:07:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.486+0000 7ff9e27fc700 1 -- 172.21.15.57:0/4112210837 <== osd.6 v2:172.21.15.181:6816/120984 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff9f00743f0 con 0x7ff9d406e790 2023-11-28T12:07:45.487 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:07:45.488 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T12:07:45.488 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:07:45.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.487+0000 7ff9f7b25700 1 -- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x7ff9d406e790 msgr2=0x7ff9d4070ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.487+0000 7ff9f7b25700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:6816/120984,v1:172.21.15.181:6817/120984] conn(0x7ff9d406e790 0x7ff9d4070ba0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.487+0000 7ff9f7b25700 1 -- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff9d4068320 msgr2=0x7ff9d406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.487+0000 7ff9f7b25700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff9d4068320 0x7ff9d406a7d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0158790 tx=0x7ff9e4009040 comp rx=0 tx=0).stop 2023-11-28T12:07:45.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.487+0000 7ff9f7b25700 1 -- 172.21.15.57:0/4112210837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9f00af4e0 msgr2=0x7ff9f009e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.487+0000 7ff9f7b25700 1 --2- 172.21.15.57:0/4112210837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9f00af4e0 0x7ff9f009e4c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00d7d0 tx=0x7ff9ec00db90 comp rx=0 tx=0).stop 2023-11-28T12:07:45.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.487+0000 7ff9f60c2700 1 -- 172.21.15.57:0/4112210837 reap_dead start 2023-11-28T12:07:45.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.487+0000 7ff9f7b25700 1 -- 172.21.15.57:0/4112210837 shutdown_connections 2023-11-28T12:07:45.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.487+0000 7ff9f7b25700 1 -- 172.21.15.57:0/4112210837 >> 172.21.15.57:0/4112210837 conn(0x7ff9f006ab90 msgr2=0x7ff9f006f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:45.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.487+0000 7ff9f7b25700 1 -- 172.21.15.57:0/4112210837 shutdown_connections 2023-11-28T12:07:45.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.487+0000 7ff9f7b25700 1 -- 172.21.15.57:0/4112210837 wait complete. 2023-11-28T12:07:45.501 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-28T12:07:45.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.637+0000 7f6db79ea700 1 Processor -- start 2023-11-28T12:07:45.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.637+0000 7f6db79ea700 1 -- start start 2023-11-28T12:07:45.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.638+0000 7f6db79ea700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6db015c810 0x7f6db0160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.638+0000 7f6db79ea700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6db006a270 0x7f6db006a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.638+0000 7f6db79ea700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db006abf0 con 0x7f6db00718e0 2023-11-28T12:07:45.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.638+0000 7f6db79ea700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db015cbe0 con 0x7f6db015c810 2023-11-28T12:07:45.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.638+0000 7f6db79ea700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01607a0 con 0x7f6db006a270 2023-11-28T12:07:45.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.638+0000 7f6db5f87700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6db006a270 0x7f6db006a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.638+0000 7f6db5786700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6db015c810 0x7f6db0160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.638+0000 7f6db4f85700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6db00718e0 0x7f6db006c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.57:49914/0 (socket says 172.21.15.57:49914) 2023-11-28T12:07:45.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.638+0000 7f6db4f85700 1 -- 172.21.15.57:0/861322494 learned_addr learned my addr 172.21.15.57:0/861322494 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:45.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.638+0000 7f6da7fff700 1 -- 172.21.15.57:0/861322494 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3735821099 0 0) 0x7f6db006abf0 con 0x7f6db00718e0 2023-11-28T12:07:45.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.639+0000 7f6db5f87700 1 -- 172.21.15.57:0/861322494 >> v2:172.21.15.181:3300/0 conn(0x7f6db015c810 msgr2=0x7f6db0160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:45.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.639+0000 7f6db5f87700 1 --2- 172.21.15.57:0/861322494 >> v2:172.21.15.181:3300/0 conn(0x7f6db015c810 0x7f6db0160260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.639+0000 7f6db5f87700 1 -- 172.21.15.57:0/861322494 >> v1:172.21.15.181:6789/0 conn(0x7f6db00718e0 legacy=0x7f6db006c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.639+0000 7f6db5f87700 1 -- 172.21.15.57:0/861322494 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0160a50 con 0x7f6db006a270 2023-11-28T12:07:45.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.639+0000 7f6db5786700 1 --2- 172.21.15.57:0/861322494 >> v2:172.21.15.181:3300/0 conn(0x7f6db015c810 0x7f6db0160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:07:45.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.639+0000 7f6db5f87700 1 --2- 172.21.15.57:0/861322494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6db006a270 0x7f6db006a6b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009d90 tx=0x7f6da800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60544a557a68a69f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:45.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.639+0000 7f6da7fff700 1 -- 172.21.15.57:0/861322494 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800e070 con 0x7f6db006a270 2023-11-28T12:07:45.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.639+0000 7f6da7fff700 1 -- 172.21.15.57:0/861322494 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da800bb20 con 0x7f6db006a270 2023-11-28T12:07:45.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.639+0000 7f6da7fff700 1 -- 172.21.15.57:0/861322494 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8012430 con 0x7f6db006a270 2023-11-28T12:07:45.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.640+0000 7f6db79ea700 1 -- 172.21.15.57:0/861322494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6db006a270 msgr2=0x7f6db006a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.640+0000 7f6db79ea700 1 --2- 172.21.15.57:0/861322494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6db006a270 0x7f6db006a6b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009d90 tx=0x7f6da800b330 comp rx=0 tx=0).stop 2023-11-28T12:07:45.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.640+0000 7f6db79ea700 1 -- 172.21.15.57:0/861322494 shutdown_connections 2023-11-28T12:07:45.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.640+0000 7f6db79ea700 1 --2- 172.21.15.57:0/861322494 >> v2:172.21.15.181:3300/0 conn(0x7f6db015c810 0x7f6db0160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.640+0000 7f6db79ea700 1 --2- 172.21.15.57:0/861322494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6db006a270 0x7f6db006a6b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.640+0000 7f6db79ea700 1 -- 172.21.15.57:0/861322494 >> 172.21.15.57:0/861322494 conn(0x7f6db006c010 msgr2=0x7f6db00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:45.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.640+0000 7f6db79ea700 1 -- 172.21.15.57:0/861322494 shutdown_connections 2023-11-28T12:07:45.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.640+0000 7f6db79ea700 1 -- 172.21.15.57:0/861322494 wait complete. 2023-11-28T12:07:45.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.641+0000 7f6db79ea700 1 Processor -- start 2023-11-28T12:07:45.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.641+0000 7f6db79ea700 1 -- start start 2023-11-28T12:07:45.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db79ea700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6db006a270 0x7f6db0151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db79ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db00718e0 0x7f6db00b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db79ea700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6db015c810 0x7f6db00c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db79ea700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db00b5750 con 0x7f6db006a270 2023-11-28T12:07:45.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db79ea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db00b4a80 con 0x7f6db00718e0 2023-11-28T12:07:45.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db79ea700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db00b44c0 con 0x7f6db015c810 2023-11-28T12:07:45.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db5786700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6db006a270 0x7f6db0151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db5786700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6db006a270 0x7f6db0151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57144/0 (socket says 172.21.15.57:57144) 2023-11-28T12:07:45.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db5786700 1 -- 172.21.15.57:0/1653198177 learned_addr learned my addr 172.21.15.57:0/1653198177 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:45.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db4f85700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db00718e0 0x7f6db00b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db5f87700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6db015c810 0x7f6db00c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db5786700 1 -- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6db015c810 msgr2=0x7f6db00c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db5786700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6db015c810 0x7f6db00c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db5786700 1 -- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db00718e0 msgr2=0x7f6db00b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db5786700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db00718e0 0x7f6db00b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.642+0000 7f6db5786700 1 -- 172.21.15.57:0/1653198177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac0094a0 con 0x7f6db006a270 2023-11-28T12:07:45.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.643+0000 7f6db5786700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6db006a270 0x7f6db0151a10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00ebe0 tx=0x7f6dac009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:45.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.643+0000 7f6da5ffb700 1 -- 172.21.15.57:0/1653198177 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00c8b0 con 0x7f6db006a270 2023-11-28T12:07:45.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.643+0000 7f6da5ffb700 1 -- 172.21.15.57:0/1653198177 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac00ca10 con 0x7f6db006a270 2023-11-28T12:07:45.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.643+0000 7f6db79ea700 1 -- 172.21.15.57:0/1653198177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da8009a70 con 0x7f6db006a270 2023-11-28T12:07:45.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.643+0000 7f6da5ffb700 1 -- 172.21.15.57:0/1653198177 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00cda0 con 0x7f6db006a270 2023-11-28T12:07:45.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.643+0000 7f6db79ea700 1 -- 172.21.15.57:0/1653198177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db0161010 con 0x7f6db006a270 2023-11-28T12:07:45.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.644+0000 7f6db79ea700 1 -- 172.21.15.57:0/1653198177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db0160d30 con 0x7f6db006a270 2023-11-28T12:07:45.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.644+0000 7f6da5ffb700 1 -- 172.21.15.57:0/1653198177 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6dac019030 con 0x7f6db006a270 2023-11-28T12:07:45.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.645+0000 7f6da5ffb700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6d940682f0 0x7f6d9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:45.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.645+0000 7f6da5ffb700 1 -- 172.21.15.57:0/1653198177 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6dac085970 con 0x7f6db006a270 2023-11-28T12:07:45.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.645+0000 7f6db4f85700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6d940682f0 0x7f6d9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:45.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.646+0000 7f6db4f85700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6d940682f0 0x7f6d9406a7a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6db00ca2f0 tx=0x7f6da0007be0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:45.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.650+0000 7f6da5ffb700 1 -- 172.21.15.57:0/1653198177 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6dac0524d0 con 0x7f6db006a270 2023-11-28T12:07:45.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.851+0000 7f6db79ea700 1 -- 172.21.15.57:0/1653198177 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f6db00ceb20 con 0x7f6d940682f0 2023-11-28T12:07:45.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.851+0000 7f6da5ffb700 1 -- 172.21.15.57:0/1653198177 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6db00ceb20 con 0x7f6d940682f0 2023-11-28T12:07:45.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.854+0000 7f6db79ea700 1 -- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6d940682f0 msgr2=0x7f6d9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.854+0000 7f6db79ea700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6d940682f0 0x7f6d9406a7a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6db00ca2f0 tx=0x7f6da0007be0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.854+0000 7f6db79ea700 1 -- 172.21.15.57:0/1653198177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6db006a270 msgr2=0x7f6db0151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:45.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.854+0000 7f6db79ea700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6db006a270 0x7f6db0151a10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00ebe0 tx=0x7f6dac009e20 comp rx=0 tx=0).stop 2023-11-28T12:07:45.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.854+0000 7f6db79ea700 1 -- 172.21.15.57:0/1653198177 shutdown_connections 2023-11-28T12:07:45.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.854+0000 7f6db79ea700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f6d940682f0 0x7f6d9406a7a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.854+0000 7f6db79ea700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6db015c810 0x7f6db00c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.854+0000 7f6db79ea700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6db006a270 0x7f6db0151a10 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.854+0000 7f6db79ea700 1 --2- 172.21.15.57:0/1653198177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db00718e0 0x7f6db00b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:45.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.854+0000 7f6db79ea700 1 -- 172.21.15.57:0/1653198177 >> 172.21.15.57:0/1653198177 conn(0x7f6db006c010 msgr2=0x7f6db0155d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:45.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.854+0000 7f6db79ea700 1 -- 172.21.15.57:0/1653198177 shutdown_connections 2023-11-28T12:07:45.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:45.854+0000 7f6db79ea700 1 -- 172.21.15.57:0/1653198177 wait complete. 2023-11-28T12:07:45.857 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 6 to deep-scrub 2023-11-28T12:07:45.869 INFO:tasks.ceph:Scrubbing osd.7 2023-11-28T12:07:45.869 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-28T12:07:46.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.007+0000 7fe17acef700 1 Processor -- start 2023-11-28T12:07:46.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.007+0000 7fe17acef700 1 -- start start 2023-11-28T12:07:46.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.008+0000 7fe17acef700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1740b0200 0x7fe1740cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.008+0000 7fe17acef700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17406b750 con 0x7fe1740cff10 2023-11-28T12:07:46.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.008+0000 7fe17acef700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe174160580 con 0x7fe1740cdbe0 2023-11-28T12:07:46.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.008+0000 7fe17acef700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1740ce080 con 0x7fe1740b0200 2023-11-28T12:07:46.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.008+0000 7fe17928c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe1740cff10 0x7fe17406bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:58932/0 (socket says 172.21.15.57:58932) 2023-11-28T12:07:46.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.008+0000 7fe17928c700 1 -- 172.21.15.57:0/1529640914 learned_addr learned my addr 172.21.15.57:0/1529640914 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:07:46.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.008+0000 7fe178a8b700 1 --2- 172.21.15.57:0/1529640914 >> v2:172.21.15.181:3300/0 conn(0x7fe1740b0200 0x7fe1740cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.008+0000 7fe1737fe700 1 -- 172.21.15.57:0/1529640914 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1260607145 0 0) 0x7fe17406b750 con 0x7fe1740cff10 2023-11-28T12:07:46.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.008+0000 7fe1737fe700 1 -- 172.21.15.57:0/1529640914 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1600035b0 con 0x7fe1740cff10 2023-11-28T12:07:46.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.009+0000 7fe1737fe700 1 -- 172.21.15.57:0/1529640914 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159502395 0 0) 0x7fe174160580 con 0x7fe1740cdbe0 2023-11-28T12:07:46.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.009+0000 7fe1737fe700 1 -- 172.21.15.57:0/1529640914 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe17406b750 con 0x7fe1740cdbe0 2023-11-28T12:07:46.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.009+0000 7fe1737fe700 1 -- 172.21.15.57:0/1529640914 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 649502874 0 0) 0x7fe1600035b0 con 0x7fe1740cff10 2023-11-28T12:07:46.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.009+0000 7fe178a8b700 1 -- 172.21.15.57:0/1529640914 >> v1:172.21.15.181:6789/0 conn(0x7fe1740cdbe0 legacy=0x7fe1740cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.009+0000 7fe178a8b700 1 -- 172.21.15.57:0/1529640914 >> v1:172.21.15.57:6789/0 conn(0x7fe1740cff10 legacy=0x7fe17406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.009+0000 7fe178a8b700 1 -- 172.21.15.57:0/1529640914 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17406b420 con 0x7fe1740b0200 2023-11-28T12:07:46.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.009+0000 7fe178a8b700 1 --2- 172.21.15.57:0/1529640914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1740b0200 0x7fe1740cd6a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe164009840 tx=0x7fe164009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb023cedb3d73a68 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:46.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.009+0000 7fe1737fe700 1 -- 172.21.15.57:0/1529640914 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c0019f0 con 0x7fe1740b0200 2023-11-28T12:07:46.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.009+0000 7fe1737fe700 1 -- 172.21.15.57:0/1529640914 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe164004030 con 0x7fe1740b0200 2023-11-28T12:07:46.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.009+0000 7fe1737fe700 1 -- 172.21.15.57:0/1529640914 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1640043c0 con 0x7fe1740b0200 2023-11-28T12:07:46.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.010+0000 7fe17acef700 1 -- 172.21.15.57:0/1529640914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1740b0200 msgr2=0x7fe1740cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.010+0000 7fe17acef700 1 --2- 172.21.15.57:0/1529640914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1740b0200 0x7fe1740cd6a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe164009840 tx=0x7fe164009b50 comp rx=0 tx=0).stop 2023-11-28T12:07:46.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.010+0000 7fe17acef700 1 -- 172.21.15.57:0/1529640914 shutdown_connections 2023-11-28T12:07:46.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.010+0000 7fe17acef700 1 --2- 172.21.15.57:0/1529640914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1740b0200 0x7fe1740cd6a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.010+0000 7fe17acef700 1 -- 172.21.15.57:0/1529640914 >> 172.21.15.57:0/1529640914 conn(0x7fe17406dcc0 msgr2=0x7fe17406e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:46.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.010+0000 7fe17acef700 1 -- 172.21.15.57:0/1529640914 shutdown_connections 2023-11-28T12:07:46.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.010+0000 7fe17acef700 1 -- 172.21.15.57:0/1529640914 wait complete. 2023-11-28T12:07:46.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.011+0000 7fe17acef700 1 Processor -- start 2023-11-28T12:07:46.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.011+0000 7fe17acef700 1 -- start start 2023-11-28T12:07:46.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe17acef700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1740b0200 0x7fe1741582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe17acef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1740cdbe0 0x7fe1741568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe17acef700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1740cff10 0x7fe1740b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe17acef700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe174158820 con 0x7fe1740b0200 2023-11-28T12:07:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe17acef700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe174157240 con 0x7fe1740cdbe0 2023-11-28T12:07:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe17acef700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe174157520 con 0x7fe1740cff10 2023-11-28T12:07:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe178a8b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1740b0200 0x7fe1741582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe178a8b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1740b0200 0x7fe1741582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57154/0 (socket says 172.21.15.57:57154) 2023-11-28T12:07:46.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe178a8b700 1 -- 172.21.15.57:0/1410323699 learned_addr learned my addr 172.21.15.57:0/1410323699 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:46.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe17928c700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1740cff10 0x7fe1740b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe173fff700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1740cdbe0 0x7fe1741568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe178a8b700 1 -- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1740cff10 msgr2=0x7fe1740b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe178a8b700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1740cff10 0x7fe1740b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe178a8b700 1 -- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1740cdbe0 msgr2=0x7fe1741568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe178a8b700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1740cdbe0 0x7fe1741568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe178a8b700 1 -- 172.21.15.57:0/1410323699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1640094a0 con 0x7fe1740b0200 2023-11-28T12:07:46.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe17928c700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1740cff10 0x7fe1740b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:46.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.012+0000 7fe173fff700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1740cdbe0 0x7fe1741568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:07:46.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.013+0000 7fe178a8b700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1740b0200 0x7fe1741582e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe1640090b0 tx=0x7fe164004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:46.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.013+0000 7fe1717fa700 1 -- 172.21.15.57:0/1410323699 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164011830 con 0x7fe1740b0200 2023-11-28T12:07:46.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.013+0000 7fe1717fa700 1 -- 172.21.15.57:0/1410323699 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16401f070 con 0x7fe1740b0200 2023-11-28T12:07:46.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.013+0000 7fe17acef700 1 -- 172.21.15.57:0/1410323699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1740d0750 con 0x7fe1740b0200 2023-11-28T12:07:46.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.013+0000 7fe1717fa700 1 -- 172.21.15.57:0/1410323699 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164011bf0 con 0x7fe1740b0200 2023-11-28T12:07:46.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.013+0000 7fe17acef700 1 -- 172.21.15.57:0/1410323699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1740d0470 con 0x7fe1740b0200 2023-11-28T12:07:46.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.014+0000 7fe17acef700 1 -- 172.21.15.57:0/1410323699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1740d0a30 con 0x7fe1740b0200 2023-11-28T12:07:46.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.014+0000 7fe1717fa700 1 -- 172.21.15.57:0/1410323699 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe16402d070 con 0x7fe1740b0200 2023-11-28T12:07:46.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.015+0000 7fe1717fa700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1540682f0 0x7fe15406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.015+0000 7fe1717fa700 1 -- 172.21.15.57:0/1410323699 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe164021070 con 0x7fe1740b0200 2023-11-28T12:07:46.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.015+0000 7fe1717fa700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x7fe15406e760 0x7fe154070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.015+0000 7fe1717fa700 1 -- 172.21.15.57:0/1410323699 --> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe154071240 con 0x7fe15406e760 2023-11-28T12:07:46.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.015+0000 7fe1717fa700 1 -- 172.21.15.57:0/1410323699 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe164089540 con 0x7fe1740b0200 2023-11-28T12:07:46.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.015+0000 7fe173fff700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1540682f0 0x7fe15406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.015+0000 7fe17928c700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x7fe15406e760 0x7fe154070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.015+0000 7fe173fff700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1540682f0 0x7fe15406a7a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe1740c7860 tx=0x7fe15c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:46.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.016+0000 7fe17928c700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x7fe15406e760 0x7fe154070b70 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:46.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.016+0000 7fe1717fa700 1 -- 172.21.15.57:0/1410323699 <== osd.7 v2:172.21.15.181:6808/120959 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe154071240 con 0x7fe15406e760 2023-11-28T12:07:46.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.031+0000 7fe17acef700 1 -- 172.21.15.57:0/1410323699 --> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe174061e80 con 0x7fe15406e760 2023-11-28T12:07:46.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.032+0000 7fe1717fa700 1 -- 172.21.15.57:0/1410323699 <== osd.7 v2:172.21.15.181:6808/120959 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe174061e80 con 0x7fe15406e760 2023-11-28T12:07:46.033 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-11-28T12:07:46.034 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T12:07:46.034 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-11-28T12:07:46.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.032+0000 7fe17acef700 1 -- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x7fe15406e760 msgr2=0x7fe154070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.032+0000 7fe17acef700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:6808/120959,v1:172.21.15.181:6809/120959] conn(0x7fe15406e760 0x7fe154070b70 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.032+0000 7fe17acef700 1 -- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1540682f0 msgr2=0x7fe15406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.032+0000 7fe17acef700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fe1540682f0 0x7fe15406a7a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe1740c7860 tx=0x7fe15c009040 comp rx=0 tx=0).stop 2023-11-28T12:07:46.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.032+0000 7fe17acef700 1 -- 172.21.15.57:0/1410323699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1740b0200 msgr2=0x7fe1741582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.032+0000 7fe17acef700 1 --2- 172.21.15.57:0/1410323699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1740b0200 0x7fe1741582e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe1640090b0 tx=0x7fe164004750 comp rx=0 tx=0).stop 2023-11-28T12:07:46.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.032+0000 7fe17928c700 1 -- 172.21.15.57:0/1410323699 reap_dead start 2023-11-28T12:07:46.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.032+0000 7fe17acef700 1 -- 172.21.15.57:0/1410323699 shutdown_connections 2023-11-28T12:07:46.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.032+0000 7fe17acef700 1 -- 172.21.15.57:0/1410323699 >> 172.21.15.57:0/1410323699 conn(0x7fe17406dcc0 msgr2=0x7fe17406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:46.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.032+0000 7fe17acef700 1 -- 172.21.15.57:0/1410323699 shutdown_connections 2023-11-28T12:07:46.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.032+0000 7fe17acef700 1 -- 172.21.15.57:0/1410323699 wait complete. 2023-11-28T12:07:46.046 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-28T12:07:46.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.183+0000 7fa90ee94700 1 Processor -- start 2023-11-28T12:07:46.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.183+0000 7fa90ee94700 1 -- start start 2023-11-28T12:07:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.183+0000 7fa90ee94700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa9080cec00 0x7fa908065fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa90ee94700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9080cac90 0x7fa9080cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa90ee94700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90815ceb0 con 0x7fa908160300 2023-11-28T12:07:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa90ee94700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9080cb650 con 0x7fa9080cec00 2023-11-28T12:07:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa90ee94700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9081608c0 con 0x7fa9080cac90 2023-11-28T12:07:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa90cc30700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa9080cec00 0x7fa908065fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa90cc30700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa9080cec00 0x7fa908065fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57158/0 (socket says 172.21.15.57:57158) 2023-11-28T12:07:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa90cc30700 1 -- 172.21.15.57:0/2404795970 learned_addr learned my addr 172.21.15.57:0/2404795970 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa90d431700 1 --2- 172.21.15.57:0/2404795970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9080cac90 0x7fa9080cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa90cc30700 1 -- 172.21.15.57:0/2404795970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9080cac90 msgr2=0x7fa9080cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa90cc30700 1 --2- 172.21.15.57:0/2404795970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9080cac90 0x7fa9080cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa90cc30700 1 -- 172.21.15.57:0/2404795970 >> v1:172.21.15.181:6789/0 conn(0x7fa908160300 legacy=0x7fa90815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa90cc30700 1 -- 172.21.15.57:0/2404795970 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908160ba0 con 0x7fa9080cec00 2023-11-28T12:07:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa90cc30700 1 --2- 172.21.15.57:0/2404795970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9080cec00 0x7fa908065fd0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80098f0 tx=0x7fa8f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73debc4e78f6338 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.184+0000 7fa9077fe700 1 -- 172.21.15.57:0/2404795970 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f800b8a0 con 0x7fa9080cec00 2023-11-28T12:07:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.185+0000 7fa9077fe700 1 -- 172.21.15.57:0/2404795970 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f800ba00 con 0x7fa9080cec00 2023-11-28T12:07:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.185+0000 7fa9077fe700 1 -- 172.21.15.57:0/2404795970 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f800bd90 con 0x7fa9080cec00 2023-11-28T12:07:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.185+0000 7fa90ee94700 1 -- 172.21.15.57:0/2404795970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9080cec00 msgr2=0x7fa908065fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.185+0000 7fa90ee94700 1 --2- 172.21.15.57:0/2404795970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9080cec00 0x7fa908065fd0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80098f0 tx=0x7fa8f8009c00 comp rx=0 tx=0).stop 2023-11-28T12:07:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.185+0000 7fa90ee94700 1 -- 172.21.15.57:0/2404795970 shutdown_connections 2023-11-28T12:07:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.185+0000 7fa90ee94700 1 --2- 172.21.15.57:0/2404795970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9080cec00 0x7fa908065fd0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.185+0000 7fa90ee94700 1 --2- 172.21.15.57:0/2404795970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9080cac90 0x7fa9080cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.185+0000 7fa90ee94700 1 -- 172.21.15.57:0/2404795970 >> 172.21.15.57:0/2404795970 conn(0x7fa9080cf400 msgr2=0x7fa9080cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:46.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.185+0000 7fa90ee94700 1 -- 172.21.15.57:0/2404795970 shutdown_connections 2023-11-28T12:07:46.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.185+0000 7fa90ee94700 1 -- 172.21.15.57:0/2404795970 wait complete. 2023-11-28T12:07:46.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.187+0000 7fa90ee94700 1 Processor -- start 2023-11-28T12:07:46.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.187+0000 7fa90ee94700 1 -- start start 2023-11-28T12:07:46.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.188+0000 7fa90ee94700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9080cac90 0x7fa908152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.188+0000 7fa90ee94700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9080cec00 0x7fa9080cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.188+0000 7fa90ee94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa908160300 0x7fa90815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.188+0000 7fa90ee94700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080c5260 con 0x7fa9080cac90 2023-11-28T12:07:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.188+0000 7fa90ee94700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080c5540 con 0x7fa908160300 2023-11-28T12:07:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.188+0000 7fa90ee94700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080c5820 con 0x7fa9080cec00 2023-11-28T12:07:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.188+0000 7fa90cc30700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9080cac90 0x7fa908152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.188+0000 7fa90cc30700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9080cac90 0x7fa908152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57166/0 (socket says 172.21.15.57:57166) 2023-11-28T12:07:46.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.188+0000 7fa90cc30700 1 -- 172.21.15.57:0/411319415 learned_addr learned my addr 172.21.15.57:0/411319415 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:46.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.188+0000 7fa90d431700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa908160300 0x7fa90815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.188+0000 7fa90cc30700 1 -- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9080cec00 msgr2=0x7fa9080cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.188+0000 7fa907fff700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9080cec00 0x7fa9080cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.189+0000 7fa90cc30700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9080cec00 0x7fa9080cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.189+0000 7fa90cc30700 1 -- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa908160300 msgr2=0x7fa90815efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.189+0000 7fa90cc30700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa908160300 0x7fa90815efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.189+0000 7fa90cc30700 1 -- 172.21.15.57:0/411319415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f8009550 con 0x7fa9080cac90 2023-11-28T12:07:46.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.189+0000 7fa90d431700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa908160300 0x7fa90815efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:46.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.189+0000 7fa907fff700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9080cec00 0x7fa9080cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:46.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.189+0000 7fa90cc30700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9080cac90 0x7fa908152420 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009c40 tx=0x7fa8f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:46.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.189+0000 7fa9057fa700 1 -- 172.21.15.57:0/411319415 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8004400 con 0x7fa9080cac90 2023-11-28T12:07:46.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.189+0000 7fa9057fa700 1 -- 172.21.15.57:0/411319415 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f801f070 con 0x7fa9080cac90 2023-11-28T12:07:46.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.189+0000 7fa90ee94700 1 -- 172.21.15.57:0/411319415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa908161160 con 0x7fa9080cac90 2023-11-28T12:07:46.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.189+0000 7fa9057fa700 1 -- 172.21.15.57:0/411319415 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80038c0 con 0x7fa9080cac90 2023-11-28T12:07:46.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.189+0000 7fa90ee94700 1 -- 172.21.15.57:0/411319415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa908160e80 con 0x7fa9080cac90 2023-11-28T12:07:46.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.190+0000 7fa90ee94700 1 -- 172.21.15.57:0/411319415 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa908066510 con 0x7fa9080cac90 2023-11-28T12:07:46.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.191+0000 7fa9057fa700 1 -- 172.21.15.57:0/411319415 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa8f802a020 con 0x7fa9080cac90 2023-11-28T12:07:46.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.191+0000 7fa9057fa700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa8f40682f0 0x7fa8f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.191+0000 7fa9057fa700 1 -- 172.21.15.57:0/411319415 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa8f8087090 con 0x7fa9080cac90 2023-11-28T12:07:46.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.191+0000 7fa907fff700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa8f40682f0 0x7fa8f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.192+0000 7fa907fff700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa8f40682f0 0x7fa8f406a7a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa9081f5a10 tx=0x7fa8fc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:46.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.196+0000 7fa9057fa700 1 -- 172.21.15.57:0/411319415 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa8f8053bf0 con 0x7fa9080cac90 2023-11-28T12:07:46.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.396+0000 7fa90ee94700 1 -- 172.21.15.57:0/411319415 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9080743f0 con 0x7fa8f40682f0 2023-11-28T12:07:46.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.397+0000 7fa9057fa700 1 -- 172.21.15.57:0/411319415 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa9080743f0 con 0x7fa8f40682f0 2023-11-28T12:07:46.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.399+0000 7fa90ee94700 1 -- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa8f40682f0 msgr2=0x7fa8f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.399+0000 7fa90ee94700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa8f40682f0 0x7fa8f406a7a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa9081f5a10 tx=0x7fa8fc009040 comp rx=0 tx=0).stop 2023-11-28T12:07:46.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.399+0000 7fa90ee94700 1 -- 172.21.15.57:0/411319415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9080cac90 msgr2=0x7fa908152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.399+0000 7fa90ee94700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9080cac90 0x7fa908152420 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009c40 tx=0x7fa8f8004750 comp rx=0 tx=0).stop 2023-11-28T12:07:46.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.399+0000 7fa90ee94700 1 -- 172.21.15.57:0/411319415 shutdown_connections 2023-11-28T12:07:46.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.399+0000 7fa90ee94700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7fa8f40682f0 0x7fa8f406a7a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.399+0000 7fa90ee94700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa908160300 0x7fa90815efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.399+0000 7fa90ee94700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9080cac90 0x7fa908152420 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.399+0000 7fa90ee94700 1 --2- 172.21.15.57:0/411319415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9080cec00 0x7fa9080cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.399+0000 7fa90ee94700 1 -- 172.21.15.57:0/411319415 >> 172.21.15.57:0/411319415 conn(0x7fa9080cf400 msgr2=0x7fa9080b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:46.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.399+0000 7fa90ee94700 1 -- 172.21.15.57:0/411319415 shutdown_connections 2023-11-28T12:07:46.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.399+0000 7fa90ee94700 1 -- 172.21.15.57:0/411319415 wait complete. 2023-11-28T12:07:46.402 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 7 to deep-scrub 2023-11-28T12:07:46.414 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:07:46.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.548+0000 7f430f5a9700 1 Processor -- start 2023-11-28T12:07:46.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.549+0000 7f430f5a9700 1 -- start start 2023-11-28T12:07:46.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.549+0000 7f430f5a9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f430806e400 0x7f430806e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.549+0000 7f430f5a9700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f43080d15f0 0x7f430815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.549+0000 7f430f5a9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f430815af30 con 0x7f430815e010 2023-11-28T12:07:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.549+0000 7f430f5a9700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f430815a9d0 con 0x7f43080d15f0 2023-11-28T12:07:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.549+0000 7f430f5a9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43081584e0 con 0x7f430806e400 2023-11-28T12:07:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.550+0000 7f430cb44700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f43080d15f0 0x7f430815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.550+0000 7f430cb44700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f43080d15f0 0x7f430815b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57178/0 (socket says 172.21.15.57:57178) 2023-11-28T12:07:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.550+0000 7f430cb44700 1 -- 172.21.15.57:0/2692765298 learned_addr learned my addr 172.21.15.57:0/2692765298 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.550+0000 7f430d345700 1 --2- 172.21.15.57:0/2692765298 >> v2:172.21.15.181:3300/0 conn(0x7f430806e400 0x7f430806e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.550+0000 7f430cb44700 1 -- 172.21.15.57:0/2692765298 >> v2:172.21.15.181:3300/0 conn(0x7f430806e400 msgr2=0x7f430806e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.550+0000 7f430cb44700 1 --2- 172.21.15.57:0/2692765298 >> v2:172.21.15.181:3300/0 conn(0x7f430806e400 0x7f430806e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.550+0000 7f430cb44700 1 -- 172.21.15.57:0/2692765298 >> v1:172.21.15.181:6789/0 conn(0x7f430815e010 legacy=0x7f43080ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:07:46.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.550+0000 7f430cb44700 1 -- 172.21.15.57:0/2692765298 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430815ef40 con 0x7f43080d15f0 2023-11-28T12:07:46.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.550+0000 7f430cb44700 1 --2- 172.21.15.57:0/2692765298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43080d15f0 0x7f430815b3c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f43080cd900 tx=0x7f4304009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe5d2de6b599dbe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:46.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.550+0000 7f42fbfff700 1 -- 172.21.15.57:0/2692765298 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304004230 con 0x7f43080d15f0 2023-11-28T12:07:46.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.550+0000 7f42fbfff700 1 -- 172.21.15.57:0/2692765298 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4304004390 con 0x7f43080d15f0 2023-11-28T12:07:46.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.550+0000 7f42fbfff700 1 -- 172.21.15.57:0/2692765298 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43040036a0 con 0x7f43080d15f0 2023-11-28T12:07:46.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.551+0000 7f430f5a9700 1 -- 172.21.15.57:0/2692765298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43080d15f0 msgr2=0x7f430815b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.551+0000 7f430f5a9700 1 --2- 172.21.15.57:0/2692765298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43080d15f0 0x7f430815b3c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f43080cd900 tx=0x7f4304009bd0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.551+0000 7f430f5a9700 1 -- 172.21.15.57:0/2692765298 shutdown_connections 2023-11-28T12:07:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.551+0000 7f430f5a9700 1 --2- 172.21.15.57:0/2692765298 >> v2:172.21.15.181:3300/0 conn(0x7f430806e400 0x7f430806e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.551+0000 7f430f5a9700 1 --2- 172.21.15.57:0/2692765298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43080d15f0 0x7f430815b3c0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.551+0000 7f430f5a9700 1 -- 172.21.15.57:0/2692765298 >> 172.21.15.57:0/2692765298 conn(0x7f430806a270 msgr2=0x7f430806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.551+0000 7f430f5a9700 1 -- 172.21.15.57:0/2692765298 shutdown_connections 2023-11-28T12:07:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.551+0000 7f430f5a9700 1 -- 172.21.15.57:0/2692765298 wait complete. 2023-11-28T12:07:46.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.552+0000 7f430f5a9700 1 Processor -- start 2023-11-28T12:07:46.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.552+0000 7f430f5a9700 1 -- start start 2023-11-28T12:07:46.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.553+0000 7f430f5a9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f430806e400 0x7f43081f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.553+0000 7f430f5a9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43080d15f0 0x7f430815b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.553+0000 7f430f5a9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f430815e010 0x7f430815b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.553+0000 7f430f5a9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430806d300 con 0x7f43080d15f0 2023-11-28T12:07:46.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.553+0000 7f430f5a9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43081f27c0 con 0x7f430806e400 2023-11-28T12:07:46.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.553+0000 7f430f5a9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43081f2a00 con 0x7f430815e010 2023-11-28T12:07:46.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.553+0000 7f430cb44700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43080d15f0 0x7f430815b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.553+0000 7f430cb44700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43080d15f0 0x7f430815b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57182/0 (socket says 172.21.15.57:57182) 2023-11-28T12:07:46.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.553+0000 7f430cb44700 1 -- 172.21.15.57:0/2795287998 learned_addr learned my addr 172.21.15.57:0/2795287998 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:07:46.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f430d345700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f430806e400 0x7f43081f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f430db46700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f430815e010 0x7f430815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f430cb44700 1 -- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f430815e010 msgr2=0x7f430815b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f430cb44700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f430815e010 0x7f430815b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f430cb44700 1 -- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f430806e400 msgr2=0x7f43081f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f430cb44700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f430806e400 0x7f43081f2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f430cb44700 1 -- 172.21.15.57:0/2795287998 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4304009550 con 0x7f43080d15f0 2023-11-28T12:07:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f430d345700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f430806e400 0x7f43081f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:07:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f430cb44700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43080d15f0 0x7f430815b2c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4304009130 tx=0x7f4304004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f42f9ffb700 1 -- 172.21.15.57:0/2795287998 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304003d80 con 0x7f43080d15f0 2023-11-28T12:07:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f42f9ffb700 1 -- 172.21.15.57:0/2795287998 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430401f070 con 0x7f43080d15f0 2023-11-28T12:07:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f430f5a9700 1 -- 172.21.15.57:0/2795287998 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430815e980 con 0x7f43080d15f0 2023-11-28T12:07:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f42f9ffb700 1 -- 172.21.15.57:0/2795287998 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304011870 con 0x7f43080d15f0 2023-11-28T12:07:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.554+0000 7f430f5a9700 1 -- 172.21.15.57:0/2795287998 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430806ed10 con 0x7f43080d15f0 2023-11-28T12:07:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.555+0000 7f430f5a9700 1 -- 172.21.15.57:0/2795287998 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430815ec60 con 0x7f43080d15f0 2023-11-28T12:07:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.556+0000 7f42f9ffb700 1 -- 172.21.15.57:0/2795287998 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4304017020 con 0x7f43080d15f0 2023-11-28T12:07:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.556+0000 7f42f9ffb700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f42ec0682f0 0x7f42ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:07:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.556+0000 7f42f9ffb700 1 -- 172.21.15.57:0/2795287998 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f43040872b0 con 0x7f43080d15f0 2023-11-28T12:07:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.556+0000 7f430d345700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f42ec0682f0 0x7f42ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:07:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.557+0000 7f430d345700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f42ec0682f0 0x7f42ec06a7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f43080b3a20 tx=0x7f42fc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:07:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.561+0000 7f42f9ffb700 1 -- 172.21.15.57:0/2795287998 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4304053e10 con 0x7f43080d15f0 2023-11-28T12:07:46.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.757+0000 7f430f5a9700 1 -- 172.21.15.57:0/2795287998 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43080743f0 con 0x7f42ec0682f0 2023-11-28T12:07:46.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.761+0000 7f42f9ffb700 1 -- 172.21.15.57:0/2795287998 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73598 (secure 0 0 0) 0x7f43080743f0 con 0x7f42ec0682f0 2023-11-28T12:07:46.763 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:07:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.763+0000 7f430f5a9700 1 -- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f42ec0682f0 msgr2=0x7f42ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.763+0000 7f430f5a9700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f42ec0682f0 0x7f42ec06a7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f43080b3a20 tx=0x7f42fc008040 comp rx=0 tx=0).stop 2023-11-28T12:07:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.763+0000 7f430f5a9700 1 -- 172.21.15.57:0/2795287998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43080d15f0 msgr2=0x7f430815b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:07:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.763+0000 7f430f5a9700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43080d15f0 0x7f430815b2c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4304009130 tx=0x7f4304004750 comp rx=0 tx=0).stop 2023-11-28T12:07:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.763+0000 7f430f5a9700 1 -- 172.21.15.57:0/2795287998 shutdown_connections 2023-11-28T12:07:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.763+0000 7f430f5a9700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f42ec0682f0 0x7f42ec06a7a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.763+0000 7f430f5a9700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f430815e010 0x7f430815b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.763+0000 7f430f5a9700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f430806e400 0x7f43081f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.763+0000 7f430f5a9700 1 --2- 172.21.15.57:0/2795287998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43080d15f0 0x7f430815b2c0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:07:46.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.763+0000 7f430f5a9700 1 -- 172.21.15.57:0/2795287998 >> 172.21.15.57:0/2795287998 conn(0x7f430806a270 msgr2=0x7f43080b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:07:46.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.764+0000 7f430f5a9700 1 -- 172.21.15.57:0/2795287998 shutdown_connections 2023-11-28T12:07:46.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:07:46.764+0000 7f430f5a9700 1 -- 172.21.15.57:0/2795287998 wait complete. 2023-11-28T12:07:46.767 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:07:46.778 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2023-11-28T12:07:45.752954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":97224,"kb_used_data":7624,"kb_used_omap":0,"kb_used_meta":89536,"kb_avail":754877496,"statfs":{"total":773094113280,"available":772994555904,"internally_reserved":0,"allocated":7806976,"data_stored":4130784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91684864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.049020"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":60,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T12:07:36.706313+0000","last_change":"2023-11-28T12:01:38.395945+0000","last_active":"2023-11-28T12:07:36.706313+0000","last_peered":"2023-11-28T12:07:36.706313+0000","last_clean":"2023-11-28T12:07:36.706313+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:07:36.706313+0000","last_undegraded":"2023-11-28T12:07:36.706313+0000","last_fullsized":"2023-11-28T12:07:36.706313+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T12:07:36.708273+0000","last_change":"2023-11-28T12:01:38.242187+0000","last_active":"2023-11-28T12:07:36.708273+0000","last_peered":"2023-11-28T12:07:36.708273+0000","last_clean":"2023-11-28T12:07:36.708273+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:07:36.708273+0000","last_undegraded":"2023-11-28T12:07:36.708273+0000","last_fullsized":"2023-11-28T12:07:36.708273+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":58,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T12:07:36.709010+0000","last_change":"2023-11-28T12:01:38.242242+0000","last_active":"2023-11-28T12:07:36.709010+0000","last_peered":"2023-11-28T12:07:36.709010+0000","last_clean":"2023-11-28T12:07:36.709010+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:07:36.709010+0000","last_undegraded":"2023-11-28T12:07:36.709010+0000","last_fullsized":"2023-11-28T12:07:36.709010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:44:22.982979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":205,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:39.431698+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:07:39.431698+0000","last_peered":"2023-11-28T12:07:39.431698+0000","last_clean":"2023-11-28T12:07:39.431698+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:07:39.431698+0000","last_undegraded":"2023-11-28T12:07:39.431698+0000","last_fullsized":"2023-11-28T12:07:39.431698+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:48.770862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":55,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T12:07:37.292831+0000","last_change":"2023-11-28T12:01:38.441774+0000","last_active":"2023-11-28T12:07:37.292831+0000","last_peered":"2023-11-28T12:07:37.292831+0000","last_clean":"2023-11-28T12:07:37.292831+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:07:37.292831+0000","last_undegraded":"2023-11-28T12:07:37.292831+0000","last_fullsized":"2023-11-28T12:07:37.292831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:39.433595+0000","last_change":"2023-11-28T12:01:38.441812+0000","last_active":"2023-11-28T12:07:39.433595+0000","last_peered":"2023-11-28T12:07:39.433595+0000","last_clean":"2023-11-28T12:07:39.433595+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:07:39.433595+0000","last_undegraded":"2023-11-28T12:07:39.433595+0000","last_fullsized":"2023-11-28T12:07:39.433595+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":57,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T12:07:37.292592+0000","last_change":"2023-11-28T12:01:38.441549+0000","last_active":"2023-11-28T12:07:37.292592+0000","last_peered":"2023-11-28T12:07:37.292592+0000","last_clean":"2023-11-28T12:07:37.292592+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:07:37.292592+0000","last_undegraded":"2023-11-28T12:07:37.292592+0000","last_fullsized":"2023-11-28T12:07:37.292592+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:59.043152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":56,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T12:07:36.741320+0000","last_change":"2023-11-28T12:01:38.441615+0000","last_active":"2023-11-28T12:07:36.741320+0000","last_peered":"2023-11-28T12:07:36.741320+0000","last_clean":"2023-11-28T12:07:36.741320+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:07:36.741320+0000","last_undegraded":"2023-11-28T12:07:36.741320+0000","last_fullsized":"2023-11-28T12:07:36.741320+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:48.759662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":56,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T12:07:36.741633+0000","last_change":"2023-11-28T12:01:38.235886+0000","last_active":"2023-11-28T12:07:36.741633+0000","last_peered":"2023-11-28T12:07:36.741633+0000","last_clean":"2023-11-28T12:07:36.741633+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:07:36.741633+0000","last_undegraded":"2023-11-28T12:07:36.741633+0000","last_fullsized":"2023-11-28T12:07:36.741633+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:37.218416+0000","last_clean_scrub_stamp":"2023-11-28T12:01:37.218416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:31.237195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11060,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360780,"statfs":{"total":96636764160,"available":96625438720,"internally_reserved":0,"allocated":1421312,"data_stored":959124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.42099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71599999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.44},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.50700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77400000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.5}]},{"osd":1,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.45000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.746,"5min":0.872,"15min":0.872},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.80700000000000005},"min":{"1min":0.48799999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.745,"5min":7.7460000000000004,"15min":7.7460000000000004},"last":0.56399999999999995}]},{"osd":2,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63300000000000001,"15min":0.79200000000000004},"min":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72699999999999998,"5min":7.7789999999999999,"15min":7.7789999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.44800000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.52200000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.47699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.75900000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60999999999999999,"15min":0.66400000000000003},"min":{"1min":0.46200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67300000000000004,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.58599999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47399999999999998,"15min":0.499},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.56599999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47999999999999998,"15min":0.48799999999999999},"min":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68500000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.378}]},{"osd":5,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.497,"15min":0.51600000000000001},"min":{"1min":0.34599999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.59399999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52300000000000002,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65700000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.39600000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54900000000000004,"15min":0.58099999999999996},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.69599999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54600000000000004,"15min":0.57399999999999995},"min":{"1min":0.39300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.65000000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9584,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362256,"statfs":{"total":96636764160,"available":96626950144,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.498},"min":{"1min":0.39800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.746,"5min":1.446,"15min":1.446},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55200000000000005,"15min":0.51400000000000001},"min":{"1min":0.44600000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.80200000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46800000000000003}]},{"osd":1,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.50900000000000001},"min":{"1min":0.45300000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65800000000000003,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.52700000000000002},"min":{"1min":0.503,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":0.88,"15min":0.88},"last":0.71499999999999997}]},{"osd":2,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.51100000000000001},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72699999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":1.254,"15min":1.254},"last":0.48999999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58399999999999996,"15min":0.53300000000000003},"min":{"1min":0.40100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.67900000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.56399999999999995},"min":{"1min":0.48099999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72399999999999998,"5min":1.228,"15min":1.228},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46400000000000002,"15min":0.45000000000000001},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.58699999999999997,"5min":1.204,"15min":1.204},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.39300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66400000000000003,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68400000000000005,"5min":0.753,"15min":0.753},"last":0.433},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.42699999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68899999999999995,"5min":1.29,"15min":1.29},"last":0.627}]},{"osd":7,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52500000000000002,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69799999999999995,"5min":1.159,"15min":1.159},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65500000000000003,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15028,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":13632,"kb_avail":94356812,"statfs":{"total":96636764160,"available":96621375488,"internally_reserved":0,"allocated":1421312,"data_stored":959124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13959168},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.371,"15min":0.39900000000000002},"min":{"1min":0.28399999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51000000000000001,"5min":1.2,"15min":1.2},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39800000000000002,"15min":0.40899999999999997},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60999999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58499999999999996}]},{"osd":2,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.379,"15min":0.373},"min":{"1min":0.26300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.44400000000000001},"min":{"1min":0.35399999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.62,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.78600000000000003}]},{"osd":3,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46200000000000002,"15min":0.48199999999999998},"min":{"1min":0.38700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68200000000000005,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.44400000000000001,"15min":0.45400000000000001},"min":{"1min":0.373,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66300000000000003,"5min":1.03,"15min":1.03},"last":0.66400000000000003}]},{"osd":4,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60999999999999999,"15min":0.60499999999999998},"min":{"1min":0.48299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.50600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.90800000000000003,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.625,"15min":0.629},"min":{"1min":0.5,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83899999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.51800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95899999999999996,"5min":0.998,"15min":0.998},"last":0.83799999999999997}]},{"osd":6,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64400000000000002,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.55500000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85799999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.88100000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64400000000000002,"15min":0.65900000000000003},"min":{"1min":0.54600000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.52600000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92300000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.92000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13872,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":13056,"kb_avail":94357968,"statfs":{"total":96636764160,"available":96622559232,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13369344},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42999999999999999,"15min":0.53900000000000003},"min":{"1min":0.27900000000000003,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54800000000000004,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42499999999999999,"15min":0.44900000000000001},"min":{"1min":0.27700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56699999999999995,"5min":1.667,"15min":1.667},"last":0.309}]},{"osd":2,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45800000000000002,"15min":0.55800000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59799999999999998,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.433,"15min":0.438},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59299999999999997,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.436}]},{"osd":3,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.44400000000000001},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.63700000000000001,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.63100000000000001},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67700000000000005,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.41599999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89100000000000001,"5min":1.536,"15min":1.536},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67600000000000005,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92300000000000004,"5min":4.5999999999999996,"15min":4.5999999999999996},"last":0.67800000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91100000000000003,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.443,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84199999999999997,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.69899999999999995}]},{"osd":6,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.76900000000000002},"min":{"1min":0.54500000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65800000000000003,"15min":0.628},"min":{"1min":0.47999999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92900000000000005,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.67000000000000004}]},{"osd":7,"last update":"Tue Nov 28 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.51400000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96299999999999997,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.66800000000000004,"15min":0.65800000000000003},"min":{"1min":0.49399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90100000000000002,"5min":1.605,"15min":1.605},"last":0.72599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13808,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94358032,"statfs":{"total":96636764160,"available":96622624768,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.377,"15min":0.375},"min":{"1min":0.27600000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.61799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61099999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.33200000000000002}]},{"osd":1,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.41999999999999998},"min":{"1min":0.33200000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.53200000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.28499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56899999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.45200000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42199999999999999,"15min":0.41299999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.44600000000000001},"min":{"1min":0.31900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.59199999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.437}]},{"osd":4,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.51500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90500000000000003,"5min":1.004,"15min":1.004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.48299999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.89000000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71099999999999997}]},{"osd":5,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.52000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84199999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54100000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.873,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85299999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Nov 28 12:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.496,"5min":0.435,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":1.004,"15min":1.004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.63200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14128,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94357712,"statfs":{"total":96636764160,"available":96622297088,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.27800000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55200000000000005,"5min":0.749,"15min":0.749},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39200000000000002,"15min":0.42999999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":1.238},"last":0.42099999999999999}]},{"osd":1,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39200000000000002,"15min":0.437},"min":{"1min":0.28799999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":1.53},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.442,"15min":0.496},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":1.7270000000000001},"last":0.38100000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45000000000000001,"15min":0.48299999999999998},"min":{"1min":0.318,"5min":0.312,"15min":0.28499999999999998},"max":{"1min":0.63200000000000001,"5min":0.77400000000000002,"15min":1.4610000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.36899999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61599999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.45200000000000001}]},{"osd":4,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88700000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64000000000000001,"15min":0.65000000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.41799999999999998},"max":{"1min":0.90500000000000003,"5min":0.91900000000000004,"15min":1.4159999999999999},"last":0.437}]},{"osd":5,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.67800000000000005},"min":{"1min":0.44600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85599999999999998,"5min":0.99299999999999999,"15min":1.4990000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65800000000000003,"15min":0.65000000000000002},"min":{"1min":0.46500000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.51800000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.66300000000000003},"min":{"1min":0.46000000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.85499999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94599999999999995,"5min":0.96999999999999997,"15min":0.98599999999999999},"last":0.503}]},{"osd":7,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.499,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66200000000000003,"15min":0.67700000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.93899999999999995,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.53100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9968,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361872,"statfs":{"total":96636764160,"available":96626556928,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.505},"min":{"1min":0.435,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68000000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72099999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.71599999999999997}]},{"osd":1,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.751,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55100000000000005,"15min":0.53000000000000003},"min":{"1min":0.35499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74399999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73399999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66400000000000003,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56799999999999995,"15min":0.54200000000000004},"min":{"1min":0.45000000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67900000000000005,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65000000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56799999999999995,"15min":0.53600000000000003},"min":{"1min":0.46300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.752,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.47399999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.748}]},{"osd":5,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.46899999999999997,"15min":0.497},"min":{"1min":0.32900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48199999999999998,"15min":0.48999999999999999},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66000000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38800000000000001}]},{"osd":6,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46800000000000003,"15min":0.48999999999999999},"min":{"1min":0.33200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69699999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.434},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52800000000000002,"15min":0.56399999999999995},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.42099999999999999}]},{"osd":7,"last update":"Tue Nov 28 12:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.51000000000000001,"15min":0.53800000000000003},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68400000000000005,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.54300000000000004,"15min":0.56499999999999995},"min":{"1min":0.34100000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67400000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.48599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9776,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362064,"statfs":{"total":96636764160,"available":96626753536,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.433,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68100000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.47499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72099999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.56699999999999995}]},{"osd":1,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69899999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.64300000000000002,"15min":0.86499999999999999},"min":{"1min":0.41499999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71599999999999997,"5min":9.0630000000000006,"15min":9.0630000000000006},"last":0.51000000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.45600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.47899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51900000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.748,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.497},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.47399999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.75,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.68600000000000005}]},{"osd":4,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.32900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.434},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.499,"15min":0.52500000000000002},"min":{"1min":0.34999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65500000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.34300000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.495,"15min":0.51400000000000001},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66400000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71399999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.38400000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.42299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68500000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.58499999999999996},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73499999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:07:46.785 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-11-28T12:01:37.218416+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=1, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=7, tm_sec=41, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T12:07:46.785 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-11-28T12:01:37.218416+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=1, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=7, tm_sec=41, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T12:07:46.785 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-11-28T12:01:37.218416+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=1, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=7, tm_sec=41, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T12:07:46.785 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-28T12:01:38.223765+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=1, tm_sec=38, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=7, tm_sec=41, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T12:07:46.786 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-11-28T12:01:37.218416+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=1, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=7, tm_sec=41, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T12:07:46.786 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-28T12:01:37.218416+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=1, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=7, tm_sec=41, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T12:07:46.786 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-11-28T12:01:37.218416+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=1, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=7, tm_sec=41, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T12:07:46.786 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-11-28T12:01:37.218416+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=1, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=7, tm_sec=41, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T12:07:46.786 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-11-28T12:01:37.218416+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=1, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=7, tm_sec=41, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T12:07:46.786 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T12:07:46.992 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:07:46.995 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:08:06.787 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:08:06.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.933+0000 7ff11597a700 1 Processor -- start 2023-11-28T12:08:06.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.933+0000 7ff11597a700 1 -- start start 2023-11-28T12:08:06.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.933+0000 7ff11597a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff110158cd0 0x7ff1100c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:06.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff11597a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1100718e0 con 0x7ff1100cd1c0 2023-11-28T12:08:06.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff11597a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11006be90 con 0x7ff1100cdb80 2023-11-28T12:08:06.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff11597a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1101590a0 con 0x7ff110158cd0 2023-11-28T12:08:06.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff10e7fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff1100cd1c0 0x7ff1100b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:42140/0 (socket says 172.21.15.57:42140) 2023-11-28T12:08:06.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff10e7fc700 1 -- 172.21.15.57:0/299667708 learned_addr learned my addr 172.21.15.57:0/299667708 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-11-28T12:08:06.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff10effd700 1 --2- 172.21.15.57:0/299667708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff110158cd0 0x7ff1100c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:06.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff10dffb700 1 -- 172.21.15.57:0/299667708 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284004857 0 0) 0x7ff0f8001110 con 0x7ff1100cd1c0 2023-11-28T12:08:06.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff10dffb700 1 -- 172.21.15.57:0/299667708 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0fc0035b0 con 0x7ff1100cd1c0 2023-11-28T12:08:06.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff10dffb700 1 -- 172.21.15.57:0/299667708 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315711137 0 0) 0x7ff11006be90 con 0x7ff1100cdb80 2023-11-28T12:08:06.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff10dffb700 1 -- 172.21.15.57:0/299667708 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0f8001110 con 0x7ff1100cdb80 2023-11-28T12:08:06.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff10dffb700 1 -- 172.21.15.57:0/299667708 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1619082070 0 0) 0x7ff0fc0035b0 con 0x7ff1100cd1c0 2023-11-28T12:08:06.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff10effd700 1 -- 172.21.15.57:0/299667708 >> v1:172.21.15.181:6789/0 conn(0x7ff1100cdb80 legacy=0x7ff110163a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:06.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff10effd700 1 -- 172.21.15.57:0/299667708 >> v1:172.21.15.57:6789/0 conn(0x7ff1100cd1c0 legacy=0x7ff1100b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:06.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.934+0000 7ff10effd700 1 -- 172.21.15.57:0/299667708 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1100cad30 con 0x7ff110158cd0 2023-11-28T12:08:06.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.935+0000 7ff10effd700 1 --2- 172.21.15.57:0/299667708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff110158cd0 0x7ff1100c92c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff100009870 tx=0x7ff100009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc4c96001ee86bfc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:08:06.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.935+0000 7ff10dffb700 1 -- 172.21.15.57:0/299667708 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8001800 con 0x7ff110158cd0 2023-11-28T12:08:06.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.935+0000 7ff10dffb700 1 -- 172.21.15.57:0/299667708 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff100004030 con 0x7ff110158cd0 2023-11-28T12:08:06.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.935+0000 7ff10dffb700 1 -- 172.21.15.57:0/299667708 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000043a0 con 0x7ff110158cd0 2023-11-28T12:08:06.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.935+0000 7ff11597a700 1 -- 172.21.15.57:0/299667708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff110158cd0 msgr2=0x7ff1100c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:06.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.935+0000 7ff11597a700 1 --2- 172.21.15.57:0/299667708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff110158cd0 0x7ff1100c92c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff100009870 tx=0x7ff100009b80 comp rx=0 tx=0).stop 2023-11-28T12:08:06.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.935+0000 7ff11597a700 1 -- 172.21.15.57:0/299667708 shutdown_connections 2023-11-28T12:08:06.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.935+0000 7ff11597a700 1 --2- 172.21.15.57:0/299667708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff110158cd0 0x7ff1100c92c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:06.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.935+0000 7ff11597a700 1 -- 172.21.15.57:0/299667708 >> 172.21.15.57:0/299667708 conn(0x7ff1100cb660 msgr2=0x7ff1100cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:08:06.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.935+0000 7ff11597a700 1 -- 172.21.15.57:0/299667708 shutdown_connections 2023-11-28T12:08:06.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.935+0000 7ff11597a700 1 -- 172.21.15.57:0/299667708 wait complete. 2023-11-28T12:08:06.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.937+0000 7ff11597a700 1 Processor -- start 2023-11-28T12:08:06.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.937+0000 7ff11597a700 1 -- start start 2023-11-28T12:08:06.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff11597a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1100cd1c0 0x7ff1101494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:06.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff11597a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1100cdb80 0x7ff1100ad300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:06.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff11597a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff110158cd0 0x7ff110159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:06.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff11597a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100b1fd0 con 0x7ff1100cd1c0 2023-11-28T12:08:06.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff11597a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100add50 con 0x7ff1100cdb80 2023-11-28T12:08:06.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff11597a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100ae030 con 0x7ff110158cd0 2023-11-28T12:08:06.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff10effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1100cd1c0 0x7ff1101494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:06.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff10effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1100cd1c0 0x7ff1101494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:32902/0 (socket says 172.21.15.57:32902) 2023-11-28T12:08:06.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff10effd700 1 -- 172.21.15.57:0/2789971711 learned_addr learned my addr 172.21.15.57:0/2789971711 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:08:06.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff10f7fe700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff110158cd0 0x7ff110159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:06.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff10e7fc700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1100cdb80 0x7ff1100ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:06.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff10effd700 1 -- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff110158cd0 msgr2=0x7ff110159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:06.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff10effd700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff110158cd0 0x7ff110159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:06.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff10effd700 1 -- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1100cdb80 msgr2=0x7ff1100ad300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:06.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff10effd700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1100cdb80 0x7ff1100ad300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:06.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff10effd700 1 -- 172.21.15.57:0/2789971711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1000094a0 con 0x7ff1100cd1c0 2023-11-28T12:08:06.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.938+0000 7ff10e7fc700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1100cdb80 0x7ff1100ad300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T12:08:06.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.939+0000 7ff10effd700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1100cd1c0 0x7ff1101494a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff11006f500 tx=0x7ff100004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:08:06.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.939+0000 7ff107fff700 1 -- 172.21.15.57:0/2789971711 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100003bb0 con 0x7ff1100cd1c0 2023-11-28T12:08:06.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.939+0000 7ff107fff700 1 -- 172.21.15.57:0/2789971711 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10001f070 con 0x7ff1100cd1c0 2023-11-28T12:08:06.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.939+0000 7ff107fff700 1 -- 172.21.15.57:0/2789971711 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100011630 con 0x7ff1100cd1c0 2023-11-28T12:08:06.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.939+0000 7ff11597a700 1 -- 172.21.15.57:0/2789971711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1100cb2f0 con 0x7ff1100cd1c0 2023-11-28T12:08:06.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.939+0000 7ff11597a700 1 -- 172.21.15.57:0/2789971711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1100cb010 con 0x7ff1100cd1c0 2023-11-28T12:08:06.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.940+0000 7ff11597a700 1 -- 172.21.15.57:0/2789971711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1100cd610 con 0x7ff1100cd1c0 2023-11-28T12:08:06.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.940+0000 7ff107fff700 1 -- 172.21.15.57:0/2789971711 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff10002a020 con 0x7ff1100cd1c0 2023-11-28T12:08:06.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.941+0000 7ff107fff700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0f00682f0 0x7ff0f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:06.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.941+0000 7ff107fff700 1 -- 172.21.15.57:0/2789971711 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff100086f50 con 0x7ff1100cd1c0 2023-11-28T12:08:06.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.941+0000 7ff10e7fc700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0f00682f0 0x7ff0f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:06.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.942+0000 7ff10e7fc700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0f00682f0 0x7ff0f006a7a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8001eb0 tx=0x7ff0f8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:08:06.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:06.945+0000 7ff107fff700 1 -- 172.21.15.57:0/2789971711 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff100053a30 con 0x7ff1100cd1c0 2023-11-28T12:08:07.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.140+0000 7ff11597a700 1 -- 172.21.15.57:0/2789971711 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1100b1e50 con 0x7ff0f00682f0 2023-11-28T12:08:07.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.143+0000 7ff107fff700 1 -- 172.21.15.57:0/2789971711 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73971 (secure 0 0 0) 0x7ff0f8007050 con 0x7ff0f00682f0 2023-11-28T12:08:07.145 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:08:07.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.145+0000 7ff11597a700 1 -- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0f00682f0 msgr2=0x7ff0f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:07.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.145+0000 7ff11597a700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0f00682f0 0x7ff0f006a7a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8001eb0 tx=0x7ff0f8009040 comp rx=0 tx=0).stop 2023-11-28T12:08:07.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.145+0000 7ff11597a700 1 -- 172.21.15.57:0/2789971711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1100cd1c0 msgr2=0x7ff1101494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:07.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.145+0000 7ff11597a700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1100cd1c0 0x7ff1101494a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff11006f500 tx=0x7ff100004750 comp rx=0 tx=0).stop 2023-11-28T12:08:07.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.145+0000 7ff11597a700 1 -- 172.21.15.57:0/2789971711 shutdown_connections 2023-11-28T12:08:07.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.145+0000 7ff11597a700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff0f00682f0 0x7ff0f006a7a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.145+0000 7ff11597a700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff110158cd0 0x7ff110159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.145+0000 7ff11597a700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1100cd1c0 0x7ff1101494a0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.145+0000 7ff11597a700 1 --2- 172.21.15.57:0/2789971711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1100cdb80 0x7ff1100ad300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.145+0000 7ff11597a700 1 -- 172.21.15.57:0/2789971711 >> 172.21.15.57:0/2789971711 conn(0x7ff1100cb660 msgr2=0x7ff1100b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:08:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.146+0000 7ff11597a700 1 -- 172.21.15.57:0/2789971711 shutdown_connections 2023-11-28T12:08:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:07.146+0000 7ff11597a700 1 -- 172.21.15.57:0/2789971711 wait complete. 2023-11-28T12:08:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:08:07.160 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2023-11-28T12:08:05.757628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":97224,"kb_used_data":7624,"kb_used_omap":0,"kb_used_meta":89536,"kb_avail":754877496,"statfs":{"total":773094113280,"available":772994555904,"internally_reserved":0,"allocated":7806976,"data_stored":4130784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91684864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003070"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:08:01.528719+0000","last_change":"2023-11-28T12:08:01.528693+0000","last_active":"2023-11-28T12:08:01.528719+0000","last_peered":"2023-11-28T12:08:01.528719+0000","last_clean":"2023-11-28T12:08:01.528719+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:08:01.528719+0000","last_undegraded":"2023-11-28T12:08:01.528719+0000","last_fullsized":"2023-11-28T12:08:01.528719+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:08:01.528579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:08:01.528579+0000","last_clean_scrub_stamp":"2023-11-28T12:08:01.528579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:36:30.798062+0000","scrub_duration":0.0038280089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:44.713258+0000","last_change":"2023-11-28T12:07:44.713224+0000","last_active":"2023-11-28T12:07:44.713258+0000","last_peered":"2023-11-28T12:07:44.713258+0000","last_clean":"2023-11-28T12:07:44.713258+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:07:44.713258+0000","last_undegraded":"2023-11-28T12:07:44.713258+0000","last_fullsized":"2023-11-28T12:07:44.713258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:07:44.713138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:07:44.713138+0000","last_clean_scrub_stamp":"2023-11-28T12:07:44.713138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:00:13.082296+0000","scrub_duration":0.0039004579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":74,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:44.132510+0000","last_change":"2023-11-28T12:07:44.132453+0000","last_active":"2023-11-28T12:07:44.132510+0000","last_peered":"2023-11-28T12:07:44.132510+0000","last_clean":"2023-11-28T12:07:44.132510+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:07:44.132510+0000","last_undegraded":"2023-11-28T12:07:44.132510+0000","last_fullsized":"2023-11-28T12:07:44.132510+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:07:44.132347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:07:44.132347+0000","last_clean_scrub_stamp":"2023-11-28T12:07:44.132347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:16.533535+0000","scrub_duration":0.0038597449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":206,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:41.767655+0000","last_change":"2023-11-28T12:01:39.245229+0000","last_active":"2023-11-28T12:07:41.767655+0000","last_peered":"2023-11-28T12:07:41.767655+0000","last_clean":"2023-11-28T12:07:41.767655+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:07:41.767655+0000","last_undegraded":"2023-11-28T12:07:41.767655+0000","last_fullsized":"2023-11-28T12:07:41.767655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:01:38.223765+0000","last_clean_scrub_stamp":"2023-11-28T12:01:38.223765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":71,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:08:00.976716+0000","last_change":"2023-11-28T12:08:00.976686+0000","last_active":"2023-11-28T12:08:00.976716+0000","last_peered":"2023-11-28T12:08:00.976716+0000","last_clean":"2023-11-28T12:08:00.976716+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:08:00.976716+0000","last_undegraded":"2023-11-28T12:08:00.976716+0000","last_fullsized":"2023-11-28T12:08:00.976716+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:08:00.976573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:08:00.976573+0000","last_clean_scrub_stamp":"2023-11-28T12:08:00.976573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:53.835041+0000","scrub_duration":0.0038555519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:54.995575+0000","last_change":"2023-11-28T12:07:54.995533+0000","last_active":"2023-11-28T12:07:54.995575+0000","last_peered":"2023-11-28T12:07:54.995575+0000","last_clean":"2023-11-28T12:07:54.995575+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:07:54.995575+0000","last_undegraded":"2023-11-28T12:07:54.995575+0000","last_fullsized":"2023-11-28T12:07:54.995575+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:07:54.995432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:07:54.995432+0000","last_clean_scrub_stamp":"2023-11-28T12:07:54.995432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:33.748611+0000","scrub_duration":0.003965817,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":73,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:53.661603+0000","last_change":"2023-11-28T12:07:53.661534+0000","last_active":"2023-11-28T12:07:53.661603+0000","last_peered":"2023-11-28T12:07:53.661603+0000","last_clean":"2023-11-28T12:07:53.661603+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:07:53.661603+0000","last_undegraded":"2023-11-28T12:07:53.661603+0000","last_fullsized":"2023-11-28T12:07:53.661603+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:07:53.661363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:07:53.661363+0000","last_clean_scrub_stamp":"2023-11-28T12:07:53.661363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:32.414542+0000","scrub_duration":0.0042274499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:46.293715+0000","last_change":"2023-11-28T12:07:46.293683+0000","last_active":"2023-11-28T12:07:46.293715+0000","last_peered":"2023-11-28T12:07:46.293715+0000","last_clean":"2023-11-28T12:07:46.293715+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:07:46.293715+0000","last_undegraded":"2023-11-28T12:07:46.293715+0000","last_fullsized":"2023-11-28T12:07:46.293715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:07:46.293566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:07:46.293566+0000","last_clean_scrub_stamp":"2023-11-28T12:07:46.293566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:54.517742+0000","scrub_duration":0.003754617,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":71,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:47.271247+0000","last_change":"2023-11-28T12:07:47.271210+0000","last_active":"2023-11-28T12:07:47.271247+0000","last_peered":"2023-11-28T12:07:47.271247+0000","last_clean":"2023-11-28T12:07:47.271247+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:07:47.271247+0000","last_undegraded":"2023-11-28T12:07:47.271247+0000","last_fullsized":"2023-11-28T12:07:47.271247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:07:47.271078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:07:47.271078+0000","last_clean_scrub_stamp":"2023-11-28T12:07:47.271078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:59.748681+0000","scrub_duration":0.0039561600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11060,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360780,"statfs":{"total":96636764160,"available":96625438720,"internally_reserved":0,"allocated":1421312,"data_stored":959124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56399999999999995,"15min":0.55100000000000005},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72999999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.40500000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":0.77400000000000002,"15min":0.88300000000000001},"last":0.72499999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.40000000000000002,"15min":0.32100000000000001},"max":{"1min":0.71299999999999997,"5min":0.872,"15min":0.872},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57799999999999996,"15min":0.78500000000000003},"min":{"1min":0.438,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76700000000000002,"5min":0.79100000000000004,"15min":7.7460000000000004},"last":0.69699999999999995}]},{"osd":2,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.77100000000000002},"min":{"1min":0.44900000000000001,"5min":0.41399999999999998,"15min":0.39800000000000002},"max":{"1min":0.69999999999999996,"5min":0.73499999999999999,"15min":7.7789999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.43099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71099999999999997,"5min":0.80400000000000005,"15min":0.99199999999999999},"last":0.73599999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72799999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.65600000000000003},"min":{"1min":0.46100000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66100000000000003,"5min":0.84199999999999997,"15min":2.8199999999999998},"last":0.624}]},{"osd":4,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63200000000000001,"5min":0.76500000000000001,"15min":1.0720000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.48599999999999999},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67600000000000005,"5min":0.68500000000000005,"15min":0.70699999999999996},"last":0.435}]},{"osd":5,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48799999999999999,"15min":0.51300000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68999999999999995,"5min":0.76500000000000001,"15min":0.81200000000000006},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.51100000000000001,"15min":0.54200000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45600000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.57499999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68600000000000005,"5min":0.75,"15min":0.97499999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53500000000000003,"15min":0.57099999999999995},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69399999999999995,"5min":0.73999999999999999,"15min":0.95599999999999996},"last":0.50600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9584,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362256,"statfs":{"total":96636764160,"available":96626950144,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56799999999999995,"15min":0.505},"min":{"1min":0.39900000000000002,"5min":0.39800000000000002,"15min":0.30199999999999999},"max":{"1min":0.78100000000000003,"5min":1.446,"15min":1.446},"last":0.495},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56399999999999995,"15min":0.51800000000000002},"min":{"1min":0.41599999999999998,"5min":0.34300000000000003,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.55000000000000004}]},{"osd":1,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55900000000000005,"15min":0.51200000000000001},"min":{"1min":0.33600000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81599999999999995,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.53300000000000003},"min":{"1min":0.46800000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71499999999999997,"5min":0.88,"15min":0.88},"last":0.51900000000000002}]},{"osd":2,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59399999999999997,"15min":0.51800000000000002},"min":{"1min":0.45100000000000001,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":0.754,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59499999999999997,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.35299999999999998},"max":{"1min":0.68300000000000005,"5min":1.254,"15min":1.254},"last":0.53000000000000003}]},{"osd":3,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60599999999999998,"15min":0.54000000000000004},"min":{"1min":0.50600000000000001,"5min":0.38800000000000001,"15min":0.36599999999999999},"max":{"1min":0.76900000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.45800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79400000000000004,"5min":1.228,"15min":1.228},"last":0.50600000000000001}]},{"osd":4,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46600000000000003,"15min":0.45100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63300000000000001,"5min":1.204,"15min":1.204},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.30399999999999999},"max":{"1min":0.72599999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.40100000000000002}]},{"osd":5,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.63300000000000001,"5min":0.753,"15min":0.753},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70399999999999996,"5min":1.29,"15min":1.29},"last":0.45300000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52400000000000002,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.35099999999999998,"15min":0.32200000000000001},"max":{"1min":0.70799999999999996,"5min":1.159,"15min":1.159},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.39700000000000002,"5min":0.34100000000000003,"15min":0.33800000000000002},"max":{"1min":0.748,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.36599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15028,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":13632,"kb_avail":94356812,"statfs":{"total":96636764160,"available":96621375488,"internally_reserved":0,"allocated":1421312,"data_stored":959124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13959168},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.37,"15min":0.39900000000000002},"min":{"1min":0.26600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":1.2},"last":0.318},{"interface":"front","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.41099999999999998},"min":{"1min":0.32000000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58499999999999996,"5min":0.60999999999999999,"15min":0.84199999999999997},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.374},"min":{"1min":0.27300000000000002,"5min":0.22900000000000001,"15min":0.214},"max":{"1min":0.53100000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.43099999999999999,"15min":0.44600000000000001},"min":{"1min":0.314,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":1.4810000000000001},"last":0.45700000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46000000000000002,"15min":0.48099999999999998},"min":{"1min":0.34999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68100000000000005,"5min":0.68200000000000005,"15min":1.3049999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29899999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66400000000000003,"5min":0.67000000000000004,"15min":1.03},"last":0.40799999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.621,"15min":0.60899999999999999},"min":{"1min":0.47899999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82799999999999996,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.52900000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.88700000000000001,"5min":0.90800000000000003,"15min":1.4390000000000001},"last":0.55200000000000005}]},{"osd":5,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63300000000000001,"15min":0.63200000000000001},"min":{"1min":0.48499999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89500000000000002,"5min":0.97699999999999998,"15min":0.98899999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.504,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90200000000000002,"5min":0.95899999999999996,"15min":0.998},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.65000000000000002,"15min":0.67000000000000004},"min":{"1min":0.497,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.58199999999999996,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.94099999999999995},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65200000000000002,"15min":0.66200000000000003},"min":{"1min":0.55500000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67200000000000004,"15min":0.69199999999999995},"min":{"1min":0.51200000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.1699999999999999},"last":0.73799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13872,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":13056,"kb_avail":94357968,"statfs":{"total":96636764160,"available":96622559232,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13369344},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.38200000000000001,"15min":0.52300000000000002},"min":{"1min":0.24399999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49299999999999999,"5min":0.68999999999999995,"15min":4.4950000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41299999999999998,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.27700000000000002,"15min":0.23599999999999999},"max":{"1min":0.56200000000000006,"5min":0.63300000000000001,"15min":1.667},"last":0.44700000000000001}]},{"osd":2,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41599999999999998,"15min":0.54400000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.51700000000000002,"5min":0.68000000000000005,"15min":4.5369999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.433,"15min":0.438},"min":{"1min":0.32500000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54700000000000004,"5min":0.70399999999999996,"15min":1.7529999999999999},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54700000000000004,"5min":0.63700000000000001,"15min":1.6319999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47799999999999998,"15min":0.61499999999999999},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.56999999999999995,"5min":0.76100000000000001,"15min":4.6260000000000003},"last":0.60099999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64100000000000001,"15min":0.63200000000000001},"min":{"1min":0.50900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.96599999999999997,"15min":1.536},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64600000000000002,"15min":0.74099999999999999},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.44600000000000001},"max":{"1min":0.84399999999999997,"5min":1.0149999999999999,"15min":4.5999999999999996},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67100000000000004,"15min":0.63900000000000001},"min":{"1min":0.55100000000000005,"5min":0.46500000000000002,"15min":0.40600000000000003},"max":{"1min":0.82099999999999995,"5min":1.0549999999999999,"15min":1.7809999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65600000000000003,"15min":0.63},"min":{"1min":0.53400000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86299999999999999,"5min":1.044,"15min":1.5740000000000001},"last":0.73999999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68600000000000005,"15min":0.76200000000000001},"min":{"1min":0.57899999999999996,"5min":0.441,"15min":0.44},"max":{"1min":0.89000000000000001,"5min":1.0669999999999999,"15min":4.5709999999999997},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67000000000000004,"15min":0.63200000000000001},"min":{"1min":0.53500000000000003,"5min":0.46300000000000002,"15min":0.42899999999999999},"max":{"1min":0.81499999999999995,"5min":1.014,"15min":1.6919999999999999},"last":0.77500000000000002}]},{"osd":7,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67800000000000005,"15min":0.65800000000000003},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.45700000000000002},"max":{"1min":0.85599999999999998,"5min":1.03,"15min":1.7190000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.68300000000000005,"15min":0.66300000000000003},"min":{"1min":0.59299999999999997,"5min":0.45700000000000002,"15min":0.45400000000000001},"max":{"1min":0.91000000000000003,"5min":1.083,"15min":1.605},"last":0.65200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13808,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94358032,"statfs":{"total":96636764160,"available":96622624768,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.375,"15min":0.375},"min":{"1min":0.252,"5min":0.24399999999999999,"15min":0.23499999999999999},"max":{"1min":0.49199999999999999,"5min":0.61799999999999999,"15min":0.70899999999999996},"last":0.307},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42399999999999999,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60499999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.497}]},{"osd":1,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.27400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.69199999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.38400000000000001},"min":{"1min":0.27600000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.28599999999999998}]},{"osd":3,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41899999999999998,"15min":0.41199999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47499999999999998,"15min":0.44700000000000001},"min":{"1min":0.35299999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.68100000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.61499999999999999},"min":{"1min":0.505,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79200000000000004,"5min":1.004,"15min":1.004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.54200000000000004,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":0.84999999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.42399999999999999}]},{"osd":5,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64700000000000002,"15min":0.64100000000000001},"min":{"1min":0.48599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81699999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.49099999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.876,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.51300000000000001}]},{"osd":6,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67600000000000005,"15min":0.66300000000000003},"min":{"1min":0.57799999999999996,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.55900000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.874,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.53300000000000003}]},{"osd":7,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.51000000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.83299999999999996,"5min":1.004,"15min":1.004},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67000000000000004,"15min":0.65600000000000003},"min":{"1min":0.51600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.88800000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.45100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14128,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94357712,"statfs":{"total":96636764160,"available":96622297088,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.27800000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55200000000000005,"5min":0.749,"15min":0.749},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39200000000000002,"15min":0.42999999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":1.238},"last":0.51000000000000001}]},{"osd":1,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39200000000000002,"15min":0.437},"min":{"1min":0.28799999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":1.53},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.442,"15min":0.496},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":1.7270000000000001},"last":0.54700000000000004}]},{"osd":2,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45000000000000001,"15min":0.48299999999999998},"min":{"1min":0.318,"5min":0.312,"15min":0.28499999999999998},"max":{"1min":0.63200000000000001,"5min":0.77400000000000002,"15min":1.4610000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.36899999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61599999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88700000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64000000000000001,"15min":0.65000000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.41799999999999998},"max":{"1min":0.90500000000000003,"5min":0.91900000000000004,"15min":1.4159999999999999},"last":0.72099999999999997}]},{"osd":5,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.67800000000000005},"min":{"1min":0.44600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85599999999999998,"5min":0.99299999999999999,"15min":1.4990000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65800000000000003,"15min":0.65000000000000002},"min":{"1min":0.46500000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.75800000000000001}]},{"osd":6,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.66300000000000003},"min":{"1min":0.46000000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.85499999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94599999999999995,"5min":0.96999999999999997,"15min":0.98599999999999999},"last":0.81999999999999995}]},{"osd":7,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.499,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66200000000000003,"15min":0.67700000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.93899999999999995,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.83599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9968,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361872,"statfs":{"total":96636764160,"available":96626556928,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54600000000000004,"15min":0.50900000000000001},"min":{"1min":0.41699999999999998,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":0.70299999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.53300000000000003},"min":{"1min":0.42299999999999999,"5min":0.38400000000000001,"15min":0.36699999999999999},"max":{"1min":0.78600000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.67300000000000004}]},{"osd":1,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.624},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.34899999999999998},"max":{"1min":0.81000000000000005,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70799999999999996}]},{"osd":2,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54300000000000004,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72999999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.54600000000000004},"min":{"1min":0.47799999999999998,"5min":0.39800000000000002,"15min":0.32700000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.75900000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58199999999999996,"15min":0.54000000000000004},"min":{"1min":0.45700000000000002,"5min":0.433,"15min":0.41299999999999998},"max":{"1min":0.80900000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.747},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60599999999999998,"15min":0.58899999999999997},"min":{"1min":0.47299999999999998,"5min":0.40799999999999997,"15min":0.39900000000000002},"max":{"1min":0.77300000000000002,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.77100000000000002}]},{"osd":5,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.45900000000000002,"15min":0.49399999999999999},"min":{"1min":0.35099999999999998,"5min":0.30099999999999999,"15min":0.27800000000000002},"max":{"1min":0.68899999999999995,"5min":0.73099999999999998,"15min":0.77700000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.34699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66900000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57999999999999996}]},{"osd":6,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.48899999999999999},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.29199999999999998},"max":{"1min":0.63100000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51900000000000002,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73799999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.505,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73899999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.56399999999999995},"min":{"1min":0.34399999999999997,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79900000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9776,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362064,"statfs":{"total":96636764160,"available":96626753536,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54000000000000004,"15min":0.51600000000000001},"min":{"1min":0.371,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70199999999999996,"5min":0.71199999999999997,"15min":0.79700000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57399999999999995,"15min":0.54600000000000004},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.33800000000000002},"max":{"1min":0.71399999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.625}]},{"osd":1,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72099999999999997,"5min":0.78900000000000003,"15min":0.85899999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55700000000000005,"15min":0.83599999999999997},"min":{"1min":0.47299999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70199999999999996,"5min":0.74299999999999999,"15min":9.0630000000000006},"last":0.55200000000000005}]},{"osd":2,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68100000000000005,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68899999999999995,"5min":0.77800000000000002,"15min":0.94299999999999995},"last":0.61399999999999999}]},{"osd":3,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.42799999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.63900000000000001,"5min":0.748,"15min":0.84599999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.45800000000000002,"5min":0.44800000000000001,"15min":0.41999999999999998},"max":{"1min":0.69099999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.63900000000000001}]},{"osd":4,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45300000000000001,"15min":0.46800000000000003},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.746,"15min":0.746},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.49099999999999999,"15min":0.52300000000000002},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66300000000000003,"5min":0.69199999999999995,"15min":0.72699999999999998},"last":0.66100000000000003}]},{"osd":6,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.49399999999999999,"15min":0.51400000000000001},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.28899999999999998},"max":{"1min":0.65800000000000003,"5min":0.73899999999999999,"15min":0.89300000000000002},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.30199999999999999},"max":{"1min":0.66600000000000004,"5min":0.72099999999999997,"15min":0.82599999999999996},"last":0.68400000000000005}]},{"osd":7,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.54200000000000004},"min":{"1min":0.35499999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65500000000000003,"5min":0.72399999999999998,"15min":0.80200000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69699999999999995,"5min":0.83899999999999997,"15min":0.86599999999999999},"last":0.67200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:08:07.163 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-28T12:01:38.223765+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=1, tm_sec=38, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=12, tm_min=7, tm_sec=41, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T12:08:07.163 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T12:08:17.108 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:08:17.111 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:08:27.164 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T12:08:27.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.307+0000 7ff295737700 1 Processor -- start 2023-11-28T12:08:27.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.308+0000 7ff295737700 1 -- start start 2023-11-28T12:08:27.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.308+0000 7ff295737700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff290158cd0 0x7ff2900c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:27.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.308+0000 7ff295737700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff2900cd1c0 0x7ff2900cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:27.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.308+0000 7ff295737700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2900ce0c0 0x7ff29015c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:27.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.308+0000 7ff295737700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2901590a0 con 0x7ff2900cd1c0 2023-11-28T12:08:27.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.308+0000 7ff295737700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2900718e0 con 0x7ff2900ce0c0 2023-11-28T12:08:27.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.308+0000 7ff295737700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2900ce500 con 0x7ff290158cd0 2023-11-28T12:08:27.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28e7fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff2900cd1c0 0x7ff2900cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:27.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28e7fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff2900cd1c0 0x7ff2900cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55308/0 (socket says 172.21.15.57:55308) 2023-11-28T12:08:27.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28e7fc700 1 -- 172.21.15.57:0/3260458485 learned_addr learned my addr 172.21.15.57:0/3260458485 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:08:27.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28f7fe700 1 --2- 172.21.15.57:0/3260458485 >> v2:172.21.15.181:3300/0 conn(0x7ff2900ce0c0 0x7ff29015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:27.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28effd700 1 --2- 172.21.15.57:0/3260458485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff290158cd0 0x7ff2900c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:27.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28e7fc700 1 -- 172.21.15.57:0/3260458485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff290158cd0 msgr2=0x7ff2900c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:08:27.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28e7fc700 1 --2- 172.21.15.57:0/3260458485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff290158cd0 0x7ff2900c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28e7fc700 1 -- 172.21.15.57:0/3260458485 >> v2:172.21.15.181:3300/0 conn(0x7ff2900ce0c0 msgr2=0x7ff29015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:08:27.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28e7fc700 1 --2- 172.21.15.57:0/3260458485 >> v2:172.21.15.181:3300/0 conn(0x7ff2900ce0c0 0x7ff29015c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28e7fc700 1 -- 172.21.15.57:0/3260458485 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29006be90 con 0x7ff2900cd1c0 2023-11-28T12:08:27.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28e7fc700 1 --2- 172.21.15.57:0/3260458485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2900cd1c0 0x7ff2900cdb80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff29006f130 tx=0x7ff284009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec449ff4e9b4604f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:08:27.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28dffb700 1 -- 172.21.15.57:0/3260458485 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284011830 con 0x7ff2900cd1c0 2023-11-28T12:08:27.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28dffb700 1 -- 172.21.15.57:0/3260458485 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff284011990 con 0x7ff2900cd1c0 2023-11-28T12:08:27.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.309+0000 7ff28dffb700 1 -- 172.21.15.57:0/3260458485 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284011d20 con 0x7ff2900cd1c0 2023-11-28T12:08:27.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.310+0000 7ff295737700 1 -- 172.21.15.57:0/3260458485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2900cd1c0 msgr2=0x7ff2900cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:27.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.310+0000 7ff295737700 1 --2- 172.21.15.57:0/3260458485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2900cd1c0 0x7ff2900cdb80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff29006f130 tx=0x7ff284009bd0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.310+0000 7ff295737700 1 -- 172.21.15.57:0/3260458485 shutdown_connections 2023-11-28T12:08:27.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.310+0000 7ff295737700 1 --2- 172.21.15.57:0/3260458485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff290158cd0 0x7ff2900c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.310+0000 7ff295737700 1 --2- 172.21.15.57:0/3260458485 >> v2:172.21.15.181:3300/0 conn(0x7ff2900ce0c0 0x7ff29015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.310+0000 7ff295737700 1 --2- 172.21.15.57:0/3260458485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2900cd1c0 0x7ff2900cdb80 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.310+0000 7ff295737700 1 -- 172.21.15.57:0/3260458485 >> 172.21.15.57:0/3260458485 conn(0x7ff2900cb660 msgr2=0x7ff2900cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:08:27.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.310+0000 7ff295737700 1 -- 172.21.15.57:0/3260458485 shutdown_connections 2023-11-28T12:08:27.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.310+0000 7ff295737700 1 -- 172.21.15.57:0/3260458485 wait complete. 2023-11-28T12:08:27.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.312+0000 7ff295737700 1 Processor -- start 2023-11-28T12:08:27.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.312+0000 7ff295737700 1 -- start start 2023-11-28T12:08:27.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff295737700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2900cd1c0 0x7ff290150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:27.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff295737700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2900ce0c0 0x7ff290153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:27.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff295737700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff290158cd0 0x7ff29013a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:27.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff295737700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290153fa0 con 0x7ff290158cd0 2023-11-28T12:08:27.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff295737700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290154280 con 0x7ff2900ce0c0 2023-11-28T12:08:27.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff295737700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff290154560 con 0x7ff2900cd1c0 2023-11-28T12:08:27.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff28f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff290158cd0 0x7ff29013a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:27.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff28f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff290158cd0 0x7ff29013a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55310/0 (socket says 172.21.15.57:55310) 2023-11-28T12:08:27.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff28f7fe700 1 -- 172.21.15.57:0/3710583489 learned_addr learned my addr 172.21.15.57:0/3710583489 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:08:27.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff28effd700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2900cd1c0 0x7ff290150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:27.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff28e7fc700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2900ce0c0 0x7ff290153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:27.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff28f7fe700 1 -- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2900cd1c0 msgr2=0x7ff290150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:27.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff28f7fe700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2900cd1c0 0x7ff290150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff28f7fe700 1 -- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2900ce0c0 msgr2=0x7ff290153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:27.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff28f7fe700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2900ce0c0 0x7ff290153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.313+0000 7ff28f7fe700 1 -- 172.21.15.57:0/3710583489 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff284009550 con 0x7ff290158cd0 2023-11-28T12:08:27.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.314+0000 7ff28f7fe700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff290158cd0 0x7ff29013a750 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff28800efc0 tx=0x7ff28800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:08:27.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.314+0000 7ff2777fe700 1 -- 172.21.15.57:0/3710583489 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28800cad0 con 0x7ff290158cd0 2023-11-28T12:08:27.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.314+0000 7ff2777fe700 1 -- 172.21.15.57:0/3710583489 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28800f040 con 0x7ff290158cd0 2023-11-28T12:08:27.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.314+0000 7ff295737700 1 -- 172.21.15.57:0/3710583489 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2900cb010 con 0x7ff290158cd0 2023-11-28T12:08:27.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.314+0000 7ff2777fe700 1 -- 172.21.15.57:0/3710583489 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28800ce70 con 0x7ff290158cd0 2023-11-28T12:08:27.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.314+0000 7ff295737700 1 -- 172.21.15.57:0/3710583489 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2900cad30 con 0x7ff290158cd0 2023-11-28T12:08:27.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.315+0000 7ff295737700 1 -- 172.21.15.57:0/3710583489 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2900cb2f0 con 0x7ff290158cd0 2023-11-28T12:08:27.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.315+0000 7ff2777fe700 1 -- 172.21.15.57:0/3710583489 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff288014480 con 0x7ff290158cd0 2023-11-28T12:08:27.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.316+0000 7ff2777fe700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff27c0682f0 0x7ff27c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:27.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.316+0000 7ff2777fe700 1 -- 172.21.15.57:0/3710583489 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff288086220 con 0x7ff290158cd0 2023-11-28T12:08:27.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.316+0000 7ff28effd700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff27c0682f0 0x7ff27c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:27.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.316+0000 7ff28effd700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff27c0682f0 0x7ff27c06a7a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff28000ac60 tx=0x7ff28000a320 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:08:27.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.320+0000 7ff2777fe700 1 -- 172.21.15.57:0/3710583489 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff288052d20 con 0x7ff290158cd0 2023-11-28T12:08:27.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.520+0000 7ff295737700 1 -- 172.21.15.57:0/3710583489 --> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2901543c0 con 0x7ff27c0682f0 2023-11-28T12:08:27.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.523+0000 7ff2777fe700 1 -- 172.21.15.57:0/3710583489 <== mgr.4102 v2:172.21.15.181:6832/120259 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73948 (secure 0 0 0) 0x7ff2901543c0 con 0x7ff27c0682f0 2023-11-28T12:08:27.526 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:08:27.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.526+0000 7ff295737700 1 -- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff27c0682f0 msgr2=0x7ff27c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:27.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.526+0000 7ff295737700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff27c0682f0 0x7ff27c06a7a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff28000ac60 tx=0x7ff28000a320 comp rx=0 tx=0).stop 2023-11-28T12:08:27.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.526+0000 7ff295737700 1 -- 172.21.15.57:0/3710583489 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff290158cd0 msgr2=0x7ff29013a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:27.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.526+0000 7ff295737700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff290158cd0 0x7ff29013a750 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff28800efc0 tx=0x7ff28800c0c0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.526+0000 7ff295737700 1 -- 172.21.15.57:0/3710583489 shutdown_connections 2023-11-28T12:08:27.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.526+0000 7ff295737700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7ff27c0682f0 0x7ff27c06a7a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.526+0000 7ff295737700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff290158cd0 0x7ff29013a750 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.526+0000 7ff295737700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2900cd1c0 0x7ff290150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.526+0000 7ff295737700 1 --2- 172.21.15.57:0/3710583489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2900ce0c0 0x7ff290153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.526+0000 7ff295737700 1 -- 172.21.15.57:0/3710583489 >> 172.21.15.57:0/3710583489 conn(0x7ff2900cb660 msgr2=0x7ff2900b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:08:27.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.526+0000 7ff295737700 1 -- 172.21.15.57:0/3710583489 shutdown_connections 2023-11-28T12:08:27.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.526+0000 7ff295737700 1 -- 172.21.15.57:0/3710583489 wait complete. 2023-11-28T12:08:27.529 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-11-28T12:08:27.540 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2023-11-28T12:08:25.762208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":97224,"kb_used_data":7624,"kb_used_omap":0,"kb_used_meta":89536,"kb_avail":754877496,"statfs":{"total":773094113280,"available":772994555904,"internally_reserved":0,"allocated":7806976,"data_stored":4130784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91684864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002837"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:08:01.528719+0000","last_change":"2023-11-28T12:08:01.528693+0000","last_active":"2023-11-28T12:08:01.528719+0000","last_peered":"2023-11-28T12:08:01.528719+0000","last_clean":"2023-11-28T12:08:01.528719+0000","last_became_active":"2023-11-28T12:01:38.395625+0000","last_became_peered":"2023-11-28T12:01:38.395625+0000","last_unstale":"2023-11-28T12:08:01.528719+0000","last_undegraded":"2023-11-28T12:08:01.528719+0000","last_fullsized":"2023-11-28T12:08:01.528719+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:08:01.528579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:08:01.528579+0000","last_clean_scrub_stamp":"2023-11-28T12:08:01.528579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:36:30.798062+0000","scrub_duration":0.0038280089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:44.713258+0000","last_change":"2023-11-28T12:07:44.713224+0000","last_active":"2023-11-28T12:07:44.713258+0000","last_peered":"2023-11-28T12:07:44.713258+0000","last_clean":"2023-11-28T12:07:44.713258+0000","last_became_active":"2023-11-28T12:01:38.241785+0000","last_became_peered":"2023-11-28T12:01:38.241785+0000","last_unstale":"2023-11-28T12:07:44.713258+0000","last_undegraded":"2023-11-28T12:07:44.713258+0000","last_fullsized":"2023-11-28T12:07:44.713258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:07:44.713138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:07:44.713138+0000","last_clean_scrub_stamp":"2023-11-28T12:07:44.713138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:00:13.082296+0000","scrub_duration":0.0039004579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":74,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:44.132510+0000","last_change":"2023-11-28T12:07:44.132453+0000","last_active":"2023-11-28T12:07:44.132510+0000","last_peered":"2023-11-28T12:07:44.132510+0000","last_clean":"2023-11-28T12:07:44.132510+0000","last_became_active":"2023-11-28T12:01:38.241905+0000","last_became_peered":"2023-11-28T12:01:38.241905+0000","last_unstale":"2023-11-28T12:07:44.132510+0000","last_undegraded":"2023-11-28T12:07:44.132510+0000","last_fullsized":"2023-11-28T12:07:44.132510+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:07:44.132347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:07:44.132347+0000","last_clean_scrub_stamp":"2023-11-28T12:07:44.132347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:16.533535+0000","scrub_duration":0.0038597449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"68'119","reported_seq":222,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:08:09.046071+0000","last_change":"2023-11-28T12:08:09.046035+0000","last_active":"2023-11-28T12:08:09.046071+0000","last_peered":"2023-11-28T12:08:09.046071+0000","last_clean":"2023-11-28T12:08:09.046071+0000","last_became_active":"2023-11-28T12:01:39.244274+0000","last_became_peered":"2023-11-28T12:01:39.244274+0000","last_unstale":"2023-11-28T12:08:09.046071+0000","last_undegraded":"2023-11-28T12:08:09.046071+0000","last_fullsized":"2023-11-28T12:08:09.046071+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'119","last_scrub_stamp":"2023-11-28T12:08:09.045880+0000","last_deep_scrub":"68'119","last_deep_scrub_stamp":"2023-11-28T12:08:09.045880+0000","last_clean_scrub_stamp":"2023-11-28T12:08:09.045880+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:41.208479+0000","scrub_duration":0.0079897089999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":71,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:08:00.976716+0000","last_change":"2023-11-28T12:08:00.976686+0000","last_active":"2023-11-28T12:08:00.976716+0000","last_peered":"2023-11-28T12:08:00.976716+0000","last_clean":"2023-11-28T12:08:00.976716+0000","last_became_active":"2023-11-28T12:01:38.441356+0000","last_became_peered":"2023-11-28T12:01:38.441356+0000","last_unstale":"2023-11-28T12:08:00.976716+0000","last_undegraded":"2023-11-28T12:08:00.976716+0000","last_fullsized":"2023-11-28T12:08:00.976716+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:08:00.976573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:08:00.976573+0000","last_clean_scrub_stamp":"2023-11-28T12:08:00.976573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:53.835041+0000","scrub_duration":0.0038555519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:54.995575+0000","last_change":"2023-11-28T12:07:54.995533+0000","last_active":"2023-11-28T12:07:54.995575+0000","last_peered":"2023-11-28T12:07:54.995575+0000","last_clean":"2023-11-28T12:07:54.995575+0000","last_became_active":"2023-11-28T12:01:38.441484+0000","last_became_peered":"2023-11-28T12:01:38.441484+0000","last_unstale":"2023-11-28T12:07:54.995575+0000","last_undegraded":"2023-11-28T12:07:54.995575+0000","last_fullsized":"2023-11-28T12:07:54.995575+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:07:54.995432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:07:54.995432+0000","last_clean_scrub_stamp":"2023-11-28T12:07:54.995432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:33.748611+0000","scrub_duration":0.003965817,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":73,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:53.661603+0000","last_change":"2023-11-28T12:07:53.661534+0000","last_active":"2023-11-28T12:07:53.661603+0000","last_peered":"2023-11-28T12:07:53.661603+0000","last_clean":"2023-11-28T12:07:53.661603+0000","last_became_active":"2023-11-28T12:01:38.441233+0000","last_became_peered":"2023-11-28T12:01:38.441233+0000","last_unstale":"2023-11-28T12:07:53.661603+0000","last_undegraded":"2023-11-28T12:07:53.661603+0000","last_fullsized":"2023-11-28T12:07:53.661603+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:07:53.661363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:07:53.661363+0000","last_clean_scrub_stamp":"2023-11-28T12:07:53.661363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:32.414542+0000","scrub_duration":0.0042274499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:46.293729+0000","last_change":"2023-11-28T12:07:46.293683+0000","last_active":"2023-11-28T12:07:46.293729+0000","last_peered":"2023-11-28T12:07:46.293729+0000","last_clean":"2023-11-28T12:07:46.293729+0000","last_became_active":"2023-11-28T12:01:38.441310+0000","last_became_peered":"2023-11-28T12:01:38.441310+0000","last_unstale":"2023-11-28T12:07:46.293729+0000","last_undegraded":"2023-11-28T12:07:46.293729+0000","last_fullsized":"2023-11-28T12:07:46.293729+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:07:46.293566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:07:46.293566+0000","last_clean_scrub_stamp":"2023-11-28T12:07:46.293566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:54.517742+0000","scrub_duration":0.003754617,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T12:07:47.271262+0000","last_change":"2023-11-28T12:07:47.271210+0000","last_active":"2023-11-28T12:07:47.271262+0000","last_peered":"2023-11-28T12:07:47.271262+0000","last_clean":"2023-11-28T12:07:47.271262+0000","last_became_active":"2023-11-28T12:01:38.235575+0000","last_became_peered":"2023-11-28T12:01:38.235575+0000","last_unstale":"2023-11-28T12:07:47.271262+0000","last_undegraded":"2023-11-28T12:07:47.271262+0000","last_fullsized":"2023-11-28T12:07:47.271262+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T12:07:47.271078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T12:07:47.271078+0000","last_clean_scrub_stamp":"2023-11-28T12:07:47.271078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:59.748681+0000","scrub_duration":0.0039561600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11060,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360780,"statfs":{"total":96636764160,"available":96625438720,"internally_reserved":0,"allocated":1421312,"data_stored":959124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56399999999999995,"15min":0.55100000000000005},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72999999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.40500000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":0.77400000000000002,"15min":0.88300000000000001},"last":0.56100000000000005}]},{"osd":1,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.40000000000000002,"15min":0.32100000000000001},"max":{"1min":0.71299999999999997,"5min":0.872,"15min":0.872},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57799999999999996,"15min":0.78500000000000003},"min":{"1min":0.438,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76700000000000002,"5min":0.79100000000000004,"15min":7.7460000000000004},"last":0.58299999999999996}]},{"osd":2,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.77100000000000002},"min":{"1min":0.44900000000000001,"5min":0.41399999999999998,"15min":0.39800000000000002},"max":{"1min":0.69999999999999996,"5min":0.73499999999999999,"15min":7.7789999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.43099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71099999999999997,"5min":0.80400000000000005,"15min":0.99199999999999999},"last":0.46500000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72799999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.65600000000000003},"min":{"1min":0.46100000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.66100000000000003,"5min":0.84199999999999997,"15min":2.8199999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63200000000000001,"5min":0.76500000000000001,"15min":1.0720000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.48599999999999999},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67600000000000005,"5min":0.68500000000000005,"15min":0.70699999999999996},"last":0.38}]},{"osd":5,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48799999999999999,"15min":0.51300000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68999999999999995,"5min":0.76500000000000001,"15min":0.81200000000000006},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.51100000000000001,"15min":0.54200000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42599999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.57499999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68600000000000005,"5min":0.75,"15min":0.97499999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53500000000000003,"15min":0.57099999999999995},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69399999999999995,"5min":0.73999999999999999,"15min":0.95599999999999996},"last":0.497}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9584,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362256,"statfs":{"total":96636764160,"available":96626950144,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56799999999999995,"15min":0.505},"min":{"1min":0.39900000000000002,"5min":0.39800000000000002,"15min":0.30199999999999999},"max":{"1min":0.78100000000000003,"5min":1.446,"15min":1.446},"last":0.62},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56399999999999995,"15min":0.51800000000000002},"min":{"1min":0.41599999999999998,"5min":0.34300000000000003,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68300000000000005}]},{"osd":1,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55900000000000005,"15min":0.51200000000000001},"min":{"1min":0.33600000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81599999999999995,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.53300000000000003},"min":{"1min":0.46800000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71499999999999997,"5min":0.88,"15min":0.88},"last":0.71899999999999997}]},{"osd":2,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59399999999999997,"15min":0.51800000000000002},"min":{"1min":0.45100000000000001,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":0.754,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59499999999999997,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.35299999999999998},"max":{"1min":0.68300000000000005,"5min":1.254,"15min":1.254},"last":0.70099999999999996}]},{"osd":3,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60599999999999998,"15min":0.54000000000000004},"min":{"1min":0.50600000000000001,"5min":0.38800000000000001,"15min":0.36599999999999999},"max":{"1min":0.76900000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.45800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79400000000000004,"5min":1.228,"15min":1.228},"last":0.71099999999999997}]},{"osd":4,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46600000000000003,"15min":0.45100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63300000000000001,"5min":1.204,"15min":1.204},"last":0.497},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.30399999999999999},"max":{"1min":0.72599999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.46700000000000003}]},{"osd":5,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.63300000000000001,"5min":0.753,"15min":0.753},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70399999999999996,"5min":1.29,"15min":1.29},"last":0.58099999999999996}]},{"osd":7,"last update":"Tue Nov 28 12:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52400000000000002,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.35099999999999998,"15min":0.32200000000000001},"max":{"1min":0.70799999999999996,"5min":1.159,"15min":1.159},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.39700000000000002,"5min":0.34100000000000003,"15min":0.33800000000000002},"max":{"1min":0.748,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.60199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15028,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":13632,"kb_avail":94356812,"statfs":{"total":96636764160,"available":96621375488,"internally_reserved":0,"allocated":1421312,"data_stored":959124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13959168},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.37,"15min":0.39900000000000002},"min":{"1min":0.26600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":1.2},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.41099999999999998},"min":{"1min":0.32000000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58499999999999996,"5min":0.60999999999999999,"15min":0.84199999999999997},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.374},"min":{"1min":0.27300000000000002,"5min":0.22900000000000001,"15min":0.214},"max":{"1min":0.53100000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.43099999999999999,"15min":0.44600000000000001},"min":{"1min":0.314,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":1.4810000000000001},"last":0.46500000000000002}]},{"osd":3,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46000000000000002,"15min":0.48099999999999998},"min":{"1min":0.34999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68100000000000005,"5min":0.68200000000000005,"15min":1.3049999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29899999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66400000000000003,"5min":0.67000000000000004,"15min":1.03},"last":0.48499999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.621,"15min":0.60899999999999999},"min":{"1min":0.47899999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82799999999999996,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.52900000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.88700000000000001,"5min":0.90800000000000003,"15min":1.4390000000000001},"last":0.69399999999999995}]},{"osd":5,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63300000000000001,"15min":0.63200000000000001},"min":{"1min":0.48499999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89500000000000002,"5min":0.97699999999999998,"15min":0.98899999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.504,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90200000000000002,"5min":0.95899999999999996,"15min":0.998},"last":0.85599999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.65000000000000002,"15min":0.67000000000000004},"min":{"1min":0.497,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.58199999999999996,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.94099999999999995},"last":0.83099999999999996}]},{"osd":7,"last update":"Tue Nov 28 12:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65200000000000002,"15min":0.66200000000000003},"min":{"1min":0.55500000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67200000000000004,"15min":0.69199999999999995},"min":{"1min":0.51200000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.1699999999999999},"last":0.878}]}]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13872,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":13056,"kb_avail":94357968,"statfs":{"total":96636764160,"available":96622559232,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13369344},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.38200000000000001,"15min":0.52300000000000002},"min":{"1min":0.24399999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49299999999999999,"5min":0.68999999999999995,"15min":4.4950000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41299999999999998,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.27700000000000002,"15min":0.23599999999999999},"max":{"1min":0.56200000000000006,"5min":0.63300000000000001,"15min":1.667},"last":0.378}]},{"osd":2,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41599999999999998,"15min":0.54400000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.51700000000000002,"5min":0.68000000000000005,"15min":4.5369999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.433,"15min":0.438},"min":{"1min":0.32500000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54700000000000004,"5min":0.70399999999999996,"15min":1.7529999999999999},"last":0.436}]},{"osd":3,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.32700000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54700000000000004,"5min":0.63700000000000001,"15min":1.6319999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47799999999999998,"15min":0.61499999999999999},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.56999999999999995,"5min":0.76100000000000001,"15min":4.6260000000000003},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64100000000000001,"15min":0.63200000000000001},"min":{"1min":0.50900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.96599999999999997,"15min":1.536},"last":0.622},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64600000000000002,"15min":0.74099999999999999},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.44600000000000001},"max":{"1min":0.84399999999999997,"5min":1.0149999999999999,"15min":4.5999999999999996},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67100000000000004,"15min":0.63900000000000001},"min":{"1min":0.55100000000000005,"5min":0.46500000000000002,"15min":0.40600000000000003},"max":{"1min":0.82099999999999995,"5min":1.0549999999999999,"15min":1.7809999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65600000000000003,"15min":0.63},"min":{"1min":0.53400000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86299999999999999,"5min":1.044,"15min":1.5740000000000001},"last":0.66500000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68600000000000005,"15min":0.76200000000000001},"min":{"1min":0.57899999999999996,"5min":0.441,"15min":0.44},"max":{"1min":0.89000000000000001,"5min":1.0669999999999999,"15min":4.5709999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67000000000000004,"15min":0.63200000000000001},"min":{"1min":0.53500000000000003,"5min":0.46300000000000002,"15min":0.42899999999999999},"max":{"1min":0.81499999999999995,"5min":1.014,"15min":1.6919999999999999},"last":0.75800000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67800000000000005,"15min":0.65800000000000003},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.45700000000000002},"max":{"1min":0.85599999999999998,"5min":1.03,"15min":1.7190000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.68300000000000005,"15min":0.66300000000000003},"min":{"1min":0.59299999999999997,"5min":0.45700000000000002,"15min":0.45400000000000001},"max":{"1min":0.91000000000000003,"5min":1.083,"15min":1.605},"last":0.72399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13808,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94358032,"statfs":{"total":96636764160,"available":96622624768,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.375,"15min":0.375},"min":{"1min":0.252,"5min":0.24399999999999999,"15min":0.23499999999999999},"max":{"1min":0.49199999999999999,"5min":0.61799999999999999,"15min":0.70899999999999996},"last":0.308},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42399999999999999,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60499999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38600000000000001}]},{"osd":1,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.27400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.69199999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.38400000000000001},"min":{"1min":0.27600000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.32400000000000001}]},{"osd":3,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41899999999999998,"15min":0.41199999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47499999999999998,"15min":0.44700000000000001},"min":{"1min":0.35299999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.68100000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.437}]},{"osd":4,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.61499999999999999},"min":{"1min":0.505,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79200000000000004,"5min":1.004,"15min":1.004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.54200000000000004,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":0.84999999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.75700000000000001}]},{"osd":5,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64700000000000002,"15min":0.64100000000000001},"min":{"1min":0.48599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81699999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.49099999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.876,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72399999999999998}]},{"osd":6,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67600000000000005,"15min":0.66300000000000003},"min":{"1min":0.57799999999999996,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.55900000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.874,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77600000000000002}]},{"osd":7,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.51000000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.83299999999999996,"5min":1.004,"15min":1.004},"last":0.629},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67000000000000004,"15min":0.65600000000000003},"min":{"1min":0.51600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.88800000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.69799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14128,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94357712,"statfs":{"total":96636764160,"available":96622297088,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.27800000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55200000000000005,"5min":0.749,"15min":0.749},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39200000000000002,"15min":0.42999999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":1.238},"last":0.35799999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39200000000000002,"15min":0.437},"min":{"1min":0.28799999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":1.53},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.442,"15min":0.496},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":1.7270000000000001},"last":0.40799999999999997}]},{"osd":2,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45000000000000001,"15min":0.48299999999999998},"min":{"1min":0.318,"5min":0.312,"15min":0.28499999999999998},"max":{"1min":0.63200000000000001,"5min":0.77400000000000002,"15min":1.4610000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.36899999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61599999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88700000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64000000000000001,"15min":0.65000000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.41799999999999998},"max":{"1min":0.90500000000000003,"5min":0.91900000000000004,"15min":1.4159999999999999},"last":0.49399999999999999}]},{"osd":5,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.67800000000000005},"min":{"1min":0.44600000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85599999999999998,"5min":0.99299999999999999,"15min":1.4990000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65800000000000003,"15min":0.65000000000000002},"min":{"1min":0.46500000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.67100000000000004}]},{"osd":6,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.66300000000000003},"min":{"1min":0.46000000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.85499999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94599999999999995,"5min":0.96999999999999997,"15min":0.98599999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Tue Nov 28 12:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.499,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66200000000000003,"15min":0.67700000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.93899999999999995,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.60099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9968,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361872,"statfs":{"total":96636764160,"available":96626556928,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54600000000000004,"15min":0.50900000000000001},"min":{"1min":0.41699999999999998,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":0.70299999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.53300000000000003},"min":{"1min":0.42299999999999999,"5min":0.38400000000000001,"15min":0.36699999999999999},"max":{"1min":0.78600000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60199999999999998}]},{"osd":1,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.34899999999999998},"max":{"1min":0.81000000000000005,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.629}]},{"osd":2,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54300000000000004,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72999999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.54600000000000004},"min":{"1min":0.47799999999999998,"5min":0.39800000000000002,"15min":0.32700000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.66000000000000003}]},{"osd":3,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58199999999999996,"15min":0.54000000000000004},"min":{"1min":0.45700000000000002,"5min":0.433,"15min":0.41299999999999998},"max":{"1min":0.80900000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60599999999999998,"15min":0.58899999999999997},"min":{"1min":0.47299999999999998,"5min":0.40799999999999997,"15min":0.39900000000000002},"max":{"1min":0.77300000000000002,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68999999999999995}]},{"osd":5,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.45900000000000002,"15min":0.49399999999999999},"min":{"1min":0.35099999999999998,"5min":0.30099999999999999,"15min":0.27800000000000002},"max":{"1min":0.68899999999999995,"5min":0.73099999999999998,"15min":0.77700000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.34699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66900000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.42699999999999999}]},{"osd":6,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.48899999999999999},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.29199999999999998},"max":{"1min":0.63100000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.498},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51900000000000002,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73799999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51600000000000001}]},{"osd":7,"last update":"Tue Nov 28 12:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.505,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73899999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.56399999999999995},"min":{"1min":0.34399999999999997,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79900000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64400000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9776,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362064,"statfs":{"total":96636764160,"available":96626753536,"internally_reserved":0,"allocated":827392,"data_stored":368756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54000000000000004,"15min":0.51600000000000001},"min":{"1min":0.371,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70199999999999996,"5min":0.71199999999999997,"15min":0.79700000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57399999999999995,"15min":0.54600000000000004},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.33800000000000002},"max":{"1min":0.71399999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57399999999999995}]},{"osd":1,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72099999999999997,"5min":0.78900000000000003,"15min":0.85899999999999999},"last":0.746},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55700000000000005,"15min":0.83599999999999997},"min":{"1min":0.47299999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70199999999999996,"5min":0.74299999999999999,"15min":9.0630000000000006},"last":0.60699999999999998}]},{"osd":2,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68100000000000005,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68899999999999995,"5min":0.77800000000000002,"15min":0.94299999999999995},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.42799999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.63900000000000001,"5min":0.748,"15min":0.84599999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.45800000000000002,"5min":0.44800000000000001,"15min":0.41999999999999998},"max":{"1min":0.69099999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.61699999999999999}]},{"osd":4,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45300000000000001,"15min":0.46800000000000003},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.746,"15min":0.746},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.49099999999999999,"15min":0.52300000000000002},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66300000000000003,"5min":0.69199999999999995,"15min":0.72699999999999998},"last":0.40100000000000002}]},{"osd":6,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.49399999999999999,"15min":0.51400000000000001},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.28899999999999998},"max":{"1min":0.65800000000000003,"5min":0.73899999999999999,"15min":0.89300000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.30199999999999999},"max":{"1min":0.66600000000000004,"5min":0.72099999999999997,"15min":0.82599999999999996},"last":0.42299999999999999}]},{"osd":7,"last update":"Tue Nov 28 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.54200000000000004},"min":{"1min":0.35499999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65500000000000003,"5min":0.72399999999999998,"15min":0.80200000000000005},"last":0.498},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69699999999999995,"5min":0.83899999999999997,"15min":0.86599999999999999},"last":0.50800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T12:08:27.543 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-28T12:08:27.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.677+0000 7f4faa90c700 1 Processor -- start 2023-11-28T12:08:27.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.677+0000 7f4faa90c700 1 -- start start 2023-11-28T12:08:27.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.678+0000 7f4faa90c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4fa415a5f0 0x7f4fa40d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:27.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.678+0000 7f4faa90c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4fa40b2ee0 0x7f4fa40ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:27.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.678+0000 7f4faa90c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa415a9c0 con 0x7f4fa40cc200 2023-11-28T12:08:27.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.678+0000 7f4faa90c700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa40c7b80 con 0x7f4fa415a5f0 2023-11-28T12:08:27.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.678+0000 7f4faa90c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa40cc990 con 0x7f4fa40b2ee0 2023-11-28T12:08:27.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.678+0000 7f4fa3fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4fa415a5f0 0x7f4fa40d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:27.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.678+0000 7f4fa3fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4fa415a5f0 0x7f4fa40d0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55314/0 (socket says 172.21.15.57:55314) 2023-11-28T12:08:27.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.678+0000 7f4fa3fff700 1 -- 172.21.15.57:0/1720399645 learned_addr learned my addr 172.21.15.57:0/1720399645 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:08:27.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.678+0000 7f4fa8ea9700 1 --2- 172.21.15.57:0/1720399645 >> v2:172.21.15.181:3300/0 conn(0x7f4fa40b2ee0 0x7f4fa40ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:27.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.679+0000 7f4fa3fff700 1 -- 172.21.15.57:0/1720399645 >> v2:172.21.15.181:3300/0 conn(0x7f4fa40b2ee0 msgr2=0x7f4fa40ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:08:27.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.679+0000 7f4fa3fff700 1 --2- 172.21.15.57:0/1720399645 >> v2:172.21.15.181:3300/0 conn(0x7f4fa40b2ee0 0x7f4fa40ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.679+0000 7f4fa3fff700 1 -- 172.21.15.57:0/1720399645 >> v1:172.21.15.181:6789/0 conn(0x7f4fa40cc200 legacy=0x7f4fa40b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:27.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.679+0000 7f4fa3fff700 1 -- 172.21.15.57:0/1720399645 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa40ccc70 con 0x7f4fa415a5f0 2023-11-28T12:08:27.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.679+0000 7f4fa3fff700 1 --2- 172.21.15.57:0/1720399645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fa415a5f0 0x7f4fa40d0ad0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40b3630 tx=0x7f4f94009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ee25c2422acb56 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:08:27.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.679+0000 7f4fa2ffd700 1 -- 172.21.15.57:0/1720399645 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94011830 con 0x7f4fa415a5f0 2023-11-28T12:08:27.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.679+0000 7f4fa2ffd700 1 -- 172.21.15.57:0/1720399645 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f94011990 con 0x7f4fa415a5f0 2023-11-28T12:08:27.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.679+0000 7f4fa2ffd700 1 -- 172.21.15.57:0/1720399645 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94011d20 con 0x7f4fa415a5f0 2023-11-28T12:08:27.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.679+0000 7f4faa90c700 1 -- 172.21.15.57:0/1720399645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fa415a5f0 msgr2=0x7f4fa40d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:27.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.679+0000 7f4faa90c700 1 --2- 172.21.15.57:0/1720399645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fa415a5f0 0x7f4fa40d0ad0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40b3630 tx=0x7f4f94009bd0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.680+0000 7f4faa90c700 1 -- 172.21.15.57:0/1720399645 shutdown_connections 2023-11-28T12:08:27.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.680+0000 7f4faa90c700 1 --2- 172.21.15.57:0/1720399645 >> v2:172.21.15.181:3300/0 conn(0x7f4fa40b2ee0 0x7f4fa40ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.680+0000 7f4faa90c700 1 --2- 172.21.15.57:0/1720399645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fa415a5f0 0x7f4fa40d0ad0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.680+0000 7f4faa90c700 1 -- 172.21.15.57:0/1720399645 >> 172.21.15.57:0/1720399645 conn(0x7f4fa406ea80 msgr2=0x7f4fa406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:08:27.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.680+0000 7f4faa90c700 1 -- 172.21.15.57:0/1720399645 shutdown_connections 2023-11-28T12:08:27.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.680+0000 7f4faa90c700 1 -- 172.21.15.57:0/1720399645 wait complete. 2023-11-28T12:08:27.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.681+0000 7f4faa90c700 1 Processor -- start 2023-11-28T12:08:27.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.681+0000 7f4faa90c700 1 -- start start 2023-11-28T12:08:27.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4faa90c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa40b2ee0 0x7f4fa415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:27.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4faa90c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa40cc200 0x7f4fa414a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:27.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4faa90c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fa415a5f0 0x7f4fa415e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:27.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4faa90c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa415c130 con 0x7f4fa415a5f0 2023-11-28T12:08:27.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4faa90c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa415b760 con 0x7f4fa40b2ee0 2023-11-28T12:08:27.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4faa90c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa414a6a0 con 0x7f4fa40cc200 2023-11-28T12:08:27.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4fa8ea9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fa415a5f0 0x7f4fa415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:27.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4fa8ea9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fa415a5f0 0x7f4fa415e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55330/0 (socket says 172.21.15.57:55330) 2023-11-28T12:08:27.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4fa8ea9700 1 -- 172.21.15.57:0/1568448940 learned_addr learned my addr 172.21.15.57:0/1568448940 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:08:27.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4fa37fe700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa40cc200 0x7f4fa414a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:27.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4fa3fff700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa40b2ee0 0x7f4fa415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:27.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4fa8ea9700 1 -- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa40cc200 msgr2=0x7f4fa414a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:27.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4fa8ea9700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa40cc200 0x7f4fa414a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4fa8ea9700 1 -- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa40b2ee0 msgr2=0x7f4fa415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:27.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4fa8ea9700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa40b2ee0 0x7f4fa415bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.682+0000 7f4fa8ea9700 1 -- 172.21.15.57:0/1568448940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f94009550 con 0x7f4fa415a5f0 2023-11-28T12:08:27.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.683+0000 7f4fa8ea9700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fa415a5f0 0x7f4fa415e110 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00efc0 tx=0x7f4f9c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:08:27.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.683+0000 7f4fa0ff9700 1 -- 172.21.15.57:0/1568448940 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c009b20 con 0x7f4fa415a5f0 2023-11-28T12:08:27.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.683+0000 7f4fa0ff9700 1 -- 172.21.15.57:0/1568448940 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c00f040 con 0x7f4fa415a5f0 2023-11-28T12:08:27.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.683+0000 7f4faa90c700 1 -- 172.21.15.57:0/1568448940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa40cd230 con 0x7f4fa415a5f0 2023-11-28T12:08:27.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.683+0000 7f4fa0ff9700 1 -- 172.21.15.57:0/1568448940 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00cc70 con 0x7f4fa415a5f0 2023-11-28T12:08:27.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.683+0000 7f4faa90c700 1 -- 172.21.15.57:0/1568448940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa40ccf50 con 0x7f4fa415a5f0 2023-11-28T12:08:27.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.684+0000 7f4faa90c700 1 -- 172.21.15.57:0/1568448940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa40cd510 con 0x7f4fa415a5f0 2023-11-28T12:08:27.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.684+0000 7f4fa0ff9700 1 -- 172.21.15.57:0/1568448940 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4f9c014480 con 0x7f4fa415a5f0 2023-11-28T12:08:27.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.685+0000 7f4fa0ff9700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4f900682f0 0x7f4f9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:27.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.685+0000 7f4fa0ff9700 1 -- 172.21.15.57:0/1568448940 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4f9c086580 con 0x7f4fa415a5f0 2023-11-28T12:08:27.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.685+0000 7f4fa3fff700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4f900682f0 0x7f4f9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:27.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.686+0000 7f4fa3fff700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4f900682f0 0x7f4f9006a7a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009c10 tx=0x7f4f94008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:08:27.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.689+0000 7f4fa0ff9700 1 -- 172.21.15.57:0/1568448940 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4f9c052ff0 con 0x7f4fa415a5f0 2023-11-28T12:08:27.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.888+0000 7f4faa90c700 1 -- 172.21.15.57:0/1568448940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f4fa40743f0 con 0x7f4fa415a5f0 2023-11-28T12:08:27.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.896+0000 7f4fa0ff9700 1 -- 172.21.15.57:0/1568448940 <== mon.0 v2:172.21.15.57:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4f9c052ff0 con 0x7f4fa415a5f0 2023-11-28T12:08:27.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.896+0000 7f4fa0ff9700 1 -- 172.21.15.57:0/1568448940 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7f4f9c018070 con 0x7f4fa415a5f0 2023-11-28T12:08:27.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.899+0000 7f4faa90c700 1 -- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4f900682f0 msgr2=0x7f4f9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:27.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.899+0000 7f4faa90c700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4f900682f0 0x7f4f9006a7a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009c10 tx=0x7f4f94008030 comp rx=0 tx=0).stop 2023-11-28T12:08:27.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.899+0000 7f4faa90c700 1 -- 172.21.15.57:0/1568448940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fa415a5f0 msgr2=0x7f4fa415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:27.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.899+0000 7f4faa90c700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fa415a5f0 0x7f4fa415e110 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00efc0 tx=0x7f4f9c009f60 comp rx=0 tx=0).stop 2023-11-28T12:08:27.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.899+0000 7f4faa90c700 1 -- 172.21.15.57:0/1568448940 shutdown_connections 2023-11-28T12:08:27.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.899+0000 7f4faa90c700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f4f900682f0 0x7f4f9006a7a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.899+0000 7f4faa90c700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fa415a5f0 0x7f4fa415e110 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.899+0000 7f4faa90c700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa40b2ee0 0x7f4fa415bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.899+0000 7f4faa90c700 1 --2- 172.21.15.57:0/1568448940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa40cc200 0x7f4fa414a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:27.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.899+0000 7f4faa90c700 1 -- 172.21.15.57:0/1568448940 >> 172.21.15.57:0/1568448940 conn(0x7f4fa406ea80 msgr2=0x7f4fa4154bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:08:27.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.899+0000 7f4faa90c700 1 -- 172.21.15.57:0/1568448940 shutdown_connections 2023-11-28T12:08:27.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:27.899+0000 7f4faa90c700 1 -- 172.21.15.57:0/1568448940 wait complete. 2023-11-28T12:08:27.913 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T12:08:27.914 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T12:08:28.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.049+0000 7f15b8895700 1 Processor -- start 2023-11-28T12:08:28.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.049+0000 7f15b8895700 1 -- start start 2023-11-28T12:08:28.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.050+0000 7f15b8895700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15b406a270 0x7f15b406a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:28.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.050+0000 7f15b8895700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15b406e570 0x7f15b4070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:28.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.050+0000 7f15b8895700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b406ab80 con 0x7f15b415cc80 2023-11-28T12:08:28.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.050+0000 7f15b8895700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b40b2490 con 0x7f15b406a270 2023-11-28T12:08:28.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.050+0000 7f15b8895700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40b1810 con 0x7f15b406e570 2023-11-28T12:08:28.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.050+0000 7f15b259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15b406a270 0x7f15b406a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:28.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.050+0000 7f15b2d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15b406e570 0x7f15b4070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:28.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.050+0000 7f15b259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15b406a270 0x7f15b406a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:51906/0 (socket says 172.21.15.57:51906) 2023-11-28T12:08:28.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.050+0000 7f15b259c700 1 -- 172.21.15.57:0/387921258 learned_addr learned my addr 172.21.15.57:0/387921258 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:08:28.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.050+0000 7f15b2d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15b406e570 0x7f15b4070d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.57:47380/0 (socket says 172.21.15.57:47380) 2023-11-28T12:08:28.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.051+0000 7f15b259c700 1 -- 172.21.15.57:0/387921258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15b406e570 msgr2=0x7f15b4070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:08:28.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.051+0000 7f15b259c700 1 --2- 172.21.15.57:0/387921258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15b406e570 0x7f15b4070d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:28.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.051+0000 7f15b259c700 1 -- 172.21.15.57:0/387921258 >> v1:172.21.15.181:6789/0 conn(0x7f15b415cc80 legacy=0x7f15b406de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:28.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.051+0000 7f15b259c700 1 -- 172.21.15.57:0/387921258 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b40b1af0 con 0x7f15b406a270 2023-11-28T12:08:28.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.051+0000 7f15b159a700 1 -- 172.21.15.57:0/387921258 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797822051 0 0) 0x7f15b406ab80 con 0x7f15b415cc80 2023-11-28T12:08:28.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.051+0000 7f15b2d9d700 1 --2- 172.21.15.57:0/387921258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15b406e570 0x7f15b4070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:08:28.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.051+0000 7f15b259c700 1 --2- 172.21.15.57:0/387921258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15b406a270 0x7f15b406a640 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f15a40098f0 tx=0x7f15a4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdecd1e535e03ee9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:08:28.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.051+0000 7f15b159a700 1 -- 172.21.15.57:0/387921258 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4011830 con 0x7f15b406a270 2023-11-28T12:08:28.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.051+0000 7f15b159a700 1 -- 172.21.15.57:0/387921258 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f15a4011990 con 0x7f15b406a270 2023-11-28T12:08:28.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.051+0000 7f15b159a700 1 -- 172.21.15.57:0/387921258 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4011d40 con 0x7f15b406a270 2023-11-28T12:08:28.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.052+0000 7f15b8895700 1 -- 172.21.15.57:0/387921258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15b406a270 msgr2=0x7f15b406a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:28.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.052+0000 7f15b8895700 1 --2- 172.21.15.57:0/387921258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15b406a270 0x7f15b406a640 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f15a40098f0 tx=0x7f15a4009c00 comp rx=0 tx=0).stop 2023-11-28T12:08:28.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.052+0000 7f15b8895700 1 -- 172.21.15.57:0/387921258 shutdown_connections 2023-11-28T12:08:28.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.052+0000 7f15b8895700 1 --2- 172.21.15.57:0/387921258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15b406a270 0x7f15b406a640 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:28.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.052+0000 7f15b8895700 1 --2- 172.21.15.57:0/387921258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15b406e570 0x7f15b4070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:28.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.052+0000 7f15b8895700 1 -- 172.21.15.57:0/387921258 >> 172.21.15.57:0/387921258 conn(0x7f15b406c010 msgr2=0x7f15b40cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:08:28.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.052+0000 7f15b8895700 1 -- 172.21.15.57:0/387921258 shutdown_connections 2023-11-28T12:08:28.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.052+0000 7f15b8895700 1 -- 172.21.15.57:0/387921258 wait complete. 2023-11-28T12:08:28.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.053+0000 7f15b8895700 1 Processor -- start 2023-11-28T12:08:28.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.053+0000 7f15b8895700 1 -- start start 2023-11-28T12:08:28.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.054+0000 7f15b8895700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15b406a270 0x7f15b4157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:28.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.054+0000 7f15b8895700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15b406e570 0x7f15b40b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:28.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.054+0000 7f15b8895700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15b415cc80 0x7f15b415f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:28.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.054+0000 7f15b8895700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b41585a0 con 0x7f15b406a270 2023-11-28T12:08:28.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.054+0000 7f15b8895700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b4157990 con 0x7f15b415cc80 2023-11-28T12:08:28.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.054+0000 7f15b8895700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b4157c70 con 0x7f15b406e570 2023-11-28T12:08:28.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.054+0000 7f15b259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15b406a270 0x7f15b4157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:28.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.054+0000 7f15b259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15b406a270 0x7f15b4157340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55336/0 (socket says 172.21.15.57:55336) 2023-11-28T12:08:28.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.054+0000 7f15b259c700 1 -- 172.21.15.57:0/723583270 learned_addr learned my addr 172.21.15.57:0/723583270 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-11-28T12:08:28.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15b2d9d700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15b415cc80 0x7f15b415f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:28.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15b1d9b700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15b406e570 0x7f15b40b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:28.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15b259c700 1 -- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15b406e570 msgr2=0x7f15b40b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:28.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15b259c700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15b406e570 0x7f15b40b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:28.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15b259c700 1 -- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15b415cc80 msgr2=0x7f15b415f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:28.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15b259c700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15b415cc80 0x7f15b415f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:28.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15b259c700 1 -- 172.21.15.57:0/723583270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac009a70 con 0x7f15b406a270 2023-11-28T12:08:28.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15b1d9b700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15b406e570 0x7f15b40b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:08:28.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15b259c700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15b406a270 0x7f15b4157340 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009c40 tx=0x7f15a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:08:28.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15a2ffd700 1 -- 172.21.15.57:0/723583270 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a40045d0 con 0x7f15b406a270 2023-11-28T12:08:28.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15a2ffd700 1 -- 172.21.15.57:0/723583270 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f15a401e070 con 0x7f15b406a270 2023-11-28T12:08:28.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15b8895700 1 -- 172.21.15.57:0/723583270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a4009550 con 0x7f15b406a270 2023-11-28T12:08:28.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15a2ffd700 1 -- 172.21.15.57:0/723583270 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4003900 con 0x7f15b406a270 2023-11-28T12:08:28.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.055+0000 7f15b8895700 1 -- 172.21.15.57:0/723583270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b40b20b0 con 0x7f15b406a270 2023-11-28T12:08:28.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.056+0000 7f15b8895700 1 -- 172.21.15.57:0/723583270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b40b1dd0 con 0x7f15b406a270 2023-11-28T12:08:28.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.057+0000 7f15a2ffd700 1 -- 172.21.15.57:0/723583270 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f15a400f050 con 0x7f15b406a270 2023-11-28T12:08:28.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.057+0000 7f15a2ffd700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15940682f0 0x7f159406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:08:28.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.057+0000 7f15a2ffd700 1 -- 172.21.15.57:0/723583270 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f15a4086850 con 0x7f15b406a270 2023-11-28T12:08:28.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.057+0000 7f15b1d9b700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15940682f0 0x7f159406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:08:28.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.058+0000 7f15b1d9b700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15940682f0 0x7f159406a7a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f15b4156e10 tx=0x7f15a8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:08:28.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.062+0000 7f15a2ffd700 1 -- 172.21.15.57:0/723583270 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f15a40533b0 con 0x7f15b406a270 2023-11-28T12:08:28.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.306+0000 7f15b8895700 1 -- 172.21.15.57:0/723583270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15b415daa0 con 0x7f15b406a270 2023-11-28T12:08:28.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.307+0000 7f15a2ffd700 1 -- 172.21.15.57:0/723583270 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f15a4015de0 con 0x7f15b406a270 2023-11-28T12:08:28.308 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-11-28T12:08:28.309 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2317025916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2317025916},{"type":"v1","addr":"172.21.15.181:6836","nonce":2317025916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3486085199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3486085199},{"type":"v1","addr":"172.21.15.181:6837","nonce":3486085199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6834/4272259116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":4272259116},{"type":"v1","addr":"172.21.15.57:6834","nonce":4272259116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6759,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/3764175384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6833","nonce":3764175384},{"type":"v1","addr":"172.21.15.57:6835","nonce":3764175384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[]} 2023-11-28T12:08:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.309+0000 7f15b8895700 1 -- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15940682f0 msgr2=0x7f159406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.309+0000 7f15b8895700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15940682f0 0x7f159406a7a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f15b4156e10 tx=0x7f15a8009040 comp rx=0 tx=0).stop 2023-11-28T12:08:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.309+0000 7f15b8895700 1 -- 172.21.15.57:0/723583270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15b406a270 msgr2=0x7f15b4157340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:08:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.309+0000 7f15b8895700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15b406a270 0x7f15b4157340 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009c40 tx=0x7f15a4004750 comp rx=0 tx=0).stop 2023-11-28T12:08:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.309+0000 7f15b8895700 1 -- 172.21.15.57:0/723583270 shutdown_connections 2023-11-28T12:08:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.309+0000 7f15b8895700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:6832/120259,v1:172.21.15.181:6833/120259] conn(0x7f15940682f0 0x7f159406a7a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.309+0000 7f15b8895700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15b415cc80 0x7f15b415f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:28.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.309+0000 7f15b8895700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15b406a270 0x7f15b4157340 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:28.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.309+0000 7f15b8895700 1 --2- 172.21.15.57:0/723583270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15b406e570 0x7f15b40b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:08:28.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.309+0000 7f15b8895700 1 -- 172.21.15.57:0/723583270 >> 172.21.15.57:0/723583270 conn(0x7f15b406c010 msgr2=0x7f15b40b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:08:28.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.309+0000 7f15b8895700 1 -- 172.21.15.57:0/723583270 shutdown_connections 2023-11-28T12:08:28.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-11-28T12:08:28.309+0000 7f15b8895700 1 -- 172.21.15.57:0/723583270 wait complete. 2023-11-28T12:08:28.313 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 35 2023-11-28T12:08:28.324 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-28T12:08:28.324 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-28T12:08:28.324 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T12:08:28.325 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.333 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T12:08:28.334 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T12:08:28.334 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.384 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T12:08:28.384 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T12:08:28.384 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.395 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T12:08:28.395 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T12:08:28.395 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.445 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T12:08:28.446 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-28T12:08:28.446 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-28T12:08:28.446 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.475 INFO:tasks.ceph.osd.0:Stopped 2023-11-28T12:08:28.475 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-28T12:08:28.475 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.541 INFO:tasks.ceph.osd.1:Stopped 2023-11-28T12:08:28.541 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-28T12:08:28.541 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.612 INFO:tasks.ceph.osd.2:Stopped 2023-11-28T12:08:28.612 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-28T12:08:28.612 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.686 INFO:tasks.ceph.osd.3:Stopped 2023-11-28T12:08:28.687 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-28T12:08:28.687 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.726 INFO:tasks.ceph.osd.4:Stopped 2023-11-28T12:08:28.726 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-28T12:08:28.727 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.792 INFO:tasks.ceph.osd.5:Stopped 2023-11-28T12:08:28.792 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-28T12:08:28.792 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.865 INFO:tasks.ceph.osd.6:Stopped 2023-11-28T12:08:28.866 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-28T12:08:28.866 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.934 INFO:tasks.ceph.osd.7:Stopped 2023-11-28T12:08:28.934 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-28T12:08:28.935 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T12:08:28.935 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.961 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T12:08:28.962 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-28T12:08:28.962 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:28.999 INFO:tasks.ceph.mgr.x:Stopped 2023-11-28T12:08:29.000 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-28T12:08:29.000 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-28T12:08:29.000 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:29.016 INFO:tasks.ceph.mon.a:Stopped 2023-11-28T12:08:29.017 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-28T12:08:29.017 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:29.052 INFO:tasks.ceph.mon.b:Stopped 2023-11-28T12:08:29.052 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-28T12:08:29.053 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T12:08:29.103 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-11-28T12:08:29.103 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-11-28T12:08:29.104 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-11-28T12:08:29.104 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-11-28T12:08:29.104 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-11-28T12:08:29.104 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-11-28T12:08:29.105 INFO:tasks.ceph.mon.c:Stopped 2023-11-28T12:08:29.105 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-28T12:08:29.106 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-28T12:08:29.148 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:08:29.148 DEBUG:teuthology.orchestra.run.smithi057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-28T12:08:29.388 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:08:29.388 DEBUG:teuthology.orchestra.run.smithi057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-28T12:08:29.587 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:08:29.587 DEBUG:teuthology.orchestra.run.smithi057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-28T12:08:29.660 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:08:29.660 DEBUG:teuthology.orchestra.run.smithi057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-28T12:08:29.789 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:08:29.789 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-28T12:08:29.952 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:08:29.952 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-28T12:08:30.072 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:08:30.073 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-28T12:08:30.128 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:08:30.129 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-28T12:08:30.202 INFO:tasks.ceph:Archiving mon data... 2023-11-28T12:08:30.204 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962/data/mon.a.tgz 2023-11-28T12:08:30.204 DEBUG:teuthology.orchestra.run.smithi057:> mktemp 2023-11-28T12:08:30.229 INFO:teuthology.orchestra.run.smithi057.stdout:/tmp/tmp.it81pn2PMw 2023-11-28T12:08:30.230 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.it81pn2PMw 2023-11-28T12:08:30.407 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0666 /tmp/tmp.it81pn2PMw 2023-11-28T12:08:30.506 DEBUG:teuthology.orchestra.remote:smithi057:/tmp/tmp.it81pn2PMw is 423KB 2023-11-28T12:08:30.583 DEBUG:teuthology.orchestra.run.smithi057:> rm -fr /tmp/tmp.it81pn2PMw 2023-11-28T12:08:30.606 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962/data/mon.b.tgz 2023-11-28T12:08:30.606 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2023-11-28T12:08:30.627 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.h9qLOcAugG 2023-11-28T12:08:30.627 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.h9qLOcAugG 2023-11-28T12:08:30.824 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.h9qLOcAugG 2023-11-28T12:08:30.912 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.h9qLOcAugG is 546KB 2023-11-28T12:08:31.015 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.h9qLOcAugG 2023-11-28T12:08:31.032 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962/data/mon.c.tgz 2023-11-28T12:08:31.032 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2023-11-28T12:08:31.089 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.gHb219OHp6 2023-11-28T12:08:31.089 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.gHb219OHp6 2023-11-28T12:08:31.259 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.gHb219OHp6 2023-11-28T12:08:31.345 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.gHb219OHp6 is 546KB 2023-11-28T12:08:31.443 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.gHb219OHp6 2023-11-28T12:08:31.460 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-28T12:08:31.461 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T12:08:31.463 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T12:08:31.526 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-28T12:08:31.526 DEBUG:teuthology.orchestra.run.smithi057:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T12:08:31.539 DEBUG:teuthology.orchestra.run.smithi181:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T12:08:31.572 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T12:08:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T12:08:31.599 INFO:tasks.ceph:Archiving crash dumps... 2023-11-28T12:08:31.601 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962/remote/smithi057/crash 2023-11-28T12:08:31.602 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T12:08:31.654 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962/remote/smithi181/crash 2023-11-28T12:08:31.655 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T12:08:31.681 INFO:tasks.ceph:Shutting down logrotate 2023-11-28T12:08:31.682 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:08:31.699 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:08:31.787 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T12:08:31.821 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T12:08:31.847 INFO:tasks.ceph:Compressing logs... 2023-11-28T12:08:31.847 DEBUG:teuthology.orchestra.run.smithi057:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T12:08:31.863 DEBUG:teuthology.orchestra.run.smithi181:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T12:08:33.903 INFO:tasks.ceph:Archiving logs... 2023-11-28T12:08:33.903 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962/remote/smithi057/log 2023-11-28T12:08:33.905 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T12:08:34.135 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-11-28T12:08:34.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-11-28T12:08:34.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-11-28T12:08:34.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-11-28T12:08:34.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-11-28T12:08:34.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-11-28T12:08:34.477 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962/remote/smithi181/log 2023-11-28T12:08:34.481 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T12:08:34.750 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-28T12:08:34.762 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-28T12:08:34.763 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T12:08:34.765 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T12:08:34.858 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-28T12:08:34.869 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-28T12:08:34.870 DEBUG:teuthology.orchestra.run.smithi057:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T12:08:34.872 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T12:08:34.892 INFO:teuthology.orchestra.run.smithi057.stderr:bash: ntpq: command not found 2023-11-28T12:08:34.892 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-11-28T12:08:34.897 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T12:08:34.897 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2023-11-28T12:08:34.897 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 55 +873us[ +873us] +/- 75ms 2023-11-28T12:08:34.897 INFO:teuthology.orchestra.run.smithi057.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -568us[ -567us] +/- 59ms 2023-11-28T12:08:34.898 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -165us[ -165us] +/- 74ms 2023-11-28T12:08:34.898 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:08:34.899 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T12:08:34.899 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-11-28T12:08:34.900 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 55 +906us[ +907us] +/- 75ms 2023-11-28T12:08:34.900 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -532us[ -531us] +/- 59ms 2023-11-28T12:08:34.900 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 -125us[ -124us] +/- 74ms 2023-11-28T12:08:34.900 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:08:34.900 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-28T12:08:34.913 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-28T12:08:34.914 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-28T12:08:34.952 DEBUG:teuthology.orchestra.run.smithi057:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T12:08:35.070 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T12:08:35.169 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T12:08:35.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:08:35.209 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T12:08:35.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:08:35.242 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-28T12:08:35.253 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-28T12:08:35.293 INFO:teuthology.task.internal:Duration was 1580.663947 seconds 2023-11-28T12:08:35.294 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-28T12:08:35.343 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-28T12:08:35.344 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T12:08:35.347 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T12:08:35.394 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T12:08:35.404 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T12:08:35.860 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-28T12:08:35.860 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi057.front.sepia.ceph.com 2023-11-28T12:08:35.861 DEBUG:teuthology.orchestra.run.smithi057:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T12:08:35.891 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2023-11-28T12:08:35.891 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T12:08:35.916 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-28T12:08:35.916 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T12:08:35.934 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T12:08:35.996 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-28T12:08:35.996 DEBUG:teuthology.orchestra.run.smithi057:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T12:08:35.999 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T12:08:36.104 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-28T12:08:36.115 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-28T12:08:36.115 DEBUG:teuthology.orchestra.run.smithi057:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T12:08:36.148 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T12:08:36.175 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-28T12:08:36.187 DEBUG:teuthology.orchestra.run.smithi057:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T12:08:36.190 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T12:08:36.222 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = core 2023-11-28T12:08:36.241 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2023-11-28T12:08:36.254 DEBUG:teuthology.orchestra.run.smithi057:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T12:08:36.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:08:36.302 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T12:08:36.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:08:36.319 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-28T12:08:36.332 INFO:teuthology.task.internal:Transferring archived files... 2023-11-28T12:08:36.332 DEBUG:teuthology.misc:Transferring archived files from smithi057:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962/remote/smithi057 2023-11-28T12:08:36.333 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T12:08:36.389 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962/remote/smithi181 2023-11-28T12:08:36.389 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T12:08:36.426 INFO:teuthology.task.internal:Removing archive directory... 2023-11-28T12:08:36.426 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T12:08:36.429 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T12:08:36.481 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-28T12:08:36.599 INFO:teuthology.task.internal:Not uploading archives. 2023-11-28T12:08:36.599 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-28T12:08:36.642 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-28T12:08:36.642 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T12:08:36.645 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T12:08:36.662 INFO:teuthology.orchestra.run.smithi181.stdout: 525836 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 12:08 /home/ubuntu/cephtest 2023-11-28T12:08:36.665 INFO:teuthology.orchestra.run.smithi057.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 12:08 /home/ubuntu/cephtest 2023-11-28T12:08:36.669 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-28T12:08:36.693 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-28T12:08:36.801 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T12:08:36.827 DEBUG:teuthology.nuke:shortname: smithi057 2023-11-28T12:08:36.827 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T12:08:36.854 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 11:31:37.858995', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA6+4W7Ok7NoF6bVdQc/c/LOeeQAcwpH+6HftP+t3kY+m8ErJj7jYo3zzEqgHMOHWK+VUvA6Fl7wmHXamG87gGs='} 2023-11-28T12:08:36.855 DEBUG:teuthology.nuke:shortname: smithi181 2023-11-28T12:08:36.856 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T12:08:36.879 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469962', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 11:31:37.857232', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBrSn8IPPHQCmw++kQtET3aogXGzcc8RD5mq/rRSwIrRlek5htPlT3joY4zEeml9nV4qK/jmu/p0jz1EVyYBsYI='} 2023-11-28T12:08:36.910 INFO:teuthology.orchestra.console.smithi057:Power off 2023-11-28T12:08:36.911 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T12:08:36.928 INFO:teuthology.orchestra.console.smithi181:Power off 2023-11-28T12:08:36.929 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T12:08:36.946 DEBUG:teuthology.orchestra.console.smithi057:power off output: Chassis Power Control: Down/Off 2023-11-28T12:08:36.946 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T12:08:36.964 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-11-28T12:08:36.965 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T12:08:36.980 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-11-28T12:08:36.991 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-11-28T12:08:40.981 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T12:08:40.992 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T12:08:41.093 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-11-28T12:08:41.103 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-11-28T12:08:45.097 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T12:08:45.106 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T12:08:45.208 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-11-28T12:08:45.216 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-11-28T12:08:45.316 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-11-28T12:08:49.208 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T12:08:49.322 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is off 2023-11-28T12:08:49.423 INFO:teuthology.orchestra.console.smithi057:Power off completed 2023-11-28T12:08:49.528 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} duration: 1580.6639473438263 failure_reason: 'Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=2b616a9481d24e0da0c47448b7a9ab16 status: fail success: false 2023-11-28T12:08:49.528 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T12:08:49.613 INFO:teuthology.run:FAIL